# Ingest Logs

### Prerequisites <a href="#prerequisites" id="prerequisites"></a>

* **Logpoint**: v7.4.0 or later
* **AWS Access**: Read access to the Amazon S3 bucket and its objects
* **AWS Credentials**: Valid AWS Access Key ID and Secret Access Key
* **Network Access**: Connectivity to AWS S3 endpoints (proxy support available)

### Install Cloud Trail <a href="#install-cloud-trail" id="install-cloud-trail"></a>

1. Download the .pak file, from the Service Desk.
2. Install the Package
   1. Go to **Settings** >> **System Settings** from the navigation bar.
   2. Click **Applications.**
   3. Click **Import.**
   4. Browse to the downloaded `.pak` file.
   5. Click **Upload.**
3. After installation, verify the integration appears under **Settings** >> **System Settings** >> **Plugins**

### Configure Cloud Trail <a href="#configure-cloud-trail" id="configure-cloud-trail"></a>

Before configuring Cloud Trail in Logpoint, obtain AWS access credentials (**Access Key ID** and **Secret Access Key**). Go to the [AWS Documentation](https://docs.aws.amazon.com/) for instructions on retrieving them.

You can configure CloudTrail using two methods:

1. **Log Source**, recommended as it provides a centralized interface for all integrations
2. **Devices**

#### Method 1: Configure via Log Source <a href="#method-1-configure-via-log-source" id="method-1-configure-via-log-source"></a>

**For Logpoint v7.4.0:**

1. Go to **Settings** >> **Log Sources** from the navigation bar.
2. Click **Browse Log Source Templates** and select **Cloud Trail.**

**For Logpoint v7.5.0 and above:**

1. Go to **Settings** >> **Log Sources** and click **+ Add Log Source.**
2. Select **Cloud Trail** from the search bar.

**Source**&#x20;

Configure the log source settings:

1. Click **Source.**
2. Enter the **Log Source's Name.**
3. Select the **Fetch Interval (min)** to set how frequently the logs are retrieved.
4. Select the **Time Zone.**

**Connector**

Configure the connection to AWS:

1. Click **Connector**.
2. **EndPoint URL**:
   * For Amazon S3: `https://s3.amazonaws.com` (default).
   * For third-party services: Enter the service URL.
3. Enter your **AWS Access Key ID**.
4. Enter your **AWS Secret Access Key**.
5. Enter the **Bucket Name** where CloudTrail logs are stored.
6. Select the **AWS Region.**

**Routing**

Set up log storage and routing:

**Create Repository**

1. Click **Routing** and **+ Create Repo.**
2. Enter a **Repo name.**
3. In **Path**, specify the location to store incoming logs.
4. In **Retention (Days)**, set how long logs are kept before automatic deletion.
5. In **Availability**, select the **Remote logpoint** and **Retention (Days).**
6. Click **Create Repo.**
7. Select the created repo in **Repo**.

**Create Routing Criteria**

1. Click **+ Add row.**
2. Enter a **Key** and **Value** for log filtering.
3. Select log handling options:
   * **Store raw message**: Store both incoming and normalized logs.
   * **Discard raw message**: Keep only normalized logs.
   * **Discard entire event**: Discard both incoming and normalized logs.
4. Select the target **Repository.**

**Normalization**

Set up log normalization:

1. Click **Normalization**.
2. Either:
   * Select a previously created normalization policy from the dropdown, or
   * Select **CloudTrailCompiledNormalizer** from the list and click the swap icon.

**Enrichment**

Configure log enrichment:

1. Click **Enrichment.**
2. Select an **Enrichment Policy.**

**Finalize Configuration:**\
Click **Create Log Source** to save all configurations.

#### Method 2: Configure via Devices <a href="#method-2-configure-via-devices" id="method-2-configure-via-devices"></a>

**Adding a Normalization Policy**

1. Go to **Settings** >> **Configuration** >> **Normalization Policies**.
2. Click **Add.**
3. Enter a **Policy Name.**
4. Select **CloudTrailCompiledNormalizer**.
5. Click **Submit.**

**Adding a Processing Policy**

1. Go to **Settings** >> **Configuration** >> **Processing Policies**.
2. Click **Add**.
3. Enter a **Policy Name**.
4. Select the previously created normalization policy.
5. Select **Enrichment Policy** and **Routing Policy**.

**Configuring the CloudTrail Log Fetcher**

1. Go to **Settings** >> **Configuration** >> **Devices**.
2. Click the **Add collectors/fetchers** icon for the localhost device.
3. Click **CloudTrail Log Fetcher**.
4. Click **Add** and enter a **Name**.
5. Enter your **AWS Access Key ID** and **Secret Access Key**.
6. Enter the **Bucket Name** from which Cloud Trail fetches the logs.
7. Select the **AWS Region**.
8. Select the frequency at which logs are retrieved in **Fetch Interval (minutes)**.
9. Select the previously created processing policy.
10. Select the **Charset.**

**Filter Configuration:**

* **Base Path**: Specify the AWS directory path. Leave the field empty to fetch from all directories.
* **Logs From**: Set the starting date for log retrieval.

**Proxy Configuration (if needed):**

1. Select **Enable Proxy**.
2. Enter proxy server **IP address** and **Port number**.
3. Select **HTTP** or **HTTPS** protocol.
4. Click **Submit**.

### Verify Ingestion <a href="#verify-ingestion" id="verify-ingestion"></a>

#### Check Log Ingestion <a href="#check-log-ingestion" id="check-log-ingestion"></a>

Use the following query to verify CloudTrail logs are being ingested:

`col_type = cloudtrail`

#### Verify Data Flow <a href="#verify-data-flow" id="verify-data-flow"></a>

1. **Check Fetcher Status**: Ensure the CloudTrail fetcher is running without errors
2. **Monitor Log Volume**: Verify expected log volumes are being processed
3. **Validate Normalization**: Confirm logs are correctly parsed and normalized. Refer to [the CloudTrail field mapping](https://docs.logpoint.com/integrations-1/aws/aws-cloudtrail-log-reference#field-mapping-reference) to understand how AWS CloudTrail logs correspond to Logpoint fields.
4. **Test Dashboards**: Access CloudTrail dashboards to verify data visualization

&#x20;
