# Ingest Logs via Director Console UI

### Prerequisites <a href="#prerequisites" id="prerequisites"></a>

* **Logpoint**: v7.8.0 and later
* **Director**: v2.9.0 and later
* **AWS Access**: Read access to the Amazon S3 bucket and its objects
* **AWS Credentials**: Valid AWS Access Key ID and Secret Access Key
* **Network Access**: Connectivity to AWS S3 endpoints (proxy support available)

### Installation

<details>

<summary>Install S3Fetcher</summary>

1. Log in to Director Console.
2. Click **ASSETS** in the navigation bar.
3. Select **Plugin** from the **Assets** drop-down.
4. Upload the S3Fetcher .pak file. Select **Replace Existing?** to replace the old file with new one.
5. Click **UPLOAD**.

Once uploaded, the .pak file is displayed in the list of available packages.

6. Select the S3Fetcher .pak file from the list.
7. Click **INSTALL**.
8. Select the pool and Logpoint to install S3Fetcher. You can select multiple Logpoint from different pools.
9. Click **NEXT**.
10. Review your changes. You can go **BACK** to make necessary changes.
11. Click **INSTALL** and click **OK** to confirm.

You are redirected to **TASKS**, which displays the installation progress.

</details>

<details>

<summary>Uninstall S3Fetcher</summary>

You must first remove S3Fetcher configurations from Logpoint and then uninstall it.

**To remove S3Fetcher configurations:**

1. Click **CONFIGURE** in the navigation bar.
2. Under **Entities**, click **LOG SOURCES**.
3. Click the **More** icon next to the S3Fetcher log source and click **Delete Log Source**.
4. Click **Delete**.

**To uninstall S3Fetcher:**

1. Click **ASSETS** in the navigation bar.
2. Click **UNINSTALL**.
3. Select the Logpoint where S3Fetcher is installed. You can select multiple Logpoint from different pools.
4. Select **S3Fetcher** from the list of available packages.
5. Click **NEXT**.
6. Review your changes. You can go **BACK** to make necessary changes.
7. Click **UNINSTALL** and click **OK** to confirm.

</details>

### Configuration <a href="#configure-cloud-trail" id="configure-cloud-trail"></a>

Before configuring S3Fetcher in Logpoint, obtain AWS access credentials (**Access Key ID** and **Secret Access Key**). Go to the [AWS Documentation](https://docs.aws.amazon.com/) for instructions on retrieving them.

1. Click **CONFIGURE** in the navigation bar.
2. Under **ENTITIES**, click **LOG SOURCES**.
3. Click **Create Log Source**.
4. Select **S3Fetcher**.
5. Select a Pool and Logpoint to configure the fetcher.
6. Click **NEXT**.

<details>

<summary>Source </summary>

Configure the log source settings:

1. Click **Source.**
2. Enter the **Log Source's Name.**
3. Select the **Fetch Interval (min)** to set how frequently the logs are retrieved.
4. Select the **Charset** and **Time Zone.**

</details>

<details>

<summary>Connector</summary>

Configure the connection to AWS:

1. Click **Connector**.
2. **EndPoint URL**:
   1. For Amazon S3: `https://s3.amazonaws.com` (default).
   2. For third-party services: Enter the service URL.
3. Enter your **AWS Access Key ID** and **AWS Secret Access Key**.
4. Enter the AWS **Bucket Name**. S3Fetcher fetches logs from this bucket.
5. In **Filter by Prefix**, enter a prefix name to fetch a specific file or folder from the bucket. Leave this field empty to fetch the entire bucket.
6. Select the **AWS Region.**
7. Select **Initial Fetch Date**. S3Fetcher fetches logs from the specified date. To change the Initial Fetch Date and re-fetch logs from the new date, enable **Reset Last Fetch Date** and select the new date.
8. If you are using a [Distributed Logpoint,](https://app.gitbook.com/s/KXIpFdXJLhdyVghVnSDh/readme/architectureconfiguration#distributed-logpoint) select Distributed Collectors from the drop-down.
9. Select a **Parser** to parse the logs.
10. **Enable Proxy** to use a proxy server.
    1. Select either **HTTP** or **HTTPS** protocol.
    2. Enter the proxy server **IP** address and the **PORT** number.

</details>

<details>

<summary>Routing</summary>

Set up log storage and routing:

Create Repository:

1. Click **Routing** and **+ Create Repo.**
2. Enter a **Repo name.**
3. In **Path**, specify the location to store incoming logs.
4. In **Retention (Days)**, set how long logs are kept before automatic deletion.
5. In **Availability**, select the **Remote logpoint** and **Retention (Days).**
6. Click **Create Repo.**
7. Select the created repo in **Repo**.

Create Routing Criteria

1. Click **+ Add row.**
2. Enter a **Key** and **Value** for log filtering.
3. Select log handling options:
   1. **Store raw message**: Store both incoming and normalized logs.
   2. **Discard raw message**: Keep only normalized logs.
   3. **Discard entire event**: Discard both incoming and normalized logs.
4. Select the target **Repository.**

</details>

<details>

<summary>Normalization</summary>

Set up log normalization:

1. Click **Normalization**.
2. Either:
   1. Select a previously created normalization policy from the dropdown, or
   2. Select a normalizer from the list and click the swap icon.

</details>

<details>

<summary>Enrichment</summary>

Configure log enrichment:

1. Click **Enrichment.**
2. Select an **Enrichment Policy.**

</details>

7. Click **Create Log Source** to save the configurations of Source, Connector, Routing, Normalization, and Enrichment.
