Ingest Logs

Prerequisites

• Logpoint: v6.7.0 or later

• Logpoint: v7.4.0 or later for log source template

• BIG-IP Access: Syslog forwarding configured on BIG-IP devices to send logs to Logpoint

Install BIG-IP

1. Download the .pak file from the Help Center.

2. Install the Package

   1. Go to Settings >> System Settings from the navigation bar.

   2. Click Applications.

   3. Click Import.

   4. Browse to the downloaded .pak file.

   5. Click Upload.

3. Verify Installation After installation, verify the integration appears under Settings >> System Settings >> Plugins.

Configure BIG-IP

You can configure BIG-IP using two methods:

1. Log Source Template (recommended), which provides a centralized interface for all integrations and minimizes setup requirements

2. Devices

Method 1: Configure via Log Source Template

For Logpoint v7.4.0 and above:

You must create a log source using the log source template to receive the normalized BIG-IP logs.

1. Go to Settings >> Log Sources from the navigation bar.

2. Click Browse Log Source Templates and select F5.

For detailed configuration steps using log source templates, refer to the Logpoint documentation on Creating Log Source via a Template.

Source Configuration

Configure the log source settings:

1. Click Source.

2. Enter the Log Source's Name.

3. Select the Device Addresses.

4. Select the Device Groups.

5. Select a Time Zone. The timezone of the device must be same as its log source.

6. Configure the Risk Values for Confidentiality, Integrity and Availability used to calculate the risk levels of the alerts generated from the device.

Connector Configuration

Configure the connection to BIG-IP:

1. Click Connector.

2. Select Syslog Parser as Parser.

3. Select the Charset.

4. In Proxy Server, select None (or configure proxy settings if needed).

Routing Configuration

Set up log storage and routing:

Create Repository

1. Click Routing and + Create Repo.

2. Enter a Repo name.

3. In Path, specify the location to store incoming logs.

4. In Retention (Days), set how long logs are kept before automatic deletion.

5. In Availability, select the Remote logpoint and Retention (Days).

6. Click Create Repo.

7. Select the created repo in Repo.

Create Routing Criteria

1. Click + Add row.

2. Enter a Key and Value for log filtering.

3. Select log handling options:

   • Store raw message: Store both incoming and normalized logs.

   • Discard raw message: Keep only normalized logs.

   • Discard entire event: Discard both incoming and normalized logs.

4. Select the target Repository.

Normalization Configuration

Set up log normalization:

1. Click Normalization.

2. Either:

   • Select a previously created normalization policy from the dropdown, or

   • Select BigIPF5CompiledNormalizer from the list and click the swap icon.

Available Normalization Packages:

• LP_BIG-IP ASM Remote Server Format

• LP_F5 Load Balancer

• LP_F5 Load Balancer v11_4_1

• LP_F5 BIG-IP APM v12_x_x

• LP_F5 BIG-IP Process

• LP_F5 BIG-IP AFM Syslog

• LP_F5 LTM and FWM

• LP_F5 Load Balancer v11_6

• LP_F5 BIG_IP Link Controller

• LP_F5 BIG-IP APM v11_x_x

• LP_BIG-IP ASM Reporting Server Format

Select the appropriate packages based on your BIG-IP modules and versions.

Enrichment Configuration

Configure log enrichment:

1. Click Enrichment.

2. Select an Enrichment Policy.

Finalize Configuration: Click Create Log Source to save all configurations.

---

Method 2: Configure via Devices

Configuring a Repo for BIG-IP

1. Go to Settings >> Configuration from the navigation bar and click Repos.

2. Click Add.

3. Enter a Repo Name.

4. Select a Repo Path to store incoming logs.

5. Set a Retention Day to keep logs in a repository before they are automatically deleted.

   Note: You can add and remove multiple Repo Path and Retention Day.

6. Select a Remote LogPoint and set a Available for (day).

7. Click Submit.

Adding a Normalization Policy for BIG-IP

Normalization policies normalize and standardize logs for efficient storage, analysis, and retrieval.

1. Go to Settings >> Configuration from the navigation bar and click Normalization Policies.

2. Click Add.

3. Enter a Policy Name.

4. Select the Compiled Normalizer and Normalization Packages for BIG-IP:

   • Compiled Normalizer: BigIPF5CompiledNormalizer

   • Normalization Packages: Select packages based on your BIG-IP deployment:

     • LP_BIG-IP ASM Remote Server Format (for ASM logs)

     • LP_F5 Load Balancer (for load balancing logs)

     • LP_F5 Load Balancer v11_4_1 or v11_6 (version-specific)

     • LP_F5 BIG-IP APM v11_x_x or v12_x_x (for APM logs)

     • LP_F5 BIG-IP AFM Syslog (for AFM logs)

     • LP_F5 LTM and FWM (for LTM logs)

     • Others as needed

5. Click Submit.

Configuring a Processing Policy for BIG-IP

Processing policy dictates how BIG-IP logs are handled, processed, and stored to enhance their usability and accessibility for monitoring, reporting, and alerting purposes.

1. Go to Settings >> Configuration from the navigation bar and click Processing Policies.

2. Click Add.

3. Enter a Policy Name.

4. Select the previously created normalization policy.

5. Select the Enrichment Policy and Routing Policy.

6. Click Submit.

Adding BIG-IP as a Device in Logpoint

1. Go to Settings >> Configuration from the navigation bar and click Devices.

2. Click Add.

3. Enter a device Name.

4. Enter the BIG-IP server IP address(es).

5. Select the Device Groups.

6. Select an appropriate Log Collection Policy for the logs.

7. Select a collector or a forwarder from the Distributed Collector drop-down.

   Note: It is optional to select the Device Groups, the Log Collection Policy and the Distributed Collector.

8. Select a Time Zone. The timezone of the device must be same as its log source.

9. Configure the Risk Values for Confidentiality, Integrity and Availability used to calculate the risk levels of the alerts generated from the device.

10. Click Submit.

Configuring the Syslog Collector for BIG-IP

1. Go to Settings >> Configuration from the navigation bar and click Devices.

2. Click the Add collectors/fetchers icon under Actions of the previously added device.

3. Click Syslog Collector.

   Note: You can select a different collector depending on your requirements and added device. To learn more about available collectors, refer to the collectors documentation. If you require assistance, contact our support team.

4. Select Syslog Parser as Parser.

5. Select the previously created Processing Policy.

6. Select the Charset.

7. In Proxy Server, select None (or configure proxy settings if needed).

8. Click Submit.

---

Verify Ingestion

Check Log Ingestion

Use the following query to verify BIG-IP logs are being ingested:

col_type = "bigip" OR col_type = "f5"

Or search by normalizer:

norm_id = "F5*" OR norm_id = "BIGIP*"

Verify Data Flow

1. Check Syslog Collector Status: Ensure the BIG-IP collector is running without errors.

2. Monitor Log Volume: Verify expected log volumes are being processed.

3. Validate Normalization: Confirm logs are correctly parsed and normalized using the BigIPF5CompiledNormalizer.

4. Test Dashboards: Access BIG-IP dashboards to verify data visualization.