Signatures and Fields
Signatures
Signatures are rules used to extract important field values from raw logs. The extracted values are indexed, making it easier to search, analyze, and work with logs. You can use signatures to compare, aggregate, and correlate data, and to generate reports from log events.
Signatures are created using fields and definers:
A field is the key in a key–value pair that identifies a specific value in normalized logs.
Definers are simplified regular expressions that make signature creation easier and patterns more readable. They are designed to detect common field types and improve consistency, such as IP addresses, strings, words, and MAC addresses.
Logpoint Vendor Normalization packages include signatures that map logs to the Logpoint taxonomy. Create your own signatures within a normalization package, integrate them into normalization policies, and apply them to your log collector. It is best to use fields and definers, but when no existing field or definer can extract the relevant information, use regex patterns to create a signature.
Example: user Bob logged in from source 192.168.2.10
Signature: user <user:word> <action:all> from source <source_address:ip>
The signature extracts the following key-value pairs:
user:Bobaction: logged insource_address: 192.168.2.10
Write Signatures
Regex patterns are always inside < and > symbols. Always use Fields and Definers (or custom re2 patterns) between these symbols.
Extracting Word(s)
Use the definers word(s) to extract a single word or several words respectively.
Log Example:
Detailed Tracking A process has exited: Process ID: 17728
Syntax Example:
<status:words>A process has <action:word>
Extracted Parameters:
Fields
Values
status
Detailed Tracking
action
exited
Extracting Numbers
It is possible to extract integers and floating values with appropriate definers. Use the int and the float keywords to extract integers and floating values respectively. If you are not sure whether the value is an integer or float, you can use the int_float keyword to extract both types of data.
Log Example:
592 Security SYSTEM User Success Audit Logpoint Detailed Tracking A new process has been created: New Process ID: 16668
Syntax Example:
<event_id:int><:all>A new<object:word>has been<action:'created'>: New Process ID:<process_id:int>
Extracted Parameters:
Fields
Values
event_id
592
object
process
action
created
process_id
16668
Extracting IPs
Use the definer IP to extract IP addresses (IPV4 or IPV6).
Log Example:
%PIX-4-403109: Rec'd packet not an PPTP packet. (ip) dest_address= 192.168.2.14, src_addr= 192.168.2.78, data: string
Syntax Example:
<event_id:'403109'><:all>dest_address=<destination_address:ip>, src_addr=<source_address:ip>
Extracted Parameters:
Fields
Values
event_id
403109
source_adddress
192.168.2.78
destination_address
192.168.2.14
Extracting Field-values Separated by Signs
Any sign, such as =,:, and -, can separate the values. Use backslash ( \ ) to escape separators key for separators as “:”, “,” “-” etc.
Log Example:
Benchmarker; reporting speed; service: normalizer_pfsense_and_openvpn; actual_mps=0; doable_mps=878;
Syntax Example:
reporting speed;<word:\::all>;<word:=:int>;<word:=:int>;
Extracted Parameters:
Fields
Values
service
normalizer_pfsense_and_openvpn
actual_mps
0
doable_mps
878
Extracting field-values using Dynamic Definers
It is possible to use dynamic mapping to extract similarly placed field-value pairs.
Log Example:
INFO: search; performed; type=audit_log; username='Bob'; query='sig_id=1012 | timechart count()'; time_range=[1326245880, 1326267480]; repos=['127.0.0.1:5504/default', '127.0.0.1:5504/_logpoint']
Syntax Example:
<severity:word>:<object:all>; <action:all>;<<word:=:all>;>
Extracted Parameters:
Fields
Values
severity
INFO
object
search
action
performed
type
audit_log
username
Bob
query
sig_id=1012 | timechart count()
time_range
1326245880, 1326267480
Extracting using a Regex Pattern
You can extract the values from the logs using the regex patterns. However, using fields and definers to write signatures is more efficient.
Log Example:
ASA-2-106006: Deny inbound UDP from 192.168.2.222/514 to 202.70.91.12/26 on interface interface_name
Syntax Example:
<event_id:'106006'>:<action:'\S+'>inbound<object:'\S+'><:'.*?'> <source_address:'\d+.\d+.\d+.\d+'><:'/'><source_port:'\d+'>to <destination_address:'\d+.\d+.\d+.\d+'><:'/'><destination_port:'\d+'>
Extracted Parameters:
Fields
Values
event_id
106006
action
Deny
object
UDP
destination_address
202.70.91.12
destination_port
26
source_address
192.168.2.222
source_port
512
Using OR on definers
You can use the OR operator with the definers if you are uncertain about the data types of the fields. Switch among the possible definers using the OR operator.
Log Example:
Metrics; Physical Memory; total=2009 MB; use=59.1%; used=1913 MB; free=95 MB
Syntax Example:
Physical Memory; <word:=:int>MB; <word:=:int OR float>%; <word:=:int>MB; <word:=:int OR float>MB
Extracted Parameters:
Fields
Values
total
2009
use
59.1
used
1913
free
95
Fields
Fields are the keys in key–value pairs that identify values in normalized logs. They act as value identifiers, providing consistent and meaningful names for extracted log data.
A field name can be a single word or multiple words separated by underscores (_). Spaces are not allowed. For example: device, source_address, destination_address, source_port, and destination_port.
Logpoint reserves specific field names, including device_ip, device_name, and msg. Before creating new fields, it is recommended to search indexed logs to verify which field names are already reserved and in use.
Add fields manually, or import a list of fields in a .pak file.
Ordering Fields
The order of fields is important because it controls the sequential arrangement of key–value pairs in the logs, determining how the normalized log entry is structured and displayed.
Exporting & Importing Fields
Exporting and Importing Fields allows you to share and reuse fields across different Logpoints or instances within your environment. By exporting fields, maintain consistency in field definitions and ordering, and import them into another system to avoid manually recreating fields. This helps standardize log normalization and simplifies management across multiple Logpoint deployments.
Available Fields
Use the following fields to write signatures:
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
U
V
W
Z
Last updated
Was this helpful?