Signatures are rules to capture important field values from the raw logs. The values are then indexed to simplify the search. You can also use them to compare, aggregate, correlate, and create reports for the logs.
For example:
user Bob logged in from source 192.168.2.10
Applying signature against the raw log above gives you concise information.
user <user:word> <action:all> from source <source_address:ip>
This signature captures the following field-value pairs from the log.
user: Bob
action: logged in
source_address: 192.168.2.10
Note
LogPoint provides normalization policies for different systems and applications. You can create your own signatures inside a normalization package, build them into normalization policies, and use them with your log collector.
Fields are names given to normalized values. Field values can be used in searches to filter specific terms.
For example,
<174>Oct 22 10:52:58 10.141.15.121 user Bob logged in from source 192.168.2.10
The important fields from the above message are user, action, and source_address.
user: Bob
action: logged in
source_address: 192.168.2.10
Values are the words (or terms) extracted from log messages.
In the above example, the field user has the value Bob, the field action has the value Logged in, and the field source_address has the value 192.168.2.10.
Definers are inbuilt customized regex provided by LogPoint.
For example:
User<user:word><action:'logged'>in from<source_address:ip>:
<source_port:port>using service<service_name:word>
The definers like word, ip, and port provide readability in the signatures and help with troubleshooting the signatures effectively.
Go to the LogPoint HelpCenter for a detailed list of definers.
Before writing a signature, remember that regex patterns are always inside < and > symbols. Make sure that your Fields and Definers (or custom re2 patterns) are always between these symbols.
Use the definers word(s) to capture a single word or several respectively.
Log message example:
Detailed Tracking A process has exited: Process ID: 17728
Syntax example:
<status:words>A process has <action:word>
Extracted parameters:
Fields
Values
status
Detailed Tracking
action
exited
It is possible to capture integers and floating values with appropriate definers. Use the int and the float keywords to capture integers and floating values respectively. If you are not sure whether the value is an integer or float, you can use the int_float keyword to capture both types of data.
Log message example:
592 Security SYSTEM User Success Audit LogPoint Detailed Tracking A new process has been created: New Process ID: 16668
Syntax example:
<event_id:int><:all>A new<object:word>has been<action:'created'>: New Process ID:<process_id:int>
Extracted parameters:
Fields |
Values |
|---|---|
event_id |
592 |
object |
process |
action |
created |
process_id |
16668 |
It is possible to capture any type of IP addresses (IPV4 or IPV6) with the definer ip.
Log message example:
%PIX-4-403109: Rec'd packet not an PPTP packet. (ip) dest_address= 192.168.2.14, src_addr= 192.168.2.78, data: string
Syntax example:
<event_id:'403109'><:all>dest_address=<destination_address:ip>, src_addr=<source_address:ip>
Extracted parameters:
Fields |
Values |
|---|---|
event_id |
403109 |
source_adddress |
192.168.2.78 |
destination_address |
192.168.2.14 |
It is possible to capture fields and values separated by signs. Any sign such as =, :, and - can separate the values.
Log messages example:
Benchmarker; reporting speed; service: normalizer_pfsense_and_openvpn; actual_mps=0; doable_mps=878;
Syntax example:
reporting speed;<word:\::all>;<word:=:int>;<word:=:int>;
Note
Use backslash ( \ ) to escape separators key for separators as “:”, “,” “-” etc.
Extracted parameters:
Fields |
Values |
|---|---|
service |
normalizer_pfsense_and_openvpn |
actual_mps |
0 |
doable_mps |
878 |
It is possible to use dynamic mapping to capture similarly placed field-value pairs.
Log message example:
INFO: search; performed; type=audit_log; username='Bob'; query='sig_id=1012 | timechart count()'; time_range=[1326245880, 1326267480]; repos=['127.0.0.1:5504/default', '127.0.0.1:5504/_logpoint']
Syntax example:
<severity:word>:<object:all>; <action:all>;<<word:=:all>;>
Extracted parameters:
Fields |
Values |
|---|---|
severity |
INFO |
object |
search |
action |
performed |
type |
audit_log |
username |
Bob |
query |
sig_id=1012 | timechart count() |
time_range |
1326245880, 1326267480 |
It is possible to capture the values from the log messages using the regex patterns. However, using fields and definers to write signatures is more efficient.
Log message example:
ASA-2-106006: Deny inbound UDP from 192.168.2.222/514 to 202.70.91.12/26 on interface interface_name
Syntax example:
<event_id:'106006'>:<action:'\S+'>inbound<object:'\S+'><:'.*?'> <destination_address:'\d+.\d+.\d+.\d+'><:'/'><destination_port:'\d+'>to <source_address:'\d+.\d+.\d+.\d+'><:'/'><source_port:'\d+'>
Extracted parameters:
Fields |
Values |
|---|---|
event_id |
106006 |
action |
Deny |
object |
UDP |
destination_address |
192.168.2.222 |
destination_port |
514 |
source_address |
202.70.91.12 |
source_port |
26 |
You can use the OR operand with the definers if you are uncertain of the type of data associated with the fields. Switch between the possible definers with the OR operand.
Log message example:
Metrics; Physical Memory; total=2009 MB; use=59.1%; used=1913 MB; free=95 MB
Syntax example:
Physical Memory; <word:=:int>MB; <word:=:int OR float>%; <word:=:int>MB; <word:=:int OR float>MB
Extracted parameters:
Fields |
Values |
|---|---|
total |
2009 |
use |
59.1 |
used |
1913 |
free |
95 |