Need to check regex – here is a cool tool  https://regexr.com/ aka building an syslog parser for CheckPoint

Hi dear,

i just want to make you aware of a cool tool I found recently at the Internet while I had the need to verify and checking REGEX. The original task was to build an 3rd party log file parser for syslog messages send from an 3rd party firewall to a central CheckPoint SmartEvent log server. While many parsers are included already with CheckPoint SmartEvent, there are still systems in the wild which sending logs not „understandable“ by CheckPoint. 

While Checkpoint provides and tool called „Eventia Log Parsing Editor“  to generate the parsers based on tons of logs you can provide, the output of this tools is a not very effective piece of regex code, which finally results on  high CPU load when processing thousands of logs per second. This is especially the case when you feed it with very specific logs and many different fields. 

Since you usually know your source logs and also the fields you would like to extract, it would make sense to build the parser by hand with can result in a very compact regex code and finally an parser which runs with a very good performance.  The syntax of the parser files and their general structure are documented in the CheckPoint Logging and Monitoring guide  – Appendix: Manual Syslog Parsing.

The core of building an good parser is at its core to build a REGEX which groups your input SYSLOG message into groups, which contain the information you would like to extract and assign to the CheckPoint log fields e.g. source IP address, destination IP address, port, service, URL…. 

For building and debugging REGEX I searched for locally install-able solution on Linux, but was not really successful. So I finally ended up to use an browser based REGEX testers and debugger hosted at https://regexr.com/

REGEXR allows you to input serveral lines of input text/logs and then develop and test your regex code and verify on the fly whether its parsing right and grouping works and extracts the information you need. You can then click the input line by line and check if the data is extracted. If you are OK with your results you can just cut&paste the REGEX into your parser file for CheckPoint.

tltr;

Go to https://regexr.com/ if you need a easy tool to check your REGEX code against your input text/logs.