In the world of firewall administration, one very common problem is a host behind a firewall has more access than what was intended.  This occurs mainly due to ‘loosely defined’ rules that happen to ‘catch’ unintended traffic and then inadvertently allows it to pass.  I recently was given a task of reducing access from a set of hosts behind a CheckPoint firewall which had a 1000 rule policy installed, with logging turned on for each one (including the cleanup rule).  My point of describing the environment is that it can quickly become overwhelming to fire up Tracker and begin to piece access information together, especially across multiple days.  In order to get started, the first item of business was to find all the rules this group of hosts were using, which had to be known before implementing the required ‘bare bones’ access.

Before I go any further, it must be stated that commercial packages exist that can do this type of analysis for you.  These software programs usually import CheckPoint logs into a larger data-source and then run various reports against it.  While those packages are extremely valuable to the firewall administrator, often times it is cost prohibitive to the company they work for.  It will be my attempt to share a Do It Yourself, bare bones, just get it done, alternative approach to buying these costly software packages.

As far as prerequisites, not much is needed: I’ll be using a Linux workstation for the utilities, such as cat,grep, and others.  The log data will be imported into a SQLite database for analysis.  Everything I have mentioned thus far is available on a Windows workstation, but will require a little bit of work to find/install it.  My point here is: If you have not taken the plunge to set up a Linux ‘utility’ workstation yet – now would be a great time to knock that out.  Anyway, I will show all my examples and reference the procedure as if it is being performed from a Linux machine.  However, I think it will easily be adaptable to the Windows only administrator.  If not, I will do my best to clarify points as questions are asked.

First, we must get the logs in the format we can work with.  This will require exporting the current CheckPoint log file type to a delimited, plain text type.  The utility required for this will be located in the Firewall1 program directory on the SmartCenter management station.

From a command line on the SmartCenter machine, we want to change to the firewall log directory:

cd \fw1_install_dir\RXX\fw1\log

where fw1_install_dir = SmartCenter installation directory and XX = the version of SmartCenter installed (i.e. R75).

running the ‘dir’ command in this directory will give you the name of the available logfiles for export.  The file names will follow the format of YYYY-MM-DD_HHMMSS_XX.log, select the file for export and run the following:

fwm logexport -n -p -m raw -i [YYYY-MM-DD_HHMMSS_XX.log] -o [YYYY-MM-DD_HHMMSS_XX.txt]

The switches are explained below:

Usage:
fwm logexport [-d delimiter |-s] [-i filename] [-o filename] [-f|-t] [-x start_p
os] [-y end_pos] [-z] [-n] [-p] [-a] [-u unification_scheme_file] [-m (initial|s
emi|raw)]
Where:
-d  - Set the output delimiter. Default is ';'.
-s  - Set the delimiter to be ASCII character #255.
-i  - Input log file name. Default is the active log file, fw.log.
-o  - Output file name. Default is printing to the screen.
-f  - Only in case of active log file - Upon reaching end of file, wait for new
records and export them as well.
-t  - Same as -f flag, only start at end of file.
-x  - Start exporting at the specified position.
-y  - End exporting at the specified position.
-z  - Continue exporting the next records, in case of an error. Default is to stop exporting.
-n  - No IP resolving. Default is to resolve all IPs.
-p  - No port resolving. Default is to resolve all ports.
-a  - Export account records only. Default is export all records.
-u  - Unification scheme file name. Default is log_unification_scheme.C.
-m  - Unification mode: initial-order, semi-unified, or raw. Default is 'initial'.

The switches used in the previous example should be self-explanatory after looking them up using the syntax help above.

Here is the command I ran in my environment

C:\Program Files\CheckPoint\R71\fw1\log>fwm logexport -n -p -m raw -i "2011-09-28_235900_98.log"
 -o "d:\2011-09-28_235900_98.txt"
Starting... There are 20492936 log records in the file
File logexport.ini was opened successfully
Processed 20492936 out of 20492936 records (99%)

Once I did this in my environment for one log file, which contained access information for a 24 hour period, the result was a 5.2G text file.  This would obviously be impossible to open with any editor, which is where our Unix utilities come into play.

At this point, I only want to load the necessary data into the db.  This keeps the database small and makes queries much more responsive.  In order to extract a subset of data from the log output, we will use awk and grep to put the desired results into a separate file.  In this example, I want traffic that was either sourced or destined to 10.16.2.20.

# awk '{q=split($0,a,";");if (NR==1){for (v=1;v<=q;v++) c[a[v]]=v} printf("%s;%s;%s;%s;%s;
%s;%s;%s;%s;%s\n",a[c["date"]],a[c["time"]],a[c["action"]],a[c["rule_uid"]],a[c["rule_name"]],
a[c["src"]],a[c["s_port"]],a[c["dst"]],a[c["service"]],a[c["xlatesrc"]])}' 2011-09-28_235900
_98.txt | grep '\<10.16.2.20\>' > windowsdc01.txt

The previous command uses ‘awk’ to process the file ’2011-09-28_235900_98.txt’ and only print the log fields we are interested in. Awk is being used because for some reason, Checkpoint does not export log files the same way twice.  For example a fwm export one day may contain 51 columns, the next day it might contain only 40.  Obviously this would play havoc on importing the same fields each time into our database.  By extracting just the columns we need, this ensures the same format each time.  This command looks very complex, the only thing you really want to consider is if additional fields are wanted in the output.  If this is the case, just make sure the additional fields are specified in order within the script.  For example, let say you want to add an additional field (i/f_name) to the output.  If you look at the first line of the original exported file, which are the column headers, you will see the “i/f_name” column is between “action” and “rule_uid”.  So hear is what you would add to the existing script (in bold)

a[c["action"]],a[c["i/f_name"]],a[c["rule_uid"]],

You will also need to add an additional %s; after the printf statement for each additional field you add

Moving on, note after the grep command the \< and the \> characters with the ip address in between.  What this does it tell grep to only match this character string if it’s the beginning or end of the word.  If the \< characters were missing from above, then we would also match other hosts like 110.16.2.20 or 210.16.2.20.  Likewise, if the \> were missing off the end, then we would match on 10.16.2.201, 10.16.2.202, and so on.  Finally, the greater than sign followed by a file name, will output the results to a file instead of to the default location of the screen.

Now, I have a separate file that contains only the data I care about at the moment and it is 28Mb vs. the 5Gb source file we started with.  The next thing to do is load it into a sqlite database.  Before we can do that, we have to create the database with a table containing the proper columns to accept the text file import.  We start by invoking sqlite and passing it a variable that will be the name of a new database, which in this example is called data.db.  Once sqlite is invoked, run the SQL script shown below, which is used to create the table. Obviously this sql statement would need to be modified if you added additional fields over what is shown in the previous example.

# sqlite3 data.db
SQLite version 3.7.5
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite>CREATE TABLE tbl_fwlogs(
f_date varchar(10),
f_time varchar(10),
f_action varchar(10),
f_rule_uid varchar(100),
f_rule_name varchar(15),
f_src varchar(10),
f_srcport varchar(10),
f_dst varchar(15),
f_service varchar(10),
f_xlatesrc varchar(10));
sqlite>

Define the separator used in the import file

sqlite> .separator ";"

Finally, import the text file into the database

sqlite> .import file.txt tbl_fwlogs

Now that you have data to query, here is a sample that displays what rules are being used in the rulebase for this particular host.

sqlite> select DISTINCT(f_rule_uid),f_rule_name from tbl_fwlogs;

{3C9A2260-8E75-4488-82C3-A3F279BB72B6};Srv to Srv access
{13385ECB-2S6F-4657-CC20-4DA76F217141};Windows Domain Resources
{3A5A0D9E-1D32-41BD-9795-829ED5CFE366};Time Requests
{5D2726D6-738A-43BA-8B5B-63FA0A7EBF78};Monitoring Servers
{A696790B-2605-46B2-BDA3-8A64A5B98C1A};DNS
{DDDCF882-8121-4E27-8A28-EA17EC5BC47E};Internal ICMP
sqlite>

In this example we see there are 6 rules in use for this host.  From here additional queries would determine src/dest addresses and protocols used so that we could take that info and build a stricter rule set for this host.