Packet capture for only certain TCP flags (for use with Fortigate 'sniffer' command):

Information extracted from the Man pages of 'tcpdump' command:

# Updated Jan 2007 for Fortinet KC by Stephen Dennis.

       There are 8 bits in the control bits section of the TCP header:

       Let's  assume  that  we  want  to  watch  packets  used  in establishing a TCP

       connection.  Recall that TCP uses a 3-way handshake protocol when it  initial-

       izes  a new connection; the connection sequence with regard to the TCP control

       bits is

              1) Caller sends SYN

              2) Recipient responds with SYN, ACK

              3) Caller sends ACK

       Now we're interested in capturing packets that have only the SYN bit set (Step

       1).   Note that we don't want packets from step 2 (SYN-ACK), just a plain ini-

       tial SYN.  What we need is a correct filter expression for tcpdump.

       Recall the structure of a TCP header without options:

        0                            15                              31

       -----------------------------------------------------------------

       |          source port          |       destination port        |

       -----------------------------------------------------------------

       |                        sequence number                        |

       -----------------------------------------------------------------

       |                     acknowledgment number                     |

       -----------------------------------------------------------------

       |  HL   | rsvd  |C|E|U|A|P|R|S|F|        window size            |

       -----------------------------------------------------------------

       |         TCP checksum          |       urgent pointer          |

       -----------------------------------------------------------------

       A TCP header usually holds 20 octets of data, unless options are present.  The

       first  line of the graph contains octets 0 - 3, the second line shows octets 4

       - 7 etc.

       Starting to count with 0, the relevant TCP control bits are contained in octet

       13:

        0             7|             15|             23|             31

       ----------------|---------------|---------------|----------------

       |  HL   | rsvd  |C|E|U|A|P|R|S|F|        window size            |

       ----------------|---------------|---------------|----------------

       |               |  14th octet   |               |               |

       Let's have a closer look at octet no. 13:

                       |               |

                       |---------------|

                       |C|E|U|A|P|R|S|F| <- TCP (and ECN) flags

                       |---------------|

                       |7   5   3     0| <- bit position

       These  are  the  TCP  control bits we are interested in.  We have numbered the

       bits in this octet from 0 to 7, right to left, so the PSH bit is bit number 3

       (4th bit from right), while the URG bit is number 5 (6th bit from right).

       Recall that we want to capture packets with only SYN set.  Let's see what hap-

       pens to octet 13 if a TCP datagram arrives with the SYN bit set in its header:

                       |C|E|U|A|P|R|S|F| <- TCP and ECN flags

                       |---------------|

                       |0 0 0 0 0 0 1 0| <- bit value

                       |---------------|

                       |7 6 5 4 3 2 1 0| <- bit position

       Looking  at  the  control  bits section we see that only bit number 1 (SYN) is

       set.

       Assuming that octet number 13 is an 8-bit unsigned  integer  in  network  byte

       order, the binary value of this octet is

              00000010

       and its decimal representation is:

             | bit  | dec.   | bit   |       | TCP

             | pos. | value  | value | total | Flag

         ____|______|________|_______|_______|_____

             |      |        |       |       |

         2   ^   7  =  128,  x   0   =   0   | CWR (ECN Congestion Window Reduced)

         2   ^   6  =   64,  x   0   =   0   | ECE (ECN Capable Echo)

         2   ^   5  =   32,  x   0   =   0   | URG

         2   ^   4  =   16,  x   0   =   0   | ACK

         2   ^   3  =    8,  x   0   =   0   | PSH

         2   ^   2  =    4,  x   0   =   0   | RST

         2   ^   1  =    2,  x   1   =   2   | SYN*

         2   ^   0  =    1,  x   0   =   0   | FIN

         ____|______|________|_______|_______|_____

                                  sum:   2

         ==========================================

       ('2 ^ 7 = 128' means '2 to the power of 7 equals 128')

       We're  almost  done, because now we know that if only SYN is set, the value of

       the 14th octet in the TCP header, when interpreted as an 8-bit unsigned integer

       in network byte order, must be exactly 2.

       This relationship can be expressed as

              'tcp[13]==2'

       The expression says "match all TCP datagrams where octet 13 (i.e. the 14th octet)

       is equal to the decimal value 2", which is exactly what we want.

       But now, let's assume that we need to capture ALL packets with the  SYN bit  set,

       regardless of whether any other TCP control bits (such as ACK, URG, PSH etc.) are

       set. Let's see what happens to octet 13 when a TCP datagram arrives with both the

       SYN and ACK bits set:

            |C|E|U|A|P|R|S|F| <- TCP (and ECN) flags

            |---------------|

            |0 0 0 1 0 0 1 0| <- bit value

            |---------------|

            |7 6 5 4 3 2 1 0| <- bit position

       Now bits 4 and 1 are set in the 14th octet.  The binary value of octet 13 is now:

                   00010010

       which translates to a decimal value of:

             | bit  | dec.   | bit   |       | TCP

             | pos. | value  | value | total | Flag

         ____|______|________|_______|____________

             |      |        |       |       |

         2   ^   7  =  128,  x   0   =   0   | CWR (ECN Congestion Window Reduced)

         2   ^   6  =   64,  x   0   =   0   | ECE (ECN Capable Echo)

         2   ^   5  =   32,  x   0   =   0   | URG

         2   ^   4  =   16,  x   1   =  16   | ACK*

         2   ^   3  =    8,  x   0   =   0   | PSH

         2   ^   2  =    4,  x   0   =   0   | RST

         2   ^   1  =    2,  x   1   =   2   | SYN*

         2   ^   0  =    1,  x   0   =   0   | FIN

         ____|______|________|_______|_______|_____

                                  sum:  18

         ==========================================

       We can't just use 'tcp[13]==18' in the tcpdump filter expression, because that

       would match only those packets where both SYN and ACK are set, but not packets

       with only SYN set. Remember that we don't care if ACK or any other control bit

       is set, as long as SYN is set.

       In order to achieve our goal, we need to logically 'AND' the binary  value  of

       octet  13 with some other value to preserve the SYN bit.  We know that we want

       SYN to be set in any case, so we'll logically AND the value in the 14th  octet

       with the binary value of a SYN:

                       v                             v

                 00010010 SYN-ACK              00000010 SYN

            AND  00000010 (we want SYN)   AND  00000010 (we want SYN)

                 --------                      --------

            =    00000010                 =    00000010

                       ^                             ^

       We see that this AND operation delivers the same result regardless of  whether

       any other TCP control bits are set or not. The decimal representation  of  the

       AND value as well as the result of this operation is 2 (binary 00000010), so we

       know that for packets with SYN set the following relation must hold true:

              ( ( value of octet 13 ) AND ( 2 ) ) == ( 2 )

       This relationship can be expressed as:

       'tcp[13]&2==2'

       The following list of expressions expand on this idea:

       'tcp[13]==1'       matches packets with only the FIN bit set to 1;

       'tcp[13]&4==4'     matches all packet with the RST bit set to 1;

       'tcp[13]&8==8'     matches all packet with the PSH bit set to 1;

       'tcp[13]==16'      matches packets with only the ACK bit set to 1;

       'tcp[13]&32==32'   matches all packets with the URG bit set to 1;

       'tcp[13]&64==64'   matches all packets with the ECE bit set to 1;

       'tcp[13]&128==128' matches all packets with the CWR bit set to 1;

       'tcp[13]==24'      matches packets with only the PSH and ACK bits set to 1;

Fortigate examples:

--------------------

Example to sniff for packets with ONLY the TCP SYN set (no other flags set):

# diag sniff packet wan1 'tcp[13]==2'

Example to sniff for ANY packets with the TCP SYN set (regardless of other flags):

# diag sniff packet wan1 'tcp[13]&2==2'

Example to sniff for any packets on port 80 (HTTP) with the TCP SYN set:

# diag sniff packet wan1 'port 80 and tcp[13]&2==2'

Example to sniff for any packets on port 80 (HTTP) or 443 (HTTPS) with the TCP SYN set:

# diag sniffer packet internal 'port 80 or 443 and tcp[13]&2==2'

#!/usr/bin/perl

# This work (and included software, documentation or other related items) is

# being provided by the copyright holders under the following license. By

# obtaining, using and/or copying this work, you (the licensee) agree that you

# have read, understood, and will comply with the following terms and conditions

#

# Permission to copy, modify, and distribute this software and its documentation

# with or without modification, for any purpose and without fee or royalty is

# hereby granted, provided that you include the following on ALL copies of the

# software and documentation or portions thereof, including modifications:

#

#   1. The full text of this NOTICE in a location viewable to users of the

#      redistributed or derivative work.

#   2. Notice of any changes or modifications to the files, including the date

#      changes were made.

#

#

# THIS SOFTWARE AND DOCUMENTATION IS PROVIDED "AS IS," AND COPYRIGHT HOLDERS

# MAKE NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT

# LIMITED TO, WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR

# PURPOSE OR THAT THE USE OF THE SOFTWARE OR DOCUMENTATION WILL NOT INFRINGE ANY

# THIRD PARTY PATENTS, COPYRIGHTS, TRADEMARKS OR OTHER RIGHTS.

#

# COPYRIGHT HOLDERS WILL NOT BE LIABLE FOR ANY DIRECT, INDIRECT, SPECIAL OR

# CONSEQUENTIAL DAMAGES ARISING OUT OF ANY USE OF THE SOFTWARE OR DOCUMENTATION.

#

# Title to copyright in this software and any associated documentation will at

# all times remain with copyright holders.

#

# Copyright: Fortinet Inc - 2005

#

# Fortinet EMEA Support

# This script converts a Fortigate verbose 3 sniffer output file to a file that

# can be opened by Ethereal. It uses the text2pcap binary included in the Ethereal package.

# It is supplied as is and no technical support will be provided for it.

  my $version               = "Dec 19 2014";

# ------------------ don't edit after this line -------------------------------

  use strict;

  use Getopt::Long;

  use FileHandle;

  use Config;

  use vars qw ($debug $help $vers $in $out $lines $demux $childProcess);

  use Data::Dumper;

# Autoflush

    $| = ;

# Global variables

  my $line_count                = ;

  my ($fh_in, $fh_out);

  my @fields                     = {};

  my @subhexa                     = {};

  my ($offset,$hexa,$garbage)   = "";

  my %outfileList;

  my %outfilenameList;

  # Get commandLine arguments

  getArgs();

  # In order to support real-time display in wireshark/ethereal, we need to pipe

  # our stdout into wireshark stdin, which is not allowed by the OS...

  # The trick consists in creating a child process with the appropriate anonymous

  # pipes already in place and delegate the work to the child.

  spawnPipedProcess();

  open(fh_in,  '<', $in)  or die "Cannot open file ".$in." for reading\n";

# Convert

  if( $debug ) {

    print STDERR "Conversion of file ".$in." phase 1 (FGT verbose 3 conversion)\n";

    print STDERR "Output written to ".$out.".\n";

  }

  my @packetArray = ();

  #Parse the entire source file

  my $DuplicateESP = ;

  my $eth0 = ;

  my $skipPacket = ;

followMode:

    while (<fh_in>) {

        s/^\d{}(\d{})/0x$/;

        #and build an array from the current packet

        if( isTimeStamp() ) {

            $skipPacket = ;

            if( not $demux and /eth0/ ) {

                $eth0++;

                $skipPacket = ;

            }

               # Select the appropriate output file for the interface.

               $fh_out = getOutputFileHandler() if defined $demux;

            $skipPacket |= convertTimeStamp();

         } elsif    ( isHexData() and not $skipPacket ) {

            buildPacketArray();

            adjustPacket();

            _startConvert();

        }

    }

    if( $out eq "-" ) {

        # no more incoming data. Wait 2 seconds and try again

        sleep ;

        goto followMode;

    }

    print "** Skipped $eth0 packets captured on eth0\n" if $eth0;

# Close files and start text2pcap if needed

    close (fh_in)  or die "Cannot close file ".$in."\n";

    my $text2pcap  = getText2PcapCmd();

    foreach my $intf( keys %outfileList ) {

        close $outfileList{ $intf };

        my $filename_in = $outfilenameList{$intf};

        my $filename_out = $filename_in;

        $filename_out    =~ s/\.tmp$/\.pcap/;

        system("$text2pcap $filename_in $filename_out");

        unlink($filename_in);

    }

    if( $debug ) {

        print STDERR "Output file to load in Ethereal is \'".$out."\'\n";

        print STDERR "End of script\n";

    }

sub isHexData   { /^(0x[-9a-f]+[ \t\xa0]+)/ }

sub isTimeStamp { /^[-]+[\.\-][-]+/      }

# /Dummy comment to workaround Komodo parsing

sub buildPacketArray {

    my $line = ;

    @packetArray = ();

    do {

        # Format offset from 0x0000 to 000000 (text2pcap requirement)

        s/^0x([\da-f]{})/$/;

        if ( s/^([\da-f]{})\s+// ) {

            # Remove ASCII translation at the end of the line

            s/\s\s+.*//;

            my @bytes  = /([\da-f]{})\s?([\da-f]{})?/g;

            $#bytes = 15;

            push @packetArray => @bytes;

        }

        $_ = <fh_in>;

    } until ( /^\s*$/ );

}

sub convertTimeStamp {

    # Keep timestamps.

    return  if /truncated\-ip \- [-]+ bytes missing!/ ;

    if ( /^([-]+)\.([-]+) / )

    {

        my $packet = ;

        my $time = $;

        # Extract days

        my $nbDays    = int($time / );

        my $day     = sprintf("%0.2d", +$nbDays);

        $time         = $time % ;

        # Extract hours

        my $hour = int($time /  );

        $time = $time % ;

        # Extract minutes

        my $minute = int( $time / );

        $time = $time % ;

        # and remaining seconds

        my $sec = $time;

        _print("01/$day/2005 " . $hour . ":" . $minute . ":" . $sec . ".$2\n");

    } elsif ( /^(\d+-\d+-\d+ \d+:\d+:\d+\.\d+) / ) {

        # absolute timestamp

        my $timestamp   = $;

        $timestamp      =~ s/(\d+)-(\d+)-(\d+)/$\/$\/$/;

        _print("$timestamp\n");

    }

    # Check if line is a duplicate ESP packet (FGT display bug)

    return ;

}

sub getOutputFileHandler

{

    my ($currIntf) =  $_ =~ / (\S+) (?:out|in) /;

    $currIntf = "[noIntf]" if $currIntf eq "";

    if( not defined( $outfileList{$currIntf} )) {

        my $filename = $out ? $out : $in;

        $filename =~ s/\.zip$//g;

        my $suffix = ".$currIntf.tmp";

        $suffix    =~ s/\//-/g;     # Escape '/' char in interface name

        $filename .= $suffix;

        open( $outfileList{$currIntf}, "> $filename");

        $outfilenameList{$currIntf} = $filename;

    }

    return $outfileList{$currIntf};

}

#----------

# name : adjustPacket

# description:

#  Applies changes to the current packetArray to make it convertible into

#  ethereal format.

#     - Removes internal Fortigate tag when capture interface is any.

#

sub adjustPacket {

  next unless $packetArray[][] =~ /0x0000/;

  stripBytes( ,  ) if $packetArray[][] =~ //;

  stripBytes( ,  ) if $packetArray[][] =~ //;

  addHdrMAC()        if $packetArray[][] =~ /[0c]/;

  $packetArray[][] =~ s/|// if $packetArray[][] =~ /0x0000/;

}

sub addHdrMAC

{

  my $nbRows = scalar @packetArray;

  # And populate 0x0000 line

  unshift @packetArray => qw(                 );

  # left shift the IP ver+IHL (4500)

  stripBytes(,);

}

sub stripBytes

{

  my $start         = shift;

  my $nbBytes       = shift;

  my @subArray = @packetArray[$start..$#packetArray-1];

  shift @subArray for ..$nbBytes;

  @packetArray = (@packetArray[..$start-],@subArray);

}

sub _startConvert {

  LINE:

    #Initialisation

    my $hexa = "";

    my $garbage = "";

    my $offset = ;

    foreach my $byte (@packetArray) {

        _print( sprintf( "%0.6x ", $offset )) unless $offset % ;

        _print( " ". $byte);

        $offset ++;

        _print("\n") unless $offset % ;

    }

    _print( "\n");

     $line_count++;

    if (defined($lines)) {

        if ($line_count >= $lines) {

            print STDERR "Reached max number of lines to write in output file\n";

            last LINE;

        }

    }

}

sub spawnPipedProcess

{

    # Spawn a new process to pipe ourselves with text2pcap

    my( $fgt2eth, $text2pcap, $wireshark );

    # Are we already in the pipelined session ?

    return if $childProcess or $demux;

    $fgt2eth    = $ . " -in \"$in\" -out \"$out\" -childProcess";

    $text2pcap  = getText2PcapCmd(). "- ";

    $wireshark  = getWiresharkCmd();

    my $cmd = "$fgt2eth | $text2pcap";

    # Prepare output filename

    if( $out eq "-" ) {

        $cmd .= "- | $wireshark";

    } else {

        $out  = $in . ".pcap" unless $out;

        $cmd .= " $out";

    }

    print STDERR $cmd if $debug;

    open( CMD, "$cmd |" );

    while( <CMD> ) {

        _print($_);

    }

    close CMD;

    # Make sure we don't go further in the parent process

    exit;

}

sub getText2PcapCmd

{

    my $cmd;

    my @windowsPath = qw(

                       c:\\Progra~\\Ethereal

                       c:\\Progra~\\Wireshark

                    );

    if ($ENV{'OS'} =~ /windows/i) {

        if( $Config{osname} =~ /cygwin/i ) {

            #OS is Windows running Cygwin

            $cmd =  "'/cygdrive/c/Program Files/Wireshark/text2pcap'";

        } else {

            # OS is windows

            my $dir;

            for $dir( @windowsPath ) {

                $cmd = "$dir\\text2pcap.exe" if -e "$dir\\text2pcap.exe";

            }

        }

    } else {

        # OS is linux :-)

        $cmd = "text2pcap";

    }

    # Sanity

    die "Text2Pcap could not be found\n" unless $cmd;

    $cmd .= " -q -t \"%d/%m/%Y %H:%M:%S.\" ";

    return $cmd;

}

sub getWiresharkCmd

{

    my $cmd;

    my @windowsPath = qw(

                       c:\\Progra~\\Wireshark

                    );

    if ($ENV{'OS'} =~ /windows/i) {

        if( $Config{osname} =~ /cygwin/i ) {

            # OS is Windows running Cygwin

            $cmd = "'/cygdrive/c/Program Files/Wireshark/wireshark'";

        } else {

            # OS is windows

            my $dir;

            for $dir( @windowsPath ) {

                $cmd = "$dir\\wireshark.exe" if -e "$dir\\wireshark.exe";

            }

        }

    } else {

        # OS is linux :-)

        $cmd = "wireshark";

    }

    # Sanity

    die "wireshark could not be found\n" unless $cmd;

    $cmd .= " -k -i -";

    return $cmd;

}

sub _print{

    my $msg = shift;

    if( defined $fh_out ) {

        print $fh_out $msg;

    } else {

        print $msg;

    }

}

sub getArgs

{

   # Control command line options

   GetOptions(

    "debug"          => \$debug,            # use -debug to turn on debug

      "version"       => \$vers,                    # use -version to display version

    "help"           => \$help,            # use -help to display help page

    "in=s"        => \$in,            # use -in  <filename> to specify an input file

    "out=s"       => \$out,            # use -out <filename> to specify an output file

        "lines=i"      => \$lines,            # use -lines <number> to stop after <number> lines written

        "demux"         => \$demux,                     # use -demux to create one pcap per intf

        "childProcess"  => \$childProcess,

    );

  if ($help) {

    Print_help();

    exit;

    }

  if ($vers) {

    Print_version();

    exit;

    }

  # Sanity checks

  if (not(defined($in))) {

    Print_usage();

    exit;

    }

}

#------------------------------------------------------------------------------

 sub Print_usage {

  print <<EOT;

Version : $version

Usage : fgt2eth.pl -in <input_file_name>

Mandatory argument are :

    -in  <input_file>     Specify the file to convert (FGT verbose  text file)

Optional arguments are :

    -help                 Display help only

    -version              Display script version and date

    -out <output_file>    Specify the output file (Ethereal readable)

                By default <input_file>.pcap is used

                - will start wireshark for realtime follow-up

    -lines <lines>        Only convert the first <lines> lines

    -demux            Create one pcap file per interface (verbose  only)

    -debug                Turns on debug mode

EOT

}

#------------------------------------------------------------------------------

 sub Print_help {

  print <<EOT;

This script permits to sniff packets on the fortigate with built-in sniffer

diag sniff interface <interface> verbose  filters '....'

and to be able to open the captured packets with Ethereal free sniffer.

* What do I need to know about this script ?

    - It can be sent to customers, but it is given as is.

    - No support is available for it as it is not an 'offical' fortinet product.

    - It should run on windows and linux as long as perl is installed.

    - To install perl on windows system,

             http://www.activestate.com/Products/ActivePerl/

    - All lines from the source file that do not begin with '0x' are ignored.

    - Do not add garbage characters to the file during the capture

    - If possible do not hit the keyboard during capture.;

Remarks concerning this script can be sent to eu_support\@fortinet.com

Thanks to Claudio for this great idea and Ellery from Vancouver Team for the timestamps

Cedric

EOT

}

#------------------------------------------------------------------------------

sub Print_version {

  print "\nVersion : ".$version."\n\n";

}

sub myDump {

    my $object = shift;

    my $dumper = new Data::Dumper([$object]);

    $dumper->Maxdepth();

    print STDERR $dumper->Dump;

}

> fgt2eth.02.2008.exe /?

Version : 1.22 (Feb 21 2008)

Assuming OS: windows

Usage : fgt2eth.pl -in <input_file_name>

Mandatory argument are :

   -in  <input_file>     Specify the file to convert (FGT verbose 3 text file)

Optional arguments are :

   -help                 Display help only

   -version              Display script version and date

   -out <output_file>    Specify the output file (Ethereal readable)

                         By default '<input_file>.eth' is used

   -lines <lines>        Only convert the first <lines> lines

   -system <system>      Can be either linux or windows

   -debug                Turns on debug mode

> fgt2eth.02.2008.exe /?

Version : 1.22 (Feb 21 2008)

Assuming OS: windows

Usage : fgt2eth.pl -in <input_file_name>

Mandatory argument are :

   -in  <input_file>     Specify the file to convert (FGT verbose 3 text file)

Optional arguments are :

   -help                 Display help only

   -version              Display script version and date

   -out <output_file>    Specify the output file (Ethereal readable)

                         By default '<input_file>.eth' is used

   -lines <lines>        Only convert the first <lines> lines

   -system <system>      Can be either linux or windows

   -debug                Turns on debug mode

PS C:\Users\lsgxt\Downloads> .\fgt2eth.12.2014.exe /?

Version : Dec 19 2014

Usage : fgt2eth.pl -in <input_file_name>

Mandatory argument are :

    -in  <input_file>     Specify the file to convert (FGT verbose 3 text file)

Optional arguments are :

    -help                 Display help only

    -version              Display script version and date

    -out <output_file>    Specify the output file (Ethereal readable)

                By default <input_file>.pcap is used

                - will start wireshark for realtime follow-up

    -lines <lines>        Only convert the first <lines> lines

    -demux            Create one pcap file per interface (verbose 6 only)

    -debug                Turns on debug mode

Mode                LastWriteTime         Length Name

----                -------------         ------ ----

-a----       2013/11/30      9:08             84 convert.bat

-a----       2013/11/30      9:08            104 convertopen.bat

-a----       2010/11/17      4:19        1808230 fgt2eth.exe

-a----       2012/10/29      3:57        1149982 libglib-2.0-0.dll

-a----       2012/10/29      3:57          46371 libgmodule-2.0-0.dll

-a----         2012/9/4      4:30         130181 libintl-8.dll

-a----        2013/11/2      2:07          33200 libwsutil.dll

-a----        2011/2/20     14:03         211280 msvcp100.dll

-a----        2011/2/19     15:40         356176 msvcr100.dll

-a----        2013/12/3      2:14           1102 README.txt

-a----        2013/11/2      2:07         338864 text2pcap.exe

-a----       2018/11/30     10:28          48968 uninst.exe