Monitoring for Large-Scale Networks

fixbuf

ipfixDump

NAME

ipfixDump - Print contents of an IPFIX file as human-readable text

SYNOPSIS

 ipfixDump [--in FILE_NAME] [--out FILE_NAME]
           [--rfc5610] [--element-file FILE_NAME] [--yaf]
           [{ --show=SHOW_LIST | --templates | --data |
              --template-counts | --file-stats }]
           [--octet-format=FORMAT] [--string-format=FORMAT]
           [--suppress=ITEM_LIST]

 ipfixDump [--version]

 ipfixDump [--help]

DESCRIPTION

ipfixDump is a tool to read an IPFIX file and print its contents as human-readable text to assist the user in analyzing the file. By default, ipfixDump prints all message headers, templates, data records, options templates, and options records to the output, plus a one line summary of the file's content.

To select different output, use the --show, --templates, --data, --template-counts, or --file-stats switches.

ipfixDump supports IPFIX structured data in the form of basicLists, subTemplateLists, and subTemplateMultiLists.

By default, ipfixDump reads the IPFIX file from the standard input and writes the text to the standard output. To specify the input or output file's location, use the --in or --out option, respectively.

ipfixDump requires the input file to contain the IPFIX templates that describe the data records within the file, and the template must appear before the records that use it. Any records that do not have a corresponding template are ignored.

The default information model used by ipfixDump includes only the standard information elements defined by IANA and provided by libfixbuf. There are three ways to augment the set of elements:

The --rfc5610 option instructs ipfixDump to watch the input for options records that define private enterprise information elements (as defined by RFC5610) and to add any unknown elements to the information model.
The --element-file=FILE_NAME option tells ipfixDump to parse the contents of FILE_NAME and add those information elements to the information model. The argument is an XML file whose schema is that used by IANA's XML Information Element Registry, with the following additions:

cert:enterpriseId

A number representing the Private Enterprise Number of the element

cert:reversible

A boolean value (true, yes, or 1 for true; false, no, or 0 for false) that specifies whether the element may have a separate identity in a reverse flow.

The --element-file option may be used multiple times to load multiple files, and the loaded elements replace existing elements with the same identifier.
The --yaf option loads the CERT private enterprise information elements into the information model. These elements are used by the NetSA tools yaf(1), pipeline(8), super_mediator(1), and rwsilk2ipfix(1). This option is implemented as a wrapper over the --element-file option where the file name is cert_ipfix.xml and ipfixDump checks several directories to attempt to find this file, stopping once it finds the first file. The list of directories, in search order, is
- the directory ../share/libfixbuf relative to the directory containing the application
- the libfixbuf subdirectory of the datadir directory specified when ipfixDump was configured (defaults to $prefix/share)
- the share/libfixbuf subdirectory installation folder for the GLib-2 library
- the libfixbuf subdirectory of the directories specified by the $XDG_DATA_DIRS environment variable, or /usr/local/share and /usr/share/libfixbuf when that variable is empty
ipfixDump exits with an error if the --yaf option is given and it is unable to find the cert_ipfix.xml file. See https://tools.netsa.cert.org/cert-ipfix-registry/ for additional information about this file.

OPTIONS

The following options are available for ipfixDump:

--in FILE_NAME

Sets the input file name to FILE_NAME. When the option is not specified, ipfixDump reads from the standard input or exits with an error when the standard input is a terminal. ipfixDump reads from the standard input if FILE_NAME is '-'. Short option: -i.

--out FILE_NAME

Sets the output file name to FILE_NAME. If FILE_NAME exists, it is overwritten. The string '-' may be used to write to standard output (the default). Short option: -o.

--rfc5610

Tells ipfixDump to scan the IPFIX input file for options records that define private enterprise information elements and to add the unknown elements to the information model.

--element-file FILE_NAME

Loads the XML file FILE_NAME and incorporates information element information found in it. The format of the file is described above. The option may be used multiple times to load multiple files, and later elements replace existing elements when they have the same identifier. Short option: -e.

--yaf

Searches for a file named cert_ipfix.xml in several locations and loads that file as if it was an argument to --element-file. ipfixDump exits with an error if it is unable to find the cert_ipfix.xml file. Short option: -y.

--show=SHOW_LIST

Causes the output to contain only the categories specified in SHOW_LIST, a comma-separated list of the names described below. When none of --show or the other output selector options is given, ipfixDump defaults to --show=templates,data. This option has precedence over the other output selectors. Short option: -s.

templates: Tells ipfixDump to print the template records that appear in the input and to print the message header and footer (statistics) for messages that contain template records.
data: Tells ipfixDump to print the data records that were read from the input and to print the message header and footer (statistics) for messages that contain data records.
messages: Tells ipfixDump to print the message header and message footer (statistics) for all messages.
template-counts: Causes ipfixDump to print, after the data and template records, a two or three column table where each row is includes a template ID defined in the input and the number of times data using that template was seen. If the input contains template description option records, the table includes the template name as the third column. The counts include uses in subTemplateLists and subTemplateMultiLists when a data record is present. If a list specifies a template ID but the list is empty, the count for that template ID is unchanged.
file-stats: Tells ipfixDump to print, at the end of the data and after the template-count table, the flowEndMilliseconds timestamps for the first and last record in the input, the minimum and maximum flowEndMilliseconds timestamps seen, and the message export times of the first and last IPFIX messages.

When --show includes any of templates, data, messages, or flow-stats, the last line of the output summarizes the number of messages, number of data records, and number of templates records contained in the input.

--templates

Tells ipfixDump to print only template definitions and the message headers and footers for messages that contain a template set. May not be used with --data, --file-stats, or --template-counts. Equivalent to --show=templates. Short option: -t.

--data

Tells ipfixDump to print only data records and the message headers and footers for messages that contain a data set. May not be used with --templates, --file-stats, or --template-counts. Equivalent to --show=data. Short option: -d.

--template-counts

Tells ipfixDump to print only a table showing the template IDs defined in the input and the number of times data using that template was seen. May not be used with --templates, --data, or --file-stats. Equivalent to --show=template-counts. Short option: -c.

--file-stats

Tells ipfixDump to print only the flowEndMilliseconds timestamps for the first and last record in the input, the minimum and maximum flowEndMilliseconds timestamps seen, and the message export times of the first and last IPFIX messages. Equivalent to --show=file-stats. Short option: -f.

--octet-format=FORMAT

Sets the output for data record elements whose type is octetArray. The output always includes the length of the octetArray surrounded by square brackets (e.g., [4]) before the displayed value, if any. If the value's length is 0, only the length is printed. The default format is short-hex.

none

Does not display anything other than the length.

hexadecimal

Outputs 0x and then the hexadecimal value for each octet in the octetArray, e.g., [4] 0x666f6f0a.

short-hex

If the octetArray is a fixed-width element and its length is eight or less, displays the value in the hexadecimal format, otherwise displays nothing.

base64

Displays the value using base-64 encoding, e.g., [4] Zm9vCg==.

hexdump

Displays the value in the style of the hexdump(1) utility when invoked with the arguments -v -C. The output displays 16 octets of the value on each line. Each line has the same prefix as that of the element name followed by a vertical bar that aligns with the element name. That is followed by the line's offset displayed in hexadecimal, up to 16 octets each presented as a hexadecimal number, a vertical bar, those 16 octets presented as ASCII with a period for any non-printable or control character, followed by another vertical bar and a newline. The final line is displays the length of the value in hexadecimal. For example:

 [4]
 | 00000000  66 6f 6f 0a                                       |foo.            |
 | 00000004

string

Displays the value as a string in double quotes ("). Octet values 34 and 92 are displayed as \" and \\ respectively, other octet values from 32 (space) to 126 (tilde) inclusive are displayed as ASCII, and all other values are shown as escaped utf-8 (\uNNNN. For example, [4] "foo\u000A" where \u000A represents a newline.

--string-format=FORMAT

Sets the output for data record elements whose type is string. The output always includes the length of the string surrounded by square brackets ([4]) before the displayed value, if any. The default format is quoted.

quoted: Displays the value as a string in double quotes ("). Octet values 34 and 92 are displayed as \" and \\, respectively, other octet values from 32 (space) to 126 (tilde) inclusive are displayed as ASCII, octet values 9 (tab), 10 (newline), and 13 (carriage return) are displayed as \t, \n, and \r respectively, and all other values are displayed as 2-character hexadecimal with a leading \x. For example, [4] "foo\n". When the value's length is 0, output is [0] "".
raw: Displays the raw string with no surrounding quotes and making no attempt to escape control characters. This is the behavior of ipfixDump before version 3.0.0.
base64: Displays the value using base-64 encoding, e.g., [4] Zm9vCg==. If the value's length is 0, only the length is printed.
hexdump: Displays the value in the style of the hexdump(1) utility when invoked with the arguments -v -C. This format is described above under --octet-format.
hexadecimal: Outputs 0x and then the hexadecimal value for each octet in the string, e.g., [4] 0x666f6f0a. If the value's length is 0, only the length is printed.
none: Does not display anything other than the length.

--suppress=ITEM_LIST

Disables the display of the items in ITEM_LIST. This may be useful when using ipfixDump to compare IPFIX files. The following items may be suppressed:

record-number: For top-level records, do not display the record number.
export-time: Do not display the export time in the message header.
stream-offset: Do not display the stream offset in the message header.
message-length: Do not display the message length in the message header.
sequence-number: Do not display the sequence number in the message header.
messages: Disables printing of the message header and message footer. The output still includes the number of messages, records, and templates seen in the file once the end of the input is reached. This overrides --show=messages.

--version

Causes ipfixDump to print the version and copyright information to standard error and exit. Short option: -V.

--help

Causes ipfixDump to print a brief usage message to the standard output and exit. Short option: -h.

EXAMPLES

In the following examples, the dollar sign ("$") represents the shell prompt. The text after the dollar sign represents the command line.

 $ ipfixDump --in - --out -

 $ ipfixDump --in /data/ipfix.ipfix --out /data/text.txt --yaf

The ipfixDump output for a complete IPFIX message that contains a single options template record is:

 --- Message Header ---
 export time: 2021-02-09 19:56:53       observation domain id: 0
 message length: 62                     sequence number: 0 (0)
 stream offset: 0

 --- Options Template Record --- tid: 53254 (0xd006), fields: 9, scope: 2 ---
   privateEnterpriseNumber              (346) <uint32>       [4] {scope}
   informationElementId                 (303) <uint16>       [2] {scope}
   informationElementDataType           (339) <uint8>        [1]
   informationElementSemantics          (344) <uint8>        [1]
   informationElementUnits              (345) <uint16>       [2]
   informationElementRangeBegin         (342) <uint64>       [8]
   informationElementRangeEnd           (343) <uint64>       [8]
   informationElementName               (341) <string>   [65535]
   informationElementDescription        (340) <string>   [65535]

 *** Msg Stats: 1 Template Records ***

Here is an example record that includes a subTemplateList:

 --- Data Record 1332161 --- fields: 7, tmpl: 53252 (0xd004) tombstone_record ---
   observationDomainId                 (149){s} : 0
   exportingProcessId                  (144){s} : 45982
   exporterConfiguredId           (6871/551){s} : 4646
   paddingOctets                       (210)    : [6]
   tombstoneId                    (6871/550)    : 0
   observationTimeSeconds              (322)    : 2021-02-09 20:01:53
   tombstoneAccessList            (6871/554)    : count: 1 {noneOf} tmpl: 53253 (0xd005) tombstone_access
    --- STL Record 1 --- fields: 2, tmpl: 53253 (0xd005) tombstone_access ---
     certToolId                     (6871/553) : 1
     observationTimeSeconds              (322) : 2021-02-09 20:01:53

AUTHORS

Emily Sarneso and the CERT Network Situational Awareness Group Engineering Team, <http://www.cert.org/netsa>.

Bug reports may be sent directly to the Network Situational Awareness team at <[email protected]>.

fixbuf

fixbuf

NAME

SYNOPSIS

DESCRIPTION

OPTIONS

EXAMPLES

AUTHORS

SEE ALSO