RACONVERT(1)

NAME
raconvert - convert CSV and JSON formatted ASCII flow data to binary argus(8) data.

SYNOPSIS
raconvert [-f conversion.map] -r input -w output

DESCRIPTION

Raconvert reads comma separated (csv) and JSON formatted flow data from a file or stdin, and converts the ASCII representation to binary argus format. There are two primary modes of operation, converting native argus ASCII output back to binary, and converting other flow data ASCII formats to argus data. These differ based on the conversion map used to convert and assign data to argus data attributes. For native argus data, the conversion map is integrated into the raconvert algorithms. For non-native flow data, raconvert.1 needs a conversion map, that identifies the (key,value) pairs in the imported data, validates the data types, and assigns the values to the mapped Argus data attributes. For (key,value) pairs that do not have a representative native Argus data attribute, (key,value) pairs can be added to the argus flow label structure.
Native Argus Data
Raconvert.1 is designed to ingest most formats of printed argusP data. CSV format is the easiest to work with, and raconvert.1 will automatically identify fields, based on the column titles in the file. For best results, to process the delimited data correctly, the file should be created by ra.1 like programs, using the "-c <char>" option to specify a field separator. For CSV, it would be "-c ,". Column delimiters can be any non-ambiguous character, and are automatically discovered from the files title string. The delimiter chosen should not be a normally occurring character, such as '.'. Because raconvert.1 expects the first valid string in the file to be a ra.1 column title line, the ra.1 program should use the equivalent of the "-L 0" option. The file can present its column oriented data in any order, and raconvert.1 will accomodate. Duplicate columns on the same line are allowed, but the final value in the resulting argus record with be the last value encountered.
Foriegn Flow Data
Raconvert.1 can convert non-argus flow data to argus binary formats. Using the "-f conversion.map" option, any ascii string can be imported into argus binary foratted data. See raconvert.zeek.conf.5 to view the structure of a raconvert.1 conversion map. raconvert.1 can read gzip and bzip2 compressed files, based on the file extension.

USES
Raconvert is predominately used for reading ASCII represented archive data. Argus data is routinely converted to ASCII for inspection and archiving, due to the good compressibility offered by gzip and others. Argus data is also converted from binary to ASCII and back to binary, as a routine process to enforce content policies, especially when sharing data. Printing the records in ASCII enables standard tools to look for keywords, specific addresses, etc ... that may not be sharable, based on policy. Converting the inspected traffic is the best way to prevent unintentional leakage. Editing flow records, or creating artificial flow data using text editors is a important part of developing training data for flow data analytics and machine learning. Creating/editing in ascii and then converting to binary argus data, enables the full set of Argus processing on generated data. And importing foreign flow data formats, such as zeek into argus data formats enables a larger set of sensors to contribute to the argus data analytics systems.

INVOCATION
ra -L0 c , -r argusfile > argus.ascii.csv
raconvert -r argus.ascii.csv -w argusfile
raconvert -f raconvert.zeek.conf -r zeek.conn.log -w argusfile

COPYRIGHT
Copyright (c) 2000-2022 QoSient. All rights reserved.

SEE ALSO
ra(1), rarc(5), raconvert.zeek.conf(5), argus(8),

FILES

AUTHORS
Carter Bullard (carter@qosient.com).

BUGS