Exporting Reporting Data to Elastic/ELK

Objective

The flow data collected by one or multiple Threat Defender installations can be collected in a single Elasticsearch instance and evaluated with Kibana.

Preparing the ELK Stack

If you do not yet have implemented an ELK stack, set up Elasticsearch, Kibana and Filebeat (see the respective documentation for further information).

Run Elasticsearch and Kibana.

Setting up Filebeat

Set up Filebeat to handle the flow data provided by Threat Defender.

In the Filebeat directory, add a new file called genua_ipfix.yml with the custom definitions of our PEN:

45480:
   10:
   - :uint16
   - :dpiProtocol
   11:
   - :uint16
   - :dpiApplication
   12:
   - :uint16
   - :dpiSrcOS
   13:
   - :uint32
   - :dpiClassification
   14:
   - :uint32
   - :dpiInSslClassification
   20:
   - :string
   - :countrySource
   21:
   - :string
   - :countryDestination
   30:
   - :string
   - :policyRuleId
   31:
   - :uint32
   - :iPSRuleId
   32:
   - :string
   - :policyRuleName
   33:
   - :uint8
   - :policyRuleAction
   34:
   - :string
   - :policyId
   35:
   - :string
   - :policyName
   36:
   - :uint8
   - :logSeverity
   37:
   - :uint8
   - :cognitixScenarioHit
   50:
   - :string
   - :url
   51:
   - :uint16
   - :urlCategory
   52:
   - :uint16
   - :urlReputation
   60:
   - :string
   - :fileTransferFilename
   70:
   - :uint16
   - :iocFeedId
   71:
   - :uint32
   - :iocIpv4
   72:
   - :string
   - :iocDomain
   73:
   - :uint64
   - :iocUrl
   74:
   - :string
   - :iocFeedName
   75:
   - :uint8
   - :iocValueType
   76:
   - :string
   - :iocValue
   80:
   - :uint8
   - :srcLocation
   81:
   - :uint8
   - :dstLocation
   90:
   - :string
   - :srcAssetId
   91:
   - :string
   - :dstAssetId
   92:
   - :string
   - :userId

Edit the filebeat.yml to contain the following element:

filebeat.inputs:
- type: netflow
  host: "0.0.0.0:2055"
  protocols: [ ipfix ]
  max_message_size: 50KiB
  custom_definitions:
    - <absolute path to genua_ipfix.yml created above>

Run Filebeat.

Setting up Threat Defender

Configure Threat Defender to send IPFIX data to Filebeat:

Go to Logging > Report Channels.
Click Add to create a new reporting channel.
On the settings screen, configure the following:
- Report Type: IPFIX
- Message Type: Select Flow Reports. You can select additional types as required.
- Observation Domain Id: can be 0 if you use only one Threat Defender. Otherwise, set a different value for each Threat Defender to be able to distinguish the reporting sources.
- Update Interval: 30 seconds
- Endpoint: UDP
- IP Address: enter the IP address of your Filebeat installation
- Port: 2055 (the port of your Filebeat installation as defined in the filebeat.yml above)
- Reconnection Delay: 15 seconds
Click SAVE to store your settings and close the settings screen. The new IPFIX channel is displayed in the list of configured reporting channels.

Click the APPLY CHANGES button at the top of the menu bar to activate your configuration changes.

As a result, the state of the newly configured IPFIX channel should change to connected and show a rising number of transmitted events.

Checking Kibana

Open the Kibana interface in your browser (see the Kibana documentation for more detailed information). In the Dashboard section, select the Netflow Overview dashboard to get a quick overview of some of the possibilities. You can also use the SIEM section to take a look into some predefined statistics; or start discovering on your own with the Discover section. You can preselect the netflow events by using input.type: netflow as a search filter.