Log Streaming Overview
Updated 2 years ago
by
admin
The Log Streaming product allows you to push enriched log data for each of the core security products that are licensed, to third party applications. The typical use case is to stream log data to an external log analysis/SIEM tool.
Supported Applications
- Amazon S3
- Google Cloud Storage
- Microsoft Azure Sentinel
- Rapid7 InsightIDR
- Splunk Enterprise or Cloud
- Sumo Logic
If you don't see your application listed above then consider that it is likely compatible with either Amazon S3 or Google Cloud Storage as a source of data. If you require a specific supported application, please contact your Service Provider to discuss further.
Technical Information
Log Streaming is activated by your Service Provider as a background service, using a HTTP push method to one of the supported third parties listed above.
The service will ship a log file every 1 minute or every 10MB of log data, whichever happens first. In busier environments, it's likely to ship more often than 1 minute due to the volume of data collected.
The log file is in json_lines format (more info - opens in external site) with one log record (JSON object) per line.
The filename format is as follows: ls.s3.{uuid}.{YYYY-MM-DDTHH.mm}.part{n}.txtwhere uuid is a uuid4, the date is the time of the file creation and n is an integer starting from one with no padding.
The JSON format varies depending on product being streamed - please see Log Streaming Record Format for more information.