Data Export

Lacework Edge supports exporting log data to external sources, enabling end users to ingest this data into their SIEM tools. Lacework Edge exports logs to AWS S3 as destination, and most popular SIEM tools can ingest data from S3 buckets. These logs are exported in newline delimited JSON format.

Data Export Types

Lacework Edge exports following types of data to the provided destination S3 bucket:

Note: All file paths begin with <s3-bucket>/<customer-id>

Type	CLI Name	Export Frequency	File Path	Description
User activity logs	ACTIVITY_LOG	every 5 minutes	/activities/YYYY/MM/dd/YYYY-MM-dd-HH-mm/activities_v1.json.gz	These logs include user activity as captured by the Lacework Edge proxy as well as from logs for the user retrieved from 3rd party integrations, such as Okta, Google Workplace, Azure AD and Microsoft 365 etc.
Browser activity logs	BROWSER_LOG	every 10 minutes	/browserLogs/YYYY/MM/dd/YYYY-MM-dd-HH-mm/browserLogs_v1.json.gz	Browser logs are collected by Lacework Edge browser plugin and include browser activity for a user without the need of MITM policies which can be resource intensive.
Event logs	EVENT_LOG	every hour	/events/YYYY/MM/dd/YYYY-MM-dd-HH-mm/events_v1.json.gz	Lacework Edge anomaly detection engine analyzes and correlates logs and finds interesting events out of them. These interesting events from logs get promoted as Events or Alerts in Lacework Edge platform.
Admin audit logs	ADMIN_AUDIT_LOG	every 5 minutes	/adminAudits/YYYY/MM/dd/YYYY-MM-dd-HH-mm/adminAudits_v1.json.gz	These logs capture the activity of users Lacework Edge admin console. Actions like addition/modification of a policy, login to Lacework Edge admin console gets captured in these logs.
Snapshot of devices	DEVICES_SNAPSHOT	every hour	/devices/YYYY/MM/dd/YYYY-MM-dd-HH-mm/devices_v1.json.gz	Full snapshot of device records (include device ID, hostname and other details about the device).
Snapshot of policies	POLICIES_SNAPSHOT	every hour	/policies/YYYY/MM/dd/YYYY-MM-dd-HH-mm/policies_v1.json.gz	Full snapshot of policy records (include policy ID and other details about the policy).
Snapshot of users	USERS_SNAPSHOT	every hour	/users/YYYY/MM/dd/YYYY-MM-dd-HH-mm/users_v1.json.gz	Full snapshot of user records (include userID,email) to enable joining userID with emails in any of the log sources.

Schema for different types of exports is listed at Data Export Schema.

Setup

Setup of Lacework Edge Data Export involves:

1. Creating the storage (AWS S3 currently supported) with the necessary permissions for Lacework Edge.
2. Configuring Lacework Edge with the storage credentials and output options.

Each of these steps are described in detail below.

Create Data Export Storage

Create an S3 bucket with versioning enabled, ideally in the same AWS region as their Lacework Edge account.

Versioning must be enabled on this S3 bucket, as Lacework Edge uses S3 replication to copy data. S3 replication requires source and destination S3 buckets to be versioned.

You can find your Lacework Edge account region via the CLI. In the response, your region will be listed in the "location": "<region>" property:

  egcli nervecenter get-account --name <account-name>

Configure Data Export

You have two options for enabling Data Export in Lacework Edge: via the UI or the CLI:.

In both cases, you will need to provide Lacework Edge with:

• S3 bucket name
• AWS account ID (where S3 bucket is hosted).
• Data Export Types you will want exported.

Once configured, log data should start flowing into your S3 bucket within the next 5 minutes.

Configure Via UI

1. In the top-right corner of the Lacework Edge UI, click the gear icon.
2. In the Settings page, click Export under the Data settings category.
3. If you have already configured the integration, it will appear here along with its status.
4. If not configured, it will first ask you to enter:
   • Name of the S3 bucket
   • AWS account ID where bucket is hosted
5. Choose the Data Export Types you will want to have synced to your bucket.
6. The Lacework Edge UI will provide you with an S3 Bucket policy.
   • In your AWS console, navigate to the S3 bucket page: Amazon S3 > Buckets > <s3-bucket-name>
   • Click on the Permissions tab, in the Bucket Policy section, add this bucket policy and click Save.

Configure Via CLI

1. Give permissions to Lacework Edge AWS IAM role to replicate data into the eventual S3 bucket.

   • Retrieve your S3 bucket policy from Lacework Edge using this CLI command:

   egcli nervecenter get-customer-bucket-policy \
     --account_name <account-name> --s3_bucket_name <s3-bucket-name> | \
     jq '.customer_bucket_policy' | jq -rc
   • In your AWS console, navigate to the S3 bucket page: Amazon S3 > Buckets > <s3-bucket-name>
   • Click on the Permissions tab, in the Bucket Policy section, add this bucket policy and click Save.

2. Create the following JSON file and fill in the missing values:

   $ cat data-export-config.json
       {
         "account_name": "<edgeguardian-account-name>",
         "s3_bucket_account_id": "<s3-bucket-AWS-account-id>",
         "s3_bucket_name": "<s3-bucket-name>",
         "data_export_types": ["ADMIN_AUDIT_LOG","EVENT_LOG","ACTIVITY_LOG","USERS_SNAPSHOT","DEVICES_SNAPSHOT","POLICIES_SNAPSHOT"] 
       }

3. Run the following CLI command to configure data export in Lacework Edge: (expects data-export-config.json is in the same directory as egcli):

     egcli nervecenter create-data-export-config --from_file data-export-config.json