Skip to main content

Stream Enrich

note

stream-enrich-kinesis and stream-enrich-kafka will soon be deprecated in favor of enrich-kinesis and enrich-kafka, respectively. We recommend switching when you have the opportunity.

Available on Terraform Registry​

A Terraform module is available which deploys Stream Enrich on a AWS EC2 without the need for this manual setup.

Run Stream Enrich​

Stream Enrich is on docker hub with several different flavours. Pull the image that matches the source/sink you are using:

docker pull snowplow/stream-enrich-kinesis:3.7.1
docker pull snowplow/stream-enrich-kafka:3.7.1
docker pull snowplow/stream-enrich-nsq:3.7.1

The application is configured by passing a hocon file on the command line:

docker run --rm \
      --log-driver awslogs \
      --log-opt awslogs-group=${log_group_name} \
      --log-opt awslogs-stream=$(ec2metadata --instance-id) \
      -v ${path_to_config_dir}:/snowplow/config \
      snowplow/stream-enrich-${message_queue}:3.7.1 \
      --config /snowplow/config/config.hocon \
      --resolver file:/snowplow/config/iglu_resolver.json \
      --enrichments file:/snowplow/config/enrichments/

Alternatively, you can download and run a jar file from the github release.

java -jar snowplow-stream-enrich-kinesis-3.7.1.jar \
  --config /path/to/config.hocon \
  --enrichments file:/path/to/enrichments/ \
  --resolver file:/path/to/iglu_resolver.json \
  --force-cached-files-download

The configuration guide describes how to construct the config, enrichments, and resolver input files.

Was this page helpful?