Home

Awesome

Raccoon

build workflow package workflow Coverage Status License Version

Raccoon is high throughput, low-latency service that provides an API to ingest clickstream data from mobile apps, sites and publish it to Kafka. Raccoon uses the Websocket protocol for peer-to-peer communication and protobuf as the serialization format. It provides an event type agnostic API that accepts a batch (array) of events in protobuf format. Refer here for proto definition format that Raccoon accepts.

<p align="center"><img src="./docs/static/assets/overview.svg" /></p>

Key Features

To know more, follow the detailed documentation

Use cases

Raccoon can be used as an event collector, event distributor and as a forwarder of events generated from mobile/web/IoT front ends as it provides an high volume, high throughput, low latency event-agnostic APIs. Raccoon can serve the needs of data ingestion in near-real-time. Some domains where Raccoon could be used is listed below

Resources

Explore the following resources to get started with Raccoon:

Run with Docker

Prerequisite

Run Docker Image

Raccoon provides Docker image as part of the release. Make sure you have Kafka running on your local and run the following.

# Download docker image from docker hub
$ docker pull raystack/raccoon

# Run the following docker command with minimal config.
$ docker run -p 8080:8080 \
  -e SERVER_WEBSOCKET_PORT=8080 \
  -e SERVER_WEBSOCKET_CONN_ID_HEADER=X-User-ID \
  -e PUBLISHER_KAFKA_CLIENT_BOOTSTRAP_SERVERS=host.docker.internal:9093 \
  -e EVENT_DISTRIBUTION_PUBLISHER_PATTERN=clickstream-%s-log \
  raystack/raccoon

Run Docker Compose You can also use docker-compose on this repo. The docker-compose provides raccoon along with Kafka setup. Then, run the following command.

# Run raccoon along with kafka setup
$ make docker-run
# Stop the docker compose
$ make docker-stop

You can consume the published events from the host machine by using localhost:9094 as kafka broker server. Mind the topic routing when you consume the events.

Running locally

Prerequisite:

# Clone the repo
$ git clone https://github.com/raystack/raccoon.git

# Build the executable
$ make

# Configure env variables
$ vim .env

# Run Raccoon
$ ./out/raccoon

Note: Read the detail of each configurations here.

Running tests

# Running unit tests
$ make test

# Running integration tests
$ cp .env.test .env
$ make docker-run
$ INTEGTEST_BOOTSTRAP_SERVER=localhost:9094 INTEGTEST_HOST=localhost:8080 INTEGTEST_TOPIC_FORMAT="clickstream-%s-log" GRPC_SERVER_ADDR="localhost:8081" go test ./integration -v

Contribute

Development of Raccoon happens in the open on GitHub, and we are grateful to the community for contributing bugfixes and improvements. Read below to learn how you can take part in improving Raccoon.

Read our contributing guide to learn about our development process, how to propose bugfixes and improvements, and how to build and test your changes to Raccoon.

To help you get your feet wet and get you familiar with our contribution process, we have a list of good first issues that contain bugs which have a relatively limited scope. This is a great place to get started.

This project exists thanks to all the contributors.

License

Raccoon is Apache 2.0 licensed.