---
title: "Kafka consumers"
url: https://develop.sentry.dev/backend/application-domains/kafka/
---

# Kafka consumers

## [Create a new Kafka topic](https://develop.sentry.dev/backend/application-domains/kafka.md#create-a-new-kafka-topic)

1. Add the topic to the `KAFKA_TOPIC_TO_CLUSTER` in [src/sentry/conf/server.py](https://github.com/getsentry/sentry/blob/master/src/sentry/conf/server.py):

* e.g. `subscription-results-eap-items`

2. Add the topic to `Topic` in [src/sentry/conf/types/kafka\_definition.py](https://github.com/getsentry/sentry/blob/master/src/sentry/conf/types/kafka_definition.py)

## [Define or re-use a processing strategy](https://develop.sentry.dev/backend/application-domains/kafka.md#define-or-re-use-a-processing-strategy)

In most cases a [Streaming Factory](https://getsentry.github.io/arroyo/getstarted.html#create-a-streaming-consumer) is what you want to when defining a consumer (see next section). You can find examples of it in [Sentry's code base](https://github.com/search?q=repo%3Agetsentry%2Fsentry+%28ProcessingStrategyFactory\&type=code).

## [Define a new Kafka consumer](https://develop.sentry.dev/backend/application-domains/kafka.md#define-a-new-kafka-consumer)

1. Add a new entry in the `KAFKA_CONSUMERS` key in [src/sentry/consumers/**init**.py](https://github.com/getsentry/sentry/blob/master/src/sentry/consumers/__init__.py):

```python
KAFKA_CONSUMERS = {
    "<your_topic_str_here>": {
        "topic": Topic.YOUR_TOPIC,
        "strategy_factory": "sentry_package_defining_your_strategy_factory_class",
    }
}
```

2. You may need optional properties (e.g. `click_options`, you will need to research them by looking at [ConsumerDefinition](https://github.com/getsentry/sentry/blob/master/src/sentry/conf/types/kafka_definition.py)'s code.

3. Make sure you can run it: `sentry run consumer <your_topic>`

4. You may need to add some devserver options [here](https://github.com/getsentry/sentry/blob/master/src/sentry/runner/commands/devserver.py).

5. Add tests for your consumer

## [Deployment](https://develop.sentry.dev/backend/application-domains/kafka.md#deployment)

Visit the Ops repo and search for `shared_config/kafka/README.md` for a full, in-depth step-by-step guide.