Service Manager (devservices)

A standalone CLI tool called devservices is used to bring up and manage service dependencies. The tool reads from a config.yml file within a repository's devservices directory. It is an abstraction built on top of Docker Compose and Docker.

usage: devservices [-h] [--version] COMMAND ...

CLI tool for managing service dependencies.

options:
  -h, --help          show this help message and exit
  --version           show program's version number and exit

commands:
  up                  Bring up a service and its dependencies
  down                Bring down a service and its dependencies
  list-dependencies   List the dependencies of a service
  list-services (ls)  List the services installed locally
  status              View status of a service
  logs                View logs for a service
  update              Update devservices to the latest version
  purge               Purge the local devservices cache
  toggle              Toggle how a service is run

Installation

Installation instructions can be found here.

Viewing logs for a service

devservices logs

Running CLI clients for redis, postgres and clickhouse

# redis
docker exec -it redis-redis-1 redis-cli

# clickhouse
docker exec -it snuba-clickhouse-1 clickhouse-client

# psql
docker exec -it sentry-postgres-1 psql -U postgres

Removing container state

Should you really bamboozle your containers or volumes, you can use devservices purge to start over.

# Remove all data (containers, volumes, and networks) associated with ALL services
devservices purge

As an example, let's say we've managed to corrupt our postgres database while working on a migration, and you want to reset your postgres data you can do:

# Remove all data (containers, volumes, and networks) associated with a single service
docker container rm sentry-postgres-1
docker volume rm sentry_postgres-data

Running devservices with a specific mode

Common modes:

• symbolicator: Bring up sentry dependencies and symbolicator
• chartcuterie: Bring up sentry dependencies and chartcuterie
• minimal: Bring up minimal services for local development
• profiling: Bring up sentry dependencies and vroom
• full: Bring up all services (symbolicator, taskbroker, snuba, vroom, etc)

devservices up --mode symbolicator

Running a dependency locally

You can run a dependency locally instead of as a container by toggling its runtime via the devservices toggle command.

Toggle to local runtime

For example, if you're running Sentry and want to run Snuba locally from source, you can toggle Snuba to the local runtime like this:

devservices toggle snuba

or you can specify the runtime explicitly:

devservices toggle snuba LOCAL

This tells devservices to treat Snuba as a service that should be started alongside dependent services like Sentry. Dependencies of Snuba, such as Redis, Kafka, and Clickhouse, will still run as containers.

What happens when you toggle

If Sentry is already running, toggling Snuba to LOCAL will:

1. Stop Snuba's containers (unless shared by another service)
2. Start Snuba's non-local dependencies (Redis, Kafka, Clickhouse) as containers

If Sentry is not yet running, the next time devservices up is run, Snuba (and any other services you toggle to LOCAL) will be automatically started as local services alongside Sentry.

If you want to bring up Snuba, or other local runtime dependencies in a non-default mode, you can:

1. Tell devservices to not bring them up automatically by passing the --exclude-local flag to devservices up.
2. Bring that mode up manually via devservices up --mode <mode> since modes are additive.
3. Bring that service down and back up again with the new mode.

Bringing down services with local runtimes

When stopping services with devservices down, the default behavior is to stop all services and their dependencies, including dependencies with local runtimes.

If you don't want dependencies with local runtimes to be brought down when bringing down a dependent service, you can pass the --exclude-local flag to devservices down. This tells devservices to only stop dependencies that are not running with local runtimes.

Toggle back to containerized runtime

To switch a service back to running in a container, you can do the following:

devservices toggle snuba

Or explicitly specify the runtime:

devservices toggle snuba CONTAINERIZED

Replace snuba with the name of the service you want to toggle.

Migrating data from the deprecated sentry devservices

Volume names are different for each service.

Clickhouse:

• old: sentry_clickhouse
• new: snuba_clickhouse-data

Postgres:

• old: sentry_postgres
• new: sentry_postgres-data

Kafka:

• old: sentry_kafka
• new: kafka_kafka-data

Redis:

• old: sentry_redis
• new: redis_redis-data

# Create a new postgres volume
docker volume create sentry_postgres-data

# Copy over the data from the old volume
docker run --rm \
  -v sentry_postgres:/old_volume \
  -v sentry_postgres-data:/new_volume \
  ubuntu \
  bash -c "cd /old_volume && cp -a . /new_volume"

# Validate that data has been copied over
docker run --rm -v sentry_postgres-data:/data ubuntu ls -l /data