---
title: "CI"
description: "Our test setup and how our CI files are generated."
url: https://develop.sentry.dev/sdk/platform-specifics/python-sdk/ci/
---

# CI

Our test suite consists of a common part that tests basic SDK functionality, and an integrations part that tests individual integrations.

Since the number of integrations to test is so large, the tests in CI are split into multiple groups (roughly by integration type) so that they can run in parallel. So there is, for example, [a GitHub workflow](https://github.com/getsentry/sentry-python/blob/master/.github/workflows/test-integrations-flags.yml) that runs all our feature flag integrations, and [another one](https://github.com/getsentry/sentry-python/blob/master/.github/workflows/test-integrations-graphql.yml) that runs the test suites of all GraphQL integrations, and so on. These workflow YAMLs are auto-generated by [this script](https://github.com/getsentry/sentry-python/blob/master/scripts/split-tox-gh-actions/split-tox-gh-actions.py).

The ultimate source of truth of what should be tested and on which versions (both Python versions as well as package versions) is stored in [`tox.ini`](https://github.com/getsentry/sentry-python/blob/master/tox.ini). This file is also auto-generated each week by a GitHub cron job that scans PyPI for new releases and updates the test matrix with them. The script that does this lives in [`scripts/populate_tox`](https://github.com/getsentry/sentry-python/tree/master/scripts/populate_tox) and is often referred to as "toxgen".

## [The Big Picture](https://develop.sentry.dev/sdk/platform-specifics/python-sdk/ci.md#the-big-picture)

There are multiple components to the test setup that runs in CI:

* on a weekly basis, [`scripts/populate_tox`](https://github.com/getsentry/sentry-python/tree/master/scripts/populate_tox) creates a PR that updates `tox.ini` with new releases
* `tox.ini` is where the configuration (what to test and what versions) is stored and read from
* `scripts/split-tox-gh-actions/split-tox-gh-actions.py` is the script that defines the [test groups](https://github.com/getsentry/sentry-python/blob/0f3e5db0c8aabcad0baf0e8b2d3e31e27e839b3e/scripts/split-tox-gh-actions/split-tox-gh-actions.py#L57) and generates the resulting YAML files
* `scripts/split-tox-gh-actions/templates` contains the jinja2 template snippets used to generate the YAML files
* `.github/workflows/test-integrations-*.yml` are the resulting workflow configuration YAMLs

If you update any of the components without keeping the rest in sync, for instance by making a change in one of the `test-integrations-*.yml` files that is not reflected in the templates in `scripts/split-tox-gh-actions/templates`, they will be overwritten next time they're auto-generated.