CI

The GitHub workflow files for testing specific integration groups (like AI, Web Frameworks, Databases, etc.) are generated by this script. Essentially, the script scans our tox.ini and generates the group test CI YAML files in .github/workflows.

There are multiple components to this:

• tox.ini is where the configuration (what to test and what versions) is read from
• scripts/split-tox-gh-actions/split-tox-gh-actions.py is the script that defines the test groups and generates the resulting YAML files
• scripts/split-tox-gh-actions/templates contains the jinja2 template snippets used to generate the YAML files
• .github/workflows/test-integrations-*.yml are the resulting workflow configuration YAMLs

If you update any of the components without keeping the rest in sync, for instance by making a change in one of the test-integrations-*.yml files that is not reflected in the templates in scripts/split-tox-gh-actions/templates, CI will error out as it actually checks if everything is in sync, meaning it runs the split-tox-gh-actions.py script and compares the result with the committed YAMLs.

AWS Lambda Test Suite

The AWS Lambda test suite will fail by default on new PRs. If you are an external contributor, this is not something you have to worry about, unless you've made changes actually affecting the AWS Lambda integration. It's a security precaution on our part.

Sensitive test suites (currently only AWS Lambda) require maintainer review to ensure that tests do not compromise our secrets. This review must be repeated after any code revisions.

Before running sensitive test suites, maintainers need to carefully check the PR. Then they will apply the Trigger: tests using secrets label. The label will be removed after any code changes to enforce our policy requiring maintainers to review all code revisions before running sensitive tests.