ABQ with Your CI
Installing the CLI
To install the ABQ CLI, you'll need
- an rwx access token
The access token is used to authenticate against the ABQ API. Visit the access-tokens documentation for more information on creating an access token.
abq will read the access token from an environment variable named
install-id is used to ensure the same version of abq is installed on all CI workers working on a single test run.
install-id should be:
- unique to this CI run
- consistent across all CI workers for this run
- if you'd like to retry individual workers without re-running your whole test
install-idshould be consistent between attempts of the CI run
Some options might be:
# Buildkite INSTALL_ID=$BULDKITE_BUILD_ID # CircleCI INSTALL_ID=$CIRCLE_WORKFLOW_JOB_ID # Generally INSTALL_ID="$CI_RUN_ID"
RWX_ACCESS_TOKEN set, download & install the CLI:
OS="$(uname --kernel-name)" # Supported values are Linux or Darwin ARCH="$(uname --machine)" # Supported values are x86_64 or aarch64 tmp="$(mktemp -d)/abq" && \ curl -o $tmp -fsSL -H "Authorization: Bearer $RWX_ACCESS_TOKEN" \ "https://abq.build/api/releases/v1/$OS/$ARCH/abq?install_id=$INSTALL_ID" && \ sudo install $tmp /usr/local/bin && \ rm $tmp && \ abq --version
If you're installing ABQ in a Docker image, we'd recommend putting it close to the execution of your tests, and avoiding caching, to ensure you have the most up to date version of the CLI installed.
Running your tests
First setup your test suite with an abq-compatible plugin (see: Test Framework Integration)
abq will need
RWX_ACCESS_TOKENenvironment variable you set earlier
- its worker's index
to use in a command like this:
abq test --worker "$WORKER_INDEX" \ --run-id "$RUN_ID" \ -- bundle exec rspec spec/features
the worker index should be 0-indexed, because the worker with index 0 will output aggregated test results.
Here are some worker indices for Buildkite and CircleCI:
# Buildkite WORKER_INDEX="$BUILDKITE_PARALLEL_JOB" # CircleCI WORKER_INDEX="$CIRCLE_NODE_INDEX"
run-id should be unique to this run, attempt or retry, and test suite
If running a single test suite, it can be the same as the
install-id you set earlier.
If running multiple test suites, it should be unique. For instance, with two rspec test suites & a jest test suite, you could use:
RSPEC_FEATURES_RUN_ID="$INSTALL_ID-rspec-features" RSPEC_UNIT_RUN_ID="$INSTALL_ID-rspec-unit" JEST_RUN_ID="$INSTALL_ID-jest"
Then you can launch your tests with:
abq test --worker "$WORKER_INDEX" \ --run-id "$RSPEC_FEATURES_RUN_ID" \ -- bundle exec rspec --tag feature abq test --worker "$WORKER_INDEX" \ --run-id "$RSPEC_UNIT_RUN_ID" \ -- bundle exec rspec --tag ~feature abq test --worker "$WORKER_INDEX" \ --run-id "$JEST_RUN_ID" \ -- bundle exec jest