ABQ with Mint
Installing the CLI
Use the rwx/install-abq leaf to install the ABQ CLI.
tasks:
- key: abq
call: rwx/install-abq 1.1.1
abq test
The abq test
command will pick up tests from a queue created for your test
run, and execute them.
An ABQ test run can consist of one or more abq test
workers. Each worker will
locally report the results of tests it's been assigned
as it runs them.
Here's a full example of configuring an ABQ test run, using Mint's parallel
configuration to determine how much
the test suite is parallelized.
Be sure to replace YOUR_TEST_COMMAND
with the command you normally use to run your tests in CI, including any arguments.
tasks:
# ... tasks that clone your repo, setup Ruby, etc. here
- key: abq
call: rwx/install-abq 1.1.1
- key: tests
parallel: 5
run: abq test --worker ${{ parallel.index }} -- YOUR_TEST_COMMAND
You can pass any command as YOUR_TEST_COMMAND
, as long as it appears after a double hyphen.
For example, if you're using RSpec and have other flags that you're using, you could call:
abq test -- bundle exec rspec --tag integration
Run ID
Different abq test
processes for a test run are connected to each other through a run-id
. On Mint, ABQ automatically sets run-id
to ${{ run.id }}
.
The default value is sufficient as long as you're only running a single test suite through ABQ in your Mint workflow.
If you're running multiple test suites through ABQ in a single Mint workflow, then you'll need to configure the run-id
so that each test suite has a separate run-id
with its own abq test
processes.
We recommend suffixing the default value with a test suite name, such as ${{ run.id }}-rspec
.
If your configured run-id
is consistent between Mint task attempts,
[retrying][retrying-tests] a single worker task will work seamlessly,
re-running just the tests allocated to that worker.
For the ABQ CLI, you can either pass the run id as a --run-id
argument, or you can set it in an environment variable named ABQ_RUN_ID
.
Captain and 'Retry failed tests'
If you use ABQ with Captain in Mint, Captain will add the ability to retry only the tests that failed for a task. To enable this feature with ABQ, you'll need to give Captain a retry command that invokes ABQ with the --local
flag.
For example:
# .captain/config.yaml
test-suites:
my-test-suite:
command: bash -c "abq test --worker $ABQ_WORKER --reporter rwx-v1-json=abq-results.json -- bundle exec rspec"
retries:
command: abq test --local --reporter rwx-v1-json=abq-results.json -- bundle exec rspec {{ tests }}
results:
path: abq-results.json
output:
print-summary: true
reporters:
rwx-v1-json: captain-results.json