Test Result Aggregation
As a test suite runs, ABQ workers will individually output and report the results of the tests they have been assigned.
It can be convenient to consolidate all of your test suite's results in one place
for easier review. ABQ provides an
abq report command to aggregate test
results in a single reporting step, after your test run completes.
abq report takes a run id via
--run-id or the
variable, and any number of reporters via the
--reporter CLI flag. For example, to consolidate all results of the test run
with run id
my-run-id, and output RWX v1 JSON and JUnit XML reports, execute
export ABQ_RUN_ID=my-run-id abq report --reporter rwx-v1-json=results.json --reporter junit-xml=results.xml
abq report aggregates and correctly reports retries,
whether they be automatic or manual retries.
No worker for a given run id may be running when
abq report is executed,
including manual retries of a worker. If a worker is running when
abq report will exit with an error. We suggest configuring
your CI so that
abq report steps only run after all workers
have explicitly completed.