Other frameworks

Captain ships built-in support for the frameworks listed in the overview, and we are always open to PRs adding support for new ones. However, even if you use a completely custom framework, you can still use Captain by having your runner emit the framework-agnostic RWX v1 Test Results Schema. This will allow you to use the full breadth of features that Captain offers for flake detection, quarantining, partitioning, and retries.

Getting started

Configure your runner to write its results as RWX v1 JSON. Then tell Captain where to find the output with test-suites.*.results.path.

test-suites:
  your-suite:
    results:
      path: tmp/rwx-results/*.json

You can optionally set results.language and results.framework (or the --language / --framework flags on captain run) to label your runner in the Captain interface. A runner that already emits RWX v1 JSON is ingested generically, so these are not required.

Flake detection, quarantining, and partitioning

These features operate on parsed results, so they work without native support once Captain is reading your output:

• Flake detection and quarantining identify and isolate unreliable tests.
• Partitioning splits your suite across parallel runs using recorded test timings.

Retrying tests

For natively supported frameworks, Captain builds the retry command from template variables like {{ tests }}, which requires it to know the framework's filter syntax. For your own runner, you write the retry command and use the {{ jsonFilePath }} template variable:

test-suites:
  your-suite:
    retries:
      attempts: 2
      command: your-runner --only-failing={{ jsonFilePath }}

Captain replaces {{ jsonFilePath }} with the path to a file containing the tests to retry, written as an RWX v1 Test Results document — the same schema your runner emits, filtered to the failures. Your runner reads that file and re-runs those tests.