RWX

RWX and Captain work together to run your tests and provide you with test result data directly in RWX.

Here, we'll walk through everything needed to integrate Captain with RWX:

  1. Installing the Captain CLI
  2. Integrating Captain with your test framework
  3. Retry failed tests from RWX

Installing Captain

Add a task that uses the rwx/install-captain package to install the Captain CLI.

# .rwx/ci.yml

tasks:
  - key: captain
    call: rwx/install-captain 1.1.6

Integrating Captain with your test framework

Captain integrates with many test frameworks. [Find instructions for your specific test framework here.][captain-test-frameworks]

This example integrates with [Ruby's RSpec][ruby-rspec] but can be used as a baseline for integrating with a different framework.

# .captain/config.yml

test-suites:
  captain-examples-rspec:
    command: bundle exec rspec --format json --out rspec-results.json --format progress
    results:
      path: rspec-results.json
    output:
      print-summary: true
      reporters:
        rwx-v1-json: captain-results.json

# .rwx/ci.yml

tasks:
  # ... tasks that clone your repo, setup Ruby, etc. here
  - key: captain
    call: rwx/install-captain 1.1.6

  - key: rspec
    use: captain # you'll likely also depend on your code, your Ruby dependencies, etc.
    run: captain run captain-examples-rspec
    outputs:
      test-results:
        - path: captain-results.json

Make sure to have Captain report rwx-v1-json to a path specified in outputs.test-results. This will allow you to view your test results directly in RWX!

There's no need to specify an RWX_ACCESS_TOKEN when using Captain with RWX. RWX automatically provides an access token to Captain with permissions to save test results to Captain Cloud.

Partitioning

Captain's partitioning feature works with RWX parallel tasks:

Start by updating your Captain configuration to enable partitioning as follows:

# .captain/config.yml

test-suites:
  captain-examples-rspec:
    # existing config...
    partition:
      command: bundle exec rspec --format json --out tmp/rspec.json --format progress {{ testFiles }}
      globs:
        - spec/**/*_spec.rb

And then add parallel: to your task. Captain reads the partition index and total directly from the RWX_PARALLEL_INDEX and RWX_PARALLEL_TOTAL environment variables that RWX sets, so no extra flags are needed.

# .rwx/ci.yml

tasks:
  - key: code
    call: git/clone 2.0.7
  - key: ruby
    call: ruby/install 1.2.30
    with:
      ruby-version: 3.3.0
  - key: deps
    use: [code, ruby]
    run: bundle install
  - key: captain
    call: rwx/install-captain 1.1.6

  - key: rspec
    use: [deps, captain]
    parallel: 8
    run: captain run captain-examples-rspec
    outputs:
      test-results:
        - path: test-results.json

Viewing test results

RWX will display your test results directly in the UI:

test results in RWX

Captain will also automatically output an RWX link linking to your test results inside of Captain, and the results in Captain will link back to RWX.

Retrying failed tests

If you configure a retry command for Captain, RWX will show a "Retry failed tests" action on the test task that re-runs only the tests that failed. Set test-suites.*.retries.command in your .captain/config.yml:

# .captain/config.yml

test-suites:
  captain-examples-rspec:
    command: bundle exec rspec --format json --out rspec-results.json --format progress
    results:
      path: rspec-results.json
    output:
      reporters:
        rwx-v1-json: captain-results.json
    retries:
      attempts: 2
      command: bundle exec rspec --format json --out rspec-results.json --format progress {{ tests }}

{{ tests }} expands to the failing test identifiers, and attempts tells Captain how many times to automatically retry failures inline before the task fails. See test-suites.*.retries for the full set of options and your test framework's Captain docs for framework-specific examples.

If you are also using ABQ, you will need to supply the --local flag to ABQ in your retry command.