👋 Email us at [email protected] with questions or feedback. We'd love to hear from you. If you have an account with RWX, view your support page for access to support via Slack.

Test Suites

Captain provides test analytics and tooling for engineering teams to make their test suites faster and more reliable.

Integration requires three steps:

  1. Configure your test framework reporter
  2. Install the Captain CLI
  3. Call the CLI when running tests

Configure your test framework reporter

You'll need to configure your test framework to write its test results to a file. Here are examples for how to configure reporters in several of the most popular testing frameworks. We're working on building support for additional frameworks. If you'd like us to prioritize any one in particular, please let us know.

Install the CLI

Next, you'll need to install the CLI.

On GitHub Actions, you can install the CLI using the setup-captain action.

uses: rwx-research/setup-[email protected]

Call the CLI when running tests

To complete the integration, call the CLI when running tests by using the captain run command.

You'll need to set your access token in an environment variable named RWX_ACCESS_TOKEN.

Two arguments are required.

--suite-id will be the identifier that Captain uses to scope your test results. Identifiers are scoped to an organization rather than a repository, so ensure that it's unique across your org. It's typical to have multiple test suites within a project, so a conventional id is {project-name}-{suite-name}, such as yourapp-jest or yourapp-cypress.

--test-results needs to point to the test results file that you configured your test framework to produce. It can support a glob pattern if you're writing results to multiple files. Take special note of ensuring that you pass the pattern in quotes so that your shell doesn't expand the pattern before running your test command, such as --test-results "tmp/jest-*.json"

Pass the command to run your tests after a double hyphen. The full command will look like this:

captain run \
  --suite-id your-project-your-suite \
  --test-results test-results-file.json \
  -- test_command_goes_here --args-to-test-command

Here's an example of what the command might look like for a Jest test suite.

captain run \
  --suite-id your-project-jest \
  --test-results tmp/jest.json \
  -- npm run test

GitHub Actions

If you're using Captain from GitHub Actions, the run command acccepts two additional arguments which you may need to pass. They're necessary due to an idiosyncracy of the GitHub API, which we've opened a PR and a discussion to fix.

--github-job-name takes the name of the job as a string. You only need this if you're setting the name: property in your GitHub Workflow YAML, like this:

run_tests:
  name: Run Tests

If you're doing that, you'll also need to pass --github-job-name "Run Tests" into captain run.

If your tests are running in a matrix, you'll also need to pass --github-job-matrix '${{ toJSON(matrix) }}'

A full command on GitHub Actions could look like:

name: run tests
run: |
  captain run \
    --suite-id your-project-jest \
    --test-results tmp/jest.json \
    --github-job-name "Run Jest Tests" \
    --github-job-matrix '${{ toJSON(matrix) }}' \
    -- npm run test
env:
  RWX_ACCESS_TOKEN: "${{ secrets.RWX_ACCESS_TOKEN }}"

Conclusion

🎉 that's it! You'll now be able to: