👋 Email us at [email protected] with questions or feedback. We'd love to hear from you. If you have an account with RWX, view your support page for access to support via Slack.

Test Partitioning

Captain can partition your test files into multiple groups for parallel execution.

As long as you've integrated your test suites into Captain, the test files will be grouped based on their predicted runtime, ensuring that each partition will have close to equal runtime.

Calling the Partition Command in the CLI

You can partition files using the partition command.

For the CLI to automatically fetch timing information to optimally partition the files, you'll need to pass the same --suite-id that you used when integrating your test suites into Captain. You'll also need to have your access token set in an environment variable named RWX_ACCESS_TOKEN.

Tell the CLI how many partitions you'd like to split the files into using the --total argument, as well as which index you'd like to generate using the --index argument. The index is zero-based.

Finally, you'll need to pass the list of files to the CLI for partitioning. In most projects, the easiest way to do this is passing a glob pattern. Because some shells don't support recursive glob expansion, pass the pattern in quotes so that the CLI can expand the pattern rather than your shell.

Some shells, such as bash, do not support recursive glob patterns. When passing a file pattern into the CLI, make sure you wrap it in quotes for recursive glob support. It's a good idea when first implementing partitioning to check the build output to make sure the total number of tests that ran across all partitions matches the number that were running pre-partitioning.

Here's a full example of using the partition command using a typical RSpec file pattern. This example splits the test files into two partitions.

captain partition \
  --suite-id your-project-your-suite \
  --index 0 --total 2 \
  "spec/**/*_spec.rb"

--index 0 is generating the first of two indexes. You'll also want a task that runs the tests for --index 1 in your CI pipeline.

Passing Files to Your Test Runner

The output of the partition command is a list of files. To actually run the tests, you'll need to pass the list of files into your testing framework's CLI. The best way to do this is set the list of files in a variable to pass into the next command. You'll also want to maintain the integration with captain run when running your tests.

A full command would look like:

test_files=$(captain partition \
  --suite-id your-project-your-suite \
  --index 0 --total 2 \
  "spec/**/*_spec.rb"
)
captain run \
  --suite-id your-project-your-suite \
  --test-results tmp/rspec.json \
  -- bundle exec rspec $test_files

Aggregating Test Output

Typically, partitioning a test suite has a disadvantage of splitting test results across multiple CI jobs, requiring engineers to dig into each partition's test output to get a comprehensive overview of failures. With Captain, any tests run using the same --suite-id will be aggregated in Captain's

test failure summaries, making it easy for engineers to see all of their failures, regardless of how many partitions the test suite is split into.

GitHub Actions Matrix Example

You can use a matrix on GitHub Actions to easily split a test suite into partitions.

run_tests:
  runs-on: ubuntu-latest
  strategy:
    fail-fast: false
    matrix:
      partition_index: [0, 1, 2]
      partition_total: [3]
  steps:
    - uses: actions/[email protected]
    - uses: ruby/setup-[email protected]
      with:
        bundler-cache: true
    - name: run tests
      run: |
        test_files=$(captain partition \
          --suite-id your-project-your-suite \
          --index ${{ matrix.partition_index }} \
          --total ${{ matrix.partition_total }} \
          "spec/**/*_spec.rb"
        )
        captain run \
          --suite-id your-project-your-suite \
          --test-results tmp/rspec.json \
          -- bundle exec rspec $test_files
      env:
        RWX_ACCESS_TOKEN: ${{ secrets.RWX_ACCESS_TOKEN }}