tmpfs for Super Fast Writes

Disk performance can be an important factor in CI/CD pipeline performance. It's affected by:

• Throughput: how much data you're writing or reading, especially sequentially
• IOPS: how many individual I/O operations are being performed, especially on small files
• Latency: how quickly each operation completes

The performance is mostly relevant in write-heavy tasks. And no task is heavier on the writes than an npm install, as hundreds of thousands of tiny files are written to disk.

tmpfs

To accelerate performance in high iops scenarios like an npm install, we built support for running a task using tmpfs.

To use tmpfs, just add tmpfs: true to the agent configuration on your task.

1
tasks:
2
- key: tmpfs-demo
3
  agent:
4
    tmpfs: true
5
  run: ...

Here's a longer example of writing a gigabyte of 0's.

Without tmpfs, it takes 3.376s to write this file.

With tmpfs, it takes 0.383s (8.8x faster).

1
tasks:
2
- key: tmpfs-demo
3
  agent:
4
    tmpfs: true
5
  run: |
6
    time dd \
7
      if=/dev/zero \
8
      of=testfile \
9
      bs=1M \
10
      count=1024 \
11
      oflag=direct \
12
      status=progress

Demo