Git Push to Run CI/CD is a Terrible Developer Experience

Traditional CI/CD platforms are designed to run when developers push commits or update their pull requests. Unfortunately, that's the only way that they can be run. And it's a terrible developer experience when implementing new workflows.

Mark Erikson @acemarke

August 26, 2021

Pro-tip:

`git commit --amend` + `git push --force` is a great way to make it look like it didn't actually take you 37 tries to get that CI job working correctly

413

Pablo Galindo Salgado @pyblogsal

March 24, 2023

Ah yes, the joy of setting up CI:

$ git commit "I swear this time I got it right" && git push origin HEAD -f

Imagine this in any other part of software engineering – not being able to test anything locally and having to git push to see if your changes work.

A third-party project trying to solve this problem for GitHub Actions has over 50k stars.

Decoupling from Version Control Events

We designed Mint to provide the ability to initiate CI/CD runs locally, without having to git push. The solution to this problem is fairly straightforward, although I expect it'll be difficult for traditional CI providers to ever fix. To initiate CI runs locally, the job or task definitions must be decoupled from version control events.

Init Parameters

Certain data attributes from version control events are often needed to implement CI workflows, such as a commit sha. However, those data attributes can be passed in as parameters, rather than having CI tasks directly coupled to version control events.

Mint achieves this using init parameters

1
2
3
4
5
6
7
8
9
10
11
12
on:
  github:
    push:
      init:
        commit-sha: ${{ event.github.push.head_commit.id }}

tasks:
  - key: code
    call: mint/git-clone 1.2.3
    with:
      repository: https://github.com/YOUR_ORG/YOUR_REPO.git
      ref: ${{ init.commit-sha }}

Rather than having the code task depend on the GitHub push event, it is parameterized to require a commit-sha. Using this approach, the commit-sha can be set based on the push event when the workflow is running in response to a push. However, it can also be set from the CLI when initiating a run locally.

1
mint run --file .mint/ci.yml --init commit-sha=25d430c2310dda51ab148184abf13a4e64f63eb8

You can also set the commit-sha value dynamically for the latest commit on your current branch using git rev-parse.

1
mint run --file .mint/ci.yml --init commit-sha=$(git rev-parse HEAD)

Concurrency Controls

Version control metadata is also commonly used for concurrency controls. In most CI implementations, pushing new commits to a branch will cancel any runs in progress on the same branch. Mint also supports using init parameters in concurrency pool definitions.

1
2
3
4
5
6
7
8
9
10
11
on:
  github:
    push:
      init:
        branch: ${{ event.github.push.ref }}

concurrency-pools:
  - id: your-org/your-repo:branch-${{ init.branch }}
    if: ${{ init.branch != 'refs/heads/main' }}
    capacity: 1
    on-overflow: cancel-running

This makes it easy to configure concurrency controls while still supporting local CLI runs.

Vaults

Often, secret values need to be protected by version control branch. For example, there may be deployment secrets which should only be available to workflows running on main.

In traditional CI systems, this can make the developer experience even worse if you not only have to push but also merge into main to exercise your changes.

With a local CLI run though, there needs to be a mechanism to allow access to protected secrets without blindly trusting the branch. Obviously when passing in a branch via an init parameter, any person can pass in any value.

Mint solves this by granting access to vaults based on the user initiating the run. To test changes involving workflows that utilize secrets in a locked vault, you can temporarily grant that user access to that vault.

For example, if you have a vault named your-repo-main, you can use this in your Mint workflow:

${{ vaults.your-repo-main.secrets.some-deployment-token }}

And then you can configure the your-repo-main vault to only be unlocked for runs on main or runs initiated by a certain user who is testing changes.

Logic Branches

As a best practice, you may want to decouple your scripts from the assumption of version control branches as much as possible. This will give you the most flexibility in how you are running your tasks, and it also prevents the awkwardness of saying that you're running on "main" just to trigger behavior that normally runs on main.

You can do this by setting boolean values in init parameters.

For example, instead of this:

1
2
3
4
5
6
7
8
9
10
on:
  github:
    push:
      init:
        branch: ${{ event.github.push.ref }}

tasks:
  - key: publish-binary
    if: ${{ init.branch == 'refs/heads/main' }}
    run: ...

You may want to do this:

1
2
3
4
5
6
7
8
9
10
on:
  github:
    push:
      init:
        publish-binary: ${{ event.github.push.ref == 'refs/heads/main' }}

tasks:
  - key: publish-binary
    if: ${{ init.publish-binary }}
    run: ...

And then via the CLI you can set the parameter for testing:

--init publish-binary=true

Demo

Here's a one minute video of how easy it is to use the Mint CLI to initiate runs locally.

Getting Started

Mint powers the fastest builds and has the best developer experience in CI/CD. Learn more about what makes Mint different or schedule time to chat with our engineers.