Add testquorum/action to your CI once

Test results in CI come in two flavours: a green check, and a wall of log text when something fails. Anything richer — which tests ran, how long they took, which one's been flaking lately — usually means picking a reporter per language, configuring it per framework, and redoing the whole thing in the next repo. We've done that enough times.

So today we're launching testquorum/action. One step in your workflow, every framework in your repo:

- uses: testquorum/action@main

No inputs, no version pin, no per-language plugin. That is the whole step.

What it actually does

The action downloads a prebuilt testquorum-runner binary and runs it. The runner looks at your repo and figures out what's testable:

• Cargo.toml → enumerates every #[test] (and a doctest run per package) via cargo metadata, then runs each one individually.
• flake.nix → evaluates your checks attrset (or any attrset you point it at), then builds each derivation as its own check.
• treefmt.toml → runs treefmt --ci.

Each of those is a manager; new ones land as we grow language coverage. There's no plugin to install, no config to write — if the manifest is in the repo, the matching manager turns on. Add a new framework next quarter, the next CI run picks it up. Same one line.

If you do want to tweak something — point Nix at a different attrset, disable a manager, use a non-root Cargo.toml — drop a .testquorum.toml in your repo. Most won't need one.

You can see this end-to-end in testquorum/rust-example: a Rust project where the only TestQuorum reference in CI is that single uses: line.

Run your tests properly

cargo test, nix flake check, treefmt — these usually run as one CI step with one pass-or-fail at the end. The runner enumerates each test up front and runs them individually, randomises order to surface order-dependent flakes, streams results as they finish, and reports timing and status per test.

Take randomisation as the worked example. Order-dependent flakes are the same problem in every language — but the fix looks different everywhere you go. GTest under CMake wants --gtest_shuffle (or GTEST_SHUFFLE=1 in the environment) on each test binary. Jest under npm takes --randomize (or randomize: true in jest.config.js). Each one is a five-line problem in isolation, but you have to know it exists, find it, and re-find it for every framework you ship.

The runner does it for you. Randomisation is just the first one — we want this to be the place where best practice for running tests in CI keeps landing, whichever CI you use, whichever languages you ship, so the improvements arrive without you finding and integrating each one yourself. More to come on that, and on the paid testquorum.dev API that builds on the test data the runner streams up.

Free for the part that runs your tests

The action and runner are open source and free forever. By default the runner makes one outbound call — a check against testquorum.dev to see whether the repo is subscribed — and then runs your tests. With a token, results stream to testquorum.dev where we keep history, surface flakes, and (more on this in future posts) build paid analytics on top. The core stays free.

If you want zero network traffic out of CI, pass local_only: true and even the subscription check is skipped:

- uses: testquorum/action@main
  with:
    local_only: true

The runner then just runs your tests and exits — nothing leaves your CI.

Updates without churn

Each release of the action bakes in a tested runner version. We'll do our best to never break @main — but your CI should ideally never break without a code change, so we recommend pinning the action to a specific commit and letting Renovate (or a similar tool) raise a PR when a new one lands. That way every bump is a code change you can review and, if necessary, revert.

- uses: testquorum/action@<sha>  # kept current by Renovate

If a tagged release is easier for your workflow, we'll ship those too — pin to @v1 (or a specific tag) and Renovate will track tags instead. Either way, the action picks up the matching runner; pass version: to override the runner specifically, or flake-ref: to build from source.

More best-practice posts on CI hygiene to follow.

Where we are

Three managers — cargo, nix, treefmt — is where the runner is today. We're growing it. If your stack isn't covered yet, or there's something you'd like the runner to handle differently, open an issue on testquorum/testquorum-rs. We're eager to work on it.