Rust code coverage

Code coverage tells you which lines of your code your tests executed. Tuist measures the line coverage of every cargo test run with LLVM's source-based coverage and folds the runs of a commit into one figure, for the project, each package and each file. The dashboard's Code Coverage page shows it per commit and as a trend for each branch.

Setup#

Turn coverage on#

Add this to the tuist.toml at your repository root:

Then run your tests as usual:

Tuist builds the tests with -C instrument-coverage, runs them, and uploads their coverage along with the run's test results. Your tests' output and exit code don't change. To override the setting for a single run, set TUIST_COVERAGE_UPLOAD=1 or TUIST_COVERAGE_UPLOAD=0.

Coverage relies on the Git history Tuist uploads with each run that collects it, which places the commit on its branch. A run outside a Git repository uploads its test results but no coverage.

Sharded runs#

When you shard your tests, every shard that runs with coverage on reports the coverage of its own targets, and Tuist merges the shards of a run into one figure. A shard runs part of the suite on purpose, so it doesn't make the commit's coverage incomplete.

Complete each commit on CI#

A commit's coverage often comes from several jobs: one per shard or per feature set. Tuist can't tell when the last of them has reported, so tell it. Once every test job for a commit has finished, run this in a job that depends on all of them:

The command reads the commit from the checkout, or the one your CI provider reports on a pull request; pass --commit <sha> to name another one. Until a commit is complete, its coverage shows as In progress and stays out of its branch's trend.

How it works#

What Tuist measures#

• Packages. Each Cargo package is a target on the dashboard, with its own coverage.
• Product code only. Integration tests, benchmarks and examples (the tests/, benches/ and examples/ directories, and targets of those kinds) are test code. Tuist stores them but leaves them out of every figure. Unit tests in a #[cfg(test)] module are part of their source file, so their lines count with it, as in cargo llvm-cov.
• Functions. Each line belongs to the function that starts closest above it. A generic function counts once, however many types it was instantiated with.
• Doctests run as usual but aren't measured.

Tuist also records which code each test binary executed, for coverage-based test selection later. cargo test runs a binary's tests in one process, so that record belongs to the binary as a whole, not to its individual tests.

Skipped tests#

When a run's own arguments leave tests out, such as a test name filter, --skip, --ignored, -p, --exclude, --lib or --test, the commit's coverage is Incomplete, unless another run of the commit that left nothing out ran too. A failure that stops cargo test before every test binary ran has the same effect. Add --no-fail-fast to keep running the remaining binaries.

Some tests are left out on purpose and don't make a commit incomplete:

• Tests that quarantine skips. They're left out as if they had been removed, so the lines only they covered count as uncovered.
• The other shards' targets, when you shard your tests.
• Ignored tests (#[ignore]), which don't run by default.

Complete and incomplete commits#

Each commit on the dashboard has a status:

A branch's trend only includes complete commits.

Performance impact#

Instrumented code runs a bit slower, and Cargo compiles it separately from your regular builds, so the first run with coverage rebuilds your crates. The Rust compilation cache doesn't reuse instrumented compilations, so runs with coverage compile every crate locally.

Limitations#

• Target-specific rustflags. Tuist adds -C instrument-coverage to Cargo's build.rustflags. A target.<triple>.rustflags entry in a Cargo configuration file replaces them, so add the flag there yourself. A run whose tests wrote no coverage says so.
• Configured target runners. Tuist starts the test binaries through a runner of its own. A target.<triple>.runner in a Cargo configuration file takes its place: the run's coverage is still uploaded, but Tuist can't tell which test binary executed what. A runner set through the CARGO_TARGET_<TRIPLE>_RUNNER environment variable keeps working.
• No coverage from pull requests from forks. Coverage is uploaded only by runs that authenticate with Tuist, and CI usually withholds secrets and identity tokens from pull requests opened from forks.

Querying coverage#

The dashboard's Code Coverage page shows each commit's figure, its packages and its files, with the functions each file holds. The same data is available to agents through the Model Context Protocol server.