Rust flaky tests

Tuist can detect inconsistent Cargo test results and help you investigate them in Tests → Flaky Tests.

How flaky detection works#

Tuist compares CI runs on the same commit. A test that passes in one run and fails in another can be marked as flaky.

Report each run through the usual command:

Cargo does not automatically retry failed tests. Re-running a CI job on the same commit provides another result for cross-run detection.

Managing flaky tests#

Automatic clearing#

Detection and recovery run through the project's "Flaky test detection" automation. By default, the flaky flag clears after 14 days without re-triggering. Edit the recovery window and actions under Settings → Automations.

Manual management#

Open Tests → Flaky Tests to inspect a test's history. You can manually mark or unmark it as flaky from its detail page.

Quarantining and stress-testing#

Mute or skip a test from its detail page, or configure quarantine in Settings → Automations. The next tuist cargo test fetches the project's current quarantine state:

• Muted tests still run and report their actual results, but their failures do not fail the command. Other test failures, compilation errors, crashes, and incomplete results still fail normally.
• Skipped tests are excluded from execution and reported as quarantined skips. Matching includes the package, target, Rust module path, and test name, so similarly named tests remain independent.

Muting preserves the selected workspace packages and features, and runs all selected targets even after a muted target fails. Skip quarantine requires one selected package: use tuist cargo test -p store rather than a multi-package workspace command. Tuist rejects multi-package skipping because narrowing the command can change Cargo's unified dependency features.

Quarantine also applies to sharded Cargo runs. If Tuist cannot fetch the state, it warns and runs without quarantine; failures are not suppressed. To deliberately run without quarantine:

Enforcement requires standard libtest harnesses and captured default output. Commands with quiet, verbose, alternate-format, or uncaptured output fail when quarantine needs to be applied. Select an exact workspace package with -p <package> if target attribution is ambiguous. Custom harnesses, package/target-name globs, and unresolved package selections are unsupported. Doctests can be muted or skipped too, but Rustdoc's test filtering cannot distinguish every possible pair of example names. If skipping an example would also skip an enabled example, or a partial doctest exclusion uses --exact, Tuist fails with a diagnostic rather than executing the skipped example or silently omitting enabled tests; use the override above while investigating.

Tuist does not stress-test new Cargo tests. Use a Cargo test filter to repeat a failing test while investigating it:

Slack notifications#

Configure Slack flaky test alerts to notify your team when a test becomes flaky.