# Bazel flaky tests {#bazel-flaky-tests}

> [!IMPORTANT]
> **Requirements**
>
> - <.localized_link href="/guides/features/test-insights/bazel">Bazel test insights</.localized_link> configured through `tuist bazel setup`.

Flaky tests are tests that produce different results (pass or fail) when run multiple times with the same code. They erode trust in your test suite and waste developer time investigating false failures. Tuist automatically detects flaky tests and helps you track them over time.


## How flaky detection works {#how-it-works}

Tuist uses the individual cases from Bazel's `test.xml` reports to detect inconsistent results across retries and across runs of the same commit and target selection in continuous integration. Local runs remain visible but do not automatically flag test cases as flaky.

### Test retries {#test-retries}

Retries are optional and independent of quarantine. `--flaky_test_attempts=3` enables up to **three total Bazel attempts** per test, not three additional retries. Keep your existing retry configuration in `.bazelrc` or pass it explicitly:

```sh
tuist bazel test -- --flaky_test_attempts=3 //app:tests
```

Retries must actually run for Tuist to observe their results. A Bazel target's flaky summary alone does not identify which individual case was flaky.

### Cross-run detection {#cross-run-detection}

Comparing runs needs commit and branch metadata. Provide it when your build environment does not already publish it:

```sh
tuist bazel test -- \
  --build_metadata=CI=true \
  --build_metadata=GIT_COMMIT="$GIT_COMMIT" \
  --build_metadata=GIT_BRANCH="$GIT_BRANCH" //app:tests
```

Cases are identified by target label, class name (falling back to the suite name), and test name, as described in <.localized_link href="/guides/features/test-insights/bazel">Bazel test insights</.localized_link>.

## Managing flaky tests {#managing-flaky-tests}

### Automatic clearing {#automatic-clearing}

Detection and clearing run through an **automation alert** on the project. Every project gets a default "Flaky test detection" automation whose *trigger* marks a test as flaky and whose *recovery* clears the flag once the test has gone the configured recovery window (default **14 days**) without re-triggering. Edit it under **Settings → Automations** to change the recovery window, change the recovery actions (for example, also release the test from quarantine), or disable recovery entirely so tests stay marked flaky until you clear them by hand.

### Manual management {#manual-management}

You can also manually mark or unmark tests as flaky from the test case detail page. This is useful when:
- You want to acknowledge a known flaky test while working on a fix
- A test was incorrectly flagged due to infrastructure issues


## Quarantining flaky tests {#quarantining}

Quarantining isolates a flaky test so it doesn't block CI while you fix it. By default it's a **manual action**: you quarantine a test, release it, and switch modes from the test case detail page in the dashboard. You can also wire it into an **automation alert** under **Settings → Automations** so a test is automatically muted (or skipped) when it crosses a flakiness threshold, and released when it recovers. Every transition, manual or automated, is recorded on the test case's audit log.

By default, recovery waits until the trigger condition no longer holds, so a test stays quarantined until its old flaky runs leave the trigger window. To recover sooner, set **Recover when** to **No new flaky runs**: the test recovers once the recovery window (days or runs) passes without a new flaky run, every new flaky run restarts that wait, and after recovering only a new flaky run quarantines it again.


`tuist bazel test` fetches the quarantine policy before each invocation and applies it to the underlying `bazel test` run. Ordinary `bazel test` still reports test results, but does not apply quarantine:

```sh
tuist bazel test -- //app:tests
```

Arguments after `--` are forwarded to `bazel test`. Use `--path` to select the project and working directory, and `--bazel` to select a different executable such as `bazelisk`. A named Bazel configuration such as `--config=ci` is optional and works the same way.

### Mute {#mute}

Use **Mute** as the primary quarantine workflow. Muted cases continue to run, their failures remain visible, and healthy sibling cases still affect the result. Tuist turns Bazel's test-failure exit into success only when the reports attribute every failure in each failed target to a muted case. Build errors, timeouts, interrupted runs, missing or incomplete reports, and failures outside the muted set keep Bazel's failure status.

### Skip {#skip}

Skipping works per Bazel target: skipping one case excludes its whole target, including healthy cases, because Bazel can't filter individual cases for every test framework. Prefer **Mute** unless you intend to exclude the entire target. The command prints every excluded target label so build logs show what the quarantine policy left out.

If a skipped target no longer exists, Tuist names it in a warning and runs without that exclusion. Return stale cases to **Enabled** to remove their policies permanently. If no test targets remain, Bazel keeps its normal "no tests found" exit status.

### Why quarantined tests can appear as passing {#quarantined-passing}

- **Muted tests** still execute. A muted test that fails is recorded as **failed** on the test case run and flagged as flaky; the per-test status is not rewritten. What gets overridden is the **overall test run**: if every failing test case in the run is muted, the run as a whole is reported as passed, so muted failures don't break CI.
- **Skipped tests** don't run at all, so the dashboard keeps showing the status from the test's last actual execution. That snapshot can be weeks old.


### Running quarantined tests again {#running-quarantined-tests-again}

To investigate or validate a fix, bypass the quarantine policy:

```sh
tuist bazel test --no-quarantine -- --nocache_test_results //app:tests
```

Return the case to **Enabled** to include its target in subsequent normal runs. Skipped cases produce no new executions until explicitly included, so an automation based on successful new runs needs a validation run before it can recover them.

### Limitations {#limitations}

- An explicit `--build_event_json_file` or `--target_pattern_file` cannot be combined with active quarantine. Pass target patterns directly.
- If Tuist cannot fetch the quarantine policy, the command stops before running tests. It never applies a partial policy. Use `--no-quarantine` when you intentionally want to run without it.
- A failure that matches a mute recorded under an older suite-based identity stays blocking, and the command names the policy to reapply.

See Bazel's [test filters](https://bazel.build/docs/user-manual#test-filter), [command options](https://bazel.build/reference/command-line-reference), and [build-event descriptions](https://bazel.build/remote/bep-glossary) for the underlying behavior.

## Slack notifications {#slack-notifications}

Get notified when a test becomes flaky by setting up <.localized_link href="/guides/integrations/slack#flaky-test-alerts">flaky test alerts</.localized_link> in your Slack integration.


## Querying flaky and quarantined state {#querying}

### CLI {#querying-cli}

The `tuist test case` command tree exposes everything Tuist tracks about a test case:

```sh
tuist test case list --flaky                       # only flaky test cases
tuist test case list --quarantined                 # only muted or skipped test cases
tuist test case show <test_case_id>                # detail: flakiness rate, last status, run counts
tuist test case events <test_case_id>              # audit log: marked_flaky, muted, skipped, ...
tuist test case run list <test_case_id>            # run history with status and duration
tuist test case run show <test_case_run_id>        # single run, including failure breakdown
```

All of these accept `--json` for scripting.

### REST API {#querying-rest-api}

The same data is available over HTTP. See the [Test Cases endpoints](https://tuist.dev/api/docs#tag/test-cases) in the API reference for the full list of routes, filters, and response fields. To change a test case's state (mark or unmark flaky, mute, skip), send `PATCH /api/projects/{account_handle}/{project_handle}/tests/test-cases/{test_case_id}` with `is_flaky` or `state`. The same change is available as `tuist test case update` and as the `update_test_case` MCP tool.