Flaky test detection at scale

const metadata = ; Flaky tests are a reliability problem disguised as a testing problem. When the same commit passes on re-run, teams learn to ignore red builds, and real failures hide in the noise. Why flake compounds Every ignored failure adds latency to the org: reviewers wait, merges stall, and on-call engineers context-switch into investigations that end with "just re-ran it." At scale, flake rate becomes a leading indicator of CI trust erosion. What to measure Track these per test or per suite: | Signal | Why it matters | | | | | Pass/fail variance on unchanged commits | Direct flake detection | | Re-run rate after failure | Proxy for developer distrust | | Time-to-quarantine | How fast bad tests leave the critical path | | Flake recidivism | Tests that return after quarantine | A practical workflow 1. Detect: flag tests that flip outcome without code changes 2. Quarantine: move offenders off the merge-blocking path 3. Assign: route to owners with failure context, not raw logs 4. Verify: require N consecutive greens before un-quarantining Agentic investigation helps at step 3: correlating flake with infrastructure changes, dependency updates, or timing assumptions in the test itself. Takeaway Reliable CI isn't zero flake, it's visible flake with a closed loop. Teams that measure and quarantine systematically recover merge confidence faster than teams that rely on heroic manual triage.