USE CASE

Flaky test triage that ends in a decision, not another rerun

A flaky test passes and fails without a code change, which makes it a triage problem before it is a coding problem. Hyperlane reruns the failing test in an isolated sandbox, collects the evidence, proposes a fix or a quarantine, and verifies the result before opening a pull request.

Rerunning the build is not triage. The agent reproduces the failure, separates a flaky test from a real regression, and returns either a patch or a documented quarantine.

Start free See pricing

Flakes are a signal problem

A flaky test passes and fails without a code change, so the first job is not fixing it; it is deciding what the failure means. Teams usually rerun the build, shrug, and move on. That works until the flake hides a real regression or trains reviewers to ignore red builds.

Reproduce, then decide

Hyperlane starts by rerunning the failing test in an isolated sandbox, using the same commit and the same commands your CI used. Repeated runs, captured logs, and the surrounding suite give the agent something to reason about: ordering, shared state, timing, or an actual bug that only appears under load.

Fix, quarantine, or delete

Not every flake deserves a code fix. A race in production code needs a patch; a test that depends on wall-clock time needs a test fix; a low-value test that keeps failing may deserve quarantine or deletion. The agent can propose any of the three, but the policy is yours to set.

Evidence in the pull request

Every change arrives as a pull request on its own branch. The verify-before-done gate reruns the relevant suite before the run can finish, and the trace records what was executed, how many times, and what changed between passes. Reviewers get evidence instead of a claim that it looks flaky.

Point it at the failures you already have

Start a run from the CI job that just failed, passing the test name and the build log through the API or CLI. Batch a week of flakes into separate runs, each with its own sandbox and pull request. Long investigations checkpoint and resume, so a slow suite does not force you to start over.

What Hyperlane ships

  • Autonomous agent loop that plans, edits, runs, and verifies work end-to-end
  • Every run executes inside an isolated, resource-capped sandbox
  • Live execution traces - diffs, terminal output, and reasoning as it happens
  • GitHub-native workflow: pushes branches and opens pull requests itself
  • Web app, REST API, TypeScript SDK, CLI, and MCP server interfaces
  • Dollar-for-dollar credits - pay $20, get $20 of agent work

Frequently asked

Can an agent prove a test is flaky?

Not in a mathematical sense. It can reproduce the failure, rerun the test under the same conditions, and report what differed between passes. That is stronger evidence than a single red build, but a human still sets the threshold for calling something flaky.

Does it fix the test or the code?

It decides from the evidence. A race in production code gets a code fix; a test that depends on wall-clock time or shared state gets a test fix. Quarantine is a third option when the right fix is not obvious yet.

Will it change our test suite without review?

Every change arrives as a pull request on a branch. Risky actions can require approval before they run, and each step is recorded in the run’s audit trail, so nothing lands without a reviewer.

How long does triage take?

It depends on how long the suite takes and how often the failure reproduces. Runs are bounded by your plan’s agent computer time, and long investigations can checkpoint and resume rather than restart.

Can we run it only on failures from CI?

Yes. Trigger a run from the CI job through the API or CLI, passing the failing test and the build log. The agent does the reruns and returns either a pull request or a triage report you can act on.

Last verified 11 Nov 2026.

Sources: Hyperlane pricing Hyperlane docs Hyperlane enterprise

Keep exploring

Delegate the ticket. Review the PR.

Start with the Free plan — 10 minutes of agent computer time every month, no card required.

Try Hyperlane free

Hyperlane AI is not affiliated with the Hyperlane blockchain protocol (hyperlane.xyz) or the Hyperlane code editor (hyperlaneide.com).