Summary
Analysis templates currently support passing the analysis after N consecutive successes and/or failing after N total failures.
We'd love the ability to fail after N consecutive failures as well.
Use Cases
This would be used to minimize "false positive" rollbacks, in cases where the metric that underlies the analysis may be slightly flaky. Requiring consecutive failures would give more confidence that something was actually wrong, rather than the system responding to a blip.
Message from the maintainers:
Need this enhancement? Give it a 👍. We prioritize the issues with the most 👍.
Summary
Analysis templates currently support passing the analysis after N consecutive successes and/or failing after N total failures.
We'd love the ability to fail after N consecutive failures as well.
Use Cases
This would be used to minimize "false positive" rollbacks, in cases where the metric that underlies the analysis may be slightly flaky. Requiring consecutive failures would give more confidence that something was actually wrong, rather than the system responding to a blip.
Message from the maintainers:
Need this enhancement? Give it a 👍. We prioritize the issues with the most 👍.