🔗 We upgraded to a Frontier model and our costs went down

Upgrading to Opus 4.6 not only improved performance but also led to lower operational costs compared to the previous Sonnet 4.0 setup. Instead of letting expensive models handle all failure investigations, the team implemented a triager pattern using the Haiku agent. The Haiku agent efficiently manages 80% of failure cases, allowing the more costly Opus model to engage only when absolutely necessary. By embracing a structured approach to log processing and leveraging smart querying techniques, the costs associated with investigating CI logs were significantly reduced. The analysis of approximately 4,000 failures highlighted the capability of the new architecture to differentiate between known issues and novel problems, streamlining operations and optimizing costs.