Running a CI agent on open-weight models

const metadata = ; Every CI/CD agent on the market wants access to your build logs, your source code, and your failure context. The question nobody asks loudly enough: where does that data go? With proprietary models, the answer is usually "our cloud." Your failed test output, your stack traces, your internal package names, processed on infrastructure you don't control, under terms that can change. Open-weight models change the calculus entirely. What "open-weight" actually means An open-weight model is a foundation model with publicly available weights, you download it, run it on hardware you control, and inference happens entirely on your side. No API calls to a third party. No training data opt-out clauses to parse. Examples teams are already running in production workflows: - Meta Llama (3.x and 4.x families) - Alibaba Qwen (strong coding and reasoning variants) - Mistral (efficient models for constrained environments) - DeepSeek (competitive coding performance at lower compute cost) These aren't research toys. [Meta reported](https://ai.meta.com/blog/) that Llama models have been downloaded hundreds of millions of times, with significant enterprise adoption for internal tooling, including code analysis and developer assistants. For CI/CD agents specifically, the requirements are narrower than general chat: read logs, reason about errors, generate patch suggestions, write structured reports. Open-weight models at 7B–70B parameters are increasingly capable at exactly this kind of task. Why CI/CD is the wrong place for closed models CI/CD failures are some of the most sensitive data in your organization: - Stack traces that reveal internal architecture - Test names that map to unreleased features - Log lines that include credentials if someone forgot to redact - Diffs that show proprietary algorithms or business logic Sending that to a proprietary API means trusting a vendor's security posture, data retention policy, and, critically, whether your code ever touches their training pipeline. Most vendors now offer enterprise agreements with data isolation guarantees. But guarantees aren't control. When compliance asks "can you prove this data never left our VPC?", self-hosted open-weight models give you a one-word answer: yes. The cost argument Proprietary API pricing for agent workloads adds up fast. A CI/CD agent that investigates every failed build might process thousands of log tokens per day across dozens of repositories. Open-weight models shift the cost model: | Factor | Proprietary API | Self-hosted open-weight | | | | | | Per-token cost | Variable, often $1–15 / 1M tokens | Fixed hardware cost | | Volume scaling | Linear cost increase | Marginal cost near zero | | Air-gapped deploys | Not possible | Native | | Model choice | Vendor's model only | Swap models freely | | Fine-tuning | Limited or unavailable | Full control | For a 50-person engineering team running CI across 20 repos, self-hosting a capable 7B–14B model on a single GPU instance can be dramatically cheaper than API-based agents at scale, especially when investigation runs on every failure, not just escalations. "But are open models good enough?" This was the right question two years ago. It's less relevant now. CI/CD investigation is a constrained task: structured input (logs, diffs, config), structured output (diagnosis, suggested fix, PR). You don't need a model that writes poetry, you need one that reads a Java stack trace and identifies the null pointer. Benchmarks like [SWE-bench](https://www.swebench.com/) show open-weight coding models closing the gap with proprietary systems on real-world software engineering tasks. For the narrower domain of "read this build failure and explain it," smaller open models often perform comparably to much larger closed ones, because the task is well-scoped. The practical advice: start with a mid-size open model (14B–32B parameters), measure investigation quality on your actual failures, and upgrade only if you hit a ceiling. Most teams don't hit the ceiling. How OpenSRE uses open-weight models OpenSRE is built on the premise that CI/CD agents should run where your code runs, on your infrastructure, with models you choose. That means: - No code or logs sent to external APIs unless you explicitly configure it - Model-agnostic architecture: run Llama, Qwen, Mistral, or whatever fits your hardware - Air-gapped compatible for teams in finance, healthcare, and defense - Open source so you can audit exactly what the agent does with your data The agent reads failed builds, traces causes through logs and diffs, and opens fix PRs. The model powers the reasoning. You power the infrastructure. The takeaway The CI/CD agent category is moving fast, and the model layer is where the most important architectural decision lives. Proprietary models offer convenience. Open-weight models offer sovereignty: your data stays yours, your costs are predictable, and your agent works the same way in a regulated data center as it does on a laptop. For teams betting on CI/CD reliability as a competitive advantage, that's not a technical preference. It's a requirement.