CI/CD Tool Selection for AI-Augmented Pipelines

There's a version of this conversation where we spend the first three paragraphs debating whether AI agents "really" belong in a CI/CD pipeline. That's not this conversation. If you're here, you've already crossed that bridge. Your team is writing code with AI assistance, PRs are opening faster than reviewers can close them, and someone has said the words "we need to rethink our pipeline" in a meeting.

So let's actually do that now.

Here's the uncomfortable reality check before we get into the framework. Roughly 90% of developers use AI in some form by early 2026. But only around 22% have deployed AI coding agents, and only about 13% have AI operating across the full software delivery lifecycle. The adoption gap is largest exactly where you'd expect it to hurt most: the CI/CD layer.

What that means in practice? Agents generate code and open PRs faster than pipelines and reviewers can process them. Research across more than 10,000 developers on high-AI-adoption teams found PR review time increasing 91% and average PR size growing 154%. If you've ever taken an intro CS course, you might recognize this as a classic Amdahl's Law problem. The system moves only as fast as its slowest link — right now, CI/CD is often that link, like a single-lane bridge carrying a six-lane highway's worth of traffic. So the question isn't whether to care about this. The question is what to actually do about it, starting with how you pick (or re-pick) your tooling.

"Agentic" Is Not a Buzzword Here. It's a Technical Distinction.

Let's separate the 2023 model from what's actually happening now, because they're genuinely different in ways that matter for infrastructure decisions.

In 2023, the typical pattern looked like this: a developer runs an AI assistant locally, reviews the output, edits it, commits it, and pushes. The CI/CD system is completely downstream of all of that. It's passive. It doesn't know or care that an AI was involved. It just sees a commit.

In 2026, GitHub Copilot's Coding Agent (launched May 2025) runs inside GitHub Actions, triggered by an issue or a PR comment. The CI system is now the agent's runtime environment. Not its validator. Its home.

That's a different relationship entirely. And it creates three concrete shifts in what a pipeline has to handle.

Volume. AI tools write a meaningful and growing share of all code in production codebases. This isn't an incremental bump in throughput. It's a qualitative change in how pipelines need to be sized and designed.

Provenance. Code arriving in the pipeline increasingly has mixed or fully autonomous authorship. The pipeline needs to carry that information, not just the diff. Right now, most pipelines have no native way to distinguish "a human wrote this" from "an agent wrote this." That's a design gap with real consequences.

Agency. Some platforms now treat the pipeline itself as a decision-making layer, not just an execution layer. Agents perceive signals, choose actions, and execute within the pipeline rather than before it. Dagger's containerized architecture is the clearest example of this model built into a platform's bones.

The Goldman Sachs and Devin pilot is a useful concrete reference point here. Picture a factory floor where human workers and robots receive tasks off the same conveyor belt, each returning finished goods — in this case, PRs with tests and documentation attached — through the same quality checkpoint. The pipeline has to authenticate and route both kinds of contributors. That's not science fiction. That's a production use case that exists right now.

One more thing worth anchoring on: "agentic CI/CD" is not a product you can buy yet. It's a set of capabilities teams are assembling from existing and emerging tools. That framing matters for how you evaluate platforms.

The Risk Profile Has Changed. The Pipelines Haven't Kept Up.

Before we get to the evaluation framework, it's worth sitting with the "why this matters" question a bit longer, because it's easy to treat agent-generated code as just "more code."

It isn't, and the data backs that up.

Analysis of PRs containing AI-generated code found roughly 1.7 times more issues than PRs of human-written code alone. Enterprises are attributing more than 10,000 new security findings per month to AI-generated code (Apiiro, 2025). Code churn, meaning rewrites of recently written code, rose from a baseline of around 3.3% in 2021 to between 5.7% and 7.1% across 2024-2025 based on GitClear data. That's a signal that AI-generated code is less stable over time, not just different in origin.

What this implies for your pipeline: the security scanning, SAST tools, and quality gates you designed for human-authored code at human velocity are now undersized. Not because they're bad tools. Because the thing they were sized for has changed. Your pipeline is a rain jacket in a hurricane — same garment, wrong storm.

Here's the subtler problem though. A study by METR (July 2025, 16 experienced developers, 246 randomly assigned issues across large open-source repositories) found that developers believed AI was speeding them up by around 20% when it was actually slowing them down by about 19%. Developer self-reporting is not a reliable signal. Which means the pipeline's telemetry has to carry the ground truth. If your pipeline can't tell you how much of your churn is agent-authored versus human-authored, you're flying blind.

And for teams still relying primarily on DORA metrics: deployment frequency and lead time don't tell you anything about defect origin or agent-authored churn. Those metrics were built for a world where all your contributors were humans. That world is ending.

The 2023 Framework Was Fine. For 2023.

Most teams evaluating CI/CD tools in 2023 used some version of the same checklist.

• Build speed and parallelism
• Plugin and integration ecosystem
• YAML or config ergonomics
• Pricing per seat or per minute
• Native support for their cloud provider

None of these criteria are wrong. They're just incomplete in ways that now matter a lot.

Not one of those questions asks: can this platform authenticate a non-human actor? Can it apply different gate policies to agent-authored PRs? Can it give an agent the observability it needs to self-diagnose a failure?

Speed is still important. But it's no longer the differentiating factor. The bottleneck has moved from build time to review and gate throughput, which is exactly what a 91% increase in PR review time is telling you.

Plugin ecosystems were designed for human-initiated integrations. They don't tell you whether a platform can serve as a reliable runtime for an autonomous agent running for hours or days.

And pricing models built around per-seat or per-minute compute don't account for agentic token spend, which can run between $200 and $2,000 or more per engineer per month. That completely reframes your total cost of ownership math.

So here's the updated framework. Five new criteria. Not instead of the old ones. In addition to them, but weighted more heavily.

Criterion 1: Can the Platform Tell the Difference Between a Human and an Agent, and Actually Care?

This is the foundational question. Everything else depends on it.

The core capability to look for: can the platform distinguish between a human committer and an agent, and apply differentiated permissions and policies to each?

Specifically, you want to see:

• Machine identities with meaningful scope. Service accounts and OIDC tokens with time-limited, scoped permissions. Not shared API keys. Not a service account with admin rights because that was easiest to set up.
• Identity metadata attached to runs. The agent's identity should be traceable through a commit, a PR, and a pipeline run so downstream gates can act on it.
• Audit logs that record actor type. Not just what happened, but what kind of actor initiated each action. This is both the telemetry foundation agents need to "see" the pipeline and what security teams need to investigate when something goes wrong.

GitHub Actions' OIDC federation model, with per-job tokens, is a reasonable reference point for what granular machine identity looks like in practice. It's not perfect, but it shows what the capability actually looks like when it's been thought through.

Why is this harder than it sounds? Most existing CI/CD systems model identity around human users with scoped tokens. Agents operating autonomously for hours may need to acquire, refresh, and scope credentials dynamically across multiple systems. That's a different pattern than "developer logs in and runs a job."

The failure mode to watch for: platforms that model all CI actors as equivalent. Human or agent, same permissions, same surface. When a misbehaving agent runs in that environment, your only options are "let it run" or "take down the whole pipeline." Neither is good.

Criterion 2: Can the Pipeline Apply Stricter Rules to Agent-Authored Code Without Slowing Everything Else Down?

Given that agent-authored code carries a measurably higher defect and security risk, teams need the pipeline to apply stricter or additional gates to those PRs without creating friction for human-authored work.

What differentiated gate logic looks like in practice:

• SAST and secrets scanning triggered automatically for all agent-authored commits, regardless of file path heuristics
• Required human approval at merge for agent-authored changes above a configurable "blast radius" threshold (touching a certain number of files, specific directories, or infrastructure config)
• Automatic assignment of a senior reviewer for agent PRs touching security-sensitive paths

The evaluation question here is specific: can gate conditions be parameterized by actor type, not just by branch name or file path?

Most platforms today can't do this natively. They require workarounds. Label-based routing. External webhooks. Custom plugins. That's not a dealbreaker if the workaround is simple, but it's a design gap worth knowing about going in.

The design principle underneath this: agents should not push to production directly. The pipeline is where that constraint lives and gets enforced. Not the agent itself. Which means the pipeline must have the gate logic to enforce it reliably, not just by convention.

A useful vendor test question: "Can we define a policy where agent-authored PRs to the release branch always require two human approvals?" If the answer requires a custom plugin, that tells you something real about where this capability sits in their roadmap.

Criterion 3: Can the Platform Run an Agent for Hours Without Falling Apart?

Traditional CI/CD is built around ephemeral jobs that complete in minutes. A refactor. A migration. A self-healing loop responding to monitoring alerts. Those tasks might run for hours. Or even days.

That's a fundamentally different runtime requirement.

What to evaluate:

• Job timeout limits. Can the platform accommodate long-running agentic tasks, or do you hit a wall at 60 minutes and start building workarounds?
• Stateful context persistence. Agents need to carry state across a run: prior decisions, partial outputs, tool call history. They shouldn't have to re-prompt from scratch because a step failed and restarted.
• Retry and resumption semantics. If a long-running agent job fails mid-execution, does the platform resume from a checkpoint, or does it restart from zero? Restarting from zero on a multi-hour task isn't just annoying. It's expensive.
• Cost controls. Token spend for agentic tasks is variable and potentially unbounded. The platform, or something adjacent to it, needs to support per-run or per-agent spend limits. Otherwise you're one runaway agent away from a very uncomfortable invoice conversation.

Dagger's containerized agent architecture is a useful reference point here. Agents operate within containerized CI environments with perception, decision, and execution separated into distinct layers. That separation is what makes long-running, stateful agentic tasks tractable.

Ask the vendor for their maximum job duration. Ask how state persists between steps. Ask whether there are native mechanisms for cost-capping agentic runs. Most traditional platforms answer these questions poorly, or not at all. That's useful information.

Criterion 4: Can the Platform Actually Show an Agent What's Happening?

This one comes from watching real teams try to build self-healing pipelines. The biggest barrier is almost never the model's capability. It's building the observability stack the agent needs to see what's actually happening.

A human can squint at a log file and figure out what went wrong. An agent trying to do the same with free-text blobs is like asking someone to navigate a city using only a photograph of a map — the information is technically there, but the format makes it nearly unusable. It needs structure.

What useful telemetry for agents actually looks like:

• Structured, machine-readable failure logs. Not wall-of-text outputs designed for a human reading a web UI. JSON. Parseable. Queryable.
• Metrics and traces in formats agents can query. Prometheus-compatible endpoints. OpenTelemetry traces accessible via API. Not just visible in a dashboard.
• Pipeline event streams. Stage transitions, gate outcomes, approval events. So an agent monitoring a run can react to state changes without constant polling.

LogSage is a relevant reference implementation here: it analyzes CI/CD logs to detect failures and support remediation actions in industrial settings. That's only possible because the pipeline exposes structured log data at sufficient granularity.

The MTTR payoff is real. One practitioner's implementation where an agent analyzes logs, identifies root cause of a production error, generates a PR for the fix, and tags a senior engineer for review reduced MTTR by nearly 60%. The enabling condition wasn't the model. It was the pipeline's observability surface.

The practical evaluation question: can an external agent subscribe to pipeline events via webhook or API without polling? Can failure logs be exported in structured JSON? If the platform's observability is designed only for human consumption in a web UI, it isn't ready for agentic use at all.

Criterion 5: When an Agent Breaks Something, Can the Platform Help You Clean It Up?

Autonomous agents will occasionally produce bad outcomes that reach the pipeline. This isn't a hypothetical. It's a guarantee, given enough volume. So the question isn't whether to trust agents completely or not at all. It's whether the platform gives teams enough control to contain and recover from mistakes when they happen.

Recovery capabilities to evaluate:

• Rollback primitives. Can the platform trigger an automatic rollback of a deployment if a post-deploy health check fails, without human initiation?
• Blast-radius limiting. Can the platform enforce that an agent-initiated deployment touches only a canary slice before full rollout, regardless of what the agent's own configuration requests? The "regardless" is important. The platform should enforce this at the infrastructure level, not just trust the agent's settings.
• Quarantine mechanisms. When an agent-authored change causes a test failure, can the platform automatically quarantine that branch and prevent it from blocking other work in the pipeline?
• Audit trail completeness. For any bad outcome, can the team reconstruct exactly what the agent did, in what order, with what inputs? Not just what the final diff contained?

One useful mental model here is what you might call the "autonomy slider." The best platforms let teams configure how much independence an agent is granted, from "agent suggests, human executes" all the way to "agent executes and self-corrects," and enforce that setting at the infrastructure level. Not by convention. Not by hoping the agent behaves.

Gartner projects that 70% of enterprises will deploy agentic AI in IT infrastructure operations by 2029, up from under 5% in 2025. That means recovery and containment tooling is rapidly becoming a standard requirement, not an edge case. If a platform isn't building toward this, it's worth asking why.

How to Actually Use This Framework When You're Sitting Across From a Vendor

The evaluation unit has changed. Don't evaluate a CI/CD platform in isolation anymore. Evaluate the platform, plus the agent runtime, plus the observability stack, as a system.

Here's what that looks like practically:

Map your current agent touch points first. Where do agents already interact with your pipeline? PR creation? Test generation? Deployment triggers? That tells you which criteria are urgent versus which are future-proofing.

Weight criteria by your current bottleneck. If your biggest pain is PR review volume, Criteria 1 and 2 matter most right now. If you're trying to build self-healing infrastructure, Criteria 4 and 5 dominate.

Run the vendor test questions. Not the demo. The specific questions. "What's your maximum job duration?" "Can gate conditions be parameterized by actor type?" "How do failure logs get exported?" Vendors who've thought seriously about agentic use cases will have answers. Vendors who haven't will get visibly uncomfortable, which is also useful information.

Check the pricing model for agentic workloads explicitly. Per-seat and per-minute pricing was designed for human developers running predictable jobs. Agentic workloads don't behave predictably. Get the cost model in writing before you're six months in and surprised.

Accept that you will need to stitch things together. There is no single platform that nails all five criteria today. The question is: what does this platform do natively, and what will you have to build or integrate around it? Knowing that upfront is better than discovering it in production.

The teams that are furthest ahead on this aren't the ones who found a perfect platform. They're the ones who mapped the capability gaps clearly, made deliberate decisions about which gaps to accept, and built toward the rest systematically. That's the move. Not waiting for the perfect tool. Knowing exactly what you're getting and what you're not.