What is AI debugging?

AI debugging is using a language model to find the cause of a defect instead of hunting for it by hand. It splits into two very different jobs. In your editor, the model reads the code and the stack trace you paste, and this already works well because everything it needs is in front of it. In production, the model has to reconstruct what happened from telemetry your system emitted at the time, so its ceiling is set by what you decided to log, not by how good the model is. Most tools marketed as AI debugging solve the first job. The second is where the time actually goes.

Can AI debug a production incident, or only code in my editor?

It can do both, but only if the evidence exists. An agent pointed at your logs can answer questions like which customers are affected or which deploy broke things, because those answers are latent in the log stream and a model can read unstructured text at a speed no human can match. What it cannot do is recover a signal you never emitted. If the failing code path logs nothing, the agent is guessing from the same silence you are. That is why instrumentation, not tool choice, is the lever for production debugging.

Do you still need metrics and traces, or are logs enough?

The traditional answer is that you need all three, logs, metrics, and traces, because each one answers a different kind of question. Sherwood Callaway, founder of the observability startup Sazabi, argues the opposite: that logs alone are enough now, because the thing that used to make logs weak, being unstructured natural language, is exactly what language models are good at reading. Treat it as a bet rather than a settled fact. For a small team the practical read is that three telemetry types you never finished instrumenting are worth less than one you actually did.

Is it safe to let an AI agent read your production logs?

Only if you treat the log stream as sensitive data, because that is what it is. Production logs routinely contain customer identifiers, tokens, internal hostnames, and personal data, and routing them to a model turns all of it into context that leaves your process and often your network. Two controls matter most: redact at the emitting service rather than at the vendor, and keep the debugging agent read-only so that anything an attacker can write into a log line cannot become an instruction the agent acts on. A log line is attacker-influenced input, and any agent reading it should be scoped accordingly.

AI Debugging: The Half AI Has Not Fixed

Search for AI debugging and page one hands you a list of tools that will read a stack trace for you. That list is real, and it is the least interesting part of the story. The interesting part is a gap that Sherwood Callaway describes from the inside, after a career spent on the maintenance side of software: the tools that write your code got dramatically better, and the tools that help you fix it barely moved.

He puts it plainly on YC Root Access: "we're building a futuristic AI product using futuristic AI coding tools and then I would go to debug it and it was the same painful manual experience that I've had for my entire career."

AI debugging means using a language model to find the cause of a defect instead of hunting for it by hand, and it splits into two jobs that are nothing alike: reading a stack trace in your editor, and reconstructing a production incident from whatever your system happened to write down at the time. The first is close to solved. The second is where the hours go.

Callaway's complaint is the reason his second company exists, and it is the reason this article is not a tool roundup. If you are a founder deciding where to spend an engineering week, the leverage in AI debugging is almost never the debugger. It is what your system tells you before anything breaks.

The half of engineering AI did not speed up

Callaway's framing of the job is worth sitting with: "a much bigger part of my job as a developer is maintaining software that's already been written." Maintenance, not creation, is where his hours actually went, and maintenance is the half that the last three years of tooling largely skipped. Coding agents attack creation. Creation was never where most of the hours went.

You can watch the same asymmetry from the other end. Garry Tan, president and CEO of Y Combinator, describes running parallel coding agents across separate branches and then hitting a wall: "I found myself sitting there doing QA. Probably the least fun part of software development." When the agent does planning, design, and implementation, the human work does not disappear. It concentrates into the parts that require judgment about whether the result is actually correct.

Tan is blunt about why this gets worse rather than better as you scale it up. Out of the box, he says, the model does not know your data well, so it guesses, and "guessing at that scale is how you get plausible looking code that silently breaks." Silently is the operative word. Code that fails loudly is cheap. Code that looks right, passes review, and quietly returns the wrong number for one customer segment is the expensive kind, and it is exactly the kind a fast agent produces more of.

Andrej Karpathy's verifiability framing explains the shape of this. Traditional computers automate what you can specify in code; this generation of models automates most reliably what you can verify. Writing code is easy to verify at the syntax level and hard to verify at the behavior level. So the volume goes up, the compile-time confidence goes up, and the runtime uncertainty goes up with it. More code, produced faster, by something that does not know your production data, is a debugging problem before it is anything else.

What AI debugging actually means in practice

Two jobs share the name, and conflating them is why founders buy the wrong thing.

The first is debugging in the editor. You have the code, the stack trace, and the failing test in one window, you hand it all to a model, and it reasons over a closed problem. This works. It is also close to solved, it ships inside the coding tools you already pay for, and it is what most of the AI bug finder pages on page one are selling.

The second is debugging in production, and it is a fundamentally different problem because the evidence is gone. The bug already happened. Whatever you can learn about it, you can only learn from what your system wrote down at the time. Callaway describes his product as the thing customers query with questions like "why is production down or what does this error mean or which customers are affected or which commit is responsible."

Read that list again as a founder rather than as an engineer. Every one of those questions is answerable only if the underlying data exists. No model recovers a signal you never emitted. This is the whole game: production AI debugging is not a tool you install after the incident, it is a decision you made weeks earlier about what your system says out loud.

Why logs became the useful telemetry

Here is the part that genuinely changed, and it is a nice example of a weakness turning into an advantage without anyone planning it.

For most of the last two decades, logs were the junk drawer of telemetry. Callaway says it directly: "logs were kind of the least valuable type of telemetry" precisely because they are unstructured, meaning a log line is a human sentence rather than a clean numeric field. You could search them, you could not really compute over them. Metrics and traces were the serious tools because a machine could aggregate them.

Language models invert that. Unstructured natural language is the one input format they handle better than any prior technology, so the property that made logs second-class is now the property that makes them the richest source you have. As Callaway puts it, "you can now have AI agents read all of your log lines and tell you what's going on."

Two consequences follow, and both are useful even if you never buy an observability product.

The first is that print-statement debugging aged well. Callaway makes the accessibility point directly: "every developer knows how to do a print statement and every developer knows how to read a log stream." His argument follows from that: "logs kind of represent the Occam's razor of observability." The simplest thing a team could possibly do turns out to be the thing that aged best. The unglamorous logging you already write is now machine-readable in a way it was not in 2019.

The second is that log quality started to matter more than log volume. A log line written for a human skimming a terminal and a log line written for an agent reconstructing a causal chain are not the same artifact. Include the request id, the customer id, the version of the code that emitted it, and the actual values that drove the branch. That is a code review standard you can adopt this week for free.

The instrumentation tax is the real decision

The orthodox answer to observability is the three pillars: logs, metrics, and traces. Callaway's objection is not philosophical, it is about who pays. As he puts it: "for every engineering team out there, they need to implement logs, metrics, and traces for all of their services." For a team of six that is three separate instrumentation projects competing with the roadmap.

His company's position is that you only need one of the three. That is a contrarian bet, published as a manifesto, and it should be read as a bet rather than a settled fact. Traces exist because reconstructing causality across service boundaries is genuinely hard, and metrics exist because they stay cheap at volumes where storing every line does not. Anyone telling you the tradeoff vanished is selling something.

But the founder version of the question is not which of the three is theoretically best. It is this: what is the smallest telemetry surface that answers the five questions you actually ask during an incident? For most early teams the honest answer is well-structured logs plus a handful of business metrics, and the two instrumentation projects you were feeling guilty about not finishing were never going to pay for themselves at your size.

Callaway's own history is the argument for doing any of this at all. By his account he was the third infrastructure engineer at Brex, roughly employee 70, and helped start the observability team once the company was running around 50 microservices in Kubernetes under different owners. At that point, by his telling, it had become hard to know what production was doing at all. He names the thing he had no word for at the time: "the answer is observability." The trigger was not a tooling trend. It was crossing the line where no single person held the system in their head.

Most startups cross that line earlier than they notice, and AI-assisted development moves the line closer, because you now ship code that nobody on the team wrote by hand. Our piece on spec-driven development covers the upstream half of that problem, deciding precisely what the agent should build. This is the downstream half: knowing what it actually did once it is running.

What AI debugging still cannot do

Callaway is clear-eyed about the limit, and it is the sentence in the interview worth putting on a wall. You can have integration tests, unit tests, QA processes, release processes, static analysis, all of it, and "nothing really prepares you for production." Once you are live, "you just can't predict what's going to happen. You can only prepare yourself to respond and that's what observability is."

That is the correct frame for every AI debugging tool on the market. None of them make your system predictable. They shorten the response, which is worth a great deal, but they do nothing about the arrival of surprises. Three practical implications:

The agent inherits your blind spots. It reads what you emitted. A code path that logs nothing is invisible to a model exactly as it is to you.
Speed of diagnosis is not the same as correctness of diagnosis. A model will produce a confident causal story from thin evidence, which is the same failure mode as confident code from thin context. Ask it for the log lines that support the claim, not just the claim.
You still define healthy. No tool knows that a small dip in one checkout step is a five alarm fire while a large spike in a background job is a Tuesday. That is product knowledge, and it has to be written down as thresholds and checks.

The last point is where debugging meets verification. Catching a defect before it ships is a different discipline from explaining one that already did, and both got cheaper at the same time. Our companion piece on AI testing for the AI coding era covers the pre-ship gate; this is the post-ship one. A lean team needs both, and neither substitutes for the other.

Callaway's stated ambition is that this eventually closes into a loop: "our vision is to create this world of self-healing software" where systems repair themselves without a human in the middle. Treat that as a direction of travel rather than a current capability. The near-term version available to you is narrower and still valuable: cut the time between a customer noticing something and an engineer understanding why.

Pointing an agent at your logs is a security decision

This part is our own read rather than the source's, and it is the question we would ask first in a security review, because the architecture that makes AI debugging work also quietly creates a new data path.

Your production logs are sensitive. Not in principle, in practice: they carry customer identifiers, email addresses, internal hostnames, session tokens that somebody logged during a debugging session two years ago and never removed. For years that was tolerable because logs sat in a system a few engineers read. Routing them to a model changes the exposure, because the whole stream becomes context, and in most setups that context leaves your process and often your network. Four things worth deciding before you wire it up:

Redact at the source, not at the vendor. Scrub personal data and secrets in the emitting service. A redaction rule configured inside a third-party tool only protects you after the data has already crossed the boundary.
Keep the debugging agent read-only. A log line frequently contains attacker-influenced text, because user input gets echoed into logs constantly. An agent that reads that text and can also execute actions is an injection path with extra steps. Let it read and recommend; keep a human on anything that changes production.
Scope its credentials like any other identity. The agent should hold read access to the log store and nothing else. It does not need database write access to explain an outage.
Answer the vendor question before a customer asks it. If your logs leave your infrastructure, that belongs in your subprocessor list and your security questionnaire responses. Enterprise buyers will find it, and finding it during procurement is worse than disclosing it up front.

None of this is an argument against AI debugging. It is an argument for treating your log pipeline as production infrastructure with a data classification, which most teams have never done because logs never mattered enough to warrant it. They do now.

What to do this week

Write down the five questions you ask during an incident. Literally list them: what broke, who is affected, when did it start, which deploy caused it, is it still happening. This is your requirements document for telemetry, and it takes twenty minutes.
Check whether your current logs can answer them. Open your last real incident and try. The gaps you find are your instrumentation backlog, prioritized by evidence rather than by best practice.
Set a logging standard for agent-written code. Request id, customer id, code version, and the values that drove the branch. Put it in the file your coding agent reads before it writes anything.
Stop instrumenting what you do not query. If you have half-finished metrics or traces nobody has looked at in a quarter, cut them and put the effort into logs you will actually read.
Classify your log stream. Find out what personal data and secrets are in it today, redact at the emitting service, and decide explicitly whether that stream is allowed to leave your network.
Make the diagnosis show its work. Whatever tool or agent you use, require it to cite the log lines behind its conclusion before anyone acts on it.

Debugging is where the cost of moving fast with agents actually lands, which makes it an operating question rather than a tooling one: what you instrument, what you verify, and who is accountable when the plausible-looking code breaks quietly. That is the operating system we teach in AI Operating System for Startups. For the wider picture of where this fits alongside product, engineering, and go-to-market, start with our pillar on AI for startups, and for the shift in how the code gets written in the first place, see from vibe coding to agentic engineering.

Sources

The End of Manual Debugging on YC Root Access, the interview with Sherwood Callaway this article distills.
Profile: Sherwood Callaway on LinkedIn, and Sazabi, the AI-native observability company he founded, on Y Combinator.
Sazabi's 8 million dollar seed round, announced June 2026 and led by J2 Ventures, Village Global, and Y Combinator: the company's announcement and Business Insider.
His first company, Opkit (Y Combinator Summer 2021), whose team was acquired by 11x in 2024, per co-founder Justin Ko.
How to Make Claude Code Your AI Engineering Team (Y Combinator), for Garry Tan on QA becoming the bottleneck and on plausible code that silently breaks.
Andrej Karpathy: From Vibe Coding to Agentic Engineering (Sequoia Capital), for the verifiability framing.