System Prompt: What It Is and When to Cut It

A system prompt is the standing instruction a model reads before it reads anything the user typed. It sets the role, the rules, the output format, and the defaults for every single call, and the user never sees it.

The harder question is not what to put in one. It is what to take out. Most system prompts running in production today were written to correct a model that has since been replaced, and nothing in a normal release process ever goes back to check.

In July 2026, Y Combinator published a talk with Boris Cherny, the creator of Claude Code at Anthropic, in which he describes what his team does when a new model ships. They do not just tune the system prompt. They delete it, then bring it back one line at a time to find out what each line is actually worth. On the most recent model, in his words, "we deleted 80% of the system prompt."

That is the part worth stealing. What follows is what a system prompt is, how it differs from the other things people call the prompt, and the maintenance discipline that almost nobody runs on theirs.

What is a system prompt?

A system prompt is the instruction block that gets prepended to every conversation with a model, independent of what the user asks. In an API call it is a separate field from the user's message. In a product, it is the thing that makes the same underlying model behave like a support agent in one app and a code reviewer in another.

A useful system prompt usually carries four things:

• Role and scope. What this assistant is, and what it will not do.
• Operating rules. Output format, tone, escalation paths, refusals.
• Tool guidance. Which tools exist and when to reach for each.
• Domain facts that never change. The handful of stable things the model cannot know, like what your product is called.

The important structural fact is the one Cherny is most direct about: this text is not free, and it is not written once. "The model is going to read this instruction every single time you use it," he says, which is why "you really want to make sure that the model needs this instruction."

System prompt vs user prompt vs context

Three different things get called "the prompt," and confusing them is how teams end up putting the wrong content in the wrong place.

The system prompt is standing instruction. It applies to every call, it rarely changes, and it is written by you rather than by the user.

The user prompt is the request for one turn. It changes constantly and you mostly do not control it.

Retrieved context is data fetched for one specific question: documents, records, search results, a file the agent just read. It changes with the question.

The test for where something belongs is how often it is true. If it is true on every call, it is a system prompt. If it is true for this question only, it is context, and it should be retrieved rather than pasted permanently into the instruction block. Teams get this backwards constantly, hard-coding a customer-specific fact into the system prompt because it was faster than building retrieval, and then paying for that fact on every request forever.

Two related failure modes are worth naming here because they are frequently blamed on the system prompt. Long sessions degrade because the context window fills up, which is a different problem with a different fix, covered in context rot. What you assemble per run, and how you decide what the model gets to see, is context engineering for AI agents. This article is about the one part that is present on every call regardless.

Why Anthropic deleted 80 percent of Claude Code's system prompt

Cherny's explanation is not about elegance. It is about the fact that a system prompt is written against a specific model, and the model keeps getting replaced underneath it.

"Every time that a new model comes out, we delete a bunch of the system prompt, change a bunch of the system prompt," he says. "We change the set of tools all the time. We change the prompts for the tools all the time. And the reason is every model is very different. So, something that you did for one model maybe 3 months ago, it just might not translate at all to the next model."

Then the diagnosis that explains the 80 percent figure: "a lot of the stuff in the system prompt was correcting for these behaviors that the model should have known, but uh it didn't."

Read that carefully, because it describes almost every system prompt in production right now. The lines accumulated as patches. Something went wrong, someone added a sentence, the sentence fixed it, and the sentence stayed. Nothing in a normal release process ever goes back and asks whether the failure it patched is still possible. So the prompt grows monotonically, and its correlation with the current model's actual weaknesses decays with every release.

There is a stronger version of this that Cherny mentions almost in passing. Claude Code has an undocumented mode that strips every system prompt, including the ones attached to tools. "We actually use this as a sort of ablation to figure out is the prompt useful?" he says. "And what's interesting is that the model is actually a little bit more intelligent without these prompts."

Not equal. Better, by a little, and consistently enough that the team keeps finding it. The most plausible reading is that instructions written for an older model do not go inert when the model improves.

What does a long system prompt actually cost?

Three costs, all of which compound with prompt length.

You pay tokens on every call. A 2,000-token system prompt on a million calls is 2 billion tokens of instruction, most of it identical, much of it obsolete. That is the visible cost and usually the smallest one.

You pay in attention. Every instruction competes with every other instruction, and with the user's actual request. Adding a line to fix one behavior can degrade a behavior you were not thinking about.

You pay in false confidence. This is the expensive one. A line in the system prompt looks like a control. It reads like policy. It is neither, and a team that believes their prompt is enforcing something will not build the thing that actually enforces it.

How to run a system prompt ablation

Ablation is a research term, and Cherny defines it plainly: "ablation essentially is a eval where you delete things to figure out the impact." Applied to a system prompt, the procedure is:

"You delete the entire system prompt and then you bring it back line by line to figure out what is the impact of each individual line."

The full loop, as he describes it:

1. Delete. Not trim. The whole thing, in a branch, against the new model. "The first step is you delete. The next step is you use it."
2. Use the product. Run real tasks against your real domain. This is not a benchmark step, it is an observation step.
3. Watch for repeated failures. Not one bad output. A pattern. "And only when you see it repeatedly stumble on the same thing, that's when you add it back."
4. Add back one line, and only that line. Then keep using it.
5. Stop when the failures stop. Whatever you did not need to add back was dead weight, and now you know it empirically rather than by intuition.

The discipline is in step 3, and Cherny is direct about why: "you don't want to guess what's the instruction that the model needs because you might not predict it correctly." Guessing is exactly what writing a system prompt from scratch is. Every line you add pre-emptively is a bet about a failure you have not observed, priced at one instruction read on every call for the life of the product.

One honest caveat he adds, and it matters if you ship this to customers: the minimal prompt is not automatically the right product. "When you use Claude Code as a product, you do actually want some of these prompts because it helps you use the product and it it helps the the product behave and the model behave in the way that you would want when when you're using it as a person." Raw capability and a coherent product are different targets. Ablation tells you which lines are load-bearing for capability. You still decide, separately, which lines are load-bearing for the experience.

What belongs in a system prompt after the cut

The lines that earn their way back are a specific shape: task, guardrails, and exit criteria, never step-by-step procedure.

Cherny's advice on how to instruct a modern model is to go up a level rather than down into detail. "You want to describe the task, you want to describe the guardrails, you want to describe like the exit criteria, and then just go with the model cook."

The failure mode on the other side is over-specification, and he names who does it: "when I look at engineers that have been, you know, coding for a long for a long time," he says, "this is a really really common failure mode is trying to over specify." His broader read on the field is blunter. "I think people tend to overthink it a little bit. I think people tend to overengineer."

There is one thing he singles out as more important than any of it: "the verification I think is probably the single most important thing that people do not get right." A model that can check its own work will grind through a task without supervision. A model that cannot will confidently produce a wrong answer and stop. If you have room for one thing in a shortened system prompt, make it the definition of done and the way to check it.

This is the same shift Harrison Chase of LangChain describes at Sequoia Capital when he says "the main job of a harness is to bring context to the model at the right point in time." The prompt is not where you encode your process. It is where you tell the model what finished looks like, and the harness is what gets it the material to get there.

For anyone tempted to build a lot of scaffolding around this, Logan Kilpatrick of Google DeepMind describes the pattern at Sequoia Capital: "the scaffolding is like often times a couple of steps ahead of like where the what is like baked directly into the model," and then "the model eats that scaffolding and it becomes part of like the native model system." Some scaffolding is a loan against the next release. Kilpatrick still sees value in external scaffolding for things like search and code execution, but he calls the agent harness "the quintessential example of this right now" and expects it to be absorbed. This is the same logic as building for the next AI model, applied to text instead of code.

Your system prompt is not a security control

A system prompt cannot enforce anything. It is guidance to a model, which makes the deletion discipline above dangerous for one specific group of teams, and they usually do not know they are in it.

Start with the standards, because they are unambiguous. OWASP's guidance on system prompt leakage states that "the system prompt should not be considered a secret, nor should it be used as a security control," and that "critical controls such as privilege separation, authorization bounds checks, and similar must not be delegated to the LLM, either through the system prompt or otherwise." The AWS Security Blog puts it flatly: "Don't use instructions in system prompts as security control."

The reason is structural. An instruction in a system prompt and an attacker's instruction embedded in a document the agent reads arrive through the same channel and compete on the same terms. You are asking a model to adjudicate between them, which is a request, not a boundary. Simon Willison's lethal trifecta names the conditions that make this expensive: access to private data, exposure to untrusted content, and a way to communicate externally. Any one alone is fine. All three together mean attacker-controlled text can move your data.

Cherny's own remarks on this are the most interesting part, and they need to be read precisely. He says of the newest model that "the model does not seem to be prompt injectable anymore," and describes three layers behind that: a model with roughly three years of alignment research in it, a prompt injection classifier run across all traffic, and a further classifier on auto mode. With all three, he says, "we just cannot demonstrate prompt injection anymore."

Two things are true about that claim at once. It is a vendor reporting its own internal testing rather than an external audit, and a team saying it can no longer demonstrate an attack is a statement about what that team found, not a proof of impossibility. But take it entirely at face value for a second, because the founder lesson survives either way: not one of those three layers is your system prompt. The defense he describes lives in model weights and in classifiers running on someone else's infrastructure. Nothing you write into your instruction block participates in it.

So the practical rules:

• Assume the system prompt is public. No credentials, no internal hostnames, no customer names, no undisclosed business logic you would not want quoted back to you.
• Enforce authorization in code. Whether this user may see this record is a decision your application makes before the model is called, not a sentence in the prompt asking the model to be careful.
• Constrain the tools, not the intentions. An agent that cannot reach a system cannot be talked into reaching it. Scope the credentials the agent holds, which is the same argument that governs where you run a self-hosted AI agent.
• Break the trifecta somewhere. If the agent touches private data and untrusted content, cut the outbound path, or gate it behind a human.

And now the specific trap. Before you delete a system prompt, find out whether any line in it is your only implementation of a control. If a sentence in your prompt is the sole thing standing between a user and data they should not see, an ablation round deletes your security posture and the tests will pass. That line was never a control, but it was doing something, and removing it without a replacement makes a bad situation worse.

The fix is the right one anyway: move it into the harness first, then ablate freely. A shorter prompt is also a smaller leak surface and a smaller place for stale exceptions to hide, so this discipline is a security win once the real controls sit where they belong.

Do evals outlast the system prompt?

If the prompt is disposable and the scaffolding is disposable, something has to be durable, and the obvious candidate is your eval set. Cherny half agrees, which is the more useful answer.

Asked whether evals carry across model releases, he says "we do until we max out the eval." Then he volunteers the limit himself: "I actually wouldn't even go this far, to be honest. I think evals, they outlive the harness a little bit, but not by that much. Like an eval might live for maybe one, two, three model generations, but nowadays the you know, we're on the exponential. The model is improving so quickly, very often we just saturate the eval, and then we have to throw it away, and we have to come up with a new eval."

So evals are the most durable artifact you have, and they still expire. A saturated eval is not a passing grade, it is a broken instrument: once everything on it passes, it has stopped telling you anything and it is actively hiding regressions elsewhere. Building and retiring these deliberately is its own discipline, covered in LLM evaluation for founders.

What genuinely persists is smaller than any artifact: the habit. Cherny's version of it is a mindset shift more than a technique. "It's not a theoretical science, it's become an empirical science," he says, and his instruction to anyone trying to get good at this is to "forget all of the things that you learned about past models."

That is the same posture behind latent demand, the other rule he works from: watch what actually happens rather than what your model of the situation predicts. There it is pointed at users. Here it is pointed at your own prompt.

How often should you rewrite your system prompt?

Tie the work to model releases, not to the calendar, because releases are what invalidate the content.

At every model release, if you build an agent product. Branch, delete the system prompt, run your evals and a week of real usage, add back only what repeatedly fails without it. Keep the diff, because it tells you what the new model learned to do on its own.

Every six months, if you use coding agents rather than build them. Cherny's line is specific: "every 6 months delete your Claude MD. Delete your skills. Delete your hooks." He means the memory file conventionally written as CLAUDE.md, plus your accumulated custom skills and hooks. "See what the model does and it might surprise you."

Never delete on the same day you ship. This is an experiment, and it needs a branch, an eval baseline, and a way back.

None of this requires a platform team. It requires a branch, a week, and a willingness to find out that most of what you wrote is no longer doing anything. For the wider picture of how this fits with agents, evals, and pricing, the AI for startups pillar is the map.

What to do this week

1. Print your system prompt and date every line. Next to each one, write the model it was written for. Any line older than the model you are running is a candidate, and you will be surprised how many there are.
2. Mark the lines that are secretly controls. Anything enforcing access, privacy, or spend. Those do not get ablated, they get rewritten as code first. If you cannot tell, assume it is a control and check.
3. Run one ablation on one surface. Pick your least critical agent. Branch, delete the whole system prompt, use it for a week against real tasks, and log every repeated failure with the exact line that would have prevented it.
4. Add back only what your log justifies. One line per observed pattern, no pre-emptive additions. Record the token count before and after, since that is the number you can show a board.
5. Write down your definition of done. If verification is the thing most teams get wrong, make the exit criteria the most explicit part of whatever prompt survives.
6. Check your prompt for secrets today. Assume it will be extracted, because OWASP does. Anything in there you would not publish gets moved out this week, not next quarter.

The pattern underneath all of this is the one this course is built around: treat the model as something you measure rather than something you argue with, and build the loop before you scale the surface. That is what AI Operating System for Startups teaches, applied to the most-read text in your product.

Sources

• Boris Cherny: We Cut 80% of Claude Code's Prompt, Y Combinator, July 2026, is the talk this article distills. All quotations attributed to Cherny are from this session unless stated otherwise.
• Background on Boris Cherny: creator and head of Claude Code at Anthropic, joined September 2024, previously a Principal Engineer at Meta, and author of O'Reilly's Programming TypeScript. Corroborated by Gergely Orosz's Building Claude Code with Boris Cherny in The Pragmatic Engineer.
• OWASP LLM07: System Prompt Leakage is the source for the guidance that a system prompt is neither a secret nor a security control, and that authorization must not be delegated to the model.
• Designing for the inevitable: System prompt leakage and mitigations in generative AI applications, the AWS Security Blog, for the parallel guidance not to use system prompt instructions as a security control.
• The lethal trifecta for AI agents by Simon Willison, June 2025, for the three conditions that make prompt injection consequential.
• Cross-source on harness design: When to Build Your Own Agent Harness with Harrison Chase of LangChain at Sequoia Capital, on what a harness is actually for, and Google DeepMind's Logan Kilpatrick: Why the Model Eats the Harness, Sequoia Capital, on scaffolding being absorbed by later models.