"Prompt engineering" made it sound like the magic words were the hard part. In every production LLM system I have watched succeed or fail, the prompt was never the bottleneck. The context was — the material you assemble around the question, on every single request. That assembly is the real engineering discipline, and the popular name badly undersells it.

The prompt is the easy ten percent

A good system prompt matters. It sets role, tone, and guardrails. But you write it once and it stops being the variable. What changes on every request, and what actually determines whether the answer is right, is the material you assemble around the question: the retrieved documents, the tool outputs, the conversation history, the schema you are asking the model to fill. Call it context engineering — the practice of getting the right information, in the right form, in the right amount, into the model's window at inference time. When an LLM feature is flaky in production, the prompt is almost never the reason. One of the following usually is.

The four failure modes

Too little context. The model was not given the fact it needed, so it guessed — and a fluent guess reads exactly like a confident answer. This is the classic retrieval miss: the answer lived in a document that ranked seventh when you only passed the top five. The model did not hallucinate from nothing; it hallucinated from absence.

Too much context. The opposite, and more common than people expect. Stuff forty documents into the window "to be safe" and two things degrade: the signal you need gets buried in the middle, where models attend to it least, and your cost and latency balloon. More context is not more accuracy. Past a point, it is less.

Wrong-shaped context. The right facts in a form the model cannot use well. A two-hundred-row table pasted as raw text when a three-row summary would have answered the question. A deeply nested data blob where the relevant field sits on line four thousand. Format is not cosmetic — it changes what the model can actually extract.

Stale or conflicting context. Two retrieved documents disagree, or one is six months out of date, and nothing in the context tells the model which to trust. It will pick one, confidently, and you will not know which until a user does.

What context engineering looks like in practice

Treating context as the primary surface changes where you spend effort:

  • Retrieval quality is the product. The highest-leverage investment in most retrieval-augmented systems is better retrieval — hybrid search, reranking, and honest evaluation of recall — not a cleverer prompt. If the right chunk is not in the top results, no prompt saves you.
  • Compress before you inject. Summarize, extract, or structure retrieved material before it hits the window. A pre-processing step that turns ten documents into one dense brief usually beats passing all ten. Let a cheap model do the compression.
  • Budget the window explicitly. Decide how many tokens each source gets — history, retrieval, tools, instructions — and enforce it. Uncontrolled context growth is how a feature that worked in the demo falls over on the long conversation.
  • Position deliberately. Put the most important material at the start or end of the context, not buried in the middle. Models have a well-documented recency and primacy bias; use it instead of fighting it.
  • Label provenance and freshness. Tag each piece of context with where it came from and how current it is, and tell the model how to arbitrate conflicts. Ambiguity you do not resolve, the model resolves for you — silently.

Why the reframing matters

If you believe the prompt is the lever, you tune wording for weeks and get marginal gains. If you believe the context is the lever, you build retrieval evaluation, compression steps, and token budgets — the boring infrastructure that actually moves accuracy. The teams shipping reliable LLM features today stopped hunting for magic words a while ago. They spend their time on the pipeline that decides what the model sees.

The model is a function. The prompt configures it. The context is the argument — and the argument is where your answer is won or lost.

None of this is glamorous. There is no clever incantation to screenshot for the team channel. There is just the unglamorous work of making sure the right information, in a usable shape, is in front of the model at the moment it answers. That work is the difference between a demo and a product.