Prompting, Context & Structured Output Questions

In plain words: Ask for a fixed format, give the document as data, then check the result with code. Allow nulls instead of guesses and test on real examples.

Full answer · Define the job and constraints, supply the document clearly delimited as data, request a strict schema (preferably via the provider's structured-output or forced-tool mechanism), tell the model to use null instead of guessing and to flag non-matching documents, and add a few diverse examples including an edge case. Then validate in code: schema, then business invariants (totals, dates, allowed vendors), with one repair attempt using the validator's errors before escalating. Evaluate on a labelled set with field-level accuracy.

In plain words: It means choosing everything the AI sees: instructions, documents, tools and history. What you leave out matters as much as what you put in.

Full answer · Designing everything the model sees at inference time — system instructions, user input, retrieved documents, tool definitions, conversation state, memory and intermediate results — not just the wording of one prompt. The skill is deciding what to include, in what order, with what authority and boundaries, and what to leave out, because relevance, recency and authority matter more than volume, and every token costs money and attention.

In plain words: You cannot fully stop it with wording. Mark outside text as data, give the AI few powers and check every action in code.

Full answer · You can't guarantee it with wording alone, so layer defenses: delimit retrieved text and label it as untrusted data; instruct the model to treat it as information only; keep the model's tools minimal and authorize every action outside the model; validate outputs; require approval for sensitive actions; and test with planted instructions in documents. The principle is that a successful injection should be unable to do anything harmful, rather than assuming it can't succeed.

In plain words: Use examples when a format is easier to show than to explain. Pick varied examples and test whether they actually help.

Full answer · When format or style is hard to describe but easy to show, or for subtle decisions such as classification boundaries. Choose diverse, representative examples including an edge case and a correct refusal. Costs: they consume context, can bias outputs toward their patterns and go stale. Compare against zero-shot on your eval set; if instructions plus a schema meet the target, skip them.

In plain words: Keep prompts in version control. Bundle them with the model and settings. Test before release, and keep a quick way to go back.

Full answer · Store them in source control as files with identifiers, review changes like code, and bundle each release with the model snapshot, tool schemas and retrieval config under one release ID that is logged on every trace. Gate changes with an eval suite and canary the rollout, with rollback as a configuration change. Keep a changelog explaining why instructions exist so future maintainers don't delete a fix for an old failure.

In plain words: First find what changed: the model, the prompt, the data or the inputs. Compare good and bad runs and re-test with a pinned version.

Full answer · First establish what changed: model snapshot or alias, prompt version, retrieval index, tool schemas, input distribution. Compare traces from good and bad periods, rerun the eval set against the old and new configurations, and look for drift in inputs. If the provider updated an unpinned model, pin a dated snapshot and re-evaluate before migrating. Fix the root cause and add the failing cases to the regression set.

In plain words: They keep your instructions apart from pasted text and make output easier to read. They help, but they are not real security.

Full answer · They separate instructions from data so untrusted text is less likely to be treated as instructions, make long prompts easier for the model to parse, and make outputs machine-readable. They are a speed bump, not a security boundary — real protection comes from least-privilege tools and validation in code.

In plain words: Put the job, rules and format in the prompt. Keep passwords, permission checks and business rules in code.

Full answer · In: the job, constraints, relevant information, output contract and failure behavior. Out: secrets, credentials, authorization decisions and business-critical invariants. Those are enforced by code, because a prompt can be ignored or manipulated, and anything in the context may leak into output or logs.

In plain words: Put the part that never changes first. The provider can then give a discount. Also cut repeated text, and use a smaller model for easy requests.

Full answer · Keep the stable prefix first and the variable content last so provider-side prompt caching can discount the repeated part; trim redundant instructions and examples; move rarely needed guidance into retrieval or conditional inclusion; and route simple requests to a cheaper model. Measure cost per successful task after each change and rerun evals to confirm quality didn't drop.

In plain words: It guarantees the shape of the answer, not that the values are true or allowed. You still check the facts and permissions.

Full answer · Constrained decoding can guarantee the output conforms to the schema's shape — types, required fields, enums. It doesn't guarantee the values are correct, plausible or authorized: a perfectly shaped invoice can have the wrong total or an unapproved vendor. So you still need domain validation, authorization and, for agents, policy checks on tool arguments.