Context, Tokens, and Modalities
Understand what fits into a model request, what can be lost, and how text, images, audio, and files change the workflow.
By the end
You will be able to
- Explain tokens and context windows in practical terms.
- Choose relevant context instead of supplying everything available.
- Recognize how multimodal inputs require their own quality and accessibility checks.
- Plan for long conversations, source boundaries, and information that must persist outside model context.
Models process tokens rather than pages
A token is a unit used to represent input and output. Tokens may be whole short words, pieces of longer words, punctuation, spaces, code fragments, or other symbols. The number of tokens is therefore not identical to a word or character count.
Tokens matter because products enforce limits, measure usage, and sometimes charge for input and output in tokens. Use the provider's current counting tools when an exact limit or cost matters.
The context window is the working set
The context window holds the information available to the model for a response: system instructions, conversation history, supplied documents, tool results, and generated tokens. Products differ in what they include and how they manage long conversations.
A larger window can hold more information, but size does not guarantee that every detail is used correctly. Relevant structure, clear priorities, source labels, and targeted retrieval still matter.
Select, structure, and label context
Supply the smallest set of trustworthy information that supports the task. Separate instructions from reference material, identify authoritative sources, remove duplicates, and place important constraints where the workflow preserves them.
When a corpus is too large, retrieve relevant material in bounded pieces and keep stable facts in an external system of record. Do not rely on a long chat as permanent memory.
Every modality has different failure modes
Multimodal systems may accept text, images, audio, video, or files. An image can be cropped or unreadable, audio can be noisy, a document parser can omit tables, and metadata can disclose sensitive information.
Provide accessible text alternatives, confirm what the system actually received, and verify modality-specific claims against the original artifact. A model description of an image or recording is an interpretation, not the artifact itself.
Practice activity
Build a trustworthy context budget
- Choose a task that uses at least three possible inputs such as a policy, notes, a spreadsheet, an image, or prior messages.
- Label each input as instruction, authoritative evidence, supporting context, untrusted content, or unnecessary material.
- Create a compact context package that includes the objective, boundaries, selected evidence, source labels, requested output, and verification method.
- Describe what belongs in an external record or retrieval system instead of the model context.
What to produce
- A context inventory with trust labels and include or exclude decisions.
- A final context package and a separate list of facts or records that must persist outside the conversation.
Reflect before continuing
What did you remove from the context, and why did removing it make the task safer or clearer?
Evidence
Sources and verification
- Key conceptsOpenAI · verified 2026-07-25
- Context windowsAnthropic · verified 2026-07-25
- Responsible Generative AI ToolkitGoogle · verified 2026-07-25
Knowledge check
Make it stick.
Choose the strongest answer for each question. Your attempts become part of your device-local transcript.