Understanding how context windows work, and how to use them well, is one of the most practical skills for getting good results from any LLM, including the models on the L&S AI Inference platform.
A model's context window is the total amount of text it can consider at once, measured in tokens (roughly 4 characters per token). Everything counts against it: the system prompt, every earlier message in the conversation, uploaded document content, web search results, and the model's own response.
| Model | Context window |
|---|---|
gemma-4-26b-a4b-it |
256k tokens |
qwen3.8-27b |
256k tokens |
gpt-oss-120b |
128k tokens |
The full conversation history is re-sent with every message, so long conversations consume context quickly.
As a conversation grows, older turns remain in the context window. Near the limit, the model may:
The model doesn't warn you before this happens. Monitoring conversation length is your responsibility.
If you're used to commercial tools like ChatGPT, Claude, or Gemini, you may have developed habits that don't translate as well to smaller open-weight models. Some key differences:
Commercial frontier models are good at inferring intent from incomplete instructions. Open-weight models are more literal — they do what you ask, not what you meant. Being explicit and specific matters more.
Instead of: "Help me with this code."
Try: "This Python function is returning None when I pass a dictionary. Find the bug and explain why it's happening."
Commercial frontier models are highly optimized for long multi-turn conversations. Open-weight models can degrade more noticeably as context grows. Shorter, more focused conversations produce more consistent results.
If a frontier model makes a mistake, it will often catch and correct itself. Open-weight models are more likely to commit to a wrong answer and build on it. If something looks wrong, explicitly correct it:
"That's not right — [explain the error]. Try again."
Don't assume the model will notice the mistake on its own.
A frontier model can usually follow a 10-part instruction reliably. Open-weight models do better with 2–3 focused instructions per message. Break complex requests into steps.
Before sending a complex message, ask yourself: