Reasoning Effort: The Dial That Sets How Long a Model Thinks

Nish · July 25, 2026

NLP   Techniques
⏱️ 20 min read

Table of Contents

Somewhere in the settings of almost every AI product you use, there is now a dial. OpenAI calls its positions things like low, medium, high and xhigh. Anthropic runs from low up to max. Gemini has thinking levels, Grok and DeepSeek have a reasoning effort field, and ChatGPT’s composer offers thinking time from Light to Heavy. Turn the dial up and the same model gets slower, more expensive, and measurably better at hard problems. Turn it down and it snaps back to being fast and cheap. This post is about what that dial actually is: not a different model, not a hard token limit, but a couple of words written into the prompt that the model has been trained to obey.

TL;DR

  • Reasoning effort controls how much test-time compute a model spends per request, mostly by changing how long its hidden chain of thought runs. One set of frozen weights serves every level.
  • Under the hood the setting is usually rendered as plain text in the system prompt or chat template (OpenAI’s open-weight models literally receive a line reading Reasoning: high), so the interesting question is how the model learned to respond to that text.
  • The recipe has two main ingredients: supervised fine-tuning on paired examples of the same model thinking and not thinking, and reinforcement learning where the reward charges a per-token price for thinking, with the price depending on the requested effort. A cheap token price teaches long deliberation; an expensive one teaches economy.
  • Alongside the learned dial there are hard, training-free controls: cut thinking off at a token budget and force the answer, or append text like “Wait” to make the model think longer. Several models are explicitly trained to survive being interrupted mid-thought.
  • Accuracy rises with effort but with sharply diminishing returns, and a small model at high effort can beat a large one at low effort. That makes effort a genuine third axis of model choice, next to which model and which prompt.

Scope. This post covers how effort dials work and how they are built. For how reasoning models came to exist at all, and the reinforcement learning recipe behind them, see the earlier post on reasoning models. The charts below are labelled schematic or toy where they are not measured data.

The controls providers actually ship

It is worth taking stock of how universal this control has become, because the convergence itself is informative. As of mid-2026:

Provider Control Values
OpenAI reasoning_effort none, minimal, low, medium, high, xhigh, max (subset varies by model)
Anthropic effort low, medium, high, xhigh, max
Google thinking_level minimal to high (Gemini 2.5 used a token-count thinkingBudget)
xAI reasoning_effort low, medium, high
DeepSeek thinking + reasoning_effort thinking on/off, then high or max
Moonshot (Kimi) thinking / reasoning_effort on/off on K2.x; low, high, max on K3
Alibaba (Qwen) enable_thinking + budget on/off, /think and /no_think soft switches, token budget

Three patterns hide in that table, and they arrived in a meaningful order. The first controls were binary toggles: Qwen3’s enable_thinking, DeepSeek’s thinking mode, a switch between “answer now” and “think first”. The second were token budgets: Gemini 2.5’s thinkingBudget and Anthropic’s budget_tokens let you cap the thinking trace at a number of tokens. The third, and the one the industry has settled on, is the categorical effort level: a behavioural label rather than a hard number.

The migration from budgets to labels is explicit in the vendors’ own documentation. Anthropic deprecated manual budget_tokens on its Claude 4.6 models and rejects it outright from 4.7 onward; the documented replacement is adaptive thinking plus the effort parameter. Google made the same move between model generations: Gemini 2.5 took a token count, Gemini 3 takes a thinking_level, and setting both in one request is an error. A budget answers “how many tokens may you spend?”; an effort level answers the question you actually care about, “how hard should you try?”, and leaves the token arithmetic to the model.1

One more product detail worth knowing: on every major API, thinking tokens are billed as output tokens, the most expensive kind, and they occupy space in the context window.2 The dial is therefore quite directly a price dial, which explains why every provider ended up shipping one.

Why the dial exists

The earlier post on reasoning models told the story of how these models are made: reinforcement learning against checkable answers, from which long chains of thought emerge because thinking longer earns more reward. DeepSeek-R1-Zero’s responses grew from hundreds of tokens to roughly ten thousand over training, and nobody told it to write more.

That emergence is also the problem. A model whose training taught it “longer thinking earns more reward” has learned a habit, not a judgement about your query. Left alone, it applies the habit indiscriminately, and the failure mode has a name in the literature: overthinking. Chen et al. measured o1-style models spending thousands of tokens double- and triple-checking trivial questions like “what is 2+3?”, with extra rounds of reflection contributing almost nothing after the first correct pass. Every one of those tokens is billed as output, adds latency one token at a time, and, as the scaling post argued from roofline arithmetic, lands on decode, exactly the memory-bound phase that serving infrastructure finds most expensive to speed up.

The dial exists because the right amount of thinking is not a property of the model; it is a property of the request. A production system triaging support tickets and the same system debugging a race condition want opposite ends of the trade. The economics are stark enough that Anthropic led its Opus 4.5 announcement with them: at medium effort, Opus 4.5 matched Sonnet 4.5’s best score on SWE-bench Verified while using 76% fewer output tokens.3 Most of the capability, a fraction of the spend, if you can control the spend.

What the setting actually changes

Here is the part that surprises most people: for the models where we can see the machinery, the effort setting is text. There is no second model, no low-effort distillation being swapped in behind the scenes, and usually no hard cutoff. The API parameter you set is rendered into the model’s context as a short instruction, and everything else follows from how the model was trained to react to it.

OpenAI’s open-weight gpt-oss models make this concrete because their prompt format (called harmony) is public. The model card says the models are trained “to support three reasoning levels”, configured “in the system prompt” by inserting a keyword line. What the model actually receives looks like this:

<|start|>system<|message|>You are ChatGPT, a large language model trained by OpenAI.
Knowledge cutoff: 2024-06
Current date: 2025-08-05

Reasoning: high

# Valid channels: analysis, commentary, final.<|end|>

Change high to low and the same weights produce a chain of thought that is shorter by an order of magnitude.4 The dial is two words.

Schematic of two identical system prompts differing only in a Reasoning: low versus Reasoning: high line, both feeding one model block labelled same frozen weights, producing a short thinking trace in one lane and a much longer one in the other before the answer.
The effort dial as the model sees it. Two requests differ by one line of text; the same frozen weights produce a short or a long thinking trace in response. Chip counts are illustrative.

The proprietary APIs are less transparent, but the seams show in the same places. Anthropic’s documentation notes that “the thinking configuration and the resolved effort level are rendered into the prompt itself”, which is why changing the effort level invalidates your prompt cache: from the model’s point of view, the prompt changed. Qwen3’s toggle works through the chat template: a /no_think flag in the message, or simply prefilling an empty <think></think> block so the model’s turn begins with its thinking already closed. The tags themselves do nothing magical. As Sebastian Raschka puts it in the article that prompted this post, the <think> tokens are not what gives the model the ability to reason; they are learned formatting, scaffolding around a behaviour that training installed.

So the honest answer to “what does the dropdown do?” is: it writes a label into the context. The interesting question is the next one.

How training gives the label meaning

A base model shown the string Reasoning: low has no reason to do anything differently. The label acquires meaning the same way everything else does: it appears during training, correlated with consequences. Two ingredients do most of the work.

Ingredient one: show it both behaviours

The direct approach is supervised fine-tuning on paired data. NVIDIA’s Llama-Nemotron models, the first open family with a reasoning toggle, built their SFT set explicitly this way: for the same prompts, reasoning-style responses (distilled from DeepSeek-R1) were tagged with the system instruction “detailed thinking on”, and concise responses (from instruction-tuned Llama models) were tagged “detailed thinking off”. One model, trained on both, learns to condition its whole response style on the tag.

Qwen3 industrialised this as a pipeline stage its report calls thinking mode fusion. After the model is first trained as a pure reasoning model with reinforcement learning, a further SFT round mixes thinking samples with non-thinking samples, the latter carrying an empty thinking block so the response format stays identical across modes. That formatting choice pays an unexpected dividend, which we will come back to: a model that has seen “thinking that stops, then an answer” learns to answer from a partial thought.

Ingredient two: put a price on thinking tokens

Pairing gets you on and off. The graded levels in between come from reinforcement learning, and the core trick is to make the reward charge for thinking. Kimi k1.5’s report published the cleanest early version: alongside the correctness reward, a length reward compares each sampled response to the shortest and longest samples for the same problem,

\[\lambda = 0.5 - \frac{\text{len} - \text{min\_len}}{\text{max\_len} - \text{min\_len}},\]

granting \(\lambda\) to correct answers (so short correct answers earn a bonus and long-winded correct answers a penalty) and \(\min(0, \lambda)\) to wrong ones, so a wrong answer can never profit from being long. Trained with this, the model keeps its accuracy while shedding the padding, because the padding now costs reward.

To turn a single length penalty into an effort dial, you make the price depend on the requested effort. During RL, each training sample carries an effort label in its system message, and the per-token cost applied to that sample is set accordingly: steep when the label says low, gentle or absent when it says high. Conceptually the reward looks like

\[R \;=\; R_{\text{task}} \;-\; \lambda(e)\, N_{\text{thinking}},\]

where \(e\) is the requested effort and \(\lambda(e)\) is the token price attached to it.5 The model is optimising one policy under many prices at once, and the best response to each price is different. Where the next thinking token stops paying for itself in expected accuracy, the reward peaks, and the model learns to stop near that point. A different price moves the peak:

Chart with thinking tokens on the x axis. A teal curve shows probability of a correct answer rising with diminishing returns. Three amber-to-ember curves show expected reward when a per-token cost is subtracted at three different rates; each peaks at a different thinking length, with the peak moving right as the token price falls.
A toy model of the training signal behind an effort dial. The chance of a correct answer (teal) always rises with more thinking, but the reward subtracts a per-token price λ that depends on the requested effort. Each price puts the reward peak, the length the model is trained toward, somewhere different. Curves are computed from the toy reward shown, not measured from a real model.

This is why effort levels feel qualitative rather than mechanical. The model at low effort is not being cut off; it has been trained that, when the context says low, brevity is the winning strategy. Anthropic’s docs describe their effort parameter in exactly these terms: “a behavioral signal, not a strict token budget”, under which the model still thinks hard about genuinely difficult problems, just less than it otherwise would.

The most aggressive version of this recipe drops the discrete levels entirely. Inkling, the open-weight model Thinking Machines released this July, conditions on a continuous effort value written into the system message, sweeping settings between 0.2 and 0.99 during an RL run of more than 30 million rollouts, with the per-token cost adjusted to match. The result is a knob rather than a switch, and an efficient one: their report shows it matching NVIDIA’s Nemotron 3 Ultra on Terminal-Bench 2.1 while spending roughly a third of the tokens.

Two variants are worth knowing about because they show the same idea pushed to its limits. L1 trains with targets instead of prices (“Think for exactly \(n\) tokens”, rewarded for hitting \(n\)), producing a model whose thinking length tracks the request almost linearly.6 And DeepSeek V4 goes the other way: rather than one model juggling every price, it trains separate specialist models for its non-thinking, thinking, and maximum-effort modes, each with its own length penalties and context budget, then distils the specialists back into a single checkpoint that serves all three.

The backstop: budgets enforced at inference

Everything above is learned behaviour, and learned behaviour comes with no guarantees on any individual request. So alongside the dial there is a blunter family of controls that need no training at all: intervene in the token stream.

The purest demonstration is the s1 paper’s budget forcing. Working with a model fine-tuned on just 1,000 curated reasoning traces, the authors controlled thinking length entirely at decode time. To stop the model thinking, append the end-of-thinking delimiter and force it to answer with whatever reasoning it has. To make it think longer, do the opposite: when it tries to close its thinking block, suppress the delimiter and append the word “Wait”, which reliably sends the model back to double-check its work. With nothing more than that, s1 beat o1-preview by up to 27% on competition-maths benchmarks, and accuracy scaled smoothly with the enforced budget.

The catch with forced stops is that a model interrupted mid-thought can panic, so the strongest recipes train for the interruption. Nemotron 3 Ultra’s SFT data includes reasoning traces truncated at random lengths while keeping the original final answers, with the appended </think> masked out of the loss, so the model practises recovering a good answer from an incomplete thought. Qwen3’s authors found they got the same robustness without asking for it: thanks to those empty-and-therefore-sometimes-short thinking blocks in the fusion stage, a budget-stop instruction inserted mid-trace (“Considering the limited time by the user, I have to give the solution based on the thinking directly now”) works even though, as the report notes, the ability “is not explicitly trained but emerges naturally”.

Seen from here, the token-budget parameters the platforms shipped (Gemini 2.5’s thinkingBudget, Anthropic’s budget_tokens, Qwen’s thinking budget) are productised versions of this backstop, and the effort dial is the productised version of the learned behaviour. The industry shipping both, then migrating its defaults from the budget to the dial, is a nice public record of which mechanism turned out to matter more.

What a level buys you

However the control is implemented, the empirical shape of what it buys is remarkably consistent: accuracy rises with thinking length, steeply at first and then with hard diminishing returns. The gpt-oss model card plots exactly this for its three effort levels, and every vendor’s documentation gestures at the same curve.

Schematic chart of accuracy versus average thinking tokens on a log scale, for a large and a small model. Markers on each curve show low, medium and high effort levels. Both curves rise with diminishing returns, and the small model at high effort sits above a dotted line marking the large model at low effort.
What effort buys, schematically (the curve shape follows the gpt-oss model card's published accuracy-versus-length figures; values are illustrative). Each level moves the model along its curve, and the curves overlap: a small model thinking hard can outscore a large model answering quickly.

Two practical consequences fall out of that picture. First, the last increments of effort are the most expensive accuracy you will ever buy: moving from high to max can multiply token spend for a benchmark gain of a point or two, which is why the top settings are framed for use cases like long-horizon agents and deep research rather than as a default. Second, because the curves of different-sized models overlap, effort becomes a real substitute for scale. A smaller, cheaper model at high effort can land in the same accuracy band as a bigger one at low effort, at a very different price and latency point, so “which model?” and “how much effort?” have become a joint decision.

The substitution only works within the model’s reach, though. Effort moves a model along its curve; it does not raise the curve’s ceiling. Snell et al., studying compute-optimal test-time scaling, found that extra inference compute pays off on questions where the base model already attains “somewhat non-trivial success rates”, while on the hardest problems the same compute is better spent on a more capable model. Apple’s “Illusion of Thinking” experiments make the point more bluntly: pushed past a complexity threshold, the reasoning models they tested collapsed to near-zero accuracy whatever the budget, and their thinking traces actually got shorter as they approached it, as if the models stopped trying.7 So a small model at high effort on a task just beyond its reliable range is the worst cell in the price matrix: you pay for maximum thinking and still do not get the answer. In an effort sweep, spend rising while accuracy stays flat is the signature of that regime, and the fix is changing models, not levels.

There is one more wrinkle: within a level, modern models modulate themselves. OpenAI’s docs note that its models “reason adaptively across reasoning efforts, using fewer tokens for simpler tasks”; Anthropic’s adaptive thinking goes further and lets the model decide whether to think at all on each request, with effort setting the overall disposition. The dial sets a stance, not a number, and the per-request judgement is increasingly delegated to the model itself.

Using the dial well

A few concrete habits, mostly lifted from the providers’ own guidance and the training story above:

  • Sweep it against your own evaluations. The levels are labels on learned behaviour, not calibrated units, so the only way to find your operating point is to run your eval suite across levels and look at the accuracy-versus-cost curve. Re-run the sweep when you change models: a given label buys different behaviour on each new release, a point the previous post on prompt rot made in more detail.
  • Know what your provider’s dial covers. On Claude, effort shapes all output, including how many tool calls the model makes and how long its visible answers run, not just hidden thinking. Dropping the level on an agent quietly changes how much work it does per turn, which may be exactly what you want, or not.
  • Match the level to the request, not the product. The clean pattern for high-volume systems is routing: a cheap default level for the easy majority, escalation to high for requests that fail, look hard, or matter. Latency-critical endpoints get none, minimal or their equivalent; overnight batch and agentic work is where xhigh and max earn their cost.
  • Mind the prompt cache. Because the effort level is rendered into the prompt, flipping it mid-conversation starts a new cache prefix on Claude. Pick a level per workload and hold it stable rather than toggling per message.
  • Stop asking for effort in prose. “Think step by step”, “think carefully before answering” and their relatives are prompt-era attempts at this dial. Now that the dial exists, set it there, and keep the prompt for the things only you know.

Where this is heading

The dial’s short history has a clear direction: from binary, to budgets, to behavioural levels, to Inkling’s continuous knob, and in parallel, from the developer choosing to the model choosing. Adaptive thinking on Claude and dynamic budgets on Gemini both hand the per-request decision to the model, with the human setting only a prior. It is easy to see where that converges: effort stops being a setting you pick and becomes a judgement the model makes about your problem, with the dial surviving as a cost ceiling and an override.

Meanwhile the word itself is already being stretched. On xAI’s multi-agent Grok model, reasoning_effort no longer controls thinking depth at all; it controls how many agents collaborate on your request. Different mechanism, same contract: more effort, more compute, better answer, bigger bill.

That contract is the durable idea. The reasoning-models story established that intelligence per query became a variable; the effort dial is that variable given a price and a user interface. However the mechanisms evolve, you should expect every serious AI system you touch from here on to expose, somewhere, a knob whose honest label is: how much is this answer worth to you?

Sources & further reading

  1. The budgets were never as hard as they looked anyway. Anthropic’s docs describe budget_tokens as “a target rather than a strict cap”, and Gemini 2.5 Pro would not let you disable thinking at all; its budget had a floor of 128 tokens. ↩

  2. OpenAI’s reasoning guide, Anthropic’s extended-thinking docs, and Google’s thinking docs all state this, and all three break reasoning tokens out in the usage response (output_tokens_details, thinking_tokens, and thoughts_token_count respectively). ↩

  3. Anthropic’s own benchmark figure from the Opus 4.5 announcement (November 2025), so treat it as a vendor measurement, but it is the cleanest published number on what effort control is worth. ↩

  4. In harmony, the thinking goes to an analysis channel and the user-facing answer to a final channel, so “how long did it think” is directly measurable. The gpt-oss model card’s accuracy-versus-length figure shows the 20B model averaging north of 20,000 thinking tokens per AIME problem at high effort. ↩

  5. This formula is Raschka’s reconstruction of the mechanism, not an equation from a paper; labs publish the idea more readily than the exact reward. The clearest first-party description is Thinking Machines’ report for its Inkling model: “We specified the model’s effort level on different samples by changing the system message and adjusting the per-token cost.” ↩

  6. Aggarwal and Welleck, “L1: Controlling How Long a Reasoning Model Thinks” (2025). A side finding worth remembering: their length-controlled models beat their own base model at short lengths, meaning long-chain-of-thought training quietly improves short answers too. ↩

  7. Shojaee et al. (2025), on controllable puzzle benchmarks with o1/o3-mini, Claude 3.7 Sonnet Thinking and DeepSeek-R1. The paper’s methodology drew published rebuttals (token limits and unsolvable puzzle instances confound some results), so treat exact collapse thresholds cautiously; the directional finding that effort stops converting into accuracy beyond a model’s reach is the part that matters here. ↩

Citation Information

If you find this content useful, please cite this work as:

Bhana, Nish. "Reasoning Effort: The Dial That Sets How Long a Model Thinks". Nish Blog (July 2026). https://www.nishbhana.com/Reasoning-Effort/

Or use the BibTeX citation:

@article{bhana2026reasoningeffort,
  title   = {Reasoning Effort: The Dial That Sets How Long a Model Thinks},
  author  = {Bhana, Nish},
  journal = {nishbhana.com},
  year    = {2026},
  month   = {July},
  url     = {https://www.nishbhana.com/Reasoning-Effort/}
}

Related Posts 🔥

The Metric Is the Spec: Choosing and Designing Evaluation Metrics
ML Jul 2026
Speculative Decoding: Two Models, One Voice
NLP Jul 2026
Mixture of Experts: Trillions of Parameters, a Fraction at a Time
NLP Jul 2026
Online Experimentation: A/B Testing a Subscription Business Without Fooling Yourself
ML Jul 2026
J-Space: How Anthropic Reads Claude's Silent Thoughts
NLP Jul 2026
How LLMs Scale: FLOPs, Bytes, and the Price of a Token
NLP Jul 2026

x.com, Facebook