LLM Settings and Parameters

Key inference parameters that control LLM output characteristics, as documented in the Prompt Engineering Guide.

Temperature (confidence: high)

Controls randomness in token selection:

• Low temperature (0-0.3): More deterministic, picks highest-probability tokens. Best for factual Q&A, code generation, structured outputs.
• High temperature (0.7-1.0): More creative/random, broader token distribution. Best for poetry, creative writing, diverse outputs.

Top-p (Nucleus Sampling) (confidence: high)

Controls the cumulative probability threshold for token selection. The model considers only tokens whose cumulative probability reaches top_p:

• Low top_p: More focused, deterministic outputs
• High top_p: More diverse outputs

Best Practice (confidence: high)

Alter one parameter at a time, not both simultaneously. Results vary by LLM version.

Task-Matching Guidance (confidence: medium)

The captured HF survey couples sampling parameters with task type rather than using them in isolation:

• Deterministic tasks (math, code, extraction): low temperature + low top-p.
• Creative tasks (brainstorming, varied output): higher temperature + higher top-p, possibly with self-consistency sampling for robustness.
• Prompt-specificity interaction: vague prompts need lower temperature to avoid chaotic outputs; a tightly-specified prompt tolerates more sampling freedom.

Implications

Temperature and top-p are not “more = more creative” dials in isolation — the correct setting depends on prompt specificity and task determinism. This connects to self-consistency, which requires temperature > 0 to generate the diverse reasoning paths it votes over.

Structured Outputs and Determinism (confidence: high)

Low temperature reduces randomness but does not guarantee shape. OpenAI’s structured-outputs feature is the complementary tool: it enforces a JSON Schema at the API level (no missing keys, no invalid enums) while temperature/top_p continue to control token-level variability within the contract. For integrations that parse output into typed objects, the schema guarantee is the determinism that matters most — combine low temperature (or none reasoning effort) with a structured-output schema instead of relying on sampling parameters alone.

Implications

Sampling parameters and output contracts solve different problems: temperature shapes distribution, structured outputs guarantee shape. Type-safe production use should treat schema enforcement as the baseline and temperature as a refinement on top — the reverse of the historical pattern where developers tuned temperature hoping for valid JSON.

Extended Thinking Settings (confidence: high)

Extended thinking is billed as output tokens and its default budget can reach tens of thousands per request. Claude Code lowers it via the effort level (/effort), disables thinking in /config, or — on models with a fixed thinking budget — caps it with the MAX_THINKING_TOKENS environment variable (e.g. MAX_THINKING_TOKENS=8000). Adaptive-reasoning models ignore nonzero budgets, so effort levels are the control there; models that always use extended thinking (Fable 5) cannot disable it.

Implications

Reasoning-effort control is a first-class cost knob, parallel to reasoning-effort on OpenAI’s API. On fixed-budget models the budget can be capped numerically; on adaptive models only effort levels work — the two mechanisms are not interchangeable. Combined with token-usage-reduction, effort tuning targets the single largest per-request output-token cost.

prompt-engineering-guide | prompt-engineering-techniques | llm-models-guide | structured-outputs | self-consistency

Sources

• raw/prompts/articles/promptingguide-pt-introduction-settings.md
• raw/prompts/articles/openai-structured-outputs-guide.md
• raw/prompts/articles/claude-code-costs-reduce-token-usage.md
• raw/external/huggingface-co-advanced-prompt-engineering-bc976d6b.md