Mahi
My notes, blogs and thoughts on tech

Important LLM parameters in a LangGraph implementation

When building applications with Large Language Models, the parameters we pass are as important as the prompts provided to the LLM. In the context of LangGraph, where LLM calls are embedded within structured, multi-node graphs, this becomes critical - e.g., poorly tuned temperature parameter in a tool-calling node can break the entire pipeline.

Unlike single-shot LLM calls, LangGraph workflows demand node-aware parameter thinking — because the same model behaves very differently depending on whether it's acting as a router, an agent, or a creative generator. In this blog, I'll walk through the key LLM parameters we need to understand, how to set them in LangGraph, and how to tune them for each type of node in the graph.

In LangGraph, LLM calls happen inside nodes, typically via LangChain's chat model integrations. Parameters are set either at model initialisation or at invocation time. Here are some of the parameters when making LLM calls in LangGraph.

Model Initialisation Parameters

These are set once when defining LLM inside a node.

from langchain_anthropic import ChatAnthropic

llm = ChatAnthropic(
    model="claude-sonnet-4-5-20250929",
    temperature=0.7,
    max_tokens=1024,
    top_p=0.9,
    top_k=40,
)

Key parameters here:

  • temperature — Controls creativity. Set low (0.1–0.3) for tool-calling or structured output nodes; higher for generative nodes
  • max_tokens — Critical in graphs where output feeds into the next node — too long can bloat LangGraph state
  • top_p / top_k — Fine-tune sampling; often left at defaults unless we notice repetition
  • model — Swappable per node, so we can use a cheaper/faster model for simple routing nodes and a more powerful one for reasoning nodes

Invocation-Time Parameters

Override per call using configurable fields.

from langchain_core.runnables import ConfigurableField

# For dynamic parameters (e.g. temperature varies per node),
# declare them configurable at model init time...
llm = ChatAnthropic(model="claude-sonnet-4-5-20250929").configurable_fields(
    temperature=ConfigurableField(id="temperature")
)

# ...then override via config at call time
response = llm.invoke(messages, config={"configurable": {"temperature": 0.0}})

This is useful when the same model instance needs different behaviour in different nodes.

System Prompt as a Parameter

In LangGraph, the system prompt is often part of the graph state or a ChatPromptTemplate, making it a dynamic "parameter".

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "{system_instruction}"),
    ("human", "{user_input}")
])

# System instruction can change per node or per run
chain = prompt | llm

We can route different system prompts based on graph state in this way, effectively changing model behaviour without touching sampling parameters.

Tool Calling Parameters

A major use case in LangGraph is binding tools to LLMs for agent nodes (Agentic AI).

tools = [search_tool, calculator_tool]
llm_with_tools = llm.bind_tools(tools)

# In the node:
def agent_node(state):
    response = llm_with_tools.invoke(state["messages"])
    return {"messages": [response]}

The temperature value at LLM initialisation should almost always be 0 (or close) for tool-calling nodes to ensure consistent, parseable outputs.

LangGraph-Specific Configuration

These aren't LLM parameters per se, but they directly affect LLM call behaviour in a graph.

Recursion Limit — Limits how many times a cycle (e.g., agent loop) can repeat:

graph.invoke(input, config={"recursion_limit": 10})

RunnableConfig — Pass metadata, tags, and callbacks per invocation.

from langchain_core.runnables import RunnableConfig

config = RunnableConfig(
    tags=["production"],
    metadata={"session_id": "abc123"},
    callbacks=[the_callback_handler]
)

Checkpointing — Persists graph state between runs; affects how LLM context is reconstructed across sessions.

from langgraph.checkpoint.memory import MemorySaver
graph = builder.compile(checkpointer=MemorySaver())

Ultimately, building reliable LangGraph applications isn't just about writing good prompts or designing clean graph structures — it's about understanding that every parameter is a decision, and every decision compounds across nodes. Treat the LLM parameters not as boilerplate to copy from a tutorial, but as levers to deliberately tune for each role a model plays in the graph.