LangGraph Performance (Python)
Making LangGraph graphs faster and cheaper: fewer model calls, parallel super-steps and Send fan-out with max_concurrency, batching, streaming, node caching, checkpoint durability and state size.
Checked against langgraph 1.2.22 and langchain 1.5.0 (2026-11-19). This page lists the levers LangGraph gives you, roughly in order of effect. It quotes no benchmark figures: measure your own graph with LangSmith traces, which show the time spent in each node and model call.
1. Make fewer model calls
Section titled “1. Make fewer model calls”In most agents the model calls dominate run time and cost, so start there.
- Cap the loop. Set
recursion_limiton every agent run (the default, 10007, does not stop a looping agent in practice). Oncreate_agent,ModelCallLimitMiddleware(run_limit=...)andToolCallLimitMiddleware(...)give finer limits. - Keep the context short.
SummarizationMiddlewareandContextEditingMiddlewareshrink long histories; in a hand-built graph, trim messages before the model call. - Offer fewer tools.
LLMToolSelectorMiddlewarepicks a subset of a large tool list per call with a small model. - Use a smaller model for simple steps. Route easy requests to
claude-haiku-5-5(or similar) in a separate node, or switch per call inwrap_model_callwithrequest.override(model=...).
The middleware are described in Chapter 8.
2. Run independent work in parallel
Section titled “2. Run independent work in parallel”LangGraph runs every node that is ready in the same super-step concurrently: two nodes with edges from the same parent, or the targets of a list of Send objects. Sync nodes run in a thread pool; async nodes run on the event loop, which is the better choice for I/O-bound work.
config["max_concurrency"] caps how many tasks run at once, which keeps a large fan-out within a provider’s rate limit:
import asyncio, operator, timefrom typing import Annotatedfrom typing_extensions import TypedDictfrom langgraph.graph import StateGraph, START, ENDfrom langgraph.types import Send
class State(TypedDict, total=False): urls: list[str] pages: Annotated[list[str], operator.add] summary: str
class FetchInput(TypedDict): url: str
running = {"now": 0, "peak": 0}
async def fetch(state: FetchInput) -> dict: running["now"] += 1 running["peak"] = max(running["peak"], running["now"]) await asyncio.sleep(0.1) # stands in for an HTTP or model call running["now"] -= 1 return {"pages": [f"content of {state['url']}"]}
def fan_out(state: State) -> list[Send]: return [Send("fetch", {"url": u}) for u in state["urls"]]
def summarise(state: State) -> dict: return {"summary": f"{len(state['pages'])} pages"}
graph = ( StateGraph(State) .add_node("fetch", fetch) .add_node("summarise", summarise) .add_conditional_edges(START, fan_out, ["fetch"]) .add_edge("fetch", "summarise") .add_edge("summarise", END) .compile())
urls = [f"https://example.com/{i}" for i in range(20)]start = time.perf_counter()result = asyncio.run(graph.ainvoke({"urls": urls}, {"max_concurrency": 5}))print(result["summary"], "peak concurrency:", running["peak"]) # 20 pages peak concurrency: 5print(f"{time.perf_counter() - start:.1f}s") # about 0.4s: 4 waves of 5Notes:
- Return the
Sendobjects from a routing function (add_conditional_edges), or from a node asCommand(goto=[Send(...), ...]); a node that returns a bare list ofSendfails. fetchwrites topages, which has anoperator.addreducer; without one, parallel writes to the same key raiseInvalidUpdateError.summariseruns once, after everyfetchtask in the super-step has finished. For fan-ins across several steps,add_node(..., defer=True)delays a node until all other work has drained (StateGraph reference).Send(..., timeout=...)limits one task; see the Command and Send reference.
3. Batch independent runs
Section titled “3. Batch independent runs”batch/abatch run many inputs through the graph; max_concurrency applies here too:
import asynciofrom typing_extensions import TypedDictfrom langgraph.graph import StateGraph, STARTfrom langgraph.checkpoint.memory import InMemorySaver
class State(TypedDict, total=False): text: str label: str
async def classify(state: State) -> dict: await asyncio.sleep(0.05) return {"label": "long" if len(state["text"]) > 20 else "short"}
graph = StateGraph(State).add_node("classify", classify).add_edge(START, "classify").compile()
inputs = [{"text": t} for t in ["hi", "a much longer piece of text here", "ok"]]results = asyncio.run(graph.abatch(inputs, config={"max_concurrency": 2}))print([r["label"] for r in results]) # ['short', 'long', 'short']
# With a checkpointer, each input needs its own thread:saved = StateGraph(State).add_node("classify", classify).add_edge(START, "classify").compile(checkpointer=InMemorySaver())configs = [{"configurable": {"thread_id": f"doc-{i}"}} for i in range(len(inputs))]print(len(asyncio.run(saved.abatch(inputs, configs, durability="exit")))) # 34. Stream for perceived latency
Section titled “4. Stream for perceived latency”Streaming does not make a run finish sooner, but users see output earlier. stream_mode="messages" yields model tokens as they arrive; "updates" yields each node’s output when it finishes; "custom" carries progress events from inside a node. See Chapter 6 and the streaming modes reference.
5. Cache deterministic steps
Section titled “5. Cache deterministic steps”add_node(..., cache_policy=CachePolicy(ttl=...)) with compile(cache=InMemoryCache()) skips a node whose input it has seen before. It suits expensive deterministic work (retrieval, parsing, embeddings) rather than model calls whose output should vary. Options, keys and clearing are in Node policies; SqliteCache and RedisCache (shared across replicas) are in the cache backends reference.
Provider-side prompt caching (for example Anthropic’s) is separate and is configured on the chat model, not in LangGraph.
6. Write fewer, smaller checkpoints
Section titled “6. Write fewer, smaller checkpoints”- Durability. The default,
durability="async", writes a checkpoint after every super-step without blocking the next one.durability="exit"writes only when the run ends or pauses, which suits short runs that need no mid-run recovery. Pass it toinvoke/stream, notcompile(). (Chapter 10.) - State size. Every checkpoint serialises the channels that changed. Keep large documents, files and embeddings out of graph state: store them elsewhere (or in the Store) and keep IDs in state.
- Append-heavy channels. A long message list or event log is re-serialised in full whenever it changes.
DeltaChannel(beta, 1.2+) stores only each step’s writes and rebuilds the value on read (Channels reference). - Old checkpoints.
PostgresSaver.prune(thread_ids, strategy="keep_latest")drops all but the latest checkpoint of finished threads (checkpointers reference). - Narrow node inputs.
add_node(..., input_schema=...)gives a node only the keys it needs, which also keeps traces small. - Connection pooling. With Postgres under concurrent load, give the saver an
AsyncConnectionPoolrather than a single connection (checkpointers reference).
7. Bound slow calls
Section titled “7. Bound slow calls”A hung HTTP or model call holds a worker. Put a timeout= (async nodes only) and a RetryPolicy on nodes that call external services; TimeoutPolicy(idle_timeout=...) cancels a call that stops producing tokens without capping a long but healthy one. See Node policies.
Measuring
Section titled “Measuring”- Turn on LangSmith tracing (
LANGSMITH_TRACING=true); each node and model call is a span with its duration and token counts. stream_mode="debug"(or"tasks") shows when each task starts and finishes, which makes it easy to see whether branches overlap.- For load tests, run a representative input set against
langgraph upor a staging deployment with your usual load tool, and compare traces before and after each change.