Advanced framework guide

Advanced DSPy

Scale DSPy from static tool demos to governed agent workloads with per-call dynamic tool selection, explicit LLM-call policy, tracing, and deployment boundaries.

← Basic DSPy usage

Advanced page: per-call dynamic tool selection

dspy.ReAct bakes each tool's name and description into its instruction prompt once, at construction, and freezes self.tools into a plain dict — there is no runtime hook to swap either mid-run. (DSPy's dspy.utils.callback.BaseCallback exposes on_tool_start/on_tool_end and on_module_start/on_module_end, but those fire for observability around an already-selected tool call or an already-finished top-level run — never before the model decides which tool to call next, unlike Strands' BeforeModelCallEvent or Google ADK's before_model_callback.) So the deepest re-selection tier DSPy permits is per top-level call: rebuild a fresh dspy.ReAct for each call, with tools re-selected for that call's query — the same tier CrewAI, Agno, and Smolagents get.

from agent_gantry.dspy import DSPyAdapter

builder = DSPyAdapter(gantry).agent_builder(
    "question -> answer",
    max_iters=5,
)

# Each call re-selects tools for THAT call's query and builds a fresh ReAct.
react = await builder.build("What's the weather in Tokyo?")
pred = react(question="What's the weather in Tokyo?")

Prefer a fixed tool set for a single run instead? The static path still works — select once and hand the framework's native objects to dspy.ReAct(signature, tools=[...]):

candidate_tools = await DSPyAdapter(gantry).select(
    query=user_message,
    limit=8,
)
react = dspy.ReAct("question -> answer", tools=candidate_tools)

Sync vs. async tool invocation

dspy.Tool supports a synchronous __call__ (the path ReAct.forward — i.e. plain react(...) — uses) and an async acall (the path ReAct.aforward / await react.acall(...) uses). DSPyAdapter wraps every tool with a synchronous function backed by Gantry's loop-safe sync bridge, so it works correctly under both entry points with zero DSPy configuration — no need to set dspy.configure(allow_tool_async_sync_conversion=True) or remember to always call await react.acall(...). Execution still always routes through gantry.execute (retries, timeouts, circuit breakers, security policy).

LLM calls and model policy

Centralize LLM calls through dspy.LM(...) (LiteLLM-backed, so one interface covers OpenAI, Anthropic, and most other providers) when possible; otherwise add middleware around the framework call via dspy.utils.callback.BaseCallback. The policy should select the provider, enforce budget ceilings, redact sensitive fields, emit token metrics, and decide whether tool results should be summarized before returning to the model.

response = llm_policy.complete(
    framework="DSPy",
    model="primary-or-fallback-model",
    messages=messages,
    tools=framework_tools,
    trace_id=trace_id,
)

Production patterns

Observability: attach trace IDs to every DSPy run, LLM call, and tool execution — dspy.utils.callback.BaseCallback's on_module_start/on_module_end and on_tool_start/on_tool_end hooks are a natural place to record them alongside gantry.on_tool_call(cb).
Safety: keep high-risk tools behind explicit allowlists, human approval, or dry-run executors — Gantry's SecurityPolicy still gates every call routed through gantry.execute, even when the tool surface is re-selected per call.
Resilience: add provider fallback for LLM calls (DSPy's dspy.LM supports multiple backends via LiteLLM) and executor fallback for remote MCP/A2A tools.
Evaluation: DSPy's own optimizers (e.g. dspy.MIPROv2) replay representative examples to tune prompts/few-shot demos — pin the tool snapshot used during optimization runs before broadening dynamic retrieval in production.
Testing offline: dspy.utils.dummies.DummyLM scripts a fixed sequence of tool calls and answers, so integration tests can exercise a full ReAct run — including a Gantry-sourced tool actually executing — with no network access or API key.