Advanced framework guide

Advanced AutoGen

Scale AutoGen from static tool demos to governed agent workloads with semantic selection, explicit LLM-call policy, tracing, and deployment boundaries.

← Basic AutoGen usage

Advanced page: dynamic tool selection

Before each AutoGen run, ask Gantry for a context-aware subset of tools. Combine request metadata, tenant policy, semantic similarity, and reranker results so the agent sees a concise, relevant tool surface.

candidate_tools = gantry.select_tools(
    query=user_message,
    tags=["support"],
    limit=8,
)
framework_tools = [AutoGenAdapter.convert(spec) for spec in candidate_tools]

LLM calls and model policy

Centralize LLM calls when AutoGen supports custom model clients; otherwise add middleware around the framework call. The policy should select the provider, enforce budget ceilings, redact sensitive fields, emit token metrics, and decide whether tool results should be summarized before returning to the model.

response = llm_policy.complete(
    framework="AutoGen",
    model="primary-or-fallback-model",
    messages=messages,
    tools=framework_tools,
    trace_id=trace_id,
)

Production patterns

• Observability: attach trace IDs to every AutoGen run, LLM call, and tool execution.
• Safety: keep high-risk tools behind explicit allowlists, human approval, or dry-run executors.
• Resilience: add provider fallback for LLM calls and executor fallback for remote MCP/A2A tools.
• Evaluation: replay representative prompts with fixed tool snapshots before broadening dynamic retrieval.