Microsoft Agent Framework (Python) — Model Providers
Imports, constructors and environment variables for every first-party chat client in agent-framework: OpenAI, Azure OpenAI, Microsoft Foundry, Foundry Local, Anthropic, Gemini, Mistral, Ollama, Bedrock, plus the Claude, GitHub Copilot and Copilot Studio agents.
Every chat client in agent-framework implements the same SupportsChatGetResponse protocol, so Agent(client=...) accepts them interchangeably. The import is always agent_framework.<provider>.<ClassName> — no Azure SDK import is required for any of these. The Azure SDK only becomes relevant for authentication (azure-identity) or for Azure-specific storage providers.
This page was verified against agent-framework 1.26.0 and the provider packages it installs (re-checked 2026-11-19; first written for 1.5.0 and the 1.0.0b260514 provider line). Each sub-package is imported lazily from the agent_framework.<provider> namespace — you install the provider package and import from agent_framework.<provider>.
Provider index
Section titled “Provider index”| Provider | Package | Import path | Status |
|---|---|---|---|
| OpenAI | agent-framework-openai | agent_framework.openai | Stable |
| Azure OpenAI | agent-framework-openai | agent_framework.openai (same client) | Stable |
| Microsoft Foundry | agent-framework-foundry | agent_framework.foundry | Stable |
| Foundry Local | agent-framework-foundry-local | agent_framework.foundry | Beta |
| Anthropic | agent-framework-anthropic | agent_framework.anthropic | Beta |
| Anthropic on Bedrock | agent-framework-anthropic | agent_framework.anthropic | Beta |
| Anthropic on Vertex | agent-framework-anthropic | agent_framework.anthropic | Beta |
| Anthropic on Foundry | agent-framework-anthropic | agent_framework.anthropic (also agent_framework.foundry) | Beta |
Claude Agent SDK (ClaudeAgent) | agent-framework-claude | agent_framework.anthropic | Beta |
| Google Gemini / Vertex AI | agent-framework-gemini | agent_framework.gemini | Beta |
| Mistral | agent-framework-mistral | agent_framework.mistral | Beta |
| Ollama | agent-framework-ollama | agent_framework.ollama | Beta |
| Amazon Bedrock (native) | agent-framework-bedrock | agent_framework.amazon | Beta |
GitHub Copilot (GitHubCopilotAgent) | agent-framework-github-copilot | agent_framework.github | Stable (2.x, Python 3.11+) |
| Copilot Studio | agent-framework-copilotstudio | agent_framework.microsoft | Beta |
OpenAI (and Azure OpenAI)
Section titled “OpenAI (and Azure OpenAI)”A single class — OpenAIChatClient — drives both OpenAI and Azure OpenAI. Passing azure_endpoint= or credential= (or an AsyncAzureOpenAI as async_client=) selects Azure. Otherwise the client uses OpenAI when an OpenAI key is available (api_key= or OPENAI_API_KEY), and falls back to the AZURE_OPENAI_* variables when it is not.
from agent_framework import Agentfrom agent_framework.openai import OpenAIChatClient
# OpenAI — reads OPENAI_API_KEY and OPENAI_CHAT_MODEL (or OPENAI_MODEL) from envagent = Agent( client=OpenAIChatClient(), instructions="You are a helpful assistant.",)response = await agent.run("Hello")print(response.text)Responses API vs Chat Completions API: OpenAIChatClient uses the Responses API (recommended — supports hosted tools like file search, code interpreter). OpenAIChatCompletionClient uses the classic Chat Completions API for OpenAI-compatible gateways that don’t support /responses.
from agent_framework.openai import OpenAIChatClient, OpenAIChatCompletionClient
responses_client = OpenAIChatClient(model="gpt-6.1-sol") # /responsescompletions_client = OpenAIChatCompletionClient(model="gpt-6.1-sol") # /chat/completionsAzure OpenAI with Entra ID (passwordless):
import osfrom agent_framework import Agentfrom agent_framework.openai import OpenAIChatClientfrom azure.identity.aio import AzureCliCredential
credential = AzureCliCredential() # or DefaultAzureCredential()agent = Agent( client=OpenAIChatClient( model=os.environ["AZURE_OPENAI_CHAT_MODEL"], azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"], api_version=os.environ.get("AZURE_OPENAI_API_VERSION"), credential=credential, ), instructions="You are a helpful assistant.",)Azure OpenAI with API key:
import osfrom agent_framework.openai import OpenAIChatClient
client = OpenAIChatClient( model=os.environ["AZURE_OPENAI_CHAT_MODEL"], azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"], api_key=os.environ["AZURE_OPENAI_API_KEY"],)Full-URL override (useful for reverse proxies): pass base_url="https://…/openai/v1" instead of azure_endpoint=.
Environment-variable cascade resolved inside the constructor:
| Argument | OpenAI env var | Azure env var |
|---|---|---|
model | OPENAI_CHAT_MODEL → OPENAI_MODEL | AZURE_OPENAI_CHAT_MODEL → AZURE_OPENAI_MODEL |
api_key | OPENAI_API_KEY | AZURE_OPENAI_API_KEY |
base_url | OPENAI_BASE_URL | AZURE_OPENAI_BASE_URL |
azure_endpoint | — | AZURE_OPENAI_ENDPOINT |
api_version | — | AZURE_OPENAI_API_VERSION |
org_id | OPENAI_ORG_ID | — |
Microsoft Foundry
Section titled “Microsoft Foundry”Microsoft Foundry (formerly Azure AI Foundry) provides project-scoped model deployments plus first-party evaluation and agent hosting. FoundryChatClient talks to the project’s OpenAI-compatible Responses endpoint. The project endpoint has the form https://<resource>.services.ai.azure.com/api/projects/<project> (copy it from the project’s overview page). An Entra ID credential is required; Foundry projects do not take API keys here.
from agent_framework import Agentfrom agent_framework.foundry import FoundryChatClientfrom azure.identity.aio import AzureCliCredential
async with AzureCliCredential() as credential: agent = Agent( client=FoundryChatClient( project_endpoint="https://<resource>.services.ai.azure.com/api/projects/<project>", model="gpt-6-luna", credential=credential, ), instructions="You are a helpful assistant.", ) response = await agent.run("Summarise the Microsoft Agent Framework in one line.")Env vars: FOUNDRY_PROJECT_ENDPOINT, FOUNDRY_MODEL.
If you already hold an AIProjectClient, pass it directly and skip endpoint/credential:
from azure.ai.projects import AIProjectClient
project = AIProjectClient(endpoint=..., credential=...)client = FoundryChatClient(project_client=project, model="gpt-6-luna")Service-managed agents. Use FoundryAgent when you want the agent’s identity, threads, and tool definitions to live in Foundry (not in your process):
from agent_framework.foundry import FoundryAgent
foundry_agent = FoundryAgent( project_endpoint="https://<resource>.services.ai.azure.com/api/projects/<project>", agent_name="contract-reviewer", agent_version="1.0", credential=credential,)response = await foundry_agent.run("Review contract.pdf")Foundry Local
Section titled “Foundry Local”FoundryLocalClient targets Foundry Local, Microsoft’s on-device inference runtime. The foundry CLI must be installed: with the default bootstrap=True the client starts the local service and, with prepare_model=True, downloads and loads the model on first use (which can take minutes). The model can also come from FOUNDRY_LOCAL_MODEL. Useful for offline development and data that must not leave the machine.
from agent_framework.foundry import FoundryLocalClientfrom agent_framework import Agent
agent = Agent( client=FoundryLocalClient(model="Phi-3.5-mini-instruct"), instructions="You are a private offline assistant.",)Anthropic
Section titled “Anthropic”Four transports — the Anthropic API, Anthropic on AWS Bedrock, Anthropic on Google Vertex AI and Anthropic models deployed in Microsoft Foundry. All implement the same chat-client protocol, so only the construction differs. AnthropicClient reads ANTHROPIC_API_KEY and ANTHROPIC_CHAT_MODEL.
from agent_framework import Agentfrom agent_framework.anthropic import ( AnthropicClient, # api.anthropic.com; reads ANTHROPIC_API_KEY AnthropicBedrockClient, # Anthropic via AWS Bedrock AnthropicVertexClient, # Anthropic via Google Vertex AI (region=, project_id=) AnthropicFoundryClient, # Anthropic models in Microsoft Foundry (resource=, api_key= or azure_ad_token_provider=))
agent = Agent( client=AnthropicClient(model="claude-sonnet-5-5"), instructions="You are a helpful assistant.",)Use ClaudeAgent (package agent-framework-claude, built on the Claude Agent SDK and the Claude Code CLI) instead of a chat client when you want Claude to drive its own tool loop, subagents and session continuity. ClaudeAgentOptions.model takes the CLI’s aliases ("sonnet", "opus", "haiku"):
from agent_framework.anthropic import ClaudeAgent, ClaudeAgentOptions
async with ClaudeAgent( instructions="You are a careful Python reviewer.", default_options=ClaudeAgentOptions(model="sonnet", permission_mode="default"),) as claude: response = await claude.run("Refactor utils.py to use dataclasses.")Ollama
Section titled “Ollama”Local models via the Ollama daemon. Reads OLLAMA_HOST and OLLAMA_MODEL.
from agent_framework import Agentfrom agent_framework.ollama import OllamaChatClient
agent = Agent( client=OllamaChatClient(model="llama3.1"), instructions="You are a helpful assistant.",)Custom base URL (non-default daemon):
OllamaChatClient(model="llama3.1", host="http://gpu-host:11434")Google Gemini
Section titled “Google Gemini”GeminiChatClient uses the google-genai SDK, so one class covers the Gemini Developer API (api_key= or GOOGLE_API_KEY) and Vertex AI (enterprise=True with project= and location=, or GOOGLE_GENAI_USE_ENTERPRISE, GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_LOCATION). The model comes from model= or GOOGLE_MODEL.
from agent_framework import Agentfrom agent_framework.gemini import GeminiChatClient
agent = Agent( client=GeminiChatClient(model="gemini-3.8-flash"), instructions="You are a helpful assistant.",)GeminiChatOptions adds Gemini settings such as thinking_config (a ThinkingConfig).
Mistral
Section titled “Mistral”from agent_framework import Agentfrom agent_framework.mistral import MistralChatClient
agent = Agent( client=MistralChatClient(model="mistral-medium-latest"), # or MISTRAL_CHAT_MODEL; key from MISTRAL_API_KEY instructions="You are a helpful assistant.",)server_url= (or MISTRAL_SERVER_URL) points the client at a self-hosted or regional endpoint.
Amazon Bedrock (native)
Section titled “Amazon Bedrock (native)”The agent_framework.amazon namespace exposes the native Bedrock Converse API (for Nova, Titan, Mistral, Cohere, DeepSeek and other models on Bedrock). Credentials come from the usual boto3 chain unless you pass access_key= / secret_key= / session_token= or a boto3_session=; region and model can come from BEDROCK_REGION and BEDROCK_CHAT_MODEL. For Claude on Bedrock you can also use AnthropicBedrockClient from the Anthropic provider, which exposes Anthropic-specific options such as extended thinking.
from agent_framework import Agentfrom agent_framework.amazon import BedrockChatClient
agent = Agent( client=BedrockChatClient(model="amazon.nova-pro-v1:0", region="us-east-1"), instructions="You are a helpful assistant.",)Guardrails are a chat option (guardrailConfig), not a constructor argument. BedrockGuardrailConfig is a TypedDict using the Bedrock API’s own camelCase keys:
from agent_framework import Agentfrom agent_framework.amazon import BedrockChatClient, BedrockGuardrailConfig
guardrail: BedrockGuardrailConfig = { "guardrailIdentifier": "gr-xyz", "guardrailVersion": "1", "trace": "enabled",}
agent = Agent( client=BedrockChatClient(model="amazon.nova-pro-v1:0", region="us-east-1"), instructions="You are a helpful assistant.", default_options={"guardrailConfig": guardrail}, # or per call via options=)GitHub Copilot
Section titled “GitHub Copilot”agent-framework-github-copilot wraps the GitHub Copilot SDK as an agent in its own right — GitHubCopilotAgent — rather than as a chat client for Agent. It drives the Copilot CLI, which needs a GitHub account with Copilot access. GitHubCopilotOptions covers model, system_message, mcp_servers, skill_directories, permission callbacks and more:
from agent_framework.github import GitHubCopilotAgent # agent_framework_github_copilot
agent = GitHubCopilotAgent( "Pair-programmer mode.", default_options={"model": "gpt-6.1-sol"},)response = await agent.run("Explain this stack trace.")Copilot Studio
Section titled “Copilot Studio”from agent_framework.microsoft import CopilotStudioAgent # agent_framework_copilotstudio
agent = CopilotStudioAgent( environment_id="<power platform environment id>", agent_identifier="<copilot schema name>", client_id="<entra app client id>", tenant_id="<tenant id>", # or pass a pre-built `client=` / `settings=`; values can also come from env vars)Swap providers at runtime
Section titled “Swap providers at runtime”Because every client satisfies SupportsChatGetResponse, the agent stays identical — only the client changes:
import osfrom agent_framework import Agentfrom agent_framework.openai import OpenAIChatClientfrom agent_framework.anthropic import AnthropicClientfrom agent_framework.ollama import OllamaChatClient
def build_client(): provider = os.environ.get("LLM_PROVIDER", "openai") if provider == "anthropic": return AnthropicClient(model="claude-sonnet-5-5") if provider == "ollama": return OllamaChatClient(model="llama3.1") return OpenAIChatClient(model="gpt-6.1-sol")
agent = Agent(client=build_client(), instructions="Helpful assistant.")Embeddings
Section titled “Embeddings”Every provider with embedding support exposes an *EmbeddingClient alongside its chat client. All satisfy SupportsGetEmbeddings and return the same GeneratedEmbeddings[list[float], EmbeddingGenerationOptions] type, so you can swap them freely.
from agent_framework.openai import OpenAIEmbeddingClientfrom agent_framework.ollama import OllamaEmbeddingClientfrom agent_framework.foundry import FoundryEmbeddingClientfrom agent_framework.amazon import BedrockEmbeddingClient
embeddings = OpenAIEmbeddingClient(model="text-embedding-3-large")result = await embeddings.get_embeddings(["hello", "world"])
for vec in result: print(vec.dimensions, vec.model, vec.vector[:4])# `result.usage` is a UsageDetails (dict-like); key names vary by provider.print("tokens used:", (result.usage or {}).get("total_tokens", 0))The Embedding and GeneratedEmbeddings types
Section titled “The Embedding and GeneratedEmbeddings types”get_embeddings always returns a GeneratedEmbeddings — it subclasses list[Embedding], so iteration, indexing, and len(...) work as you’d expect. Each Embedding is generic over the vector type (usually list[float], sometimes list[int] or bytes for quantised providers):
from agent_framework import Embedding, GeneratedEmbeddings
# Constructing an Embedding directly — dimensions default to len(vector).single = Embedding(vector=[0.1, 0.2, 0.3], model="text-embedding-3-small")assert single.dimensions == 3
# Wrapping a list of them as a GeneratedEmbeddings — this is the shape your# code should handle from every *EmbeddingClient.batch = GeneratedEmbeddings( [single, Embedding(vector=[0.4, 0.5, 0.6])], usage={"prompt_tokens": 10, "total_tokens": 10},)assert len(batch) == 2Picking dimensions (OpenAI text-embedding-3-*)
Section titled “Picking dimensions (OpenAI text-embedding-3-*)”The OpenAI text-embedding-3-* models support a dimensions parameter that lets you request a shorter vector without a separate model. Pass it through OpenAIEmbeddingOptions:
from agent_framework.openai import OpenAIEmbeddingClient, OpenAIEmbeddingOptions
client = OpenAIEmbeddingClient(model="text-embedding-3-large")
# 256-dim embeddings — cheaper to store, 4x smaller vector DB footprint.result = await client.get_embeddings( ["hello"], options=OpenAIEmbeddingOptions(dimensions=256, encoding_format="float"),)assert result[0].dimensions == 256Provider-neutral duck typing
Section titled “Provider-neutral duck typing”Any code that embeds can take the SupportsGetEmbeddings protocol instead of a concrete class — type checkers will accept every first-party client and any subclass of BaseEmbeddingClient you write yourself:
from agent_framework import SupportsGetEmbeddings
async def index(client: SupportsGetEmbeddings, docs: list[str]) -> list[list[float]]: result = await client.get_embeddings(docs) return [e.vector for e in result]Custom embedding client
Section titled “Custom embedding client”Subclass BaseEmbeddingClient when you need to wrap a provider that isn’t first-party or want to add batching/caching/shadowing on top of an existing one. The full pattern lives in the Advanced page; the short version:
from agent_framework import BaseEmbeddingClient, Embedding, GeneratedEmbeddings
class StubEmbeddingClient(BaseEmbeddingClient): OTEL_PROVIDER_NAME = "stub"
async def get_embeddings(self, values, *, options=None): return GeneratedEmbeddings( [Embedding(vector=[0.0] * 8, model="stub") for _ in values], options=options, )Provider-neutral request options — ChatOptions
Section titled “Provider-neutral request options — ChatOptions”Every provider-specific options TypedDict (OpenAIChatOptions, AnthropicChatOptions, etc.) extends the generic ChatOptions base. When you’re writing code that should work against any client, type against ChatOptions — it captures the common denominator across all providers. All fields are optional (total=False), so you only set what you need.
from agent_framework import ChatOptions
common: ChatOptions = { "model": "gpt-6-luna", "temperature": 0.2, "top_p": 0.9, "max_tokens": 2_000, "stop": ["\n\nUSER:"], "seed": 1337, "frequency_penalty": 0.0, "presence_penalty": 0.1, "user": "user-42", # end-user id for provider-side abuse tracking "metadata": {"env": "prod"}, # attached to the request; provider may echo it back}
response = await agent.run("Summarise the dataset.", options=common)The fields ChatOptions defines — every first-party client accepts at least this subset:
| Field | Type | Purpose |
|---|---|---|
model | str | Override the model for this one call |
temperature / top_p | float | Sampling temperature and nucleus probability |
max_tokens | int | Upper bound on output tokens |
stop | str | Sequence[str] | Stop sequences |
seed | int | Reproducibility hint (providers may ignore) |
logit_bias | dict[str | int, float] | Per-token bias map |
frequency_penalty / presence_penalty | float | Repetition / novelty penalties |
tools | Sequence[FunctionTool | Callable | …] | None | Per-call tool list (additive over the agent’s) |
tool_choice | ToolMode | "auto" | "required" | "none" | Force a tool, require any tool, or disable tools |
allow_multiple_tool_calls | bool | Permit the model to request more than one tool per turn |
response_format | type[BaseModel] | Mapping | None | Structured output — pass a Pydantic class or a JSON schema |
metadata | dict[str, Any] | Free-form metadata the provider round-trips on the request |
user | str | End-user identifier (OpenAI / Anthropic use it for abuse detection) |
store | bool | Provider-side conversation storage (OpenAI Responses API, Foundry) |
conversation_id | str | Continue a provider-managed conversation |
instructions | str | Per-call system instructions override |
Forcing a specific tool with ToolMode
Section titled “Forcing a specific tool with ToolMode”tool_choice accepts either the shorthand literal strings or a ToolMode TypedDict for fine-grained control over when and which tools the model may call. ToolMode has three optional keys:
| Key | Type | When to use |
|---|---|---|
mode | "auto" | "required" | "none" | Required. "auto" lets the model decide; "required" forces at least one tool call; "none" disables all tools. |
required_function_name | str | Only valid with mode="required". Forces the model to call exactly this tool. |
allowed_tools | list[str] | Valid with "auto" or "required". Restricts which tools the model can see for this call. |
Force any tool — mode="required"
Section titled “Force any tool — mode="required"”Use when you want to guarantee the model calls at least one tool (not just returns a text response):
import asynciofrom typing import Annotatedfrom agent_framework import Agent, ChatOptions, ToolMode, toolfrom agent_framework.openai import OpenAIChatClient
@tooldef get_weather(location: Annotated[str, "City name"]) -> str: """Return current weather for a city.""" return f"The weather in {location} is 22°C."
@tooldef search_web(query: Annotated[str, "Search query"]) -> str: """Search the web for up-to-date information.""" return f"Results for {query}: ..."
agent = Agent( client=OpenAIChatClient(), instructions="You are a research assistant.", tools=[get_weather, search_web],)
async def main() -> None: # Model MUST call a tool — useful when the pipeline expects a structured tool result response = await agent.run( "What's the weather in Amsterdam?", options=ChatOptions(tool_choice="required"), ) print(response.text)
asyncio.run(main())Pin to one specific tool — required_function_name
Section titled “Pin to one specific tool — required_function_name”Use when you know exactly which tool should run. The model fills in the arguments:
import asynciofrom agent_framework import Agent, ChatOptions, ToolMode, toolfrom agent_framework.openai import OpenAIChatClient
@tooldef lookup_customer(customer_id: str) -> str: """Look up a customer by ID.""" return f"Customer {customer_id}: Alice Smith"
@tooldef cancel_subscription(customer_id: str, reason: str) -> str: """Cancel a customer's subscription.""" return f"Subscription for {customer_id} cancelled: {reason}"
agent = Agent( client=OpenAIChatClient(), instructions="You are a billing assistant.", tools=[lookup_customer, cancel_subscription],)
async def main() -> None: # Force lookup_customer — the model must call this tool first regardless of what it thinks response = await agent.run( "Customer c-4821 wants to cancel.", options=ChatOptions( tool_choice=ToolMode(mode="required", required_function_name="lookup_customer") ), ) print(response.text)
asyncio.run(main())Restrict visible tools — allowed_tools
Section titled “Restrict visible tools — allowed_tools”Use when only a subset of the agent’s tools is relevant for a particular turn. The model can only see and call the listed tools:
import asynciofrom agent_framework import Agent, ChatOptions, ToolMode, toolfrom agent_framework.openai import OpenAIChatClient
@tooldef search_products(query: str) -> str: """Search the product catalog.""" return f"Found 12 results for {query}"
@tooldef add_to_cart(product_id: str, qty: int) -> str: """Add a product to the shopping cart.""" return f"Added {qty}x {product_id} to cart"
@tooldef checkout(cart_id: str) -> str: """Complete the checkout flow.""" return f"Order placed for cart {cart_id}"
agent = Agent( client=OpenAIChatClient(), instructions="You are a shopping assistant.", tools=[search_products, add_to_cart, checkout],)
async def main() -> None: # Discovery phase — only search is allowed await agent.run( "Find red sneakers under $100.", options=ChatOptions( tool_choice=ToolMode(mode="auto", allowed_tools=["search_products"]) ), )
# Add-to-cart phase — search is still available but checkout is blocked await agent.run( "Add product SKU-4821 to my cart.", options=ChatOptions( tool_choice=ToolMode(mode="required", allowed_tools=["add_to_cart"]) ), )
# Checkout phase — only checkout is visible await agent.run( "Place the order.", options=ChatOptions( tool_choice=ToolMode(mode="required", allowed_tools=["checkout"]) ), )
asyncio.run(main())Disable tools entirely — mode="none"
Section titled “Disable tools entirely — mode="none"”Use when you want a plain text response without any tool calls — for example a final summary step after all research is done:
import asynciofrom agent_framework import Agent, ChatOptions, toolfrom agent_framework.openai import OpenAIChatClient
@tooldef search_web(query: str) -> str: """Search the web.""" return f"Results for {query}: ..."
agent = Agent( client=OpenAIChatClient(), instructions="You are a research assistant.", tools=[search_web],)
async def main() -> None: session = agent.create_session()
# Research phase — tools enabled await agent.run("Research the history of quantum computing.", session=session)
# Summary phase — force plain text, no tool calls summary = await agent.run( "Now give me a concise two-paragraph summary of what you found.", session=session, options=ChatOptions(tool_choice="none"), ) print(summary.text)
asyncio.run(main())mode="none" disables tools entirely for one call (useful when you want a pure summary of the conversation without further tool-use); mode="required" without required_function_name forces the model to pick some tool.
Structured output in one line
Section titled “Structured output in one line”response_format accepts a Pydantic model — the response comes back as a typed object via response.value:
from pydantic import BaseModelfrom agent_framework import ChatOptions
class Extracted(BaseModel): sentiment: str score: float topics: list[str]
options: ChatOptions = {"response_format": Extracted}response = await agent.run( "Summarise this review: 'Fast shipping, but the fabric snagged.'", options=options,)print(response.value.sentiment, response.value.score)response.value is parsed from the response text on first access, so a provider that ignores the schema produces a validation error there rather than at the call. Check the provider’s own documentation for which models honour structured output.
Merging with client-level defaults
Section titled “Merging with client-level defaults”Defaults live in the agent’s default_options= (the same TypedDict). The call-level options= is a shallow merge on top: keys you set win, keys you omit keep the agent’s default.
from agent_framework import Agentfrom agent_framework.openai import OpenAIChatClient
agent = Agent( client=OpenAIChatClient(model="gpt-6-luna"), default_options={"temperature": 0.7},)
# Inherits temperature=0.7; only max_tokens is overridden.await agent.run("Draft a tweet.", options={"max_tokens": 280})Swapping the model on an existing client
Section titled “Swapping the model on an existing client”Every client accepts model= at construction and remembers it. But you can also override the model for a single call without building a new client — use options= on the agent run:
from agent_framework import Agentfrom agent_framework.openai import OpenAIChatClient, OpenAIChatOptions
default_client = OpenAIChatClient(model="gpt-6-luna")
agent = Agent(client=default_client, instructions="…")
# Upgrade to a bigger model just for this one tricky question.response = await agent.run( "Prove Fermat's last theorem in two sentences.", options=OpenAIChatOptions(model="gpt-6.1-sol", temperature=0.2),)options is a provider-specific TypedDict — OpenAIChatOptions, OpenAIChatCompletionOptions, OpenAIEmbeddingOptions, and the equivalents under agent_framework.anthropic, agent_framework.amazon, agent_framework.ollama, etc. IDE autocomplete drives you through every tunable. The values merge with the client’s defaults; anything you omit stays as the client was constructed.
When to reach for the provider-specific TypedDict
Section titled “When to reach for the provider-specific TypedDict”Use the generic ChatOptions whenever the knobs you need are common across providers — that keeps the call site interoperable. Drop to the provider-specific dict only when you need a feature the base can’t describe:
- OpenAI (
OpenAIChatOptions):reasoning,verbosity,prompt_cache_key,prompt_cache_retention,prompt_cache_options,service_tier,safety_identifier,top_logprobs,truncation,background,continuation_token,include,max_tool_calls,prompt,stream_options. - Anthropic (
AnthropicChatOptions):thinking,top_k,container,service_tier,additional_beta_flags. - Bedrock (
BedrockChatOptions, Converse API camelCase):guardrailConfig,additionalModelRequestFields,performanceConfig,promptVariables,requestMetadata.
Mixing them is fine — a provider-specific dict is a superset of ChatOptions, so code typed against the base still accepts it.
Building your own chat client
Section titled “Building your own chat client”For a provider that isn’t in the first-party list, or to wrap an existing client with caching / shadow traffic / logging, subclass BaseChatClient. Implement one method — _inner_get_response — and inherit middleware, telemetry, and the function calling loop for free. See the full recipe in Advanced → Custom chat client.
Picking a provider
Section titled “Picking a provider”- Prototyping —
OpenAIChatClient()orOllamaChatClient(model="llama3.1"). Neither requires Azure tooling. - Azure-native deployments —
OpenAIChatClientwithazure_endpoint+credential, orFoundryChatClientif you’re already on a Foundry project (evaluation, service-managed agents, private networking). - Cross-cloud Claude —
AnthropicClientfor Anthropic direct;AnthropicBedrockClient,AnthropicVertexClientorAnthropicFoundryClientto keep traffic in AWS, Google Cloud or Azure. - Offline / compliance —
OllamaChatClientorFoundryLocalClient. - Multi-provider fallback — build a thin factory (example above) and let an env var pick at startup; the rest of your agent code stays unchanged.
Revision history
Section titled “Revision history”| Date | Version | Changes |
|---|---|---|
| 2026-10-24 | 1.22.0 | Added Gemini, Mistral and AnthropicFoundryClient; corrected the Foundry project-endpoint format, the GitHub Copilot status (stable 2.x), Azure/OpenAI routing, and the provider-specific option names (Bedrock options are camelCase; Anthropic has no cache-control option); removed an unsupported claim about structured-output fallback. |