Skip to content
PaPy

PydanticAI: Direct API — Raw Model Calls

model_request, model_request_sync, model_request_stream, model_request_stream_sync, StreamedResponseSync — make bare model calls without an Agent when you need minimal abstraction.

Verified against pydantic-ai 2.68.0 (checked 2026-11-19) — source module: pydantic_ai.direct.

pydantic_ai.direct exposes four functions that talk to a language model with no Agent overhead: no tool execution, no output validation, no message-history management, no retry loop. The only abstraction is translating your ModelMessage list into the provider’s wire format and back. This is the right layer when you’re building custom pipelines, comparing model outputs, writing SDK-level unit tests, or calling a model as part of a custom capability.

Use caseUse
Structured output validation, tool calls, retry loopAgent
History management, multi-turn chatAgent
Raw model comparison / benchmarkingmodel_request
Building a custom capability or wrappermodel_request
REPL / notebook one-shotsmodel_request_sync
Synchronous CLI toolmodel_request_stream_sync
LLM-powered library helper (no event loop expected)model_request_sync
import asyncio
from pydantic_ai import ModelRequest
from pydantic_ai.direct import model_request
async def main():
response = await model_request(
'openai:gpt-6-luna',
[ModelRequest.user_text_prompt('What is the capital of France?')],
)
print(response)
# ModelResponse(
# parts=[TextPart(content='The capital of France is Paris.')],
# usage=RequestUsage(input_tokens=14, output_tokens=8),
# model_name='gpt-6-luna',
# timestamp=datetime.datetime(...),
# )
asyncio.run(main())
from pydantic_ai import ModelRequest, ModelResponse
from pydantic_ai.direct import model_request
from pydantic_ai.models import ModelRequestParameters
from pydantic_ai.settings import ModelSettings
async def ask(prompt: str) -> ModelResponse:
return await model_request(
'anthropic:claude-haiku-5-5',
[ModelRequest.user_text_prompt(prompt)],
model_settings=ModelSettings(max_tokens=512, temperature=0.2),
)

Signature (direct.py)

async def model_request(
model: Model | KnownModelName | str,
messages: Sequence[ModelMessage],
*,
model_settings: ModelSettings | None = None,
model_request_parameters: ModelRequestParameters | None = None,
instrument: InstrumentationSettings | bool | None = None,
) -> ModelResponse: ...
  • model — any KnownModelName string, a Model instance, or a raw string.
  • messages — a sequence of ModelMessage objects (see Building messages below).
  • model_settings — same ModelSettings TypedDict you’d pass to Agent.
  • model_request_parameters — control tools, output schemas, sampling parameters.
  • instrument — True / InstrumentationSettings to emit OTel spans. Defaults to the global Agent._instrument_default.

model_request_sync — synchronous, non-streaming

Section titled “model_request_sync — synchronous, non-streaming”

Drop-in synchronous wrapper around model_request. It runs the request with run_until_complete() on the thread’s event loop (creating one if there is none), so do not call it from code that is already running inside an event loop — use await model_request(...) there.

from pydantic_ai import ModelRequest
from pydantic_ai.direct import model_request_sync
response = model_request_sync(
'google:gemini-3.8-flash',
[ModelRequest.user_text_prompt('Explain backpropagation in two sentences.')],
)
print(response.parts[0].content) # TextPart
print(response.usage.input_tokens, response.usage.output_tokens)

Useful in CLI scripts, Jupyter cells, or libraries that must expose a synchronous API.

Returns an async context manager that yields a StreamedResponse. Iterate to get ModelResponseStreamEvent objects.

import asyncio
from pydantic_ai import ModelRequest
from pydantic_ai.direct import model_request_stream
from pydantic_ai.messages import PartStartEvent, PartDeltaEvent, TextPartDelta
async def stream_response(prompt: str) -> str:
text = ''
messages = [ModelRequest.user_text_prompt(prompt)]
async with model_request_stream('openai:gpt-6-luna', messages) as stream:
async for event in stream:
if isinstance(event, PartDeltaEvent) and isinstance(event.delta, TextPartDelta):
text += event.delta.content_delta
print(event.delta.content_delta, end='', flush=True)
print()
return text
asyncio.run(stream_response('Tell me a short joke.'))

Stream events (messages.py):

EventWhen emitted
PartStartEventA new part (text, thinking, tool call) begins
PartDeltaEventToken delta arrives; .delta is TextPartDelta, ThinkingPartDelta, or ToolCallPartDelta
PartEndEventThe part is complete
FinalResultEventThe final result tool call was selected

Inside the async with block, stream.get() returns the ModelResponse assembled so far; after draining the stream it is the complete response.

import asyncio
from pydantic_ai import ModelRequest
from pydantic_ai.direct import model_request_stream
async def main():
messages = [ModelRequest.user_text_prompt('Name three rivers.')]
async with model_request_stream('openai:gpt-6-luna', messages) as stream:
async for _ in stream:
pass # drain
response = stream.get() # the ModelResponse assembled so far (all of it, once drained)
print(response.usage, response.finish_reason)
asyncio.run(main())

model_request_stream_sync — synchronous streaming

Section titled “model_request_stream_sync — synchronous streaming”

The synchronous peer of model_request_stream. Uses a background thread with its own event loop to run the async producer, exposing a standard context-manager + iterator interface.

from pydantic_ai import ModelRequest
from pydantic_ai.direct import model_request_stream_sync
from pydantic_ai.messages import PartDeltaEvent, TextPartDelta
messages = [ModelRequest.user_text_prompt('Count from 1 to 5.')]
with model_request_stream_sync('openai:gpt-6-luna', messages) as stream:
for event in stream:
if isinstance(event, PartDeltaEvent) and isinstance(event.delta, TextPartDelta):
print(event.delta.content_delta, end='', flush=True)
print()

model_request_stream_sync must be used as a context manager — accessing it outside with raises RuntimeError.

StreamedResponseSync — the sync stream wrapper

Section titled “StreamedResponseSync — the sync stream wrapper”

model_request_stream_sync returns a StreamedResponseSync. It runs no thread of its own: on entering the with block it keeps the async stream open in a long-lived task on the calling thread’s event loop and drives it as you iterate, so async clients keep their event-loop affinity:

from pydantic_ai.direct import StreamedResponseSync, model_request_stream_sync
from pydantic_ai import ModelRequest
messages = [ModelRequest.user_text_prompt('Who invented the telephone?')]
with model_request_stream_sync('openai:gpt-6-luna', messages) as stream:
# Iterate to consume
chunks = list(stream)
# After consuming the stream, inspect the assembled response
print(stream.response) # ModelResponse
print(stream.model_name) # e.g. 'gpt-6-luna'
print(stream.timestamp) # datetime of the response

Key attributes (source: direct.py):

AttributeTypeNotes
responseModelResponseCurrent / final assembled response
model_namestrModel name from the provider
timestampdatetimeTimestamp from the provider

The background thread is joined on __exit__; it is safe to inspect the response immediately after the with block.

ModelRequest (messages.py) is the standard container. The convenience constructor ModelRequest.user_text_prompt(...) builds a request with a UserPromptPart:

from pydantic_ai import ModelRequest
from pydantic_ai.messages import (
ModelRequest,
UserPromptPart,
SystemPromptPart,
InstructionPart,
BinaryImage,
ImageUrl,
)
# Plain text turn
turn = ModelRequest.user_text_prompt('What is 2 + 2?')
# Text turn with a system prompt
multi = ModelRequest(parts=[
SystemPromptPart(content='You are a concise assistant. Reply in one sentence.'),
UserPromptPart(content='What is the capital of Spain?'),
])
# Add system-level instructions (preferred over SystemPromptPart for multi-turn)
with_instr = ModelRequest(
instructions='Answer in bullet points.',
parts=[UserPromptPart(content='List three advantages of Python.')],
)
# Multimodal: image URL
with_image = ModelRequest(parts=[
UserPromptPart(content=[
ImageUrl(url='https://upload.wikimedia.org/wikipedia/commons/thumb/4/47/PNG_transparency_demonstration_1.png/200px-PNG_transparency_demonstration_1.png'),
'What do you see in this image?',
]),
])

Chain turns by passing the prior history as additional messages:

import asyncio
from pydantic_ai import ModelRequest
from pydantic_ai.messages import ModelMessage
from pydantic_ai.direct import model_request
async def chat():
history: list[ModelMessage] = [ModelRequest.user_text_prompt('My favourite colour is green.')]
resp1 = await model_request('openai:gpt-6-luna', history)
history.append(resp1) # ModelResponse also is a ModelMessage
history.append(ModelRequest.user_text_prompt('What is my favourite colour?'))
resp2 = await model_request('openai:gpt-6-luna', history)
print(resp2.parts[0].content) # 'Your favourite colour is green.'
asyncio.run(chat())

ModelRequestParameters — tools and output schemas

Section titled “ModelRequestParameters — tools and output schemas”

Pass model_request_parameters when you need to advertise tool schemas or set an output schema:

import asyncio
from pydantic import BaseModel
from pydantic_ai import ModelRequest
from pydantic_ai.direct import model_request
from pydantic_ai.models import ModelRequestParameters
from pydantic_ai.tools import ToolDefinition
class WeatherResult(BaseModel):
temperature_c: float
condition: str
# Advertise a tool schema (the model will emit a ToolCallPart if it wants to call it)
weather_tool = ToolDefinition(
name='get_weather',
description='Get current weather for a city.',
parameters_json_schema={
'type': 'object',
'properties': {'city': {'type': 'string'}},
'required': ['city'],
},
)
async def main():
resp = await model_request(
'openai:gpt-6-luna',
[ModelRequest.user_text_prompt('What is the weather in Paris?')],
model_request_parameters=ModelRequestParameters(
function_tools=[weather_tool],
),
)
for part in resp.parts:
if part.part_kind == 'tool-call':
print(f'Model wants to call: {part.tool_name}({part.args_as_dict()})')
asyncio.run(main())

Pass instrument=True (or a custom InstrumentationSettings) to emit spans:

import asyncio
from pydantic_ai import ModelRequest
from pydantic_ai.direct import model_request
from pydantic_ai.models.instrumented import InstrumentationSettings
settings = InstrumentationSettings(include_content=True) # version 5 is the default
async def main():
response = await model_request(
'anthropic:claude-haiku-5-5',
[ModelRequest.user_text_prompt('Hello!')],
instrument=settings,
)
print(response.parts[0].content)
asyncio.run(main())
from pydantic_ai.messages import ModelResponse, TextPart, ToolCallPart, ThinkingPart
def inspect(response: ModelResponse) -> None:
print(f'model: {response.model_name}')
print(f'timestamp: {response.timestamp}')
print(f'input tokens: {response.usage.input_tokens}')
print(f'output tokens: {response.usage.output_tokens}')
print(f'finish reason: {response.finish_reason}') # 'stop' | 'length' | 'content_filter' | 'tool_call' | 'error' | None
for part in response.parts:
if isinstance(part, TextPart):
print(f'text: {part.content[:80]}')
elif isinstance(part, ThinkingPart):
print(f'thinking: {part.thinking[:80]}')
elif isinstance(part, ToolCallPart):
print(f'tool call: {part.tool_name}({part.args_as_json_str()})')
import asyncio
from pydantic_ai import ModelRequest
from pydantic_ai.direct import model_request
MODELS = [
'openai:gpt-6-luna',
'anthropic:claude-haiku-5-5',
'google:gemini-3.8-flash',
]
async def compare(prompt: str) -> None:
messages = [ModelRequest.user_text_prompt(prompt)]
results = await asyncio.gather(*[
model_request(m, messages) for m in MODELS
])
for model_name, resp in zip(MODELS, results):
text = next((p.content for p in resp.parts if p.part_kind == 'text'), '')
print(f'{model_name}: {text[:120]}')
print(f' tokens: in={resp.usage.input_tokens} out={resp.usage.output_tokens}')
asyncio.run(compare('Explain async/await in Python in one sentence.'))

2. Synchronous embedding-style call (library helper)

Section titled “2. Synchronous embedding-style call (library helper)”
from pydantic_ai import ModelRequest
from pydantic_ai.direct import model_request_sync
from pydantic_ai.messages import SystemPromptPart, TextPart, UserPromptPart
def classify(text: str, categories: list[str]) -> str:
"""Synchronous helper for non-async callers."""
cats = ', '.join(categories)
resp = model_request_sync(
'openai:gpt-6-luna',
[
ModelRequest(
parts=[
SystemPromptPart(f'Classify the input into exactly one of: {cats}. Reply with the category name only.'),
UserPromptPart(text),
]
)
],
)
return ''.join(p.content for p in resp.parts if isinstance(p, TextPart)).strip()
print(classify('I love the new Python 3.13 release', ['tech', 'sports', 'politics']))

3. Async stream with early exit and response inspection

Section titled “3. Async stream with early exit and response inspection”
import asyncio
from pydantic_ai import ModelRequest
from pydantic_ai.direct import model_request_stream
from pydantic_ai.messages import PartDeltaEvent, TextPartDelta
async def stream_until_marker(prompt: str, stop_token: str = '[DONE]') -> str:
collected = ''
async with model_request_stream('openai:gpt-6-luna',
[ModelRequest.user_text_prompt(prompt)]) as stream:
async for event in stream:
if isinstance(event, PartDeltaEvent) and isinstance(event.delta, TextPartDelta):
collected += event.delta.content_delta
if stop_token in collected:
break
return collected
asyncio.run(stream_until_marker('Say "Hello [DONE] World"'))

4. Mixing direct calls inside a custom capability

Section titled “4. Mixing direct calls inside a custom capability”

The direct API is ideal inside a WrapModelRequestHandler or a custom AbstractCapability where you want to call the model a second time (e.g. an inner-monologue step) without going through the full agent machinery:

import asyncio
from pydantic_ai import ModelRequest
from pydantic_ai.direct import model_request
from pydantic_ai.messages import ModelResponse
async def self_critique(original_response: ModelResponse, model: str) -> str:
"""Ask the model to critique its own previous response."""
original_text = next(
(p.content for p in original_response.parts if p.part_kind == 'text'), ''
)
critique_resp = await model_request(
model,
[
ModelRequest.user_text_prompt(
f'Critique the following response and point out any inaccuracies:\n\n{original_text}'
)
],
)
return next((p.content for p in critique_resp.parts if p.part_kind == 'text'), '')
import asyncio
from pydantic_ai import ModelRequest
from pydantic_ai.direct import model_request
from pydantic_ai.exceptions import ModelAPIError
from pydantic_ai.messages import ModelResponse
async def resilient_request(
model: str, prompt: str, retries: int = 3
) -> ModelResponse:
messages = [ModelRequest.user_text_prompt(prompt)]
for attempt in range(retries):
try:
return await model_request(model, messages)
except ModelAPIError as exc:
if attempt == retries - 1:
raise
wait = 2 ** attempt
print(f'Attempt {attempt + 1} failed ({exc}). Retrying in {wait}s…')
await asyncio.sleep(wait)
raise RuntimeError('unreachable')
  • No tool execution. If the model returns a ToolCallPart, it’s your responsibility to run the function and continue the conversation (or ignore it). Use Agent if you want the loop managed for you.
  • No output validators. The response is the raw ModelResponse; Pydantic validation runs only if you parse parts yourself.
  • model_request_sync / model_request_stream_sync block the thread. Do not call them from inside asyncio.run or from an async function — they create a new event loop and will deadlock or raise.
  • StreamedResponseSync must be used as a context manager. Calling iter(stream) outside a with block raises RuntimeError.
  • instrument=None inherits the global default. Set Agent.instrument_all(...) once at startup to instrument all direct calls automatically.
SymbolModuleNotes
model_requestpydantic_ai.directAsync, non-streamed
model_request_syncpydantic_ai.directSync wrapper around model_request
model_request_streampydantic_ai.directAsync context manager → StreamedResponse
model_request_stream_syncpydantic_ai.directSync context manager → StreamedResponseSync
StreamedResponseSyncpydantic_ai.directSync wrapper over the async stream; use inside with, iterate synchronously, then read .response
ModelRequest.user_text_promptpydantic_ai.messagesConvenience constructor for a plain text user turn
ModelRequestParameterspydantic_ai.modelsTool schemas, output schema, sampling
ModelResponsepydantic_ai.messagesProvider response with parts + usage
InstrumentationSettingspydantic_ai.models.instrumentedOTel settings; pass to instrument=