LlamaIndex Comprehensive Technical Guide
Complete LlamaIndex (Python) reference — ingestion, indexing, retrieval, query engines, workflow-based agents, tools, memory, structured output, multi-agent workflows and evaluation.
Verified against
llama-index/llama-index-core0.14.26 (versions checked 2026-11-19). Every import and constructor on this page was checked against an installed copy of that release.
LlamaIndex has two halves that are worth keeping separate in your head:
- The data half — readers, node parsers, ingestion pipelines, indexes, retrievers, post-processors and query engines. This is the part LlamaIndex is best known for, and it has been stable for a long time.
- The agent half — everything is now built on Workflows (the event-driven runtime in the
llama-index-workflowspackage, re-exported asllama_index.core.workflow). The agent classesFunctionAgent,ReActAgent,CodeActAgentand the multi-agentAgentWorkflowall live inllama_index.core.agent.workflow.
If you learnt LlamaIndex before mid-2025, note that the older agent stack (OpenAIAgent, FunctionCallingAgent, AgentRunner, step workers, the old ReActAgent.from_tools(...) / agent.chat(...) API) and QueryPipeline were removed in llama-index-core 0.13.0. See section 22.
Table of contents
Section titled “Table of contents”- Installation and setup
- Architecture overview
- Core concepts
- Agent classes and types
- LLM and embedding configuration
- Creating basic agents
- Tool integration with agents
- Query engine tools
- Context, state and memory
- Structured output
- Streaming responses
- Multi-agent systems with AgentWorkflow
- Custom workflows
- Serving and deploying workflows (LlamaAgents)
- Human-in-the-loop
- FunctionTool in detail
- LlamaHub tools and MCP
- Data connectors
- Advanced retrieval
- Prompts
- Evaluation
- Migrating from the legacy agent API
1. Installation and setup
Section titled “1. Installation and setup”The llama-index package is a small convenience bundle: it installs llama-index-core plus the OpenAI LLM and embedding integrations. Everything else is a separate integration package (llama-index-llms-*, llama-index-embeddings-*, llama-index-vector-stores-*, llama-index-readers-*, llama-index-tools-*).
# Starter bundle (core + OpenAI LLM and embeddings)pip install "llama-index==0.14.26"
# Or build up from core yourselfpip install "llama-index-core==0.14.26" llama-index-llms-openai llama-index-embeddings-openaillama-index and llama-index-core are released in lockstep and share a version number. Integration packages have their own version numbers; let pip resolve them against the core you pinned rather than pinning them by hand.
Python 3.10 or newer is required.
Typical extras by use case
# PDF/DOCX/etc. support for SimpleDirectoryReaderpip install llama-index-readers-file
# RAG with a persistent vector storepip install llama-index-vector-stores-chroma chromadb
# Other model providerspip install llama-index-llms-anthropic llama-index-llms-ollama llama-index-embeddings-huggingface
# Hybrid (BM25 + vector) retrievalpip install llama-index-retrievers-bm25
# Use MCP servers as agent toolspip install llama-index-tools-mcp
# Draw workflow graphspip install llama-index-utils-workflowVerify the install
import llama_index.corefrom llama_index.core import VectorStoreIndex, Documentfrom llama_index.core.agent.workflow import FunctionAgent, AgentWorkflowfrom llama_index.llms.openai import OpenAI
print(llama_index.core.__version__)Set OPENAI_API_KEY in the environment (or pass api_key= to the LLM and embedding classes).
2. Architecture overview
Section titled “2. Architecture overview”┌─────────────────────────────────────────────────────────┐│ Application layer ││ FastAPI / CLI / notebooks / llamactl-served apps │└────────────────────┬────────────────────────────────────┘ │┌────────────────────┴────────────────────────────────────┐│ Orchestration: Workflows ││ Workflow, @step, Context, events ││ FunctionAgent · ReActAgent · CodeActAgent ││ AgentWorkflow (multi-agent hand-off) │└────────────────────┬────────────────────────────────────┘ │ tools┌────────────────────┴────────────────────────────────────┐│ Query layer ││ Query engines · chat engines · routers ││ Retrievers · node post-processors · synthesisers │└────────────────────┬────────────────────────────────────┘ │┌────────────────────┴────────────────────────────────────┐│ Index & storage ││ VectorStoreIndex · SummaryIndex · PropertyGraphIndex ││ StorageContext: docstore, index store, vector store │└────────────────────┬────────────────────────────────────┘ │┌────────────────────┴────────────────────────────────────┐│ Ingestion ││ Readers → Documents → node parsers → Nodes ││ IngestionPipeline (transformations, caching, dedupe) │└────────────────────┬────────────────────────────────────┘ │┌────────────────────┴────────────────────────────────────┐│ Models (global Settings or per component) ││ LLMs · embedding models · tokenisers │└─────────────────────────────────────────────────────────┘| Concept | What it is |
|---|---|
Document | A unit of source data (a file, a page, a row) with text and metadata |
Node (TextNode) | A chunk of a document — the unit that gets embedded and retrieved |
| Index | A structure over nodes; VectorStoreIndex is the default |
| Retriever | Given a query, returns NodeWithScore objects |
| Query engine | Retriever + post-processors + response synthesiser |
| Tool | A callable an agent may choose to invoke (FunctionTool, QueryEngineTool) |
| Workflow | An event-driven async program made of @step methods |
| Agent | A prebuilt workflow that loops LLM → tool calls → LLM |
3. Core concepts
Section titled “3. Core concepts”3.1 Documents
Section titled “3.1 Documents”from llama_index.core import Document, SimpleDirectoryReader
# Build documents by handdoc = Document( text="LlamaIndex is a framework for building LLM applications over your data.", metadata={"source": "manual", "topic": "overview"},)
# Load a folder (txt, md, pdf, docx, csv, ... depending on installed readers)documents = SimpleDirectoryReader("data", recursive=True).load_data()
# Control what the LLM and the embedding model seedoc.excluded_llm_metadata_keys = ["source"]doc.excluded_embed_metadata_keys = ["source"]3.2 Nodes
Section titled “3.2 Nodes”Nodes are chunks of documents. You rarely build them by hand, but it is useful to know their shape.
from llama_index.core.schema import TextNode, NodeRelationship, RelatedNodeInfo
node1 = TextNode(text="First paragraph.", id_="node-1", metadata={"page": 1})node2 = TextNode(text="Second paragraph.", id_="node-2", metadata={"page": 1})
node1.relationships[NodeRelationship.NEXT] = RelatedNodeInfo(node_id=node2.node_id)node2.relationships[NodeRelationship.PREVIOUS] = RelatedNodeInfo(node_id=node1.node_id)
print(node1.node_id, node1.metadata, node1.get_content())3.3 Node parsers and the ingestion pipeline
Section titled “3.3 Node parsers and the ingestion pipeline”from llama_index.core import Documentfrom llama_index.core.ingestion import IngestionPipelinefrom llama_index.core.node_parser import SentenceSplitterfrom llama_index.embeddings.openai import OpenAIEmbedding
pipeline = IngestionPipeline( transformations=[ SentenceSplitter(chunk_size=512, chunk_overlap=50), OpenAIEmbedding(model="text-embedding-3-small"), ],)
nodes = pipeline.run(documents=[Document(text="A long document ...")])print(len(nodes), nodes[0].embedding is not None)Attach a docstore and a vector_store to the pipeline to get de-duplication and upserts across runs; attach a cache to avoid re-embedding unchanged chunks.
3.4 Indexes
Section titled “3.4 Indexes”from llama_index.core import VectorStoreIndex, SummaryIndex, Document
documents = [ Document(text="LlamaIndex is a data framework for LLM applications."), Document(text="RAG grounds an LLM's answer in retrieved context."),]
# Embedding-based semantic search — the default choicevector_index = VectorStoreIndex.from_documents(documents)
# Returns every node (optionally LLM-filtered) — good for "summarise all of this"summary_index = SummaryIndex.from_documents(documents)
# Build directly from nodes you produced with an IngestionPipeline# vector_index = VectorStoreIndex(nodes)ListIndex was the old name for SummaryIndex; use SummaryIndex. TreeIndex and KeywordTableIndex still exist but are rarely the right choice today. For graph-shaped data use PropertyGraphIndex (section 3.9); for images alongside text, MultiModalVectorStoreIndex (section 3.10).
3.5 Persistence
Section titled “3.5 Persistence”from llama_index.core import StorageContext, load_index_from_storage
vector_index.storage_context.persist(persist_dir="./storage")
storage_context = StorageContext.from_defaults(persist_dir="./storage")index = load_index_from_storage(storage_context)With an external vector store the embeddings live in the store, and you reconnect with VectorStoreIndex.from_vector_store(vector_store):
import chromadbfrom llama_index.core import StorageContext, VectorStoreIndexfrom llama_index.vector_stores.chroma import ChromaVectorStore
client = chromadb.PersistentClient(path="./chroma_db")collection = client.get_or_create_collection("docs")vector_store = ChromaVectorStore(chroma_collection=collection)
# First run: buildstorage_context = StorageContext.from_defaults(vector_store=vector_store)index = VectorStoreIndex.from_documents(documents, storage_context=storage_context)
# Later runs: reconnectindex = VectorStoreIndex.from_vector_store(vector_store)Changing documents later needs care. Give each Document a stable doc_id (for example the file path) and delete by it with index.delete_ref_doc(doc_id), which removes that document’s chunks from the vector store, including after from_vector_store. index.refresh_ref_docs(documents) (re-insert changed documents) relies on a docstore that remembers earlier hashes; an index reconnected with from_vector_store has none, so it inserts the updated document as new and leaves the stale chunks behind. For incremental updates across runs, use an IngestionPipeline with a persisted docstore and DocstoreStrategy.UPSERTS, as shown in the production guide.
3.6 Retrievers
Section titled “3.6 Retrievers”from llama_index.core.vector_stores import MetadataFilters, ExactMatchFilter
retriever = vector_index.as_retriever( similarity_top_k=5, filters=MetadataFilters(filters=[ExactMatchFilter(key="topic", value="overview")]),)
for hit in retriever.retrieve("What is LlamaIndex?"): print(f"{hit.score:.3f} {hit.node.get_content()[:80]}")ExactMatchFilter only does equality. MetadataFilter takes an operator (FilterOperator.EQ, NE, GT, GTE, LT, LTE, IN, NIN, ANY, ALL, CONTAINS, TEXT_MATCH, IS_EMPTY and others) and MetadataFilters combines filters with condition=FilterCondition.AND, OR or NOT:
from llama_index.core.vector_stores import FilterCondition, FilterOperator, MetadataFilter, MetadataFilters
recent_hr_or_finance = MetadataFilters( filters=[ MetadataFilter(key="year", operator=FilterOperator.GTE, value=2024), MetadataFilter(key="dept", operator=FilterOperator.IN, value=["hr", "finance"]), ], condition=FilterCondition.AND,)retriever = vector_index.as_retriever(similarity_top_k=5, filters=recent_hr_or_finance)The in-memory vector store applies filters itself; external stores translate them into their own query language, and not every store supports every operator, so check the integration’s documentation before relying on one. For diverse results, as_retriever(vector_store_query_mode="mmr", vector_store_kwargs={"mmr_threshold": 0.5}) selects results by maximal marginal relevance instead of raw similarity (store support varies here too).
3.7 Query engines
Section titled “3.7 Query engines”query_engine = vector_index.as_query_engine( similarity_top_k=3, response_mode="compact", # also: "refine", "tree_summarize", "accumulate", ...)
response = query_engine.query("What is LlamaIndex?")print(response.response)for source in response.source_nodes: print("-", source.node.get_content()[:60], source.score)
# Async variant# response = await query_engine.aquery("What is LlamaIndex?")Token streaming from a query engine:
streaming_engine = vector_index.as_query_engine(streaming=True)streaming_response = streaming_engine.query("Summarise the documents")for token in streaming_response.response_gen: print(token, end="", flush=True)similarity_top_k defaults to 2, which is low for most corpora; start at 4 to 6 and measure. response.source_nodes carries the retrieved chunks with their scores, so you can show citations or log what the answer was based on.
Response modes. After retrieval, a response synthesiser turns the chunks into an answer. response_mode selects how, and it decides how many LLM calls a query costs:
response_mode | What it does | Use it for |
|---|---|---|
compact (default) | Packs as many chunks as fit into each prompt, answers from the first prompt, then refines the answer with the rest | General Q&A; fewest calls for a given context |
refine | One LLM call per chunk, each refining the previous answer | Long answers that must consider every chunk; slowest |
tree_summarize | Summarises batches of chunks, then summarises the summaries | ”Summarise these documents”; pairs well with SummaryIndex |
simple_summarize | Merges all chunks into a single prompt | Small result sets; fails if they exceed the context window |
accumulate / compact_accumulate | Answers the query once per chunk (or packed chunk) and concatenates the answers | When each chunk should be answered independently |
no_text | Retrieves only; no LLM call. The chunks are in response.source_nodes | Debugging retrieval cheaply |
context_only | Returns the retrieved text concatenated, with no LLM call | Feeding your own prompt |
generation | Ignores the context and asks the LLM directly | Comparing against “no RAG” |
A query engine is a retriever, optional node post-processors and a synthesiser. as_query_engine() wires them for you; to choose each part yourself, build the engine explicitly:
from llama_index.core import get_response_synthesizerfrom llama_index.core.postprocessor import SimilarityPostprocessorfrom llama_index.core.query_engine import RetrieverQueryEngine
retriever = vector_index.as_retriever(similarity_top_k=6)synthesizer = get_response_synthesizer(response_mode="tree_summarize", use_async=True)
engine = RetrieverQueryEngine( retriever=retriever, response_synthesizer=synthesizer, node_postprocessors=[SimilarityPostprocessor(similarity_cutoff=0.3)],)print(engine.query("Summarise the documents."))
# Retrieval only, no LLM call: useful when tuning chunk sizes and top_kdry_run = vector_index.as_query_engine(response_mode="no_text").query("What is LlamaIndex?")print([(n.score, n.node.node_id) for n in dry_run.source_nodes])use_async=True makes the synchronous query() issue the per-chunk calls of tree_summarize and accumulate concurrently; aquery() already does so for tree_summarize. Streaming (streaming=True) works with compact, refine, tree_summarize and simple_summarize; accumulate raises ValueError: Unable to stream in Accumulate response mode. The upstream changelog lists “restore compact and refine streaming” under 0.14.25, so on older releases test streaming first.
3.8 Chat engines
Section titled “3.8 Chat engines”For a conversational RAG bot that does not need tools, a chat engine is simpler than an agent. It keeps the conversation history, retrieves on every turn and answers with the retrieved text in the prompt. Since 0.13.0 the default as_chat_engine() (chat_mode="best") returns a CondensePlusContextChatEngine:
chat_engine = vector_index.as_chat_engine(chat_mode="condense_plus_context", similarity_top_k=4)print(chat_engine.chat("What is LlamaIndex?"))print(chat_engine.chat("And how does it relate to RAG?")) # the follow-up is rewritten into a standalone question firstprint(len(chat_engine.chat_history)) # user and assistant messages so farchat_engine.reset()chat_mode | Engine | Behaviour |
|---|---|---|
condense_plus_context (also best) | CondensePlusContextChatEngine | Rewrites the latest message and history into a standalone question, retrieves for it, answers with the context in the system prompt. The usual choice. |
context | ContextChatEngine | Retrieves using the latest message as written. Cheaper (no rewrite call) but follow-ups like “and the second one?” retrieve poorly. |
condense_question | CondenseQuestionChatEngine | Rewrites the question, then runs a full query engine. Does not accept system_prompt (it raises NotImplementedError). |
simple | SimpleChatEngine | No retrieval; the LLM alone. |
react and openai were removed in 0.13.0 and raise a ValueError that points you to ReActAgent or FunctionAgent with a QueryEngineTool. Extra keyword arguments such as similarity_top_k and node_postprocessors are passed to the retriever and engine. Streaming and async variants exist for every engine:
streaming = chat_engine.stream_chat("Summarise our conversation.")for token in streaming.response_gen: print(token, end="", flush=True)print(len(streaming.source_nodes)) # sources are available once streaming has finished
# async: response = await chat_engine.achat(...); stream = await chat_engine.astream_chat(...)# async for token in stream.async_response_gen(): ...Pass memory= to control history length or to persist it: ChatMemoryBuffer.from_defaults(token_limit=3000) keeps the most recent messages within a token budget, and a database-backed Memory (section 9.3) with a stable session_id survives restarts. A chat engine holds one conversation, so create one per user or session; the index and retriever underneath can be shared.
3.9 Property graphs
Section titled “3.9 Property graphs”PropertyGraphIndex extracts entities and relations from each chunk with “knowledge graph extractors”, stores them as a labelled property graph next to the text, and retrieves by following graph paths as well as by vector similarity. Use it when questions are about relationships (“who worked with whom”, “which systems depend on this service”), not only about passages.
from llama_index.core import PropertyGraphIndexfrom llama_index.core.indices.property_graph import ImplicitPathExtractor, SimpleLLMPathExtractorfrom llama_index.llms.openai import OpenAI
llm = OpenAI(model="gpt-6-luna")graph_index = PropertyGraphIndex.from_documents( documents, llm=llm, # used by the LLM-based extractor kg_extractors=[ SimpleLLMPathExtractor(llm=llm, max_paths_per_chunk=10), # (subject, relation, object) triples ImplicitPathExtractor(), # links from node relationships, no LLM ],)
retriever = graph_index.as_retriever(include_text=True) # matched triples plus their source textfor hit in retriever.retrieve("Who did Ada Lovelace work with?"): print(hit.text[:80])
graph_engine = graph_index.as_query_engine(include_text=True)graph_index.storage_context.persist(persist_dir="./graph_storage") # reload with load_index_from_storageBy default the LLM extractors (SimpleLLMPathExtractor, DynamicLLMPathExtractor, SchemaLLMPathExtractor) log a failed extraction and carry on, so a chunk can silently contribute no triples. Pass raise_on_error=True while building an index if you would rather fail fast.
SchemaLLMPathExtractor restricts extraction to entity and relation types you list, which gives a cleaner graph than free-form triples. The default store is in-memory (SimplePropertyGraphStore); for real data use a graph database integration such as Neo4jPropertyGraphStore (llama-index-graph-stores-neo4j), passed as property_graph_store=. Extraction makes one LLM call per chunk, so building the index costs far more than a vector index over the same documents.
3.10 Multimodal documents and queries
Section titled “3.10 Multimodal documents and queries”Since 0.14.23 the standard query engines can synthesise answers from images as well as text: pass multimodal=True and an LLM that accepts image content. MultiModalVectorStoreIndex embeds images with a separate image embedding model and stores them beside the text.
# pip install llama-index-readers-file llama-index-embeddings-clip git+https://github.com/openai/CLIP.gitfrom llama_index.core import SimpleDirectoryReaderfrom llama_index.core.indices import MultiModalVectorStoreIndexfrom llama_index.core.query_engine import RetrieverQueryEnginefrom llama_index.embeddings.clip import ClipEmbedding
documents = SimpleDirectoryReader("./report").load_data() # .png/.jpg files become ImageDocumentsmm_index = MultiModalVectorStoreIndex.from_documents(documents, image_embed_model=ClipEmbedding())
engine = RetrieverQueryEngine.from_args( retriever=mm_index.as_retriever(similarity_top_k=3, image_similarity_top_k=2), llm=OpenAI(model="gpt-6.1-sol"), # must accept image input multimodal=True,)print(engine.query("What does the revenue chart show?"))Calling mm_index.as_query_engine() instead returns SimpleMultiModalQueryEngine, which is deprecated in favour of this form. Any MultiModalEmbedding works as image_embed_model; CLIP runs locally and needs PyTorch. To send an image to an LLM directly, put it in a message as a content block:
from llama_index.core.llms import ChatMessage, ImageBlock, TextBlock
msg = ChatMessage( role="user", blocks=[ImageBlock(path="./report/chart.png"), TextBlock(text="Describe this chart.")],)print(llm.chat([msg]).message.content)For PDFs and slides where the images are embedded in pages, parsing with LlamaParse (section 18.2) usually works better than extracting the images yourself.
3.11 Chunking strategies and metadata extraction
Section titled “3.11 Chunking strategies and metadata extraction”The chunk is the unit that gets embedded, retrieved and shown to the LLM, so chunking decides retrieval quality more than most other settings. SentenceSplitter is the default: it splits on sentence boundaries into chunks of up to chunk_size tokens (1024 unless you change Settings.chunk_size), with chunk_overlap tokens repeated between neighbours (200 by default). The overlap must be smaller than the chunk size, or the constructor raises ValueError. Other parsers solve specific problems:
| Parser | Splits by | Use it when |
|---|---|---|
SentenceSplitter | Token budget, respecting sentences | The default for prose |
TokenTextSplitter | Token budget only | You need hard token limits and do not care about sentence boundaries |
MarkdownNodeParser | Markdown headings; records the heading path in header_path metadata | Documentation and notes written in Markdown |
SentenceWindowNodeParser | One sentence per node, with the surrounding sentences stored in metadata | Precise matching with wider context for the LLM (see 19.6) |
HierarchicalNodeParser | Several chunk sizes at once, linked parent to child | Merging small hits into their parent chunk (see 19.6) |
SemanticSplitterNodeParser | Breaks where the embedding similarity between neighbouring sentence groups drops | Text with topic shifts and no structure; needs an embedding call per sentence group |
CodeSplitter, JSONNodeParser, HTMLNodeParser | Syntax of the format | Source code, JSON, HTML |
from llama_index.core import Document, Settingsfrom llama_index.core.node_parser import MarkdownNodeParser, SemanticSplitterNodeParser, SentenceSplitter
doc = Document(text="# Leave\n\nStaff get 25 days. " * 30 + "\n\n# Expenses\n\nClaims are paid in 30 days. " * 30, metadata={"file_name": "handbook.md"})
nodes = SentenceSplitter(chunk_size=256, chunk_overlap=32).get_nodes_from_documents([doc])print(len(nodes), nodes[1].prev_node is not None) # neighbouring chunks are linked
md_nodes = MarkdownNodeParser().get_nodes_from_documents([doc])print([n.metadata.get("header_path") for n in md_nodes])
semantic = SemanticSplitterNodeParser( embed_model=Settings.embed_model, # any embedding model; called once per sentence group buffer_size=1, # sentences grouped when comparing neighbours breakpoint_percentile_threshold=95, # lower values give more, smaller chunks)semantic_nodes = semantic.get_nodes_from_documents([doc])Every parser is a transformation, so it can sit in an IngestionPipeline (section 3.3) or be passed to an index with VectorStoreIndex.from_documents(docs, transformations=[...]). Settings.node_parser sets the default for indexes built without one.
Metadata extractors add LLM-generated fields to each chunk: TitleExtractor (document_title), QuestionsAnsweredExtractor (questions_this_excerpt_can_answer), SummaryExtractor (section_summary, and with summaries=["prev", "self", "next"] the neighbours’ summaries too) and KeywordExtractor (excerpt_keywords). They run after the splitter, one LLM call per chunk per extractor:
from llama_index.core.extractors import KeywordExtractor, QuestionsAnsweredExtractor, TitleExtractorfrom llama_index.core.ingestion import IngestionPipeline
pipeline = IngestionPipeline( transformations=[ SentenceSplitter(chunk_size=256, chunk_overlap=0), TitleExtractor(nodes=3, llm=llm), # title from the first three chunks of each document QuestionsAnsweredExtractor(questions=2, llm=llm), KeywordExtractor(keywords=5, llm=llm), ])enriched = pipeline.run(documents=[doc])print(sorted(enriched[0].metadata))By default metadata is included in the text that is embedded and shown to the LLM, so extracted questions and keywords change what the chunk matches. That is often the point, but it also increases token use; use excluded_embed_metadata_keys and excluded_llm_metadata_keys (section 3.1) to hide fields from either side. Extraction multiplies ingestion cost by the number of extractors, so run it in the ingestion job with the pipeline cache enabled, not at query time.
4. Agent classes and types
Section titled “4. Agent classes and types”All agents are workflows. They share one calling convention: handler = agent.run(user_msg=..., ctx=..., memory=...) returns a WorkflowHandler; await handler gives an AgentOutput, and handler.stream_events() yields intermediate events.
| Class | Use it when |
|---|---|
FunctionAgent | The LLM supports native tool/function calling (OpenAI, Anthropic, Gemini, Mistral, most hosted models). The default choice. |
ReActAgent | The LLM has no native tool calling, or you want an explicit Thought/Action/Observation trace. Works with any LLM. |
CodeActAgent | The agent should write and execute Python to solve the task. You supply the code_execute_fn sandbox. |
AgentWorkflow | Several agents hand control to each other, sharing state. |
All live in llama_index.core.agent.workflow (and are re-exported from llama_index.core.agent).
4.1 FunctionAgent
Section titled “4.1 FunctionAgent”import asynciofrom llama_index.core.agent.workflow import FunctionAgentfrom llama_index.llms.openai import OpenAI
def multiply(a: float, b: float) -> float: """Multiply two numbers.""" return a * b
def add(a: float, b: float) -> float: """Add two numbers.""" return a + b
agent = FunctionAgent( tools=[multiply, add], # plain functions are wrapped as FunctionTools llm=OpenAI(model="gpt-6-luna"), system_prompt="You are a careful calculator. Always use the tools.",)
async def main(): response = await agent.run(user_msg="What is 7 * 8 + 12?") print(str(response))
asyncio.run(main())Model names in this guide (gpt-6-luna, text-embedding-3-small) are examples; use whatever your provider currently offers.
4.2 ReActAgent
Section titled “4.2 ReActAgent”from llama_index.core.agent.workflow import ReActAgent
react_agent = ReActAgent( tools=[multiply, add], llm=OpenAI(model="gpt-6-luna"),)
# inside an async functionresponse = await react_agent.run(user_msg="What is 7 * 8 + 12?")ReActAgent is not removed — only the pre-0.13 implementation (ReActAgent.from_tools(...), .chat()) is. The current class is constructed directly and run with .run().
4.3 CodeActAgent
Section titled “4.3 CodeActAgent”from llama_index.core.agent.workflow import CodeActAgent
async def execute_code(code: str) -> str: """Run code in YOUR sandbox (container, Jupyter kernel, remote executor) and return stdout.""" raise NotImplementedError("Wire this to a real sandbox; never exec() untrusted code in-process.")
code_agent = CodeActAgent( code_execute_fn=execute_code, tools=[multiply, add], llm=OpenAI(model="gpt-6-luna"),)4.4 AgentWorkflow
Section titled “4.4 AgentWorkflow”AgentWorkflow coordinates several named agents. Each agent declares which others it may hand off to; see section 12. For a single agent you can also write:
from llama_index.core.agent.workflow import AgentWorkflow
workflow = AgentWorkflow.from_tools_or_functions( [multiply, add], llm=OpenAI(model="gpt-6-luna"), system_prompt="You are a careful calculator.",)response = await workflow.run(user_msg="What is 3 * 4?")There is no AgentWorkflow.from_tools(...) and AgentWorkflow is not importable from llama_index.core.workflow.
5. LLM and embedding configuration
Section titled “5. LLM and embedding configuration”5.1 LLMs
Section titled “5.1 LLMs”from llama_index.llms.openai import OpenAIfrom llama_index.llms.anthropic import Anthropic # pip install llama-index-llms-anthropicfrom llama_index.llms.google_genai import GoogleGenAI # pip install llama-index-llms-google-genaifrom llama_index.llms.ollama import Ollama # pip install llama-index-llms-ollama
openai_llm = OpenAI(model="gpt-6-luna")claude_llm = Anthropic(model="claude-sonnet-5-5")gemini_llm = GoogleGenAI(model="gemini-3.8-flash")local_llm = Ollama(model="llama3.1", request_timeout=120.0)
print(openai_llm.complete("Say hello in one word."))Chat-style calls use ChatMessage:
from llama_index.core.llms import ChatMessage
messages = [ ChatMessage(role="system", content="You are terse."), ChatMessage(role="user", content="What is RAG?"),]print(openai_llm.chat(messages))
# Async and streaming variants: achat, astream_chat, stream_complete, ...5.2 Embeddings
Section titled “5.2 Embeddings”from llama_index.embeddings.openai import OpenAIEmbeddingfrom llama_index.embeddings.huggingface import HuggingFaceEmbedding # pip install llama-index-embeddings-huggingface
openai_embed = OpenAIEmbedding(model="text-embedding-3-small")local_embed = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
vector = openai_embed.get_text_embedding("hello")batch = openai_embed.get_text_embedding_batch(["a", "b", "c"])5.3 Global Settings
Section titled “5.3 Global Settings”Settings replaced the old ServiceContext. Anything you set here is the default for indexes, query engines and agents that are not given an explicit model.
from llama_index.core import Settingsfrom llama_index.core.node_parser import SentenceSplitter
Settings.llm = OpenAI(model="gpt-6-luna")Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")Settings.node_parser = SentenceSplitter(chunk_size=512, chunk_overlap=50)Prefer passing models explicitly in libraries and tests; use Settings in applications and notebooks.
5.4 Offline testing with mock models
Section titled “5.4 Offline testing with mock models”from llama_index.core import Settingsfrom llama_index.core.embeddings import MockEmbeddingfrom llama_index.core.llms import MockLLMfrom llama_index.core.llms.mock import MockFunctionCallingLLM
Settings.embed_model = MockEmbedding(embed_dim=8)Settings.llm = MockLLM()
# A tool-calling mock suitable for exercising FunctionAgent plumbingtest_llm = MockFunctionCallingLLM()6. Creating basic agents
Section titled “6. Creating basic agents”6.1 Minimal agent
Section titled “6.1 Minimal agent”import asynciofrom datetime import datetimefrom llama_index.core.agent.workflow import FunctionAgentfrom llama_index.llms.openai import OpenAI
def get_current_time() -> str: """Return the current local time as an ISO-8601 string.""" return datetime.now().isoformat(timespec="seconds")
def add_hours(iso_time: str, hours: int) -> str: """Add a number of hours to an ISO-8601 timestamp.""" from datetime import timedelta return (datetime.fromisoformat(iso_time) + timedelta(hours=hours)).isoformat()
agent = FunctionAgent( tools=[get_current_time, add_hours], llm=OpenAI(model="gpt-6-luna"), system_prompt="You answer questions about time. Use the tools; do not guess.",)
async def main(): response = await agent.run(user_msg="What time will it be in 5 hours?") print(response)
asyncio.run(main())The docstring and type hints become the tool description and JSON schema, so write them for the model.
6.2 Multi-turn conversations
Section titled “6.2 Multi-turn conversations”An agent keeps no memory between run() calls unless you give it a Context (or a Memory):
from llama_index.core.workflow import Context
ctx = Context(agent)
async def chat(): print(await agent.run(user_msg="My name is Ada.", ctx=ctx)) print(await agent.run(user_msg="What is my name?", ctx=ctx)) # remembers "Ada"6.3 Iteration limits and timeouts
Section titled “6.3 Iteration limits and timeouts”response = await agent.run(user_msg="...", max_iterations=10)
agent = FunctionAgent(tools=[multiply, add], llm=llm, timeout=120) # workflow timeout in secondsmax_iterations defaults to 20 LLM-and-tool rounds. When a run reaches the limit, the default (early_stopping_method="force") raises a WorkflowRuntimeError; early_stopping_method="generate" instead asks the LLM for a best-effort final answer from what it has gathered. Set it per run (agent.run(..., early_stopping_method="generate")), or on the agent or AgentWorkflow constructor to change the default.
6.4 Inspecting the result
Section titled “6.4 Inspecting the result”result = await agent.run(user_msg="What is 6 * 7?")print(result.response.content) # final assistant messagefor call in result.tool_calls: # ToolCallResult objects print(call.tool_name, call.tool_kwargs, call.tool_output.content)7. Tool integration with agents
Section titled “7. Tool integration with agents”7.1 Kinds of tool
Section titled “7.1 Kinds of tool”from llama_index.core.tools import FunctionTool, QueryEngineTool
def web_search(query: str) -> str: """Search the web and return a short list of results.""" return f"Results for {query!r}: ..."
search_tool = FunctionTool.from_defaults(fn=web_search)
kb_tool = QueryEngineTool.from_defaults( query_engine=vector_index.as_query_engine(), name="knowledge_base", description="Answers questions about the internal product documentation.",)
agent = FunctionAgent( tools=[search_tool, kb_tool, multiply], # FunctionTool, QueryEngineTool and a bare function llm=OpenAI(model="gpt-6-luna"),)There is no generic Tool class in llama_index.core.tools; use FunctionTool, QueryEngineTool, RetrieverTool, or a ToolSpec’s .to_tool_list().
7.2 Tool chaining
Section titled “7.2 Tool chaining”The agent loop chains tools for you: it calls one, reads the result, and decides on the next call. Keep each tool small and deterministic, and let the LLM do the sequencing.
def extract_entities(text: str) -> list[str]: """Extract organisation names from text.""" return ["Acme Ltd", "Globex"]
def lookup_company(name: str) -> dict: """Look up basic facts about a company by exact name.""" return {"name": name, "employees": 1200}
agent = FunctionAgent( tools=[extract_entities, lookup_company], llm=OpenAI(model="gpt-6-luna"), system_prompt="Extract entities first, then look each one up, then summarise.",)7.3 Parallel tool calls
Section titled “7.3 Parallel tool calls”FunctionAgent allows the LLM to emit several tool calls in one turn (allow_parallel_tool_calls=True by default). Set it to False when tools have side effects that must be ordered.
7.4 Returning directly
Section titled “7.4 Returning directly”FunctionTool.from_defaults(fn=..., return_direct=True) ends the agent loop with that tool’s output — useful for “lookup and return verbatim” tools.
8. Query engine tools
Section titled “8. Query engine tools”8.1 One knowledge base
Section titled “8.1 One knowledge base”from llama_index.core import VectorStoreIndex, Documentfrom llama_index.core.tools import QueryEngineTool
docs = [ Document(text="Machine learning is a subset of AI ..."), Document(text="Deep learning uses multi-layer neural networks ..."),]index = VectorStoreIndex.from_documents(docs)
kb_tool = QueryEngineTool.from_defaults( query_engine=index.as_query_engine(similarity_top_k=3), name="ai_knowledge_base", description="Definitions and background on AI, ML and deep learning.",)
agent = FunctionAgent(tools=[kb_tool], llm=OpenAI(model="gpt-6-luna"))response = await agent.run(user_msg="How does deep learning relate to ML?")8.2 Several knowledge bases
Section titled “8.2 Several knowledge bases”Give each tool a precise name and a description that says what is in it — that is what the LLM routes on.
from llama_index.core.tools import QueryEngineTool, ToolMetadata
tools = [ QueryEngineTool( query_engine=hr_index.as_query_engine(), metadata=ToolMetadata(name="hr_policies", description="Leave, benefits and HR policy documents."), ), QueryEngineTool( query_engine=eng_index.as_query_engine(), metadata=ToolMetadata(name="engineering_handbook", description="Coding standards, on-call, deployment process."), ),]agent = FunctionAgent(tools=tools, llm=OpenAI(model="gpt-6-luna"))8.3 Retriever tools
Section titled “8.3 Retriever tools”When you want the agent to see raw chunks rather than a synthesised answer, wrap a retriever:
from llama_index.core.tools import RetrieverTool
retriever_tool = RetrieverTool.from_defaults( retriever=index.as_retriever(similarity_top_k=5), name="search_docs", description="Return the most relevant passages from the documentation.",)9. Context, state and memory
Section titled “9. Context, state and memory”9.1 Context
Section titled “9.1 Context”Context holds a workflow run’s state: the agent’s chat memory, any values you store, and the event stream. Reuse one Context per conversation while the process is alive:
from llama_index.core.workflow import Context
ctx = Context(agent)await agent.run(user_msg="Remember that my favourite colour is green.", ctx=ctx)await agent.run(user_msg="What is my favourite colour?", ctx=ctx) # remembersA context can be serialised with ctx.to_dict(serializer=JsonSerializer()) and restored with Context.from_dict(agent, data, serializer=JsonSerializer()), which keeps your own state keys. It does not keep the prebuilt agents’ chat history: the memory entry is skipped during serialisation (you will see an UnserializableKeyWarning). To persist conversations across processes, use a database-backed Memory (section 9.3) and pass it on every run().
9.2 Reading and writing state from tools
Section titled “9.2 Reading and writing state from tools”A tool whose first parameter is annotated ctx: Context receives the running context automatically; the parameter is hidden from the LLM.
from llama_index.core.workflow import Context
async def add_to_cart(ctx: Context, item: str) -> str: """Add an item to the user's shopping cart.""" async with ctx.store.edit_state() as state: state.setdefault("cart", []).append(item) return f"Added {item}."
async def view_cart(ctx: Context) -> str: """Show the shopping cart.""" cart = await ctx.store.get("cart", default=[]) return ", ".join(cart) or "Cart is empty."
shop_agent = FunctionAgent(tools=[add_to_cart, view_cart], llm=OpenAI(model="gpt-6-luna"))ctx = Context(shop_agent)Inside an AgentWorkflow, shared state seeded with initial_state= lives under the "state" key (state["state"]) and is also rendered into the prompt via state_prompt.
9.3 Memory
Section titled “9.3 Memory”Memory (in llama_index.core.memory) is the current memory class: a token-bounded short-term chat history plus optional long-term memory blocks. ChatMemoryBuffer still exists but Memory is preferred.
from llama_index.core.memory import Memory, StaticMemoryBlock, FactExtractionMemoryBlock, VectorMemoryBlockfrom llama_index.core.llms import ChatMessage
memory = Memory.from_defaults( session_id="user-123", token_limit=40_000, memory_blocks=[ StaticMemoryBlock(name="profile", static_content="The user is a data engineer in London."), FactExtractionMemoryBlock(name="facts", llm=OpenAI(model="gpt-6-luna"), max_facts=50), ],)
response = await agent.run(user_msg="Hi!", memory=memory)By default the chat history lives in an in-memory SQLite database. Pass async_database_uri= (for example postgresql+asyncpg://... or sqlite+aiosqlite:///memory.db) and a stable session_id to persist it; a new Memory object built with the same two values on the next request picks the conversation up where it left off:
async def handle_turn(user_id: str, message: str) -> str: memory = Memory.from_defaults( session_id=user_id, async_database_uri="postgresql+asyncpg://app:secret@db/agents", ) return str(await agent.run(user_msg=message, memory=memory))To use a store other than the default SQLAlchemyChatStore, pass any AsyncDBChatStore subclass as chat_store=. Since 0.14.24 it takes precedence over table_name= and async_database_uri=, which are then ignored. You can also build the SQLAlchemyChatStore yourself to control the table and engine:
from llama_index.core.storage.chat_store.sql import SQLAlchemyChatStore
store = SQLAlchemyChatStore( table_name="agent_memory", async_database_uri="sqlite+aiosqlite:///memory.db",)memory = Memory.from_defaults(session_id="user-123", chat_store=store, token_limit=40_000)10. Structured output
Section titled “10. Structured output”10.1 From an agent
Section titled “10.1 From an agent”from pydantic import BaseModel, Field
class Calculation(BaseModel): expression: str result: float explanation: str = Field(description="One sentence explaining the steps")
agent = FunctionAgent( tools=[multiply, add], llm=OpenAI(model="gpt-6-luna"), output_cls=Calculation,)
result = await agent.run(user_msg="Compute (15 * 3) + 9")calc = result.get_pydantic_model(Calculation) # or result.structured_response (a dict)print(calc.result)10.2 From an LLM directly
Section titled “10.2 From an LLM directly”from pydantic import BaseModelfrom llama_index.core.prompts import PromptTemplate
class Employee(BaseModel): name: str title: str start_year: int
llm = OpenAI(model="gpt-6-luna")
employee = llm.structured_predict( Employee, PromptTemplate("Extract the employee from: {text}"), text="Jane Doe joined as Staff Engineer in 2021.",)
# Or wrap the LLM so every chat/complete call returns the modelsllm = llm.as_structured_llm(Employee)resp = sllm.complete("Jane Doe joined as Staff Engineer in 2021.")print(resp.raw) # Employee(...)10.3 Programs and output parsers
Section titled “10.3 Programs and output parsers”For LLMs without tool calling, LLMTextCompletionProgram uses a prompt plus a Pydantic output parser:
from llama_index.core.program import LLMTextCompletionProgram
program = LLMTextCompletionProgram.from_defaults( output_cls=Employee, prompt_template_str="Extract the employee as JSON from: {text}", llm=llm,)employee = program(text="Jane Doe joined as Staff Engineer in 2021.")10.4 From a query engine
Section titled “10.4 From a query engine”query_engine = index.as_query_engine(output_cls=Employee, response_mode="compact")structured = query_engine.query("Who is the staff engineer?").response # an Employee11. Streaming responses
Section titled “11. Streaming responses”11.1 Token streaming from an agent
Section titled “11.1 Token streaming from an agent”from llama_index.core.agent.workflow import AgentStream
handler = agent.run(user_msg="Explain RAG in three sentences.")async for event in handler.stream_events(): if isinstance(event, AgentStream): print(event.delta, end="", flush=True)
final = await handler11.2 Event-based streaming
Section titled “11.2 Event-based streaming”from llama_index.core.agent.workflow import AgentInput, AgentOutput, ToolCall, ToolCallResult
handler = agent.run(user_msg="What is (3 * 4) + 5?")async for event in handler.stream_events(): if isinstance(event, ToolCall): print(f"→ calling {event.tool_name}({event.tool_kwargs})") elif isinstance(event, ToolCallResult): print(f"← {event.tool_name} returned {event.tool_output.content}") elif isinstance(event, AgentOutput): print(f"[{event.current_agent_name}] {event.response.content}")
result = await handlerFor logging and tracing rather than UI updates, use the instrumentation module instead of the event stream — see the observability page.
12. Multi-agent systems with AgentWorkflow
Section titled “12. Multi-agent systems with AgentWorkflow”Earlier editions of this guide described the 2024 llama-agents package (ControlPlaneServer, AgentService, message queues). That package was retired; its successor llama-deploy has in turn been folded into LlamaAgents and the llamactl CLI (see section 14). In-process multi-agent orchestration is done with AgentWorkflow.
12.1 Hand-off between specialists
Section titled “12.1 Hand-off between specialists”import asynciofrom llama_index.core.agent.workflow import AgentWorkflow, FunctionAgent, AgentOutput, ToolCallResultfrom llama_index.core.workflow import Contextfrom llama_index.llms.openai import OpenAI
llm = OpenAI(model="gpt-6-luna")
async def search_web(query: str) -> str: """Search the web for information on a topic.""" return f"(search results for {query})"
async def record_notes(ctx: Context, notes: str, title: str) -> str: """Save research notes under a title for the other agents.""" async with ctx.store.edit_state() as ctx_state: ctx_state["state"]["research_notes"][title] = notes return "Notes recorded."
async def write_report(ctx: Context, report: str) -> str: """Write the final report in markdown.""" async with ctx.store.edit_state() as ctx_state: ctx_state["state"]["report"] = report return "Report written."
research_agent = FunctionAgent( name="ResearchAgent", description="Searches the web and records notes on a topic.", system_prompt="Research the topic, record notes, then hand off to WriteAgent.", tools=[search_web, record_notes], llm=llm, can_handoff_to=["WriteAgent"],)
write_agent = FunctionAgent( name="WriteAgent", description="Writes a markdown report from the research notes.", system_prompt="Write a concise report from the notes in the state.", tools=[write_report], llm=llm, can_handoff_to=["ResearchAgent"],)
workflow = AgentWorkflow( agents=[research_agent, write_agent], root_agent=research_agent.name, initial_state={"research_notes": {}, "report": None},)
async def main(): handler = workflow.run(user_msg="Write a short report on retrieval-augmented generation.") async for event in handler.stream_events(): if isinstance(event, ToolCallResult): print(f"[tool] {event.tool_name}") elif isinstance(event, AgentOutput) and event.response.content: print(f"[{event.current_agent_name}] {event.response.content[:80]}") await handler state = await handler.ctx.store.get("state") print(state["report"])
asyncio.run(main())How hand-off works: AgentWorkflow gives every agent a handoff tool listing the agents in its can_handoff_to. When the LLM calls it, control (and the shared memory and state) passes to the named agent.
12.2 Orchestrator pattern: agents as tools
Section titled “12.2 Orchestrator pattern: agents as tools”When you want one agent firmly in charge, wrap the others as tools instead of using hand-off:
async def call_research_agent(ctx: Context, prompt: str) -> str: """Ask the research agent to investigate something and return its findings.""" result = await research_agent.run(user_msg=prompt) return str(result)
orchestrator = FunctionAgent( system_prompt="Plan the work and delegate research before answering.", tools=[call_research_agent], llm=llm,)12.3 Choosing a pattern
Section titled “12.3 Choosing a pattern”| Pattern | Control | Good for |
|---|---|---|
AgentWorkflow hand-off | Decentralised: the active agent decides who goes next | Conversational assistants with specialist modes |
| Orchestrator with agents-as-tools | One agent plans and delegates | Research/report tasks with a clear owner |
Custom Workflow | You write the control flow in Python | Fixed pipelines, fan-out/fan-in, strict ordering |
13. Custom workflows
Section titled “13. Custom workflows”When the control flow is yours rather than the model’s — a fixed pipeline, fan-out and fan-in, strict ordering, approval gates — write a Workflow. A workflow is a class whose @step methods consume and emit typed events; the runtime routes events by the types in the step signatures and validates the graph before running. The agent classes above are themselves workflows.
from llama_index.core import VectorStoreIndexfrom llama_index.core.workflow import Event, StartEvent, StopEvent, Workflow, stepfrom llama_index.llms.openai import OpenAI
class RetrievedEvent(Event): query: str context: str
class RAGWorkflow(Workflow): def __init__(self, index: VectorStoreIndex, **kwargs): super().__init__(**kwargs) self.retriever = index.as_retriever(similarity_top_k=3) self.llm = OpenAI(model="gpt-6-luna")
@step async def retrieve(self, ev: StartEvent) -> RetrievedEvent: nodes = await self.retriever.aretrieve(ev.query) return RetrievedEvent(query=ev.query, context="\n\n".join(n.get_content() for n in nodes))
@step async def synthesise(self, ev: RetrievedEvent) -> StopEvent: prompt = f"Answer using only this context.\n\n{ev.context}\n\nQuestion: {ev.query}" return StopEvent(result=str(await self.llm.acomplete(prompt)))
async def main(): wf = RAGWorkflow(index=vector_index, timeout=60) print(await wf.run(query="What is LlamaIndex?"))Workflows come from the separate llama-index-workflows package (imported as workflows), which llama_index.core.workflow re-exports. The Workflows page covers it in full: events and typed start/stop events, Context and typed state, send_event / collect_events and list fan-out, streaming, human-in-the-loop, retry policies and @catch_error, timeouts, resources, checkpointing and resuming, nested workflows, drawing and testing. Workflow patterns has longer worked examples.
14. Serving and deploying workflows (LlamaAgents)
Section titled “14. Serving and deploying workflows (LlamaAgents)”A workflow or agent is an ordinary async object, so you can call it from any web framework (the FastAPI streaming page does that). LlamaIndex also ships its own serving layer, LlamaAgents:
WorkflowServer(pip install llama-agents-server,from llama_agents.server import WorkflowServer) wraps registered workflows in a REST API: start runs, stream their events, send events in (for example a human approval), fetch results, cancel. With a durable workflow store, unfinished runs resume after a restart.WorkflowClient(pip install llama-agents-client,from llama_agents.client import WorkflowClient) calls that API from Python.llamactlscaffolds a project from templates, serves itsWorkflowServerlocally with reload, packages it as a container, and deploys it to LlamaCloud.
The older import paths workflows.server and workflows.client (the [server] and [client] extras of llama-index-workflows) still work but emit a deprecation warning pointing to the packages above.
from llama_agents.server import WorkflowServer
server = WorkflowServer() # in-memory store; see the production guide for SQLiteserver.add_workflow("calculator", agent) # any Workflow, including FunctionAgent / AgentWorkflowserver.add_workflow("rag", RAGWorkflow(index=vector_index, timeout=60))
if __name__ == "__main__": import asyncio asyncio.run(server.serve(host="0.0.0.0", port=8080))curl -X POST localhost:8080/workflows/calculator/run \ -H 'Content-Type: application/json' \ -d '{"start_event": {"user_msg": "What is 7 * 8?", "chat_history": [], "max_iterations": 20}}'For agents, send chat_history and max_iterations as well as user_msg; with user_msg alone the agent’s start event cannot be serialised and the run does not complete.
With llamactl:
uv tool install llamactlllamactl init # choose a template; creates a new project directorycd my-app # the directory you named during initllamactl serve # serves the project's workflows locally, with reloadllamactl auth login # once, before deploying to LlamaCloudllamactl deployments createA project tells llamactl which server to run in pyproject.toml, under [tool.llamaagents], for example app = "my_app.server:server". Durable stores, authentication, the serializer setting agents need, and containers are covered in the production guide.
What survives from the old llama-agents architecture is the idea, not the API: if you need agents in separate processes, run each as its own service and call it from a tool.
15. Human-in-the-loop
Section titled “15. Human-in-the-loop”A step or a tool can pause the run and wait for a human. The run emits an InputRequiredEvent; your application answers by sending a HumanResponseEvent into the context.
from llama_index.core.workflow import Context, InputRequiredEvent, HumanResponseEvent
async def send_email(ctx: Context, to: str, body: str) -> str: """Send an email. Requires human approval.""" answer = await ctx.wait_for_event( HumanResponseEvent, waiter_event=InputRequiredEvent(prefix=f"Send email to {to}? (yes/no) "), ) if answer.response.strip().lower() != "yes": return "The user declined; the email was not sent." # ... actually send it ... return f"Email sent to {to}."
agent = FunctionAgent(tools=[send_email], llm=OpenAI(model="gpt-6-luna"))
handler = agent.run(user_msg="Email bob@example.com to say the build is green.")async for event in handler.stream_events(): if isinstance(event, InputRequiredEvent): reply = input(event.prefix) handler.ctx.send_event(HumanResponseEvent(response=reply))
print(await handler)Within one process you can keep the handler alive and call handler.ctx.send_event(...) whenever the answer arrives. Approval that must survive a restart is easier with a small custom workflow whose state is all serialisable — see durable approval; a prebuilt agent paused inside a tool cannot currently be resumed from a serialised context, because its memory is not serialised.
16. FunctionTool in detail
Section titled “16. FunctionTool in detail”from typing import Annotatedfrom pydantic import BaseModel, Fieldfrom llama_index.core.tools import FunctionTool
def search_orders( customer_id: Annotated[str, "The customer's ID, e.g. C-1042"], limit: int = 5,) -> list[dict]: """Return the customer's most recent orders, newest first.""" return [{"id": "O-1", "total": 42.0}][:limit]
tool = FunctionTool.from_defaults(fn=search_orders)print(tool.metadata.name, tool.metadata.description)print(tool.metadata.fn_schema.model_json_schema())Explicit schema, name and description
class SearchInput(BaseModel): query: str = Field(description="Full-text query") max_results: int = Field(default=10, ge=1, le=50)
def search_database(query: str, max_results: int = 10) -> str: return f"{max_results} results for {query!r}"
db_tool = FunctionTool.from_defaults( fn=search_database, name="database_search", description="Full-text search over the product database.", fn_schema=SearchInput,)Async tools and sync/async pairs
import httpx
async def fetch_status(url: str) -> int: """Return the HTTP status code for a URL.""" async with httpx.AsyncClient(timeout=10) as client: return (await client.get(url)).status_code
status_tool = FunctionTool.from_defaults(async_fn=fetch_status)Binding parameters the LLM should not set
def get_user_orders(user_id: str, limit: int = 5) -> list[dict]: """Return the current user's orders.""" ...
orders_tool = FunctionTool.from_defaults( fn=get_user_orders, partial_params={"user_id": "C-1042"}, # injected, hidden from the schema)Errors. If a tool raises, the exception text is returned to the LLM as the tool output so it can recover. Put retries for transient failures inside the tool itself (for example with tenacity).
17. LlamaHub tools and MCP
Section titled “17. LlamaHub tools and MCP”17.1 Tool specs from LlamaHub
Section titled “17.1 Tool specs from LlamaHub”Tool integrations are packages named llama-index-tools-<name>; browse them at llamahub.ai. A ToolSpec turns into a list of tools:
# pip install llama-index-tools-wikipediafrom llama_index.tools.wikipedia import WikipediaToolSpec
wiki_tools = WikipediaToolSpec().to_tool_list()agent = FunctionAgent(tools=wiki_tools, llm=OpenAI(model="gpt-6-luna"))Check each spec’s README for the exact tool names and constructor arguments; they vary by package.
17.2 MCP servers as tools
Section titled “17.2 MCP servers as tools”# pip install llama-index-tools-mcpfrom llama_index.tools.mcp import BasicMCPClient, McpToolSpec
mcp_client = BasicMCPClient("http://127.0.0.1:8000/mcp") # or a command for stdio serversmcp_tools = await McpToolSpec(client=mcp_client).to_tool_list_async()
agent = FunctionAgent(tools=mcp_tools, llm=OpenAI(model="gpt-6-luna"))The same package can expose a LlamaIndex workflow as an MCP server (workflow_as_mcp).
18. Data connectors
Section titled “18. Data connectors”18.1 Files
Section titled “18.1 Files”from llama_index.core import SimpleDirectoryReader
reader = SimpleDirectoryReader( input_dir="./docs", required_exts=[".md", ".pdf"], recursive=True, filename_as_id=True,)documents = reader.load_data() # or: await reader.aload_data()SimpleDirectoryReader picks a reader per extension; plain-text formats work out of the box, while PDF, DOCX, images and other formats need llama-index-readers-file (it is not part of the llama-index bundle). For difficult PDFs (tables, scans, slides), a layout-aware parser gives much better chunks; see the next section, or the open-source LiteParse parser (liteparse) if documents must not leave your infrastructure.
18.2 LlamaParse and LlamaCloud
Section titled “18.2 LlamaParse and LlamaCloud”LlamaParse is LlamaIndex’s hosted document parser, part of LlamaCloud (which also offers extraction, classification and managed indexes). Use the llama-cloud SDK; the older llama-cloud-services package is deprecated and was maintained only until May 2026.
# pip install llama-cloud (reads LLAMA_CLOUD_API_KEY from the environment)from llama_cloud import LlamaCloudfrom llama_index.core import Document
client = LlamaCloud()with open("annual_report.pdf", "rb") as f: result = client.parsing.parse( tier="agentic", # also "fast", "cost_effective", "agentic_plus" version="latest", upload_file=f, expand=["markdown"], )
documents = [ Document(text=page.markdown, metadata={"file_name": "annual_report.pdf", "page": page.page_number}) for page in result.markdown.pages if page.success]parse() uploads the file, polls until the job finishes and returns the result; AsyncLlamaCloud has the same methods for async code. The page-level Markdown keeps tables as Markdown tables, which MarkdownNodeParser or SentenceSplitter then chunk as usual. See the LlamaParse documentation for tiers and options.
18.3 Other sources
Section titled “18.3 Other sources”Readers for databases, SaaS tools and the web are separate packages; PyPI lists more than 200 llama-index-readers-* packages (counted 2026-11-19). Some common ones:
| Source | Package | Class |
|---|---|---|
| Web pages | llama-index-readers-web | SimpleWebPageReader, TrafilaturaWebReader |
| SQL databases | llama-index-readers-database | DatabaseReader |
| GitHub | llama-index-readers-github | GithubRepositoryReader |
| Notion | llama-index-readers-notion | NotionPageReader |
| Confluence | llama-index-readers-confluence | ConfluenceReader |
| Slack | llama-index-readers-slack | SlackReader |
| Google Drive/Docs | llama-index-readers-google | GoogleDriveReader, GoogleDocsReader |
# pip install llama-index-readers-webfrom llama_index.readers.web import SimpleWebPageReader
pages = SimpleWebPageReader(html_to_text=True).load_data(["https://example.com"])18.4 Writing a reader
Section titled “18.4 Writing a reader”from llama_index.core import Documentfrom llama_index.core.readers.base import BaseReader
class TicketReader(BaseReader): """Load support tickets from an internal API."""
def __init__(self, client): self.client = client
def load_data(self, project: str) -> list[Document]: return [ Document(text=t["body"], metadata={"ticket_id": t["id"], "project": project}) for t in self.client.list_tickets(project) ]19. Advanced retrieval
Section titled “19. Advanced retrieval”19.1 Hybrid retrieval (vector + BM25) with reciprocal-rank fusion
Section titled “19.1 Hybrid retrieval (vector + BM25) with reciprocal-rank fusion”# pip install llama-index-retrievers-bm25from llama_index.core.retrievers import QueryFusionRetrieverfrom llama_index.retrievers.bm25 import BM25Retriever
vector_retriever = index.as_retriever(similarity_top_k=5)bm25_retriever = BM25Retriever.from_defaults(docstore=index.docstore, similarity_top_k=5)
hybrid = QueryFusionRetriever( [vector_retriever, bm25_retriever], similarity_top_k=5, num_queries=1, # >1 also generates query variations with the LLM mode="reciprocal_rerank", use_async=True,)nodes = hybrid.retrieve("How do I rotate API keys?")BM25Retriever is not in llama_index.core.retrievers; it comes from the separate package above.
19.2 Node post-processors and reranking
Section titled “19.2 Node post-processors and reranking”from llama_index.core.postprocessor import SimilarityPostprocessor, LLMRerank, MetadataReplacementPostProcessor
query_engine = index.as_query_engine( similarity_top_k=10, node_postprocessors=[ SimilarityPostprocessor(similarity_cutoff=0.5), LLMRerank(top_n=3, llm=OpenAI(model="gpt-6-luna")), ],)StructuredLLMRerank does the same job as LLMRerank but asks the LLM for a structured list of document numbers and relevance scores instead of parsing free text, which is more reliable with function-calling models. It has a native async path (since 0.14.25), so an async query engine reranks batches without blocking the event loop. Documents are scored in batches of choice_batch_size (default 10).
from llama_index.core.postprocessor import StructuredLLMRerank
reranker = StructuredLLMRerank( llm=OpenAI(model="gpt-6-luna"), # a non-function-calling LLM logs a warning top_n=3, choice_batch_size=10, raise_on_structured_prediction_failure=False, # default True raises ValueError)With raise_on_structured_prediction_failure=False, a batch whose structured output cannot be parsed is logged and its nodes are kept with a score of 0 rather than failing the query.
Other useful post-processors: SentenceTransformerRerank (local cross-encoder), LongContextReorder, PrevNextNodePostprocessor, MetadataReplacementPostProcessor (pair with SentenceWindowNodeParser), and provider rerankers such as llama-index-postprocessor-cohere-rerank.
A custom post-processor:
from llama_index.core.postprocessor.types import BaseNodePostprocessorfrom llama_index.core.schema import NodeWithScore, QueryBundle
class RecentOnly(BaseNodePostprocessor): min_year: int = 2024
def _postprocess_nodes(self, nodes: list[NodeWithScore], query_bundle: QueryBundle | None = None): return [n for n in nodes if n.node.metadata.get("year", 0) >= self.min_year]19.3 Router query engine
Section titled “19.3 Router query engine”from llama_index.core.query_engine import RouterQueryEnginefrom llama_index.core.selectors import LLMSingleSelectorfrom llama_index.core.tools import QueryEngineTool
router = RouterQueryEngine( selector=LLMSingleSelector.from_defaults(), query_engine_tools=[ QueryEngineTool.from_defaults( query_engine=vector_index.as_query_engine(), description="Specific facts and details from the documents.", ), QueryEngineTool.from_defaults( query_engine=summary_index.as_query_engine(response_mode="tree_summarize"), description="Summaries of the documents as a whole.", ), ],)print(router.query("Give me an overview of the whole corpus."))LLMMultiSelector lets the router pick several engines and combine their answers.
19.4 Sub-question decomposition
Section titled “19.4 Sub-question decomposition”from llama_index.core.query_engine import SubQuestionQueryEnginefrom llama_index.core.question_gen import LLMQuestionGenerator
llm = OpenAI(model="gpt-6-luna")engine = SubQuestionQueryEngine.from_defaults( question_gen=LLMQuestionGenerator.from_defaults(llm=llm), llm=llm, query_engine_tools=[ QueryEngineTool.from_defaults(query_engine=tesla_index.as_query_engine(), name="tesla_10k", description="Tesla annual report"), QueryEngineTool.from_defaults(query_engine=ford_index.as_query_engine(), name="ford_10k", description="Ford annual report"), ],)print(engine.query("Compare Tesla's and Ford's revenue growth."))Pass question_gen explicitly. Without it, from_defaults tries to import the OpenAI function-calling question generator from llama-index-question-gen-openai, which is not part of the llama-index bundle and cannot be installed alongside current releases (it pins an old llama-index-llms-openai). LLMQuestionGenerator works with any LLM.
19.5 Custom query engine
Section titled “19.5 Custom query engine”from llama_index.core.query_engine import CustomQueryEnginefrom llama_index.core.retrievers import BaseRetrieverfrom llama_index.core.llms import LLMfrom llama_index.core import PromptTemplate
QA_PROMPT = PromptTemplate("Context:\n{context}\n\nAnswer the question: {query}\n")
class SimpleRAG(CustomQueryEngine): retriever: BaseRetriever llm: LLM
def custom_query(self, query_str: str) -> str: nodes = self.retriever.retrieve(query_str) context = "\n\n".join(n.get_content() for n in nodes) return str(self.llm.complete(QA_PROMPT.format(context=context, query=query_str)))
engine = SimpleRAG(retriever=index.as_retriever(), llm=OpenAI(model="gpt-6-luna"))19.6 Small-to-big retrieval: sentence windows and auto-merging
Section titled “19.6 Small-to-big retrieval: sentence windows and auto-merging”Small chunks match a question precisely but give the LLM too little context; large chunks give context but match vaguely. Two patterns retrieve small and answer with big.
Sentence windows. SentenceWindowNodeParser makes one node per sentence and stores the surrounding sentences in metadata. The embedding is computed from the single sentence; MetadataReplacementPostProcessor swaps in the window before synthesis:
from llama_index.core import VectorStoreIndexfrom llama_index.core.node_parser import SentenceWindowNodeParserfrom llama_index.core.postprocessor import MetadataReplacementPostProcessor
window_parser = SentenceWindowNodeParser.from_defaults( window_size=3, # three sentences each side window_metadata_key="window", original_text_metadata_key="original_text",)window_nodes = window_parser.get_nodes_from_documents(documents)window_index = VectorStoreIndex(window_nodes)
window_engine = window_index.as_query_engine( similarity_top_k=4, node_postprocessors=[MetadataReplacementPostProcessor(target_metadata_key="window")],)print(window_engine.query("What does the handbook say about leave?"))The window and original_text metadata are excluded from the embedding and the LLM prompt automatically, so they do not distort retrieval.
Auto-merging. HierarchicalNodeParser produces parent chunks and nested children (chunk_sizes=[2048, 512, 128] by default). Index only the leaves, but keep every level in the docstore. When enough sibling leaves are retrieved together, AutoMergingRetriever replaces them with their parent:
from llama_index.core import StorageContext, VectorStoreIndexfrom llama_index.core.node_parser import HierarchicalNodeParser, get_leaf_nodesfrom llama_index.core.retrievers import AutoMergingRetrieverfrom llama_index.core.query_engine import RetrieverQueryEngine
hierarchy = HierarchicalNodeParser.from_defaults(chunk_sizes=[1024, 256, 64])all_nodes = hierarchy.get_nodes_from_documents(documents)leaf_nodes = get_leaf_nodes(all_nodes)
storage_context = StorageContext.from_defaults()storage_context.docstore.add_documents(all_nodes) # parents must be in the docstoreleaf_index = VectorStoreIndex(leaf_nodes, storage_context=storage_context)
retriever = AutoMergingRetriever( leaf_index.as_retriever(similarity_top_k=6), storage_context, simple_ratio_thresh=0.5, # merge when more than half of a parent's children were retrieved verbose=True,)engine = RetrieverQueryEngine.from_args(retriever)print(engine.query("How are expenses reimbursed?"))Persist storage_context (section 3.5) and reload it, rather than re-chunking, so parent and child IDs stay stable. With an external vector store, only the leaves go to the store; the docstore with all levels still has to be kept somewhere durable (a Redis, MongoDB or Postgres docstore integration, or the local JSON file).
19.7 Query transformations
Section titled “19.7 Query transformations”Short or vague questions embed poorly. HyDEQueryTransform asks the LLM to write a hypothetical answer and embeds that (together with the original query when include_original=True), because an answer resembles the relevant passage more than the question does. TransformQueryEngine applies a transform in front of any query engine:
from llama_index.core.indices.query.query_transform import HyDEQueryTransformfrom llama_index.core.query_engine import TransformQueryEngine
hyde = HyDEQueryTransform(include_original=True)hyde_engine = TransformQueryEngine(vector_index.as_query_engine(), query_transform=hyde)print(hyde_engine.query("notice period"))HyDE adds one LLM call per query and can hurt when the model does not know the domain, since a wrong hypothetical answer pulls retrieval in the wrong direction. Compare hit rate with and without it on your evaluation set (section 21.2) before adopting it. StepDecomposeQueryTransform and DecomposeQueryTransform in the same module break a question into sub-steps; for multi-document questions SubQuestionQueryEngine (19.4) is usually simpler.
19.8 Answers with citations
Section titled “19.8 Answers with citations”CitationQueryEngine splits the retrieved chunks into smaller numbered sources and prompts the LLM to cite them as [1], [2]. The numbered sources are returned in response.source_nodes, so you can map each marker back to a document:
from llama_index.core.query_engine import CitationQueryEngine
citation_engine = CitationQueryEngine.from_args( vector_index, similarity_top_k=4, citation_chunk_size=256, # size of each numbered source)response = citation_engine.query("What is LlamaIndex?")print(response) # text containing markers such as [1]for i, source in enumerate(response.source_nodes, start=1): print(i, source.node.metadata.get("file_name"), source.node.get_content()[:60])The model decides where to place markers, so check that cited numbers exist before showing them to users, and do not treat a marker as proof that the cited text supports the claim; FaithfulnessEvaluator (section 21.1) tests that.
19.9 Recursive retrieval over summaries
Section titled “19.9 Recursive retrieval over summaries”IndexNode is a node that points at something else by index_id. Index short summaries (or table captions, or questions a chunk answers) at the top level, and RecursiveRetriever follows the matching pointer to a query engine or retriever that holds the detail. Use it when whole documents are too large to embed usefully but a summary identifies them well:
from llama_index.core import VectorStoreIndexfrom llama_index.core.query_engine import RetrieverQueryEnginefrom llama_index.core.retrievers import RecursiveRetrieverfrom llama_index.core.schema import IndexNode, TextNode
keys_engine = VectorStoreIndex([TextNode(text="Rotate keys every 90 days.")]).as_query_engine()billing_engine = VectorStoreIndex([TextNode(text="Invoices are issued monthly.")]).as_query_engine()
top = VectorStoreIndex([ IndexNode(text="Key rotation and credential policy", index_id="keys"), IndexNode(text="Billing and invoices", index_id="billing"),])
retriever = RecursiveRetriever( "root", retriever_dict={"root": top.as_retriever(similarity_top_k=1)}, query_engine_dict={"keys": keys_engine, "billing": billing_engine},)engine = RetrieverQueryEngine.from_args(retriever)index_id values must match a key in query_engine_dict (the target is queried and its answer returned as a node) or in retriever_dict (the target is retrieved from and its nodes returned). A chain of IndexNodes can be followed more than one level deep. Pass verbose=True to print each hop while tuning.
19.10 Document summary index
Section titled “19.10 Document summary index”DocumentSummaryIndex asks the LLM for one summary per document at build time, then retrieves whole documents by matching the query against those summaries. It stores one summary per document, but producing it costs at least one LLM call per document up front, and several for a document too long for the context window (tree_summarize summarises each chunk, then summarises those summaries). That suits a modest collection of long documents where chunk-level retrieval loses the document’s theme:
from llama_index.core import DocumentSummaryIndex, get_response_synthesizerfrom llama_index.core.indices.document_summary import DocumentSummaryIndexLLMRetriever
summary_index = DocumentSummaryIndex.from_documents( documents, # give each Document a stable doc_id response_synthesizer=get_response_synthesizer(response_mode="tree_summarize", use_async=True), show_progress=True,)print(summary_index.get_document_summary(documents[0].doc_id))
# Default retriever: embeds the query and compares it with the summariesretriever = summary_index.as_retriever(similarity_top_k=2)
# Alternative: let the LLM choose among summaries (more calls, no embeddings involved)llm_retriever = DocumentSummaryIndexLLMRetriever(summary_index, choice_batch_size=10, choice_top_k=2)Retrieval returns the chunks of the chosen documents, so pair it with a post-processor or a small similarity_top_k if the documents are long. The summaries are persisted with the index; re-ingesting a changed document means re-summarising it.
20. Prompts
Section titled “20. Prompts”from llama_index.core import PromptTemplate, ChatPromptTemplatefrom llama_index.core.llms import ChatMessage
qa_template = PromptTemplate( "Use the context to answer.\n---\n{context_str}\n---\nQuestion: {query_str}\nAnswer: ")
chat_template = ChatPromptTemplate([ ChatMessage(role="system", content="You are a precise technical writer."), ChatMessage(role="user", content="Explain {topic} in two sentences."),])messages = chat_template.format_messages(topic="vector databases")Inspecting and overriding a query engine’s prompts
query_engine = index.as_query_engine()for key in query_engine.get_prompts(): print(key) # e.g. response_synthesizer:text_qa_template
query_engine.update_prompts({"response_synthesizer:text_qa_template": qa_template})Agents take a system_prompt; the ReAct prompt itself can be replaced with react_agent.update_prompts({"react_header": ...}).
Few-shot examples are just text in the template — or, for many examples, retrieve the most similar ones per query with a small vector index and format them into the prompt via function_mappings.
21. Evaluation
Section titled “21. Evaluation”21.1 Evaluating responses
Section titled “21.1 Evaluating responses”from llama_index.core.evaluation import FaithfulnessEvaluator, RelevancyEvaluator
judge = OpenAI(model="gpt-6-luna")faithfulness = FaithfulnessEvaluator(llm=judge)relevancy = RelevancyEvaluator(llm=judge)
query = "What is LlamaIndex?"response = query_engine.query(query)
f = faithfulness.evaluate_response(response=response)r = relevancy.evaluate_response(query=query, response=response)print(f.passing, r.passing, r.feedback)Also available: CorrectnessEvaluator (against a reference answer), AnswerRelevancyEvaluator, ContextRelevancyEvaluator, SemanticSimilarityEvaluator, and BatchEvalRunner to run many evaluators over many questions concurrently. Each of these calls the judge LLM per question.
21.2 Evaluation datasets
Section titled “21.2 Evaluation datasets”An evaluation set built from your own documents is worth more than any generic benchmark. The cheapest useful one is a retrieval set: for each chunk, ask an LLM for questions that the chunk answers, and record that chunk as the expected hit. DatasetGenerator generates the questions; RetrieverEvaluator scores a retriever against them.
import json
from llama_index.core import VectorStoreIndexfrom llama_index.core.evaluation import DatasetGenerator, RetrieverEvaluatorfrom llama_index.core.node_parser import SentenceSplitter
nodes = SentenceSplitter(chunk_size=512).get_nodes_from_documents(documents)index = VectorStoreIndex(nodes)
async def build_retrieval_set(nodes, llm, per_node=2): """Generate questions per chunk; the chunk that produced a question is the expected hit.""" pairs = [] for node in nodes: gen = DatasetGenerator([node], llm=llm, num_questions_per_chunk=per_node) for question in await gen.agenerate_questions_from_nodes(): pairs.append({"query": question, "expected_ids": [node.node_id]}) return pairs
async def main(): pairs = await build_retrieval_set(nodes, llm) with open("retrieval_eval.json", "w") as f: json.dump(pairs, f, indent=2) # review by hand, then commit it
evaluator = RetrieverEvaluator.from_metric_names( ["hit_rate", "mrr"], retriever=index.as_retriever(similarity_top_k=3) ) results = [await evaluator.aevaluate(**p) for p in pairs] hit_rate = sum(r.metric_vals_dict["hit_rate"] for r in results) / len(results) mrr = sum(r.metric_vals_dict["mrr"] for r in results) / len(results) print(f"hit rate {hit_rate:.2f} MRR {mrr:.2f} ({len(results)} questions)")Notes:
- The expected IDs are node IDs, so they must stay stable when you re-ingest. Give the node parser a deterministic
id_func, as the production guide does. - Generated questions are often too easy (they reuse the chunk’s wording). Read through the set, delete weak questions and add real user questions from your logs.
- For a question-and-reference-answer set,
DatasetGenerator(nodes, llm=llm).agenerate_dataset_from_nodes()returns aQueryResponseDataset(save_json/from_json) to use withCorrectnessEvaluator. It emits a deprecation warning that points toLabelledRagDataset, but thellama_datasetmodule (LabelledRagDataset,RagDatasetGenerator) is not included inllama-index-core0.14.26, soDatasetGeneratoris what the current release offers. Itsfrom_documents()constructor needs spaCy installed; building it from nodes, as above, does not.
22. Migrating from the legacy agent API
Section titled “22. Migrating from the legacy agent API”llama-index-core 0.13.0 (July 2025) removed the old agent stack; 0.14.0 (September 2025) moved to llama-index-workflows 2.x, which removed workflow checkpointers, sub-workflows and the deprecated Workflow.send_event / Workflow.stream_events methods (both remain on Context / the handler).
| Old (≤ 0.12) | Current |
|---|---|
from llama_index.core.agent import ReActAgent; ReActAgent.from_tools(tools, llm=llm) | from llama_index.core.agent.workflow import ReActAgent; ReActAgent(tools=tools, llm=llm) |
OpenAIAgent.from_tools(...), FunctionCallingAgent.from_tools(...) | FunctionAgent(tools=..., llm=...) |
AgentRunner, step workers, StructuredPlanningAgent | FunctionAgent / AgentWorkflow / a custom Workflow |
agent.chat("...") (sync) | await agent.run(user_msg="...", ctx=ctx) |
agent.stream_chat(...) | handler = agent.run(...); async for ev in handler.stream_events() |
memory=ChatMemoryBuffer(...) on the agent | Context(agent) per conversation, or memory=Memory.from_defaults(...) on run() |
index.as_chat_engine() returning an agent | Returns CondensePlusContextChatEngine; build an agent explicitly if you need tools |
ServiceContext | Settings |
QueryPipeline | A custom Workflow |
llama-agents (ControlPlaneServer, AgentService) | AgentWorkflow in-process; llamactl / LlamaAgents for deployment |
run() must be called while an event loop is running — asyncio.run(agent.run(...)) fails with RuntimeError: no running event loop. From synchronous code, wrap the call in a coroutine: async def main(): return await agent.run(user_msg=...) and then asyncio.run(main()).
Revision history
Section titled “Revision history”| Date | Version | Changes |
|---|---|---|
| 2026-10-29 | 0.14.25 / workflows 2.26.1 | Added recursive retrieval with IndexNode and RecursiveRetriever (19.9) and DocumentSummaryIndex (19.10). |
| 2026-10-24 | 0.14.25 / workflows 2.26.1 | Added response modes and explicit query-engine assembly (3.7), chat engine modes and streaming (3.8), metadata filter operators (3.6), chunking strategies and metadata extractors (3.11), sentence-window and auto-merging retrieval (19.6), HyDE query transformation (19.7) and citation query engine (19.8). Documented max_iterations default and early_stopping_method, and the refresh_ref_docs behaviour after from_vector_store. |
| 2026-10-24 | 0.14.25 / workflows 2.26.1 | Section 13 cut to an introduction; Workflows now have their own page. Its retry example used the deprecated ConstantDelayRetryPolicy. |
| 2026-10-21 | 0.14.25 | Added property graphs (3.9), multimodal (3.10), WorkflowServer/WorkflowClient/llamactl serving (14), LlamaParse via the llama-cloud SDK (18.2) and evaluation datasets (21.2). SubQuestionQueryEngine example now passes question_gen (the default needs a package that cannot be installed with current releases). |
| 2026-10-21 | 0.14.25 | Rewritten. Previous edition used APIs that do not exist (AgentWorkflow.from_tools, from llama_index.core.workflow import AgentWorkflow, llama_index.core.tools.Tool, BM25Retriever in core, llama_agents) and wrongly stated that ReActAgent raises ImportError. All examples now use the workflow-based agents and were checked against an installed llama-index-core. Version banner now tracks the site’s version registry. |
| 2026-05-19 | 0.14.22 | Patch-version bump. |
| 2026-04-03 | 0.14.20 | Agent sections moved off the removed legacy classes (incompletely). |
| November 2025 | 0.14.8 | First documented version. |