TL;DR:
- Deep research agents differ from basic RAG by running multiple search iterations, evaluating sources, and synthesising across many results — not just retrieving and reading
- The core loop is: decompose question → search → evaluate results → identify gaps → refine query → synthesise → verify
- Practical implementations use Exa or Perplexity Sonar for search, a reasoning model for gap identification, and a synthesis model for the final output
Single-query RAG has a fundamental ceiling. You retrieve documents matching your query, pass them to the model, and get an answer scoped to whatever you happened to retrieve. Deep research agents break that ceiling by running the retrieval-reasoning loop multiple times, adapting their search strategy based on what they find and what gaps remain.
The commercial systems that popularised this pattern — Perplexity’s Deep Research, OpenAI’s o3-based research mode, Gemini Deep Research — all share the same underlying architecture: a reasoning model orchestrating multiple search rounds, building a picture iteratively rather than in a single pass. The pattern is reproducible with open tools.
What Makes Research “Deep”
The distinction between a RAG pipeline and a deep research agent comes down to three capabilities:
Iterative query refinement. A single-pass system searches once with your original question. A deep research agent evaluates whether the first round of results actually answered the question, identifies what’s still unknown, and generates new search queries targeting those gaps. A question like “how has the EU AI Act affected LLM deployment practices in financial services?” might require separate searches on the regulation’s requirements, enforcement guidance, actual compliance implementations, and current industry responses — none of which a single query captures well.
Source quality evaluation. Not all search results are equally useful. A research agent evaluates sources for relevance, recency, credibility, and potential bias. It deprioritises obviously low-quality content and flags contradictions between sources for resolution rather than blindly accepting the first result.
Structured synthesis over raw retrieval. The final output isn’t a dump of retrieved passages. It’s a synthesised response that resolves contradictions, cites sources, distinguishes established facts from uncertain claims, and structures the answer in a form the user actually needs.
The Core Research Loop
A minimal deep research agent runs this loop:
async def deep_research(question: str, max_iterations: int = 5) -> ResearchResult:
research_state = ResearchState(
original_question=question,
findings=[],
sources=[],
gaps=[question] # Start with the question itself as the initial gap
)
for iteration in range(max_iterations):
if not research_state.gaps:
break
# Generate search queries from identified gaps
queries = await generate_search_queries(
gaps=research_state.gaps,
existing_findings=research_state.findings
)
# Execute searches in parallel
search_results = await asyncio.gather(*[
search(query) for query in queries[:3] # Limit parallel searches
])
# Evaluate and filter results
useful_results = await evaluate_results(
results=flatten(search_results),
question=question,
existing_findings=research_state.findings
)
research_state.findings.extend(useful_results)
research_state.sources.extend([r.url for r in useful_results])
# Identify what's still unknown
research_state.gaps = await identify_gaps(
question=question,
current_findings=research_state.findings
)
if await sufficient_coverage(question, research_state.findings):
break
return await synthesise(question, research_state)Each step is a separate LLM call with a focused prompt. The query generation step is where a reasoning model adds real value — it needs to understand what the first round found and what’s still missing to produce genuinely useful follow-up queries.
Prompt Patterns for Each Step
Query decomposition. The initial step breaks the original question into sub-questions. For a research question like “what’s the current state of open-source LLM fine-tuning tooling?”, a good decomposition produces: what are the leading tools (Axolotl, LLaMA-Factory, Unsloth, TRL), what are their relative strengths and use cases, what hardware do they require, and what’s changed in the last six months. These become the first search queries.
You are a research strategist. Given a research question, identify the 3-5
specific sub-questions that need to be answered to fully address the question.
Focus on: facts that can be looked up, comparisons that need data, and
recent developments that require current sources.
Question: {question}
Return as a JSON array of search queries, ordered by priority.Gap identification. After each search round, the agent needs to honestly assess what’s still unknown. This prompt should be skeptical — it’s easy to conclude coverage is sufficient prematurely.
You are evaluating research coverage. Given the original question and the
findings gathered so far, identify what important aspects are still
unaddressed, contradicted without resolution, or only partially covered.
Original question: {question}
Current findings summary: {findings_summary}
Return a list of specific gaps. If coverage is genuinely complete, return
an empty list. Be conservative — missing information is worse than an
extra search round.Source quality evaluation. Pass the raw search results through an evaluation step before incorporating them into your findings:
Evaluate these search results for usefulness in answering the research question.
For each result, assess: relevance (0-10), likely credibility (high/medium/low),
recency adequacy (for this type of question), and any contradictions with
existing findings.
Return only results with relevance >= 6. For contradictions, flag them
explicitly rather than silently choosing one version.Tooling Choices
Search APIs. Exa’s neural search API works well for research tasks because it retrieves semantically relevant content rather than just keyword matches — useful when your queries are exploratory rather than looking for specific known pages. Perplexity’s Sonar API adds LLM-powered synthesis on top of web retrieval. Tavily is a lighter-weight option with good developer ergonomics. For many research tasks, combining a broad web search (Google via SerpAPI or Bing via Azure) with a more targeted API like Exa for deep dives produces better coverage than either alone.
Models by step. Use a reasoning model (o3, claude-3-7-sonnet with extended thinking, Gemini 2.5 Pro) for the gap identification and query planning steps — these require actual multi-step reasoning about what’s known and unknown. Use a faster model for source evaluation and the synthesis drafting pass.
Orchestration. LangGraph is the most natural fit for this pattern because the research loop is genuinely stateful — you need to carry findings, sources, and gaps across iterations. The graph nodes map directly to the loop steps, and the conditional edge from gap identification (continue or stop) is cleanly expressed in LangGraph’s routing pattern.
Handling Common Failure Modes
Search loops. Without a clear stopping condition, agents can loop indefinitely, generating increasingly marginal queries. The sufficient_coverage check should be a genuine semantic evaluation, not just a count of sources found. Alternatively, add a hard cap on iterations and rely on the synthesis step to flag insufficient coverage explicitly.
Source hallucination. When models summarise retrieved content, they sometimes confuse details across sources or add plausible-sounding specifics not present in any source. Counter this by having the synthesis step cite specific passages from retrieved content rather than summarising from memory, and running a verification pass that checks each claim against the cited source.
Recency drift. Research agents will surface older content if it’s highly relevant by other measures. For time-sensitive topics, add explicit recency filters to your search queries and include a step that identifies findings where recency is material and flags them for freshness verification.
Contradictory sources. Real-world research frequently produces genuine contradictions between sources. Rather than silently resolving them in favour of the most confident-sounding source, flag contradictions explicitly in the final output. “Sources disagree on this: [source A says X, source B says Y]” is more useful than a false resolution.
A Working Example: Market Research Agent
A practical implementation for competitive market research uses this structure: an initial decomposition into product features, pricing, customer reviews, recent news, and competitor positioning; three rounds of parallel search; a contradiction resolution step specifically for pricing and capability claims (where sources frequently disagree); and a structured synthesis that produces a comparison table plus a narrative summary.
The key insight from production deployments is that the synthesis step matters as much as the retrieval. An agent that gathers excellent sources but produces a disorganised or uncited synthesis is not useful. Structuring the synthesis output — with explicit source citations, confidence levels for uncertain claims, and clear separation between established facts and analyst interpretations — is what turns retrieved content into actionable research.
The pattern scales from a simple five-step loop in a few hundred lines of Python to a full agent framework with parallelised search, multi-model routing, and citation management. The core logic doesn’t change.