TL;DR:
- Hatchet is an open-source durable task queue that runs on Postgres — no Redis or separate queue server required — giving AI agent workflows step-level persistence and automatic retries
- Python and TypeScript SDKs let you define workflows as decorated functions with explicit step dependencies, making it easy to express multi-stage agent pipelines with fan-out, fan-in, and conditional branching
- Built-in concurrency controls, rate limiting, and a real-time dashboard make Hatchet practical for production AI workloads that need observability and failure recovery
Building reliable AI agent pipelines in production exposes a fundamental tension: LLM calls are slow, expensive, and fail in ways that ordinary web request handling doesn’t account for. A document processing pipeline that chains extraction, summarization, and classification steps across three LLM calls shouldn’t restart from scratch when the third step fails on a transient API error. An agent that spawns sub-agents for parallel research should have visibility into each sub-task’s status. A content moderation workflow that processes 1,000 items per hour should not overwhelm a downstream API by sending everything at once.
Hatchet is an open-source durable task queue designed to solve exactly these problems for AI agent workflows. It stores workflow state in Postgres rather than in-memory, which means step results persist across process restarts. Steps are retried automatically based on configurable policies. Fan-out and fan-in are first-class primitives. And the whole thing ships with a dashboard that shows you exactly what’s happening.
Why Postgres, Not Redis
Most background job systems — BullMQ, Celery, RQ, Sidekiq — use Redis as their backing store. Redis is fast, but it’s an additional infrastructure dependency, and job state is transient by design. When your Redis instance restarts or a job exceeds memory limits, the state is gone.
Hatchet uses Postgres as its sole backing store. Every workflow run, step result, and event is a row in the database. This makes job state durable by default, auditable via SQL, and compatible with existing backup and recovery procedures. For teams already running Postgres, Hatchet introduces zero new infrastructure.
The tradeoff is throughput: Postgres-backed queues can’t match Redis for extremely high-volume, low-latency job dispatch. For AI agent workloads — where each step involves at minimum a few hundred milliseconds of LLM inference — this is not a meaningful constraint.
Defining Workflows
Hatchet workflows are Python or TypeScript classes where each method decorated with @hatchet.step becomes a durable step. Dependencies between steps are declared explicitly, which lets Hatchet build a DAG and execute independent steps in parallel.
from hatchet_sdk import Hatchet, Context
hatchet = Hatchet()
@hatchet.workflow(name="research-agent", on_events=["research:start"])
class ResearchWorkflow:
@hatchet.step()
async def search_web(self, context: Context) -> dict:
query = context.workflow_input()["query"]
results = await web_search(query)
return {"results": results}
@hatchet.step(parents=["search_web"])
async def extract_key_facts(self, context: Context) -> dict:
results = context.step_output("search_web")["results"]
facts = await llm_extract(results, "Extract key facts")
return {"facts": facts}
@hatchet.step(parents=["search_web"])
async def assess_source_quality(self, context: Context) -> dict:
results = context.step_output("search_web")["results"]
quality = await llm_assess(results, "Rate source credibility")
return {"quality_scores": quality}
@hatchet.step(parents=["extract_key_facts", "assess_source_quality"])
async def compile_report(self, context: Context) -> dict:
facts = context.step_output("extract_key_facts")["facts"]
quality = context.step_output("assess_source_quality")["quality_scores"]
report = await llm_compile(facts, quality)
return {"report": report}In this example, extract_key_facts and assess_source_quality execute in parallel after search_web completes, and compile_report waits for both. Hatchet builds this execution plan from the parents declarations — you don’t manage thread pools or asyncio coordination directly.
Fan-Out for Parallel Sub-Agent Execution
One of the most useful patterns in AI agent workflows is spawning multiple sub-tasks and collecting their results. Hatchet supports this with spawn_workflow, which creates child workflow runs that can be awaited individually or in bulk.
@hatchet.step()
async def parallel_research(self, context: Context) -> dict:
queries = context.workflow_input()["queries"]
# Spawn a child workflow for each query
child_runs = [
context.spawn_workflow(
"research-agent",
input={"query": q},
key=f"research-{i}"
)
for i, q in enumerate(queries)
]
# Await all results
results = await asyncio.gather(*[run.result() for run in child_runs])
return {"all_results": results}Child workflow runs are independently tracked in the dashboard and retried according to their own retry policies. If one sub-agent fails, the others continue, and the parent workflow receives the failure rather than silently dropping it.
Concurrency Controls and Rate Limiting
AI agent workflows frequently need to limit concurrency — either to respect API rate limits or to control cost. Hatchet provides concurrency controls at the workflow and step level.
@hatchet.workflow(
name="llm-pipeline",
concurrency=hatchet.concurrency(
max_runs=10,
limit_strategy=ConcurrencyLimitStrategy.GROUP_ROUND_ROBIN,
expression="input.customer_id"
)
)
class LLMPipeline:
...The GROUP_ROUND_ROBIN strategy ensures no single customer’s workflows can starve others by monopolizing the concurrency limit. For rate limiting against external APIs, Hatchet’s rate limit primitives let you declare token-per-second or request-per-minute constraints that apply globally across all workers.
Retry Configuration
Step-level retry configuration is where Hatchet’s durability model pays off for AI workloads. LLM providers return rate limit errors, transient 5xx responses, and timeouts in ways that make blind retry inappropriate — you want exponential backoff with jitter, and you want to distinguish retryable errors from logical failures.
@hatchet.step(
retries=3,
backoff_factor=2.0,
backoff_max_seconds=30
)
async def call_llm(self, context: Context) -> dict:
try:
response = await anthropic.messages.create(
model="claude-opus-4-7",
max_tokens=2048,
messages=[{"role": "user", "content": context.step_output("prepare")["prompt"]}]
)
return {"response": response.content[0].text}
except anthropic.RateLimitError:
raise # Will be retried with backoff
except anthropic.BadRequestError as e:
context.log(f"Non-retryable error: {e}")
raise hatchet.NonRetryableError(str(e)) # Will not retryNonRetryableError propagates immediately to the workflow run as a failure without consuming retry budget.
The Dashboard
Hatchet ships with a real-time dashboard that shows every workflow run, its current step, step outputs, retry counts, and timing. For AI agent workflows — where a single run might involve dozens of LLM calls over several minutes — this visibility is essential for debugging production failures.
The dashboard is particularly useful for understanding fan-out behaviour: child workflow runs are displayed as trees under their parent, making it easy to see which sub-agents succeeded, which are still running, and which failed.
Self-Hosting vs. Cloud
Hatchet is fully open-source under the MIT license. Self-hosting requires a Postgres database, a Redis instance (for pub/sub event broadcasting to workers — distinct from job storage), and the Hatchet engine binary, which is distributed as a Docker image.
# docker-compose.yml
services:
hatchet-engine:
image: ghcr.io/hatchet-dev/hatchet/hatchet-engine:latest
environment:
DATABASE_URL: "postgresql://hatchet:password@postgres:5432/hatchet"
REDIS_URL: "redis://redis:6379"
depends_on:
- postgres
- redis
hatchet-dashboard:
image: ghcr.io/hatchet-dev/hatchet/hatchet-dashboard:latest
ports:
- "8080:80"Hatchet also offers a managed cloud service at hatchet.run if you’d rather not operate the engine yourself. Pricing is consumption-based on workflow runs rather than a flat seat fee.
TypeScript SDK
The TypeScript SDK follows the same decorator-based pattern, making it natural to use in Next.js API routes, Express apps, or serverless environments.
import Hatchet from "@hatchet-dev/typescript-sdk";
const hatchet = await Hatchet.init();
const researchWorkflow = hatchet.workflow({
name: "research-agent",
steps: [
{
name: "search",
run: async (ctx) => {
const { query } = ctx.workflowInput();
const results = await webSearch(query);
return { results };
},
},
{
name: "summarize",
parents: ["search"],
run: async (ctx) => {
const { results } = ctx.stepOutput("search");
const summary = await llm.summarize(results);
return { summary };
},
},
],
});
// Trigger from an API route
await hatchet.run("research-agent", { query: "edge computing trends 2026" });When to Choose Hatchet
Hatchet sits in a specific niche: teams that want durable execution for AI workflows without the operational complexity of Temporal or the Kubernetes dependency of some alternatives.
It makes the most sense when:
- Your stack already runs Postgres and you want minimal new infrastructure
- You’re building in Python or TypeScript
- Your AI agent workflows involve multi-step pipelines where partial failures should resume, not restart
- You need fan-out to parallel sub-agents with result collection
- You want a built-in dashboard rather than instrumenting your own observability
It’s probably not the right choice if you need sub-second job latency at high volume, or if you’re operating at a scale where Temporal’s more mature ecosystem is worth the overhead.
For most production AI agent workflows — document processing, research pipelines, content generation chains, multi-step data extraction — Hatchet provides the durability guarantees you need with a small operational footprint.