The gap between what AI agents can do with text and what they can do on the web has been surprisingly persistent. Large language models got very good at reasoning, coding, and generating content — but asking an agent to fill in a form, click through a checkout flow, or navigate a site that dynamically loads content was still awkward and brittle.

That’s genuinely changing. A new category of AI-native browser automation tools has emerged that makes building web-browsing agents feel more like writing a conversation than configuring a scraper. If you’re building agents that need to interact with web UIs — whether for monitoring, data collection, workflow automation, or testing — here’s the lay of the land.

Why Traditional Browser Automation Falls Short

Tools like Selenium and the older versions of Playwright are excellent for automating predictable, well-structured web interactions. Write a script that clicks this button, fills that field, waits for this selector, extracts that text. It works brilliantly until the site changes, until you hit a CAPTCHA, until the page renders differently depending on user history, or until you need the agent to exercise judgement about what to do when something unexpected appears.

The fundamental problem is that traditional automation is rigid. It follows a script; it can’t improvise. An AI agent browsing the web needs to be able to look at a page it’s never seen before and figure out how to accomplish a goal — not follow a pre-written selector path.

What Stagehand Does Differently

Stagehand, built by Browserbase, is probably the most talked-about new tool in this space. Its core insight is that instead of writing automation scripts, you describe what you want to happen in natural language — and the library uses a vision-capable LLM to figure out how to execute it.

The API is deliberately simple. You have three main actions: act (take an action on the page), extract (pull structured data from the page), and observe (get a description of what’s on the page). You write instructions the way you’d describe a task to a person.

import { Stagehand } from "@browserbasehq/stagehand";

const stagehand = new Stagehand({ env: "LOCAL" });
await stagehand.init();

const page = stagehand.page;
await page.goto("https://example-ecommerce.com");

// Natural language actions  --  no CSS selectors required
await stagehand.act({ action: "Search for wireless headphones" });
await stagehand.act({ action: "Click the first result under £100" });
await stagehand.act({ action: "Add it to the basket" });

// Structured extraction
const productInfo = await stagehand.extract({
  instruction: "Get the product name, price, and delivery date",
  schema: z.object({
    name: z.string(),
    price: z.number(),
    deliveryDate: z.string()
  })
});

The agent isn’t following a static script — it’s looking at the actual rendered page and using computer vision to identify where the search box is, what the first result looks like, and where the add-to-basket button is. If the site redesigns tomorrow, the same natural language instructions will still work.

Playwright as the Execution Layer

Stagehand runs on top of Playwright, which handles the actual browser control (Chromium, Firefox, or WebKit). This is a sensible architecture choice: Playwright is battle-tested, maintained by Microsoft, and handles the hard parts of browser automation — JavaScript rendering, network interception, waiting for dynamic content, handling iframes.

For teams already using Playwright for testing, Stagehand slots in naturally. You can mix explicit Playwright selectors (for the parts of your workflow where you know exactly what you’re doing) with Stagehand’s natural language actions (for the parts where you need adaptability).

Plain Playwright with an LLM layer also remains a valid approach. Several teams are using Playwright’s page.evaluate() to extract semantic information from pages and feeding it to a language model for decision-making, rather than using vision-based approaches. This tends to be faster and cheaper than vision, though less robust on highly visual or canvas-based interfaces.

Browserbase for Production Deployments

Running a headful browser on your server is manageable for occasional tasks but creates real problems at scale: memory usage, blocked IPs, CAPTCHA challenges, and the complexity of managing browser pools. Browserbase provides managed browser infrastructure — essentially a cloud service where you send browser sessions rather than running Chrome on your own infrastructure.

The integration with Stagehand is built-in, so switching from local development to production Browserbase infrastructure is a one-line config change. The platform also handles rotating residential proxies, CAPTCHA solving via third-party services, and session state persistence (useful for agents that need to maintain login state across runs).

For production web agents that need to run reliably at scale, this kind of managed infrastructure makes a significant difference to operational complexity.

When to Use AI Browser Automation

It’s worth being honest about the trade-offs. AI browser automation is slower and more expensive than traditional selector-based automation. Every act call costs LLM API tokens and adds latency. If you’re automating a high-volume, well-structured workflow on a site that never changes — like a daily data export from a known internal tool — plain Playwright with explicit selectors is almost always the right call.

Where AI browser automation shines is adaptability and coverage:

Monitoring competitor websites for price or content changes, where the structure varies. Automating interactions with external tools and portals you don’t control. Building agents that need to research topics across multiple sources and collate findings. Handling customer-facing workflows that need to navigate UIs as a real user would. Integration testing for complex dynamic applications where the UI logic is hard to express in static selectors.

The agent-plus-browser combination is also becoming genuinely useful for automating tasks that previously required RPA tools like UiPath or Automation Anywhere, at a fraction of the cost and with dramatically better handling of unexpected states.

Getting Started

Stagehand is open source (MIT licensed) and published as an npm package. The repository includes examples for common patterns: form filling, multi-step navigation, data extraction with Zod schemas, and integration with LangChain and other agent frameworks. The free tier of Browserbase provides enough credits to prototype and test without spending money upfront.

If you’ve been frustrated by the brittleness of traditional browser automation, or you’re building agents that need to work with the web as it actually is rather than as you wish it were, this is a meaningful step forward. The technology isn’t magic — you still need to think carefully about what you’re automating and why — but the gap between “the AI can reason about this” and “the AI can actually do this on a real website” is closing fast.