Intent in the loop: an agent that lives inside the browser

Despite three decades of web evolution, the way we use a browser has barely changed: it renders the page, and we drive every atomic action.

Ask it to compare product specifications across five shops, pull structured citations from a dozen linked PDFs, or fill a long form with repetitive entries, and it offers nothing beyond navigation. Every objective is decomposed by hand into clicks, scrolls and keystrokes.

Why the existing answers fall short

• Headless automation — Selenium, Playwright, Puppeteer — gives precise control but runs on hand-written scripts with no understanding, and breaks on dynamic interfaces.
• Cloud agents offer a natural-language interface but send session cookies, history and page content to remote servers — a privacy cost that rules them out for many workflows.
• Browser extensions are sandboxed by Manifest V3 and content-security policies, which block deep protocol access and hardware-accurate input.

One architectural decision

NetSurf runs the reasoning loop directly inside the Electron main process, next to the browser host, with full access to the Chrome DevTools Protocol. Nothing leaves the device except the task prompt and sanitized page observations, sent to the model provider the user chooses — no cloud execution cluster, no background daemon, no extension sandbox.

Two mechanisms make it work. Physical event synthesis turns the agent’s plans into coordinate-level input through CDP, so interactions pass modern anti-bot defences. And the intent-in-the-loop design keeps the agent’s reasoning visible as it acts, with a single reply field for correcting it mid-task without interrupting the loop.

What we measured

On a 50-task subset of WebArena, NetSurf completes 76% of tasks — 24 points above a deterministic Playwright baseline — in roughly 350 MB of memory, comparable to an unmodified Chromium. Network-level DOM sanitization, which uses Ghostery filter lists to drop ad and tracker requests before the DOM is built, cuts the median context the model reads by 38% on commercial sites; ablations show that it and visual grounding each contribute to the margin independently.

NetSurf is joint work with Kritavya Patel, Yugen Jarwal and Prof. Kiran Sharma at BML Munjal University. The paper is on the research page.