AIP-63: BROWSER (browser/v1): browser provider manifest, instance lifecycle, profile consent
A profile over AIP-30 DRIVER for browsers that agents drive. A browser manifest is ordinary AIP-30 frontmatter of any kind plus a `browser:` block declaring location, capabilities, profile modes, lifecycle and sinks. Fixes the tool contracts with a capability gate, an instance lifecycle with supervision, a consent model (per-domain grants, append-only ledger, revoke) for seeding a browser with a user's logged-in state, and the `browser.recording/v1` artifact that makes what an agent did in a browser replayable, and site recipes.
| Field | Value |
|---|---|
| AIP | 63 (provisional, editors assign the final number) |
| Title | BROWSER (browser/v1): browser provider manifest, instance lifecycle, profile consent |
| Status | Draft |
| Type | Schema |
| Domain | drivers.sh |
| Requires | AIP-1, AIP-2, AIP-7 (GOVERNANCE), AIP-14 (TOOL), AIP-17 (RUNNER), AIP-19 (SECRETS), AIP-30 (DRIVER), AIP-59 (pairing) |
| Reference Impl | @agentproto/driver-browser |
| Resources | ./resources/aip-63: BROWSER.schema.json three example manifests under examples/ (camofox, chromium, browserbase), RECORDING.schema.json and a sample recording under examples/recording-sample/, RECIPE.schema.json and a LinkedIn recipe example under examples/recipe-linkedin/ |
Abstract
A browser that an agent drives is a different thing from a tool that an
agent calls: it is a long-lived process with a lifecycle, it carries
identity (cookies, storage), and it can be local or someone else's
machine. BROWSER.md declares such a browser as an
AIP-30 DRIVER of any kind, plus a browser: block that
states where it runs, what it can do, how it starts and stops, which
profile modes it supports and where session material may go. The tools
it serves are AIP-14 tools with ids such as browser.navigate, gated by
the declared capabilities: a tool the provider cannot serve fails with a
typed unsupported error that names the missing capability.
The AIP also fixes the instance lifecycle (with supervision rules that prevent silent restart loops and orphaned processes) and the consent model for seeding a browser from a user's profile: explicit per-domain grants, an append-only ledger, and a revoke that deletes derived material.
Finally it fixes the recording artifact (browser.recording/v1): a
directory of JSON Lines and JPEG files that lets a human watch, live or
afterwards, what an agent did in a browser. One format serves both
capture modes (one frame per action, continuous screencast), masks
password fields, and expires on a retention clock.
Motivation
Three parallel models of "a browser for agents" tend to coexist in a codebase: a process-lifecycle handle (start it, get a port), a page-control driver (navigate, click, evaluate), and per-vendor client code for hosted browsers. None of them can say, in one declarative place, that a provider has no CDP endpoint, or that a provider is a credential sink. The visible failures are ordinary and repeated:
- A network inspection tool fails with an opaque transport error on a
provider that never had a CDP endpoint, instead of saying "not
supported here, capability
cdp". - A supervisor restarts a browser that dies on every launch, once a minute, with no signal to anyone. Or it kills a slow but progressing launch and starts a second one on top of it.
- An idle reaper closes the tab in which a human was halfway through a login.
- A tool that "imports the user's Chrome session" copies every cookie for every domain, with no record of what was taken and no way to take it back.
- Chrome refuses
--remote-debugging-porton its default user-data directory, so the obvious way to drive the user's real profile is closed, and the workarounds tend to copy the whole profile.
A shared, declarative profile lets independent providers (a stealth Firefox variant behind REST, Playwright-managed Chromium, the user's system Chrome, a hosted browser service) be interchangeable behind one tool surface, and lets a host enforce the same consent rules whichever provider is in use.
Terminology
| Term | Meaning |
|---|---|
| Provider | A concrete browser implementation described by one BROWSER.md manifest. Not an AIP-52 BRIDGE. |
| Instance | One running browser (process tree, container or remote session) launched from a provider. |
| Target | One tab or page inside an instance. The unit tools act on. Tools take a target handle. |
| Driver handle | The object attach returns: a page-control interface bound to an instance. |
| Context | An isolated cookie and storage jar inside an instance. |
| Profile | Persistent or seeded browser state (cookies, storage) a context starts from. |
| Grant | A recorded consent, by a human, that lets named session material enter a store or an instance, or (an attach grant, S1) lets a host attach to a browser the user already runs. |
| Account | A host record of one account on one site inside one profile (linkedin.com, as some user, in a given profile), with the observed state of its login (session), the grants that authorize its use and where its cookies came from (source). A record, not a manifest field; it never authorizes anything by itself (C17). |
| Attached instance | An instance the host did not launch: a browser the user already runs, reached through a user-mediated opt-in (for Chrome, chrome://inspect/#remote-debugging). |
| Profile reference | profileRef: <instance>/<profile>, naming one profile inside one instance (for example chrome-user/perso). |
| Caller session | The agent session on whose behalf a tool call runs, identified by callerSessionId. Distinct from the ledger actor (who decided) and from a grant's sessionId (a saved browser identity). |
| Sink | Somewhere session material can end up: local (this host) or remote (a third party's machine). |
| Host | The process that runs providers and exposes the tools: an agentproto daemon, or a standalone browser server. |
| Supervisor | The host component that health-checks, restarts and reaps instances. |
Provider versus ADAPTER. In this AIP a provider is the inbound side:
it implements browser tools. AIP-52 ADAPTER is the
outward mirror of DRIVER: it projects a handle into a foreign
framework. They are unrelated. Reference npm packages are named
adapter-browser-* only by repository layout precedent (AIP-45's
reference implementations already live under adapters/); the name
carries no AIP-52 meaning.
Specification
The key words MUST, MUST NOT, SHOULD, SHOULD NOT and MAY are to be interpreted as described in AIP-1 (RFC 2119).
Rules are numbered so they can be cited and tested. Every rule in § Manifest, § Tool contracts, § Instance lifecycle, § Profile and consent and § Recording artifact names in parentheses the conformance level that tests it (see § Conformance). Business concerns such as metering, leases and licences are out of scope for this AIP.
1. Manifest
Relation to AIP-30. A browser manifest is AIP-30 DRIVER frontmatter
of any kind (sdk, http, cli, mcp, builtin) plus a
browser: block. kind keeps its AIP-30 meaning: it is the
transport axis (how the host reaches the provider), not the
"what is it" axis. A camofox-style REST service is kind: http;
Playwright in process is kind: sdk; a spawned browser binary with a
wrapper is kind: cli. Whatever the kind, that kind's own subtype
validation still applies unchanged (for kind: cli, the AIP-29 schema;
for http and sdk, the subtype AIP when it lands), in addition to
this AIP's block.
This differs from AIP-45, which layers over exactly one
subtype (AIP-29 CLI.md) and so can $ref that subtype's schema
directly. A browser identity spans kinds, so BROWSER.schema.json
composes with the AIP-30 supertype schema via allOf and adds the
browser: block; it does not replace or restrict kind.
M1 (core). A manifest MUST validate against
BROWSER.schema.json
(which includes the AIP-30 DRIVER schema) and against the schema of its
own kind.
The browser: block
| Field | Required | Type | Description |
|---|---|---|---|
location | Yes | local | remote | local: the browser runs on the host's machine. remote: it runs on a third party's machine, so the provider is a credential sink. |
display | Yes | (headless | headed)[] | Display modes the provider can launch. |
instances | Yes | single | multi | Whether more than one instance of this provider can run at once. |
endpoints | No | object | rest and cdp, each { url?, discovered? }. discovered: true means the URL is only known after launch (for example a remote session's CDP URL). A local provider SHOULD default to a loopback URL. |
capabilities | Yes | object | See below. Absent boolean means false. |
profiles | Yes | object | modes (ephemeral, persistent, import, native-login, clone, attach) and optional storedBy (host or provider). |
lifecycle | No | object | launch, health, stop, supervision. See § Instance lifecycle. |
sinks | Iff remote | object[] | { id, kind, operator?, region?, ack }. Every remote sink MUST declare ack: required. |
conformance | No | string[] | Levels the provider claims, always including core. |
Capabilities:
| Capability | Meaning |
|---|---|
cdp | A Chrome DevTools Protocol endpoint is available (endpoints.cdp). |
downloads | File downloads can be captured. |
recording | none, screencast or video. screencast: the provider can push frames continuously, so a recorder can write mode: screencast. video: it can also produce a video file natively. A recorder can always write mode: per-action from browser.screenshot, whatever this value is. The artifact is § 5. |
observers (inside core) | hosts that expose the observer seam |
stealth | The provider is designed to resist bot detection. A design goal, never a promise (see § Security considerations). |
fullPageScreenshot | Screenshots beyond the viewport. |
trustedInput | Input is produced at browser or OS level, so page script sees it as user input. |
multiTarget | More than one target per instance. |
aiActions | Natural-language actions are served by the provider. |
cookies | The host can set and read cookies on a context. |
storageState | A full storage state can be loaded into a context. |
persistentProfile | A named profile survives instance stop. |
profiles.storedBy says who persists session material between runs. A
provider that keeps its own jar (provider) puts that material outside
the host's store, and outside the host's at-rest protections (C11).
M2 (core). A provider with location: remote MUST declare at least
one remote sink, and every remote sink MUST declare ack: required.
M3 (core). capabilities.cdp: true and endpoints.cdp MUST appear
together.
M4 (core). A manifest MUST NOT list in implements[] a tool whose
capability gate is closed: browser.list_requests,
browser.get_request_body and browser.cdp_send need cdp;
browser.download needs downloads; browser.act needs aiActions.
M5 (core). profiles.modes containing persistent requires
capabilities.persistentProfile; containing import or clone
requires capabilities.cookies or capabilities.storageState.
M6 (core). Each claimed conformance level needs the capability it
tests: network needs cdp, download needs downloads, profile
needs cookies, storageState, or attach in profiles.modes, recording needs capabilities.recording
other than none.
Manifest keys and the AIP-30 universal fields cooperate rather than
duplicate: AIP-30 health_check remains the resolver's cheap
reachability probe, and browser.lifecycle.health is the supervisor's
loop. When health_check is absent a host MAY derive the probe from
lifecycle.health.
2. Tool contracts
A provider serves browser tools by binding AIP-14 tool ids in
implements[]. The ids below are reserved by this AIP. The contract
files (TOOL.md) are published with the reference implementation;
this AIP fixes ids, gates and levels.
| Tool id | Gate | Level | Summary |
|---|---|---|---|
browser.navigate | none | core | Load a URL in a target. |
browser.evaluate | none | core | Evaluate an expression in the page. |
browser.get_dom | none | core | Return the page DOM or a selector's subtree. |
browser.screenshot | fullPageScreenshot for the full-page option | core | Capture a viewport or full-page image. |
browser.click, browser.fill | trustedInput for the trusted option | interaction | Pointer and text input. |
browser.act | aiActions | interaction | Natural-language action. |
browser.tabs | multiTarget for more than one target | interaction | List, open, close targets; set keepAlive. |
browser.list_requests | cdp | network | List network requests seen by a target. |
browser.get_request_body | cdp | network | Fetch a captured response body. |
browser.cdp_send | cdp | network | Send a raw CDP command. |
browser.download | downloads | download | Capture a download to a host path. |
browser.profile.list | cookies or storageState | profile | List grants (never values). |
browser.profile.refresh | cookies or storageState | profile | Re-read granted domains from the granted source. |
browser.profile.revoke | cookies or storageState | profile | Revoke a grant or some of its domains. |
browser.list_instances | none | core | List the instances the host serves: name, engine, state, profiles, purpose. |
browser.list_accounts | none | core | List account records: site, account, profile, session, source, grant ids. Never a value. |
browser.list_owned_tabs | none | interaction | List the targets owned by the caller session, or, for an operator, every target grouped by owner. |
browser.close_tab | none | interaction | Close one target the caller session owns. Input { tabId?: string }. |
browser.press_key | none | interaction | Send a key press to a target the caller session owns. Input { key: string, modifiers?: ("Alt" | "Control" | "Meta" | "Shift")[], tabId?: string }. |
There is deliberately no tool that creates or widens a grant (C7).
browser.tabs keeps its meaning and gate (T1); the three list_* ids
are new and do not change it. browser.close_tab and browser.press_key
are tab-scoped and inherit L15 ownership; browser.press_key takes
key as a DOM KeyboardEvent.key name (Escape, Enter, Tab,
ArrowDown, or a single character).
T1 (core). Tool ids in this table MUST keep their meaning and gates. A provider MAY add vendor tools under its own namespace.
T2 (core). When the AIP-14 resolver picks a provider for a call that needs a capability, it MUST consider only providers whose manifest declares that capability, and MUST fall back per AIP-30's preference order among those.
T3 (core). A call that reaches a provider lacking the tool (not in
implements[]) or lacking the gate MUST fail with the AIP-14 error
envelope, code: "browser:unsupported", a message naming the
capability, and cause: { capability: "<name>" }. The failure MUST NOT
be an opaque transport, timeout or 404 error. The canonical example: a
provider without a CDP endpoint answers browser.list_requests with
browser:unsupported, cause.capability: "cdp".
T4 (core). Browser-specific error codes are browser:unsupported,
browser:starting (retryable), browser:not_ready,
browser:crash_looping, browser:target_gone and
browser:consent_required (names the domain), browser:transform_failed (a required observer transform failed, section 7),
browser:awaiting_consent (retryable: an attach waits for the user's
click in the browser, L14) and browser:not_owner (the target belongs to
another caller session, L15), browser:command_refused (a protocol
method outside an attached instance's allowlist, § Security considerations)
and browser:target_obscured (a click whose point does not hit the
resolved element or one of its descendants, T7). They follow AIP-14's
domain-prefix convention. A browser:target_obscured error carries the
body { code, error, tool, covering: { tag, role?, text? } };
covering.text is cut at 80 characters and any URL in it is reduced to
its origin. The recipe codes (§ 8) are
browser:origin_denied, browser:approval_required,
browser:confirm_required, browser:account_required,
browser:account_denied and recipe:error; they follow the same
convention and are tested at the recipes level.
T5 (core). browser.list_instances and browser.list_accounts MUST
NOT return a cookie value, a storage value, a token or anything derived
from one (C9). list_instances returns, per instance, its name, its
engine (provider id), its L1 state, its profile names and the operator's
free-text purpose when one is configured. list_accounts returns account
records ($defs/account) only. Both are read-only and MUST NOT launch or
attach; list_accounts MAY perform the read-only login probe C17 permits,
and neither tool probes anything else (L4).
T6 (interaction). browser.list_owned_tabs called by an agent MUST
return only the targets whose owner is its caller session (L15). Called
from an operator channel it MAY return every target, grouped by
callerSessionId, with targets of unknown owner in their own group.
T7 (interaction). A host MUST NOT dispatch a selector click whose
point does not hit the resolved element or a node inside it (crossing
open shadow roots; a label of the element counts), and MUST return
browser:target_obscured (T4) instead. The point is the centre of the
element's first client rect after scrolling it into view; the hit node
is what document.elementFromPoint returns there.
3. Instance lifecycle
An instance is in exactly one state:
absent -> starting -> healthy <-> degraded -> stopping -> stopped
| ^ |
+-----------> crash-looping +-----------+ (stopped -> starting on relaunch)| Transition | Trigger |
|---|---|
absent to starting | launch |
starting to healthy | health probe succeeds |
starting to stopped | launch failed or budget exhausted (counts toward L6 only per L9) |
healthy to degraded | health probe fails or the provider reports partial service |
degraded to healthy | probe recovers |
healthy or degraded to stopping | stop |
stopping to stopped | process tree gone and orphan sweep done |
stopped to starting | launch again |
starting to crash-looping | L6 threshold reached |
crash-looping to starting | operator reset only (L7) |
crash-looping is terminal until an operator resets it.
Provider primitives, in TypeScript notation (normative for the shape; the reference package is authoritative for the exact types):
interface BrowserProvider {
manifest: BrowserManifest
launch(opts: LaunchOptions, ctx: HostContext): Promise<BrowserInstance>
}
interface BrowserInstance {
id: string
state: InstanceState
endpoints: { rest?: string; cdp?: string }
pid?: number
wasAlreadyRunning: boolean
health(): Promise<{ state: InstanceState; reason?: string; checkedAt: string; bootId?: string }>
attach(opts?: AttachOptions): Promise<BrowserDriver>
stop(opts?: { force?: boolean }): Promise<{ swept: number }>
}L1 (core). health() and every host status surface MUST report one
of the seven states, and transitions MUST follow the table.
L2 (core). launch MUST be idempotent. When a healthy instance
already exists for the same instance key (provider id plus profile or
label), launch MUST return it with wasAlreadyRunning: true, MUST NOT
spawn a second process, and MUST NOT change its configuration. This
includes a healthy service on the provider's default endpoint that the
host did not start.
L3 (core). attach MUST return a driver handle bound to a target.
On an instance that is not healthy or degraded, attach MUST fail
with browser:not_ready carrying the current state.
L4 (core). health() MUST answer within lifecycle.health.timeoutMs
and MUST NOT create pages or targets as a side effect.
L5 (core). stop MUST be idempotent and best effort: request a
graceful stop, wait up to lifecycle.stop.graceMs, then force. It MUST
then sweep orphans: terminate every process carrying the instance's
supervision.orphanMarker, and MUST NOT terminate processes without
the marker. stop on an instance the host did not launch (a
wasAlreadyRunning attach) MUST NOT terminate it unless force: true.
On an attached instance (L14), force: true MUST come from an operator
channel: a tool call or any other agent-controlled input MUST NOT set it.
L6 (core). The supervisor MUST count consecutive failed launches
within crashLoop.windowMs (defaults: 3 failures, 300000 ms). On
reaching crashLoop.maxFailures it MUST enter crash-looping, MUST stop
spawning, and MUST surface the last failure reason in health() and in
browser:crash_looping errors.
L7 (core). Leaving crash-looping MUST require an explicit reset
from an operator channel (host CLI or UI). Timers, retries and agent
tool calls MUST NOT clear it.
L8 (core). On host start, before its first launch, the supervisor MUST sweep marker-carrying processes left by a previous host run.
L9 (core). Launch has one budget, launch.budgetMs (default
120000). The budget is a soft deadline: while progress signals continue
(process alive and a bound port, or health reporting starting, within
the trailing launch.progressWindowMs), the supervisor MUST keep
waiting, up to launch.hardCeilingMs (default three times the budget).
A slow but progressing launch MUST NOT count toward L6 and MUST NOT be
killed and replaced by a second launch. Only a launch that exhausts the
budget with no progress, or reaches the hard ceiling, is a failed launch.
L10 (core). A call that arrives while the instance is starting
MUST wait up to the remaining budget, then fail with retryable
browser:starting. It MUST NOT surface as an opaque timeout.
L11 (interaction). A target opened with keepAlive: true MUST NOT
be closed by any idle reaper (host or provider, idleReapMs or
otherwise). It closes only on an explicit close, stop, revoke of the
session's grant (C8), or instance loss.
L12 (interaction). After an instance restart (bootId change, or
the process was replaced), target handles issued by the previous boot
MUST fail with browser:target_gone. A host MUST NOT silently rebind a
stale handle to a new target. A provider SHOULD expose bootId in
health() so clients can detect the restart.
L13 (core). A provider that launches a Chrome-family browser to
drive it over CDP MUST launch it on a fresh, non-default user-data
directory. Chrome 136
and later refuse --remote-debugging-port on the default user-data
directory, so this is required, not a preference. Session material
reaches such an instance only through granted injection (C1), for
example CDP Network.setCookies; a provider MUST NOT satisfy a
profile request by copying a whole user profile unless a recorded
full-profile grant covers it (C3). L13 governs launching; attaching to a
browser the user already runs is L14.
L14 (core). An attached instance is one the host did not launch:
the user's own browser, reached through an opt-in the user performs in
that browser (for Chrome 144 and later, chrome://inspect/#remote-debugging,
then a per-connection "Allow"). A host that attaches:
- MUST hold an active
attachgrant for that browser and profile (C14) before it connects; - MUST report the instance as
startingwhile the browser waits for the user's click. A call that arrives meanwhile MUST wait up to the host's consent bound, then fail with retryablebrowser:awaiting_consent; while the instance waits for consent, that code replaces L3'sbrowser:not_readyand L10'sbrowser:starting. If no click comes within the bound, the attempt fails and the instance moves tostopped(L1); it does not count toward L6. There is no eighth state (L1).browser:awaiting_consentis distinct frombrowser:consent_required, which stays the per-domain consent of § 4; - MUST report
wasAlreadyRunning: true, and MUST NOT terminate, restart or close the browser, its windows or any target it did not open (L5 forforce); - MUST NOT copy the profile, and MUST NOT send material from it to a
remotesink (C16).
How the host finds the endpoint (a port file, a fixed endpoint), whether it shares one upstream connection between callers, and how it reconnects after the browser restarts are implementation choices.
L15 (interaction). A target opened through a tool call belongs to the
caller session that opened it (its callerSessionId, established as in
AIP-59 § 8, never from a header or a tool argument). A tool call from
another caller session that names that target MUST fail with
browser:not_owner and MUST NOT act on it. A popup opened by an owned
target belongs to the same owner. A target the host did not open (the
user's own tabs in an attached instance) has no owner and MUST NOT be
driven, read or closed by an agent call; that call MUST fail with
browser:not_owner, and so MUST a close of another caller's target.
browser.close_tab and browser.press_key are scoped by this rule; an
unknown tab MUST fail with browser:target_gone. A host MAY close an
owned target itself (for example through Target.closeTarget when raw
target-closing CDP methods are refused to agents). An operator channel is
not bound by this rule.
L16 (interaction). Ownership MUST be persisted with the instance's
bootId, so a host restart does not forget who opened what. When a caller
session ends, the host MAY close the targets that the session is proven to
have opened during the current boot, except a target opened with
keepAlive: true (L11), and MUST NOT close any other target. A target whose
owner is unknown (opened by the page), or that was opened in a previous boot
(bootId differs), MUST NOT be closed by cleanup, by the same reasoning as
L12.
4. Profile and consent
This section applies to a provider or host that can seed a context from
a user's profile or persist session material: profiles.modes includes
import, native-login, clone or attach, or capabilities.cookies or
capabilities.storageState is set. Its rules are tested at the
profile level.
Session material is cookies, storage state, profile directories and anything derived from them (a decrypted temporary copy, a warm tab or context logged in with them, a session created at a remote provider from them). A store is any persistence of session material or its transfer into a remote instance.
Grant and ledger records have the shape of $defs/grant and
$defs/ledgerRecord in BROWSER.schema.json. A grant's scope is either
an explicit domains list, fullProfile: true, or attach: true beside a
profileRef (an attached browser and profile, C14); a wildcard is not a
value of domains.
C1 (profile). Session material MUST enter a store only through a grant.
C2 (profile). A grant's domains MUST be explicit registrable
domains or hosts, lowercase, at least two labels. A grant covers the
named domain and its subdomains. A host MUST reject *, *.example.com,
a leading dot, an empty list, a bare TLD and any public suffix (checked
against the Public Suffix List; the schema pattern is necessary, not
sufficient).
C3 (profile). The only exception to C2 is a recorded full-profile
grant (fullProfile: true). It MUST be requested by name (a dedicated
flag or control, never implied by omitting domains), MUST show a warning
that the entire profile including every domain is exposed, MUST have a
ledger record, and MUST NOT include a remote sink. An attach grant (C14)
is a separate scope, not an exception to C2.
C4 (profile). A non-interactive request without an explicit domain list or the named full-profile request MUST fail. It MUST NOT default to "all domains".
C5 (profile). An interactive import MUST show per-domain cookie counts from a presence scan that does not decrypt or read values, and MUST ask per domain before any value is read.
C6 (profile). Before session material is sent to a remote sink,
a human MUST acknowledge that sink (its id and operator), producing a
sink-ack ledger record. A location: remote provider MUST refuse to
receive material for a grant whose remote sink has no such
acknowledgement (ackSeq).
C7 (profile). An agent-initiated call MUST NOT create a grant, add
a domain, change the source profile, add a sink, or turn a local
grant into a remote one. It MAY list grants, refresh material within an
existing grant (same source profile, same or fewer domains), and revoke.
A refused attempt SHOULD append a deny ledger record. An agent that
needs more access MAY file an AIP-7 approval request;
only a human decision on a human channel counts, and the widening itself
is then performed through the operator channel, not through a tool.
C8 (profile). Revoke MUST take effect before it reports success. It
MUST delete the derived material for the revoked scope (stored cookies
and storage for those domains, decrypted temporary copies, warm tabs and
contexts using them). For a remote sink it MUST also ask the provider
to end or delete the remote session and record whether that was
confirmed, unavailable or failed. It MUST append a revoke record
with both outcomes. A domain-level revoke narrows the grant.
C9 (profile). Cookie values, storage contents and any secret derived from them MUST NOT appear in tool results, error messages, logs, the ledger or telemetry. Counts, domain names and (in the ledger) salted name hashes are allowed.
C10 (profile). The consent ledger MUST be append-only: one JSON
record per line, a monotonic seq, records for grant, refresh,
revoke, sink-ack, attach-ack and deny. The host MUST NOT rewrite or truncate
it. Records SHOULD carry prev, the SHA-256 of the previous line.
C11 (profile). On POSIX, files holding session material, the ledger
and the store directory MUST be created with mode 0600 (files) and 0700
(directories). Encryption of session material at rest is a SHOULD in
agentbrowser/v1 and a MUST in agentbrowser/v2. A provider with
profiles.storedBy: provider SHOULD state in its body how its own jar is
protected; a host cannot promise more than the provider does.
C12 (profile). A grant MAY carry deviceId, the identity of a paired
device under AIP-59. When present, calls from any other
paired device MUST NOT use that grant's material, and MUST see it as
absent (browser:consent_required).
C13 (profile). When AIP-59 revokes a pairing, the host MUST stop honoring grants scoped to that device at once and SHOULD revoke them under C8.
Attach. C3 does not cover an attach: an attach copies nothing into a store, yet it lets an agent act in the user's own, logged-in profile. It therefore has its own grant and ledger record.
C14 (profile). Before it attaches to a browser it did not launch
(L14), a host MUST hold an active grant with attach: true naming the
browser instance and the profile (profileRef), created by a human on
an operator channel, and the ledger MUST hold an attach-ack record for
it carrying instance, profileRef, deviceId when the grant is device
scoped (C12), and purpose. C7 applies: an agent MUST NOT create, widen
or re-target an attach grant.
C15 (profile). Revoking an attach grant MUST detach the host from
that browser and close every target agents opened under it (L15) before
it reports success, MUST delete the recordings that list the grant (R13,
R16), and MUST append a revoke record. It MUST NOT close the user's own
targets.
C16 (profile). An attached profile is never session material for a
sink: a host MUST NOT export its cookies or storage, and MUST NOT route
an attached instance through a remote sink.
C17 (profile). An account record ($defs/account) describes one
account on one site in one profile: site, optional account (a display
name or an address), profileRef, session (valid, expired or
unknown) with verifiedAt when it was checked, source (native: the
user logged in there; synced: material copied in under a grant, with
syncedFrom, and syncedAt when known) and grantIds. Three rules:
- an account authorizes nothing: only grants do (C1, C14);
sessionis indicative. A host MAY probe it, read-only; this AIP does not require a probe;accountmay be personal data (an address). It is not a secret and may be listed; C9 still forbids every cookie, token and storage value.
A grant MAY carry profileRef next to sessionId. sessionId keeps its
meaning, and no ledger line is rewritten (C10).
5. Recording artifact
A recording is what a host writes when a human asked to see, or to
keep, what an agent does in a browser. It is a directory, named by the
recording id, holding a manifest, two append-only JSON Lines files and
the frame images. It is a host artifact, not a provider tool: a provider
declares recording (§ 1) and the host's recorder writes the directory.
The files are validated by
RECORDING.schema.json
($defs/manifest, $defs/frame, $defs/action); a complete sample is
under resources/aip-63/draft/examples/recording-sample/.
rec_01JSAMPLE0000000000000001/
manifest.json schema browser.recording/v1: who, when, mode, status, redaction, retention
frames.jsonl one line per stored frame: i, tMs, file, url, tabId, w, h
actions.jsonl one line per tool call: seq, tMs, tool, url, target, summary, status, durationMs
frames/000001.jpg the images, named by i
exports/ optional derived video (mp4, webm); regenerable, not part of the formatA line of frames.jsonl and of actions.jsonl looks like this:
{"i":5,"tMs":3900,"file":"frames/000005.jpg","url":"https://app.example.test/login","tabId":"tab-1","w":320,"h":180,"actionSeq":4,"masked":"fields"}
{"seq":4,"tMs":3600,"tool":"fill","url":"https://app.example.test/login","tabId":"tab-1","target":{"selector":"input[type=password]","role":"textbox","name":"Password","sensitive":true},"summary":"Type [redacted] into Password","status":"ok","durationMs":190,"redacted":true}Modes. per-action writes one frame after each action that can
change the page (the "one frame per action" model). screencast writes
frames as the browser pushes them (CDP Page.screencastFrame or the
provider's equivalent), subject to a rate limit and de-duplication.
Both modes write the same files; a viewer needs no knowledge of the mode
to play a recording. In per-action mode every frame carries
actionSeq; in screencast mode a frame carries actionSeq when it was
captured right after an action and omits it otherwise.
R1 (recording). A recording MUST be a directory whose name equals
manifest.id, matching the recordingId pattern of the schema (no dot,
no slash). It MUST contain manifest.json, frames.jsonl,
actions.jsonl and frames/. A reader MUST resolve every path inside
the recording against the recording directory and MUST reject a file
or exports[].file that leaves it.
R2 (recording). manifest.json MUST validate against
$defs/manifest and carry schema: "browser.recording/v1". A reader
MUST refuse a manifest with an unknown schema value instead of guessing.
R3 (recording). frames.jsonl and actions.jsonl are append-only,
UTF-8, one JSON object per line, LF-terminated. A writer MUST NOT
rewrite or truncate them. A frame image MUST be completely written
(write to a temporary name, then rename) before its line is appended. A
reader MUST ignore a final line that is not valid JSON, since a crash can
leave one. Lines SHOULD carry prev, the SHA-256 of the previous line, as
for the consent ledger (C10). manifest.json is the only file that may
be replaced after creation, and only by atomic rename. While status is
recording, its counts and durationMs MAY lag the JSONL files, which are
authoritative.
R4 (recording). i and seq MUST start at 1 and increase by exactly
tMsis an integer count of milliseconds sincemanifest.startedAtand MUST NOT decrease from one line to the next within a file. Frames and actions share this one time base, which is what lets a viewer overlay one on the other.
R5 (recording). Every frame line's file MUST be frames/ plus i
zero-padded to at least six digits plus .jpg, and the file MUST exist
and be a JPEG. w and h are the pixel size of the stored image; they
MAY be smaller than manifest.viewport when the writer downscales. When
a recording is closed, frameCount and actionCount MUST equal the line
counts of the two files.
R6 (recording). A frame of per-action mode MUST carry actionSeq
naming an action that exists. A screencast writer SHOULD cap its rate
(screencast.maxFps) and SHOULD skip a frame byte-identical to the
previous stored one, so an idle page costs no storage. Neither mode may
use a file or field the other does not.
R7 (recording). An action line records a locator and an outcome,
never a value. target holds a selector, a role and an accessible name,
and MUST NOT hold the content of an input. For a fill (or any tool that
types) into a sensitive field, summary MUST NOT contain the typed
value, a fragment of it or its length; the line MUST carry
target.sensitive: true and redacted: true. A field is sensitive when it
is input[type=password], or its autocomplete contains password,
one-time-code or a cc- token, or the writer cannot tell. The
expression of an evaluate SHOULD be summarized, not stored. url fields
MUST drop the fragment and any userinfo, and a writer SHOULD drop query
parameters it knows to be credentials. An error carries a code (T4 or a
vendor code) and a short message, scrubbed as in § Security considerations.
R8 (recording). Unless the operator turned it off for that
recording, manifest.redaction.passwordFields MUST be masked, and no
frame may show the content of a sensitive field (R7): the writer MUST mask
those fields before the pixels reach the disk, either in the page before
capture (CSS or DOM change over the debugging protocol) or by painting
over their regions in the image. A frame in which a sensitive field was
masked MUST carry masked: "fields". passwordFields: "none" is an
explicit operator choice, MUST be recorded, and MUST NOT be reachable from
an agent call (C7 applies by analogy).
R9 (recording). Masking fails closed. If a writer cannot establish
that a frame is free of sensitive content (an unmasked cross-origin frame,
a failed mask injection, a field it cannot locate), it MUST blur the whole
frame and mark it masked: "full", or drop it and count it in
redaction.droppedFrames. It MUST NOT store the frame as is.
R10 (recording). retentionDays MUST be recorded at creation and
SHOULD default to 7. A recording is expired once retentionDays full days
have passed since endedAt (since startedAt if it never ended). A host
MUST delete an expired recording as a whole: manifest, both JSONL files,
every frame and every export, with no residue. It MUST run that purge when
it starts and SHOULD run it at least daily. expiresAt, when written, is
informational and MUST agree with this rule.
R11 (recording). A recording is private. A host MUST serve it only
over an authenticated surface (the AIP-59 device credential, § Security
considerations) and MUST NOT expose any of its files, a listing of
recordings, or a live feed of one on an unauthenticated route. When
manifest.deviceId is set, a call from any other paired device MUST see
the recording as absent. Responses carrying frames SHOULD send
Cache-Control: private, no-store.
R12 (recording). On POSIX the recording directory MUST be created with
mode 0700 and its files with mode 0600 (C11 applies unchanged). A default
location under the user's data directory (for example
~/.pygmalion/data/browser-recordings/ in the reference host) is fine; a
location another account can read is not.
R13 (recording). grantIds lists the profile grants that seeded the
recorded session. A recorded session can show whatever a logged-in page
shows, so when a grant is revoked (C8) the host SHOULD delete the
recordings that list it, and if it does MUST report them as derived
material in the revoke record's derived outcome.
R14 (recording). status moves only from recording to complete,
failed or interrupted. On start, before it accepts a new recording, a
host MUST finalize every recording still marked recording from a
previous run: set status: "interrupted", set endedAt to the last
tMs seen, and correct the counts from the JSONL files (L8 by
analogy). A closed recording is immutable except for exports and the
purge of R10.
R15 (recording). A video export (exports/*.mp4, exports/*.webm) is
derived. A host MAY build it on demand and MAY cache it under exports/,
listing it in manifest.exports. It MUST be reproducible from frames and
tMs, MUST be deleted with the recording, and MUST NOT contain pixels
that the frames do not (so it inherits R8 and R9).
R16 (recording). A recording of an attached instance (L14) MUST be
off by default; the operator turns it on per instance. Its manifest
MUST list the attach grant in grantIds, so that revoking that grant
makes it derived material to delete (R13, C15). Manifests and action lines SHOULD carry
callerSessionId, the caller session (L15) that drove the target, so a
viewer can tell sessions apart. Any copy of a recording off the host
stays under R10 to R12; a host SHOULD link to it rather than copy it.
6. Conformance
Conformance is claimed per level in browser.conformance. A claimed
level MUST pass the suite for it. The reference package ships the suite
at @agentproto/driver-browser/conformance; it runs against any
provider, including one backed by a fake server, and against a
deliberately broken fake, which it MUST fail.
| Level | Applies to | Tests |
|---|---|---|
core | every provider | manifest rules M1 to M6; tool gate T1 to T4 (including that a provider without cdp answers browser.list_requests with browser:unsupported); lifecycle L1 to L10, L13 and L14; T5; browser.list_instances, browser.list_accounts; browser.navigate, browser.evaluate, browser.get_dom, browser.screenshot |
interaction | providers serving input tools | browser.click, browser.fill, browser.act, browser.tabs, browser.list_owned_tabs, browser.close_tab, browser.press_key; L11, L12, L15, L16; T6, T7 |
network | providers with cdp | browser.list_requests, browser.get_request_body, browser.cdp_send |
download | providers with downloads | browser.download |
profile | providers with cookies or storageState, and hosts that attach | C1 to C17; browser.profile.list, .refresh, .revoke |
recording | providers with recording other than none, and the host recorder that writes their recordings | R1 to R16, against the sample recording and against a deliberately broken one (a value in a password fill, a gap in i, an unmasked password frame), which it MUST reject |
recipes | hosts that load recipes | P1 to P15; browser.list_recipes and browser.recipe_guide; the origin guard, both write gates and the account resolution, including that no tool call creates or decides an approval |
Rule index (every rule names its level; this table is the fixture):
| Rule | Level | Test hook |
|---|---|---|
| M1 | core | core.manifest.schema |
| M2 | core | core.manifest.remote-sink |
| M3 | core | core.manifest.cdp-endpoint |
| M4 | core | core.manifest.implements-gate |
| M5 | core | core.manifest.profile-modes |
| M6 | core | core.manifest.conformance-claims |
| T1 | core | core.tools.ids |
| T2 | core | core.tools.resolver-filter |
| T3 | core | core.tools.unsupported |
| T4 | core | core.tools.error-codes |
| T5 | core | core.tools.list-never-values |
| T6 | interaction | interaction.tools.owned-tabs-scope |
| T7 | interaction | interaction.tools.click-hit-test |
| L1 | core | core.lifecycle.states |
| L2 | core | core.lifecycle.launch-idempotent |
| L3 | core | core.lifecycle.attach-not-ready |
| L4 | core | core.lifecycle.health-pure |
| L5 | core | core.lifecycle.stop-sweep |
| L6 | core | core.supervision.crash-loop |
| L7 | core | core.supervision.reset-operator-only |
| L8 | core | core.supervision.reap-on-start |
| L9 | core | core.supervision.launch-budget |
| L10 | core | core.supervision.starting-wait |
| L11 | interaction | interaction.targets.keepalive |
| L12 | interaction | interaction.targets.stale-handle |
| L13 | core | core.lifecycle.chrome-fresh-userdir |
| L14 | core | core.lifecycle.attach-never-stopped |
| L15 | interaction | interaction.targets.not-owner |
| L16 | interaction | interaction.targets.cleanup-proven-owned |
| C1 | profile | profile.consent.grant-required |
| C2 | profile | profile.consent.domains-explicit |
| C3 | profile | profile.consent.full-profile |
| C4 | profile | profile.consent.no-default-all |
| C5 | profile | profile.consent.presence-scan |
| C6 | profile | profile.consent.sink-ack |
| C7 | profile | profile.consent.agent-cannot-widen |
| C8 | profile | profile.consent.revoke-deletes |
| C9 | profile | profile.consent.never-print |
| C10 | profile | profile.consent.ledger-append-only |
| C11 | profile | profile.consent.perms-and-encryption |
| C12 | profile | profile.consent.device-scope |
| C13 | profile | profile.consent.pairing-revoked |
| C14 | profile | profile.consent.attach-grant |
| C15 | profile | profile.consent.attach-revoke-detaches |
| C16 | profile | profile.consent.attach-no-sink |
| C17 | profile | profile.consent.account-not-authority |
| R1 | recording | recording.layout.directory |
| R2 | recording | recording.manifest.schema |
| R3 | recording | recording.jsonl.append-only |
| R4 | recording | recording.timeline.monotonic |
| R5 | recording | recording.frames.files |
| R6 | recording | recording.modes.same-format |
| R7 | recording | recording.actions.no-values |
| R8 | recording | recording.redaction.password-fields |
| R9 | recording | recording.redaction.fail-closed |
| R10 | recording | recording.retention.purge |
| R11 | recording | recording.access.authenticated |
| R12 | recording | recording.access.perms |
| R13 | recording | recording.access.grant-revoke |
| R14 | recording | recording.lifecycle.interrupted |
| R15 | recording | recording.export.derived |
| R16 | recording | recording.attach.off-by-default |
| P1 | recipes | recipes.layout.directory |
| P2 | recipes | recipes.manifest.schema |
| P3 | recipes | recipes.tools.prefix |
| P4 | recipes | recipes.workflows.prefix |
| P5 | recipes | recipes.writes.subset |
| P6 | recipes | recipes.load.entry-matches-manifest |
| P7 | recipes | recipes.platformkit.compose |
| P8 | recipes | recipes.guard.origin |
| P9 | recipes | recipes.writes.confirm |
| P10 | recipes | recipes.writes.approve |
| P11 | recipes | recipes.accounts.resolve |
| P12 | recipes | recipes.errors.shape |
| P13 | recipes | recipes.discovery.tools |
| P14 | recipes | recipes.guide.untrusted |
| P15 | recipes | recipes.probes.verdicts |
| O1 | core | core.observers.optional-empty |
| O2 | core | core.observers.operator-only |
| O3 | core | core.observers.process-scope |
| O4 | core | core.observers.best-effort |
| O5 | core | core.observers.every-call |
| O6 | core | core.observers.screenshot-transform |
| O7 | core | core.observers.args-redacted |
| O8 | core | core.observers.module-trust |
The C9 hook plants a canary cookie value and greps tool results, error
messages, logs and the ledger for it. The C7 hook drives every tool as
an agent caller and asserts that no call changes the domain set. The L15 hook opens a target as one caller session and drives, reads and closes it as another, expecting browser:not_owner each time. The L16 hook restarts the host between opening and session end and asserts no target is closed. The O1 hook runs the full tool suite with and without the seam and diffs the results. The O7 hook fills a password field through the seam and searches every event for the typed canary and for its length. The O6 hook registers a transform that halves the image and asserts the caller never receives the original size. The R7
and R8 hooks plant a canary password in a test page, fill it through the
recorder and search every file of the recording for it, and check the
pixels of the masked region.
7. Observers
Several behaviours around a browser session need to see the same moments and must not each patch the tools: showing the user that an agent is attached to their tab, journaling actions, writing a recording (section 5), shrinking a screenshot before it reaches the model. A host that wants them exposes one optional seam, the observer. This section only fixes the seam. What an observer does with an event is private to the operator.
An observer is an object with any of these members, all optional.
| Member | Called | Receives |
|---|---|---|
onAttach | after the provider instance is registered and usable (start_browser returned a session) | sessionId, provider, instanceId, adapter, and capture (screenshot, evaluate) when the provider implements both. capture is for the observer's own frames and page probes; it is never reachable from a tool call |
onDetach | when the session ends for any reason: stop_browser, crash, host shutdown | sessionId, reason (stop, crash, shutdown) |
beforeCall | before a browser.* tool executes | sessionId, callerSessionId when the host established one (L15), tool (without the browser. prefix), args, tabId when known, callId |
afterCall | after it executed, before the result is returned | the beforeCall fields plus durationMs, ok, and error.code when it failed. Never the result body |
transformScreenshot | in browser.screenshot, after capture, before the image is placed in the result | sessionId, tabId, image (data base64, mimeType); returns the replacement image |
O1 (observers). The seam MUST be optional and empty by default. With no observer registered, every tool MUST return a result identical to the one it returned before the seam existed.
O2 (observers). Observers are registered by the operator, in process or from host configuration. A tool call, a tool argument or any other agent-controlled input MUST NOT register, replace or remove an observer. Registration is keyed by an observer id and idempotent: registering the same id replaces the previous observer.
O3 (observers). Registration state MUST live at module or process scope, not inside the per-request tool registration. A host that rebuilds its tool surface for each MCP request (the stateless transport) would otherwise lose the observer on the next call.
O4 (observers). onAttach, onDetach, beforeCall and afterCall
fail open. An observer that throws or exceeds its time budget (the host
SHOULD cap it at 2 seconds) MUST NOT change the outcome of the tool call,
MUST NOT delay it beyond that budget, and MUST be logged without the
arguments. The single exception is a required screenshot transform
(O6). onDetach MUST fire exactly once per attached session,
including when the instance is stopped by the registry rather than by
stop_browser.
O5 (observers). beforeCall and afterCall run in registration
order and see every browser.* call of an attached session, over every
host path that executes the tool (MCP and HTTP). callId pairs a
beforeCall with its afterCall. A call that is refused by the gate
(T3) or by consent (section 4) is reported to afterCall with its error
code.
O6 (observers). transformScreenshot runs before the image is
serialized into the result, so the original bytes never reach the caller
or its model. Transforms chain in registration order. A transform is
registered required or not. When a required transform fails, the tool
MUST return browser:transform_failed and MUST NOT return the original
image. When a non-required one fails, the original passes and the failure
is logged.
O7 (observers). The host redacts call arguments before any observer
sees them, not after. args carries locators and flags only (selector,
url, tab, button, click count, scroll position). The value of a fill
and the expression of an evaluate are removed, and no field stands in
for them: not their length, not a hash, not a boolean that says the value
was empty. Whether a target is a password field is not known to the host
from the arguments; an observer that needs it asks the page itself, and
an observer that persists anything applies R7 and R8 on top. Tool results
are never passed to afterCall.
O8 (observers). A host that loads observers from a file does so from the operator's own environment only, never from a tool argument, an agent-supplied path or a workspace file. The path MUST be absolute and MUST resolve to a regular file owned by the daemon user and not writable by group or others. A load failure MUST be logged at boot with the cause. The host MAY continue without observers, and MUST offer a switch that turns the failure into a refusal to boot, for operators whose observers are a safety control (a required masking transform).
8. Recipes
A recipe packages everything a host knows about one site: a
PlatformKit slice for that site, its tools, its workflows, its guide for
agents and its probes, under one version. It is a packaging unit, not a
new registry: login and account logic stays in the PlatformKit, account
records stay host records (C17), and a recipe owns exactly one site slice.
A recipe is named by its id.
Recipe code runs inside the host next to logged-in profiles. The origin guard and the write gates below (P8 to P10) are a contract against mistakes, not a sandbox: page JavaScript can still send data cross-origin. A host that loads third-party recipes MUST isolate recipe code first (separate process, narrow RPC, no CDP), and v1 loads only recipes the operator trusts. Guide content is untrusted data (P14).
P1 (recipes). A recipe MUST be a directory named <id>/ holding
RECIPE.md (YAML frontmatter plus a one-paragraph summary body) and the
files the manifest declares: guide/GUIDE.md, optional guide/reference/*.md
(each file is one more section), and the entry module (default
dist/index.js, ESM with all dependencies bundled). The installed form MUST
NOT contain src/, test/ or fixtures/.
P2 (recipes). RECIPE.md frontmatter MUST validate against
RECIPE.schema.json
and declare schema: recipe/v1 exactly. An unknown top-level key is a load
error, never a silent ignore.
The recipe/v1 manifest
| Field | Required | Type | Rule |
|---|---|---|---|
schema | Yes | recipe/v1 | Exact value. |
id | Yes | string | ^[a-z][a-z0-9-]{1,30}$. |
version | Yes | semver | One recipe, one version. |
title | Yes | string | Display name. |
api | Yes | integer | The recipe API version this recipe targets. A host MUST refuse an api it does not know. |
sites | Yes | string[] | Non-empty registrable domains, lowercase. A recipe's guard covers the domain and its subdomains exactly as C2 does for grants. |
accounts | No | string[] | Subset of sites. The recipe needs an Account (C17) on these sites. |
tools | Yes | string[] | Exact set of tool names the entry returns (P6). |
workflows | No | string[] | Exact set of workflow ids the entry returns (P6). |
writes | No | object | Tool name to approve or confirm (P5, P9, P10). Every tool not listed is a read. |
probes | No | string[] | Exact set of probe ids the entry exports (P6, P15). |
entry | No | string | Entry module path. Default dist/index.js. |
guide | No | string | Guide path. Default guide/GUIDE.md. |
publisher | Yes | string | Who publishes the recipe. Shown in discovery and in the guide's provenance. |
The manifest is checked by the recipes repository's CI and by the host loader with the same rules. The LinkedIn recipe:
schema: recipe/v1
id: linkedin
version: 1.0.0
title: LinkedIn
api: 1
sites: [linkedin.com]
accounts: [linkedin.com]
tools: [linkedin_profile, linkedin_message]
workflows: [linkedin-job-search]
writes: { linkedin_message: approve }
probes: [profile]
entry: dist/index.js
guide: guide/GUIDE.md
publisher: agentikP3 (recipes). A tool name MUST start with the recipe's tool prefix: the
recipe id with - replaced by _, then _. linkedin therefore owns
linkedin_*. The generic name pattern is ^[a-z][a-z0-9_]*$; the prefix
rule is normative prose, not a schema pattern, because it composes with the
id.
P4 (recipes). Every workflow id MUST start with <id>-.
P5 (recipes). Every writes key MUST be one of tools, and every
writes value MUST be approve or confirm. A tool whose entry kind is
write MUST appear in writes, and every key in writes MUST have
kind: write in the entry. accounts MUST be a subset of sites.
P6 (recipes). Loading MUST check the entry against the manifest and fail
as a load error, naming the recipe and the rule, on any mismatch:
module.id equals id; tools(ctx) names equal tools exactly; each
tool's kind agrees with writes (P5); workflow ids equal workflows;
probe ids equal probes; api is known. A tool name that collides with an
existing host tool is also a load error.
P7 (recipes). Recipe PlatformKits are composed next to the host's base
kit. The composed kit merges platforms; a platform key declared twice is a
load error naming both owners. It dispatches loginSpec, autofill and
accounts.* to the kit declaring that key, and takes openHumanSession
and makeVisionResolver from the base kit, else from the first recipe kit.
P8 (recipes). Every RecipePage and every tool call passes through a
guard:
navigateandgotoPacedMUST refuse a URL whose host is not one ofsitesor a subdomain of one (the domain matching of C2 applies), and any scheme other thanhttps:orhttp:, withbrowser:origin_denied.evaluate,readNextData,type,click,pressandscreenshotMUST first check the page's current origin the same way, so a redirect off-site stops the call, withbrowser:origin_denied.
The guard is a contract against mistakes, not a sandbox; see
## Security considerations.
P9 (recipes). A kind: write tool with level confirm MUST advertise
an optional boolean confirm in its input schema. Without confirm: true
the call MUST return browser:confirm_required and run nothing.
P10 (recipes). A kind: write tool with level approve MUST advertise
an optional string approvalId. Without it, the call MUST store a pending
request (recipe, version, tool and input) and MUST return
browser:approval_required with requestId and approveWith (a host CLI
command that approves it); nothing runs. A human approves in a terminal:
the command prints recipe, tool and input, asks y/N, and signs an
AIP-7 approval request with the approver key, reusing the host's
existing approval creation and consumption for a second payload kind,
recipe-write. The agent calls again with approvalId: <requestId> and the
same input; the host consumes the approval once, checks the payload (recipe,
tool and the SHA-256 of the canonical JSON of the input without approvalId
and confirm) and runs. No tool callable by an agent creates or decides
an approval; only a human channel does. A host MUST strip confirm and
approvalId from input before the tool runs.
P11 (recipes). page({ account }) MUST resolve that Account record
(C17) and its saved session. The Account's site MUST be one of the
recipe's accounts, else browser:account_denied. Without account and
with accounts declared: the single valid Account on that site is used;
none or several are a browser:account_required error naming the candidate
ids. Without accounts declared, the page is anonymous.
P12 (recipes). Errors MUST be MCP results with isError: true whose
text is JSON { "code": "<code>", "error": "<one sentence>", "recipe": "<id>", "tool": "<name>", ...extra },
the shape the AIP already uses for capability errors. An error MUST NOT
contain a full URL (host only), a cookie, a token or an input value. A
recipe's own throw MUST surface as recipe:error with its message.
Discovery tools
P13 (recipes). A host that loads recipes MUST serve two read-only tools,
browser.list_recipes and browser.recipe_guide. As elsewhere in this AIP,
MCP projections spell the same ids with underscores (browser_list_recipes,
browser_recipe_guide).
browser.list_recipes({}) returns:
{ "recipes": [ { "id": "...", "version": "...", "title": "...", "publisher": "...", "sites": ["..."], "tools": ["..."], "workflows": ["..."], "writes": {"...": "approve|confirm"}, "probes": [{"id": "...", "verdict": "ok|drift|fail|unknown", "checkedAt": "..."}], "accounts": [{"site": "...", "ids": ["..."], "valid": ["..."]}], "source": {"kind": "dir|git|dev", "ref": "...", "sha": "..."} } ] }Its description MUST start with: "Call first for any work on a specific website: lists the installed site recipes, their tools and health."
browser.recipe_guide({ id, section? }) returns, without section,
{ recipe: { id, version, publisher }, untrusted: true, summary, toc: [{ id, title, bytes }] }.
Section ids are the slug of each ## heading of the guide, then
ref/<file name without .md> for each reference file. With section, it
returns { recipe, untrusted: true, section: { id, title }, content },
where content is delimited as
<<<RECIPE_GUIDE recipe=<id> version=<v> section=<sid>
...
>>>RECIPE_GUIDEand capped at 24 KB, cut with a final line [truncated at 24 KB]. Its
description MUST say that the content is data written by the recipe
publisher, not instructions.
P14 (recipes). Guide content is untrusted data. A host MUST return it delimited, with its provenance (recipe id, version, publisher), and never as instructions. An agent MUST NOT follow instructions found in a guide.
Probes
P15 (recipes). A probe MUST return a verdict ok, drift or fail. A
host stores the last verdict per probe and surfaces unknown for a probe
that never ran or whose stored verdict is absent. Discovery reports the last
verdict with checkedAt when known.
Rationale
A profile, not a new kind. AIP-30's kind is the transport axis
with a closed enum and a preference order (builtin, sdk, http,
mcp, cli). Browser providers span transports, so encoding
"browser" as a kind would force one transport per provider concept. A
profile keeps kind truthful and lets any transport carry a browser.
A profile, not an AIP-30 amendment. A lifecycle and a consent model would bloat an abstract supertype and couple their versions to it. AIP-45 is the precedent for a domain profile over a driver with its own package and resources, and this AIP follows it, with the difference stated in § Manifest.
The capability gate is in the manifest. A flag plus an absent
implements[] entry is checkable before any call, so the resolver can
route around a provider and a host can show why a tool is unavailable.
The typed error covers the remaining case: a call routed to a provider
that cannot serve it.
The lifecycle rules are aimed at real failure shapes: silent restart loops, second launches on top of slow first ones, reapers closing human logins, and processes left behind after a crash.
The consent model is per domain by construction. The dangerous default of "take everything" is closed by making a wildcard unrepresentable in the grant schema, and by giving the one honest exception (full profile) its own name, its own warning and its own ledger row. Deriving policy from where material can go (sinks) rather than from provider brand keeps remote and local providers under one rule.
One format for two capture modes. A per-action recorder is easy to
make safe (it chooses its moments) and a screencast recorder is easy to
make watchable (it never misses a transition). Keeping them in one file
layout, with actionSeq as an optional join, lets a viewer be written
once and lets a host start cheap and add continuous capture later.
JSON Lines plus plain JPEG files were chosen over a container video
because they can be appended during a run, survive a crash, are
inspectable with ordinary tools, and let masking be verified per frame;
a video is a derived export (R15).
Masking is in the format, not left to the viewer. A recording is the most complete record of a logged-in session that a host will ever hold. Redacting at read time would leave the secret on disk, so the rules require redaction before write (R8), fail closed (R9), and give the action log no field in which a value could even be placed (R7).
Encryption at rest is SHOULD in v1. Some providers persist their own session jar in plaintext and the host cannot change that. A MUST the ecosystem cannot yet meet would be ignored; the v2 MUST is a signal.
Security considerations
Auth model. The auth model for reaching a browser host is AIP-59 pairing, and nothing else. A browser host is an "AIP-59 host": every client, local ones included, is a paired device and presents the device credential. A host MUST NOT invent a static token scheme, and a provider MUST NOT be reachable through a provider-specific static client token. Upstream credentials a provider needs to call a third party (for example an API key for a hosted browser service) are AIP-19 secrets used by the provider; they are not client auth. Per-domain grants are scoped per paired device (C12). AIP-59 describes the daemon side of pairing today; a standalone browser host implements the same host side as a library, and the AIP-59 scope sentence is amended separately.
Network exposure. A host MUST bind loopback by default, and MUST
refuse a request whose Host or Origin header is not an expected
loopback or paired-origin value, so that a web page cannot reach a local
browser host through DNS rebinding. This is the same check the agentproto
daemon applies to its HTTP surface. Binding elsewhere MUST be an
explicit operator choice, and pairing remains required.
Chrome remote debugging. A CDP endpoint is full control of the
browser and every logged-in session in it. Chrome 136 and later refuse
--remote-debugging-port on the default user-data directory. A chrome
provider MUST therefore use a fresh user-data directory (L13) and MUST
bind the debugging port to loopback.
Attaching to the user's own browser. Chrome 144 and later let the
user expose their everyday profile through chrome://inspect/#remote-debugging
and a click per connection. That reaches the most sensitive state the
user has, so an attach needs its own grant and ledger record (C14), is
never stopped or copied (L14, C16), and its agents only see the targets
they opened (L15). A host SHOULD hold a single upstream connection per
attached browser, so the user clicks once per browser launch rather than
once per agent. It MUST filter what reaches that browser with an allowlist
of protocol methods: the attach grant authorizes driving pages, not reading
the cookie jar. The allowlist MUST NOT contain a cookie or storage read or
write (Network.getAllCookies, Network.getCookies, Network.setCookies,
Storage.getCookies and the like), Page.close,
Target.createBrowserContext, Target.disposeBrowserContext, or a target
command on a target outside the caller's own set (L15). On an attached
instance browser.cdp_send MUST refuse those methods with
browser:command_refused (C9, C16).
Caller identity. A host that serves several agent sessions through
one paired device MUST NOT take the caller session from a header or a
tool argument; only a sub-session token minted under AIP-59 § 8
establishes callerSessionId. Without it, any session could drive or
close another session's targets (L15).
Threat model
- Clone leakage. A full-profile clone exposes every domain, including ones the user never thought about (banking, mail). It exists only as a named, warned, ledgered grant that cannot go to a remote sink (C3), and it is a poor default because localStorage and IndexedDB logins cannot travel by cookie injection anyway.
- Agent-driven consent escalation. An agent that can widen its own access makes consent meaningless. The tool surface has no widening tool, agent callers are refused and logged (C7), and approvals are human decisions (AIP-7, "agents never decide").
- Plaintext storageState in providers that persist their own jar. A
provider with
storedBy: providermay hold cookies in a plaintext file the host cannot protect, and may load them by file path. C11 limits what the host controls (permissions, host-side encryption) and asks the provider to disclose the rest. A short-lived plaintext copy handed to such a provider MUST be created 0600 in a 0700 directory and removed as soon as the provider has loaded it. - Trust in remote providers. A
remoteprovider sees everything sent to its browsers, including injected cookies and typed secrets. The sink acknowledgement (C6) makes that visible to the human, revoke asks the provider to delete the session (C8), and the host cannot verify what the provider does afterwards. - Stealth is a capability, not a promise.
stealth: truestates intent. Detection evolves; a provider that blocks less today may block more tomorrow. Callers SHOULD NOT treat it as an authorization or a safety property, and hosts SHOULD NOT weaken any consent rule because a provider is stealthy. - Recordings are sensitive by construction. A recording of a session seeded from a profile grant shows what the user sees, including data of domains the agent was only meant to act on. R8 and R9 keep password fields and unmaskable frames out, R10 bounds how long the rest lives, R11 and R12 keep it off any unauthenticated route and out of other accounts' reach, and R13 ties it to revoke. A page can still display secrets that are not in a password field (an API key on a settings page, a one-time code in text), and no masking rule finds them all: do not record a session that must never show them. A recording the user shares is outside this AIP's control.
- Attach is not a copy, and not a C3 grant. An attach copies nothing, so the C3 ledger (cookie counts, derived-material deletion) cannot describe it. Treating C3 as covering an attach would leave the riskiest case unledgered. C14 and C15 give it a grant whose revoke detaches and closes the agents' targets.
- Host-side secrecy. Cookie values never appear in results, logs, ledger or telemetry (C9). Error messages from providers SHOULD be scrubbed before they cross the host boundary, since a provider may echo a request that contained a value.
Recipes. A recipe (§ 8) is code running inside the host next to
logged-in profiles. The origin guard and the write gates (P8 to P10) are a
contract against mistakes, not a sandbox: page JavaScript can still send
data cross-origin (a no-cors fetch, an image beacon), and origin scoping
of a tab does not stop it. A host that loads third-party recipes MUST
therefore isolate recipe code first: a separate process or worker behind a
narrow RPC (per-origin navigate and evaluate, a same-origin fetch bridge),
with no raw CDP and no direct access to the profile store. V1 loads only
first-party recipes, and that is a trust decision, not an isolation claim.
A guide is content written by the recipe publisher: a host MUST return it
as delimited data with provenance, never as instructions, and an agent MUST
NOT follow instructions found in it (P14).
Reference Implementation
@agentproto/driver-browser at
packages/driver/browser
provides defineBrowser (built on the AIP-30 defineDriver machinery),
the manifest schema, the BrowserDriver page-control port, the
supervisor, the conformance suite and tool projection. Providers live in
adapter packages named @agentproto/adapter-browser-<id>
(a layout precedent only, see § Terminology). A profiles package holds
the grant, ledger and revoke implementation.
Compatibility
With AIP-30. A browser manifest is a valid DRIVER manifest. A host that knows only AIP-30 can resolve and run a provider's tools; it does not enforce this AIP's gates, lifecycle or consent until it implements them. AIP-30's preference order and the AIP-14 resolver are unchanged; this AIP adds the capability filter of T2.
With AIP-14. Tool ids are ordinary AIP-14 ids. The error codes of T4 follow AIP-14's domain-prefix convention.
With AIP-52. No relation (see § Terminology).
Attach, accounts and owned targets (2026-11). L14 to L16, T5, T6,
C14 to C17, R16 and the three browser.list_* ids are additive: an
existing provider that never attaches and serves one caller is
unaffected. sessionId on grants and recordings keeps its meaning;
profileRef, attach, callerSessionId and $defs/account are new and
optional. The tightenings are L5 (force: true on an attached instance is
operator-only) and L15/L16 (a target belongs to the caller session that
opened it): a host that serves several sessions through one device
credential either establishes caller sessions under AIP-59 § 8 or answers
browser:not_owner across them.
Click hit-test, close tab and key press (2026-11). T7,
browser.close_tab, browser.press_key and browser:target_obscured are
additive to browser/v1: no existing tool id or schema changes shape. A
provider that dispatches a selector click MUST now hit-test it first and
answer browser:target_obscured when the point does not hit the target,
so a click that previously landed on an overlay is refused instead of
being sent to the overlay. The two new tools are interaction-level and
follow L15: closing an unowned or another caller's tab answers
browser:not_owner, an unknown tab answers browser:target_gone, and a
host MAY close an owned target itself even when raw target-closing CDP
methods stay refused to agents.
Since 2026-10-23 (AIP-70) the canonical schema id is browser.recording/v1; documents that declare browser-recording/v1 remain valid, and hosts MUST accept both.
Open questions
- Windows and Linux cookie decryption for
import: the consent rules are platform neutral; the mechanics are not specified here. - Native
videoproviders: whether the provider's own file should be importable into the artifact (it would need the same masking proof as frames, R8 and R9), rather than only host-built exports (R15). - Whether a viewer protocol (live feed of a recording in progress) needs its own section; today only the files are fixed, and a host streams them as it sees fit (R11).
- The exact form of
deviceIdin AIP-59, which leaves device identity to the host. - Whether
browser.recorddeserves a tool contract of its own (today the recorder is a host component, § 5). - Whether grants should be able to carry an expiry the host enforces by
automatic revoke, rather than only recording
expiresAt. - Profile choice in an attached browser with several profiles open. CDP
Target.createTargetcannot name a profile: Chrome opens the target in the last active window, soprofileRefon an attach grant states intent but a host cannot yet guarantee which profile a new target lands in. - Third-party recipes: isolation model and signatures (v2).
See also
- AIP-30 DRIVER.md (the supertype this profiles)
- AIP-45 HARNESS.md (the precedent for a profile over a driver)
- AIP-14 TOOL.md (tool ids, error envelope, capability requirements)
- AIP-7 GOVERNANCE.md (approvals: agents never decide)
- AIP-59 PAIRING (the auth model)
- AIP-17 RUNNER.md and AIP-19 SECRETS.md
./resources/aip-63/draft/BROWSER.schema.json
Resources
Supporting artifacts for AIP-63. Links open the file on GitHub — markdown and JSON render natively in GitHub's viewer. Browse the full resource tree →
- BROWSER.schema.jsonaip-63/draft/BROWSER.schema.json
- RECIPE.schema.jsonaip-63/draft/RECIPE.schema.json
- RECORDING.schema.jsonaip-63/draft/RECORDING.schema.json
- DRIVER.mdaip-63/draft/examples/browserbase/DRIVER.md
- DRIVER.mdaip-63/draft/examples/camofox/DRIVER.md
- DRIVER.mdaip-63/draft/examples/chromium/DRIVER.md
- RECIPE.mdaip-63/draft/examples/recipe-linkedin/RECIPE.md
- actions.jsonlaip-63/draft/examples/recording-sample/actions.jsonl
- 000001.jpgaip-63/draft/examples/recording-sample/frames/000001.jpg
- 000002.jpgaip-63/draft/examples/recording-sample/frames/000002.jpg
- 000003.jpgaip-63/draft/examples/recording-sample/frames/000003.jpg
- 000004.jpgaip-63/draft/examples/recording-sample/frames/000004.jpg
- 000005.jpgaip-63/draft/examples/recording-sample/frames/000005.jpg
- 000006.jpgaip-63/draft/examples/recording-sample/frames/000006.jpg
- 000007.jpgaip-63/draft/examples/recording-sample/frames/000007.jpg
- 000008.jpgaip-63/draft/examples/recording-sample/frames/000008.jpg
- 000009.jpgaip-63/draft/examples/recording-sample/frames/000009.jpg
- 000010.jpgaip-63/draft/examples/recording-sample/frames/000010.jpg
- frames.jsonlaip-63/draft/examples/recording-sample/frames.jsonl
- manifest.jsonaip-63/draft/examples/recording-sample/manifest.json
AIP-62: REVIEW (review/v1): attested verdicts over a content range
Names the review — a declared set of command and agent checks run over a frozen git range and folded into a `pass | block | incomplete` verdict — as a first-class primitive, and fixes the attestation that binds that verdict to the exact manifest and content it is about. Specifies the REVIEW.md manifest (`kind: review`; `target`, `checks`, `bindings`, `uses`, `verdict`, and every cross-field rule), reusable review packs (`kind: review-pack`) with their trust rules and the `review.pack-digest/v1` digest, the execution model (prepare before the range freezes, parallel bounded lanes, the trinary fold in which `incomplete` is never a pass), the `review.attestation/v1` document, the ledger key and cache rule, ssh-ed25519 signing over canonical JSON, delta composition via `composedFrom`, and the step-by-step verification algorithm.
AIP-64: PACK (pack/v1): the installable capability bundle: plugin, apps, knowledge, playbook, pricing
Names the pack, the commercial assemblage that ships a plugin (built inline from `./skills/` or merged from published `@agentproto/skill-pack-*` includes) together with zero or more AIP-53 apps, catalog members (AIP-47 roles, AIP-25 personas, AIP-38 policies) that become resolvable by ref once the pack is installed, a knowledge workspace selection, an optional playbook to generate, legacy pricing data, and non-technical blockers, as a first-class doctype. Instances (an AIP-6 company, an AIP-22 office, an AIP-9 operator) are never pack content. Fixes the PACK.md frontmatter (`pack/v1`), every cross-field rule a conformant parser enforces, the two authoring paths (`definePack` and `parsePackManifest`) over one shared zod schema, the frozen `PackHandle` with its derived `status` (`planned | assembling | ready | gated`), the `aip://64/<name>[@version]` artifact-ref grammar and the git pin grammar reused from AIP-62, the `review.pack-digest/v1` content digest over the PACK.md, each declared app's APP.md source and each declared catalog member's file, and the installation contract (apps install through AIP-42 `app_install`, the plugin installs through the skill install verb, catalog members register with the host's role, persona and policy sources, knowledge resolves at query time, the playbook generates on demand, and re-applying a pack replaces it, never duplicates it).