# Latest host agent release (auto-update) Source: https://docs.glasswarp.com/api-reference/agent/latest-host-agent-release-auto-update /openapi.yaml get /v1/agent/latest Public. Windows host agents poll this when idle to download a newer signed zip. See `docs/AGENT_AUTO_UPDATE.md`. # Create an API key Source: https://docs.glasswarp.com/api-reference/api-keys/create-an-api-key /openapi.yaml post /v1/keys Create a new API key. The full key is returned once in the response and never stored — only a SHA-256 hash is persisted. # List API keys Source: https://docs.glasswarp.com/api-reference/api-keys/list-api-keys /openapi.yaml get /v1/keys List all active API keys (prefix only — full key is never stored). # Revoke an API key Source: https://docs.glasswarp.com/api-reference/api-keys/revoke-an-api-key /openapi.yaml delete /v1/keys/{keyId} Immediately revoke an API key. Existing sessions are not affected. # Kill apps launched in this session Source: https://docs.glasswarp.com/api-reference/apps/kill-apps-launched-in-this-session /openapi.yaml post /v1/sessions/{sessionId}/app/kill Terminate processes previously spawned via `/app/launch`. Omit `pid` to kill all session-launched apps; pass `pid` to kill one. Uses `taskkill /T /F` with protected-process guards. # Launch an application on the rig Source: https://docs.glasswarp.com/api-reference/apps/launch-an-application-on-the-rig /openapi.yaml post /v1/sessions/{sessionId}/app/launch Spawn a process on the host by absolute path or PATH name (e.g. `notepad.exe`). Does not enter Game Mode shell/display policy — desktop API session stays active. Returns the new PID. Tracked apps are killed on session end or via `/app/kill`. # List click targets (grounding v1) Source: https://docs.glasswarp.com/api-reference/grounding/list-click-targets-grounding-v1 /openapi.yaml get /v1/sessions/{sessionId}/targets Sparse click targets in native capture coords from the host focused-window UI Automation snapshot. Empty if UIA finds nothing or the host is unavailable. Clients may still inject CV/grid targets via the SDK. # Send input events Source: https://docs.glasswarp.com/api-reference/input/send-input-events /openapi.yaml post /v1/sessions/{sessionId}/input Send mouse and keyboard events to the remote desktop. Events are executed in order on the host. Optional per-event `delay_ms` pauses between events. **Batching:** prefer a short ordered list when the next screens are predictable. Cap is 100 events per call (MCP `send_actions` uses the same endpoint with a tighter cap of 10 and an optional verification observe in the same MCP turn). **Rate limits:** each event counts individually against the per-plan input quota — a 10-event batch consumes 10 tokens, not 1. Responses use HTTP 429 with `Retry-After` when exceeded. **Safety:** the owner kill switch and session end interrupt an in-flight batch mid-sequence on the host (agent ≥ 0.2.21 with `api_input_ack`). The HTTP call returns **410** with `executed` / `aborted` counts — not 200. # API reference Source: https://docs.glasswarp.com/api-reference/overview The Glasswarp Platform API — real-time vision and native input on Windows. Every endpoint below is generated from the live OpenAPI spec and comes with an interactive request builder. The SDK wraps these same endpoints — see [Install the SDK](/get-started/install-sdk). ## Base URL ``` https://signal.glasswarp.com ``` ## Authentication All `/v1/*` endpoints require a bearer token: ```bash theme={null} Authorization: Bearer gw_live_sk_... ``` Create and scope keys in [Console → API Keys](https://www.glasswarp.com/console/keys). Keys can reach any rig you own that has API access enabled. ## Conventions * **JSON** request and response bodies (observe returns JPEG as base64 plus metadata). * **Sessions meter usage** — always end them (`POST /v1/sessions/{id}/end`). * **Rate limits** apply per key; handle `429` with backoff. Input limits count **each event** in a batch (not each HTTP call) — a 10-event batch consumes 10 of the input quota. * **Some errors mean stop, not retry.** Three are worth handling explicitly: | Status | `code` | What happened | What to do | | ------ | -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- | | `410` | `aborted_by_owner` | The rig owner killed the session mid-batch. The body reports `executed` and `aborted` counts — only `executed` events reached the machine. | Stop. The session is gone; do not retry or start a new one without the human. | | `409` | `rig_busy` | That rig already has an active session (one per rig, every tier). `active_session_id` names it. | Wait for the idle auto-end (\~15 min) or end it in the Console. | | `402` | `concurrent_limit` | Your plan's concurrent-session cap is reached (Free 1 / Builder 3 / Growth 10). This is a **per-account** limit and is separate from `rig_busy`; the tier cap only buys parallelism across **different** rigs. | End a session, wait for the \~15 min idle auto-end, or upgrade. | | `503` | `capture_recovering` | Host screen capture is reacquiring after a display loss, so no frame is trustworthy. | Retry observe for a few seconds. Never act on your last known frame. | Browse the full endpoint list in the **Endpoints** section of the sidebar — each has an interactive request builder. Start with `GET /v1/rigs` to list the machines your key can reach. Prefer the [Python SDK](/get-started/install-sdk) for auth, retries, and the vision/grounding helpers. Use raw REST from any other language. # Get rig details Source: https://docs.glasswarp.com/api-reference/rigs/get-rig-details /openapi.yaml get /v1/rigs/{rigId} Get details and capabilities for a specific rig. # List rigs Source: https://docs.glasswarp.com/api-reference/rigs/list-rigs /openapi.yaml get /v1/rigs List all rigs on your account with online status and capabilities. # Get accumulated dirty rectangles Source: https://docs.glasswarp.com/api-reference/screenshots/get-accumulated-dirty-rectangles /openapi.yaml get /v1/sessions/{sessionId}/dirty Returns DXGI `GetFrameDirtyRects` accumulated since the last poll, in capture-buffer coordinates `[x, y, w, h]`. When `available` is false (GDI capture fallback), dirty metadata is absent — treat as assume changed, never as "no change". Empty `rects` with `available: true` means no visual change. Pair with ROI screenshot (`x,y,w,h`) when dirty is available. # Observe: frame, changes, and click targets in one round trip Source: https://docs.glasswarp.com/api-reference/screenshots/observe:-frame-changes-and-click-targets-in-one-round-trip /openapi.yaml get /v1/sessions/{sessionId}/observe One-round-trip agent observe: JPEG (same options as screenshot), dirty rects take, and sparse click targets from the host UI Automation snapshot. Response includes structured `text` (window title + readable target lines), `changed` (false only when DXGI dirty is available and empty; true when dirty is unavailable — assume changed), and `capture_mode` (`dxgi` | `gdi_fallback` | `unknown`; legacy `gdi` = `gdi_fallback`). Password / masked fields are redacted in `text` and target `value` (`masked=true`, value always `[redacted]`). Set `image=0` for text/targets-only verification (skips JPEG encode — much faster). Prefer `max_width=960` and `quality=60` for cheap verification-grade frames when you do need an image. SDK applies Set-of-Mark when `mark=1` and targets are present. Coords are native capture space. # Take a screenshot Source: https://docs.glasswarp.com/api-reference/screenshots/take-a-screenshot /openapi.yaml get /v1/sessions/{sessionId}/screenshot Return a JPEG of the current desktop from the host's last-frame buffer (DXGI capture, not VNC). Capture is always native resolution (including 4K). Use `max_width` / `quality` to downscale on the host for faster LLM ingest. Response headers include both JPEG size and native desktop size for click coordinate mapping. # Create a session Source: https://docs.glasswarp.com/api-reference/sessions/create-a-session /openapi.yaml post /v1/sessions Start a new desktop session on a rig. The rig must be online and have API access enabled by the owner. # End a session Source: https://docs.glasswarp.com/api-reference/sessions/end-a-session /openapi.yaml delete /v1/sessions/{sessionId} End a session. Triggers safety_restore on the host (restores desktop state, disarms input). # Get session details Source: https://docs.glasswarp.com/api-reference/sessions/get-session-details /openapi.yaml get /v1/sessions/{sessionId} Get the status, duration, and action count for a session. # List active sessions Source: https://docs.glasswarp.com/api-reference/sessions/list-active-sessions /openapi.yaml get /v1/sessions List all active sessions for the authenticated API key. # Get live stream connection Source: https://docs.glasswarp.com/api-reference/streaming/get-live-stream-connection /openapi.yaml get /v1/sessions/{sessionId}/stream Get WebRTC signaling credentials for a live 60fps video stream of the remote desktop. Returns a one-time connect URL — open a WebSocket to it, send an SDP OFFER, and receive an ANSWER to start receiving H.264 video over WebRTC. The stream is the Live View media path (DXGI → NVENC or Media Foundation → WebRTC H.264). Separate from agent observe/screenshot JPEGs. Sub-30 ms glass-to-glass on LAN is a mission target, not a guaranteed measurement. # Create a session webhook Source: https://docs.glasswarp.com/api-reference/webhooks/create-a-session-webhook /openapi.yaml post /v1/webhooks Register a public HTTPS endpoint. Management uses the **console JWT** (same as `/v1/keys`) — API keys (`gw_*`) cannot create webhooks. The response includes `webhook.secret` once — verify deliveries with `X-Glasswarp-Signature` over `"{timestamp}.{body}"` and `X-Glasswarp-Timestamp` (reject skew > 5 minutes). URLs are DNS-resolved at create and again at every delivery; private / loopback / link-local / cloud-metadata addresses are rejected. Redirects are not followed. Max 5 webhooks per account. # Delete a session webhook Source: https://docs.glasswarp.com/api-reference/webhooks/delete-a-session-webhook /openapi.yaml delete /v1/webhooks/{webhookId} # List session webhooks Source: https://docs.glasswarp.com/api-reference/webhooks/list-session-webhooks /openapi.yaml get /v1/webhooks List outbound HTTPS endpoints that receive `session.started` and `session.ended` events. Secrets are never returned on list. Management auth: console JWT (not API keys). # Eyes and hands Source: https://docs.glasswarp.com/concepts/eyes-and-hands The one idea behind Glasswarp — and the line between platform and agent. Glasswarp gives an AI agent **eyes and hands on a real Windows machine**. That's the whole product. You bring the **brain**. ## The split
### Glasswarp (eyes + hands) * **Eyes** — `screenshot`, `observe`, dirty rects, native-resolution JPEG * **Hands** — `click`, `type_text`, `key_press`, `drag`, `move_mouse`, `scroll` * **Grounding** — UIA targets and Set-of-Mark marks * **A safe session** — consent, on-screen indicator, kill switch, audit, Live View
### You (the brain) * The model and prompts * CV, OCR, and solvers * Planning and the agent loop * What to click and what to type Your task logic never runs in the platform.
## Why it matters This split keeps demos honest and integrations clean. When something drives Notepad or solves a board, **the agent did it through the Glasswarp API** — Glasswarp provided the transport, not the intelligence. Practically, it means: * You can swap models, prompts, and CV without touching the transport. * The platform stays a thin, fast, auditable I/O layer. * Safety controls live where they belong: around the session, not inside your agent's decisions. ## Versus a sandbox or VNC A cloud sandbox gives you a **fresh, throwaway Linux VM**. VNC gives you a **soft framebuffer** where every click is a pixel guess. Glasswarp gives you the **real Windows PC you own** — native resolution, structured UIA targets so clicks land in real controls, and a Live View path that can use GPU encode when the host has NVENC. Agent observe/screenshot eyes are host capture + CPU JPEG (separate from the Live View media track). Glass-to-glass LAN latency under 30 ms is a **mission target** for Live View, not a published measurement. Continue to [Rigs and sessions](/concepts/rigs-and-sessions) and [Safety and consent](/concepts/safety-and-consent) (including privacy and data flow). # Grounding Source: https://docs.glasswarp.com/concepts/grounding Click targets, not guessed pixels. The difference between a reliable agent and a flaky one is **grounding**: acting on structured targets instead of guessed coordinates. VNC only sees pixels; Glasswarp surfaces real UI structure when the app exposes it. ## UIA targets When a Windows app exposes its UI through UI Automation, the host returns sparse click targets with roles, names, and bounds. Host **v0.2.14+** also includes popup menus and combo dropdown lists (not just the focused window chrome). ```python theme={null} targets = gw.list_targets(sid) # [] if the app exposes none for t in targets: print(t.id, t.role, t.name, t.bounds) gw.click_target(sid, target_or_id=targets[0].id, targets=targets) # clicks bbox center ``` Target ids are valid for the **observe / targets response that produced them**. Do not persist them across sessions, and re-observe (or call `list_targets` again) before acting when the screen may have changed — especially lists, file browsers, and tables. Scripts should re-resolve targets each loop, not hardcode ids. ## Set-of-Mark (SoM) Draw numbered marks on a frame so a vision model can pick a target by **id** instead of estimating pixels. ```python theme={null} from glasswarp import targets_from_grid, som_annotate # From host UIA, client CV, or a grid over a board: targets = targets_from_grid(x0=100, y0=200, x1=900, y1=800, rows=8, cols=8) obs = gw.observe(sid, max_width=1280, mark=True, targets=targets) # Send obs.jpeg to your model; it returns a target id → gw.click_target(sid, "12", targets=targets) ``` `som_annotate` and the grid helpers need the grounding extra (`pip install "glasswarp[grounding]"`, which pulls in Pillow). ## The grounding ladder Most reliable — real controls with names and bounds. For canvases and games that expose no UIA, build targets client-side. `click(x, y)` still works, but avoid guessing when a target exists. Continue to [Safety and consent](/concepts/safety-and-consent) and [Safe agent sessions](/guides/safe-agent-sessions). # Rigs and sessions Source: https://docs.glasswarp.com/concepts/rigs-and-sessions The two core objects: the machine and the connection to it. ## Rigs A **rig** (console: **PC**) is a Windows machine running the host agent, paired to your account. The UI says PC; the API still uses `list_rigs` / `rig_id`. The person at the machine installs without a Glasswarp login; you pair the tray code. Your API key can reach any PC you own that has API access enabled. ```python theme={null} rigs = gw.list_rigs() for rig in rigs: print(rig.id, rig.online, rig.api_access_enabled) ``` | Field | Meaning | | -------------------- | -------------------------------------------- | | `id` | Stable rig identifier | | `online` | Host agent currently connected | | `api_access_enabled` | Owner turned on API access (per-rig consent) | To keep a rig reachable for agents, enable **Keep rig awake** in the rig's settings. Windows stays awake and connected while the physical display can rest. See [Keep a rig awake](/guides/keep-rig-awake). API sessions also manage **Windows HDR** for capture quality: Live View is SDR, so the host temporarily turns HDR off for the session and restores it afterward. See [Live View → Color / HDR](/guides/live-view#color--hdr). ## Sessions A **session** is an active, metered connection to a rig. It owns the host's capture/encode/input pipeline for its lifetime. ```python theme={null} session = gw.create_session(rig_id=rig.id, mode="desktop") sid = session.session_id status = gw.get_session(sid) # current status active = gw.list_sessions() # all active sessions gw.end_session(sid) # end + safety_restore ``` ### Modes | Mode | Use | | --------- | -------------------------------------------------------- | | `desktop` | The full Windows desktop — the default for agents and QA | | Game Mode | A frozen demo at `/play`; not a general API surface | ### Lifecycle and cleanup Sessions meter usage. Always end them in a `finally` block. The host also runs `safety_restore` automatically on disconnect, launch failure, `EXIT_GAME`, `FORCE_QUIT`, and API `END_SESSION`. ```python theme={null} session = gw.create_session(rig_id=rig.id, mode="desktop") try: ... # your loop finally: gw.end_session(session.session_id) ``` Continue to [Vision](/concepts/vision). # Safety and consent Source: https://docs.glasswarp.com/concepts/safety-and-consent Consent, observability, encryption, and what Glasswarp can and cannot see. Glasswarp drives a real machine, so consent and observability are built into the platform — not left to the agent. Glasswarp provides the **controls**; you remain responsible for your agent's **behavior**. ## Per-rig consent API access is **off by default**. The rig owner must explicitly enable `api_access_enabled` in **Console → Rigs**. No session can route to a rig without it. ```python theme={null} rig = next( (r for r in gw.list_rigs() if r.online and r.api_access_enabled), None, ) ``` ## Visible session indicator While an API session is active, the host shows an on-screen **"API session active"** indicator on the real machine. The owner always sees when an agent has eyes and hands. ## Kill switch and safety\_restore The owner can end any session from the host or Console. On **every** exit path — disconnect, launch failure, and API `END_SESSION` — the host runs `safety_restore` to return the machine to a safe state. ## Audit log Every API session creates an audit row (who, when, which rig, session duration, action counts). Sessions are observable live via [Live View](/guides/live-view) and after the fact via the Console. ## Privacy and data flow Glasswarp is the transport between your agent and a Windows PC you own. Be clear about what travels where. ### What is encrypted today | Path | Encryption | Who can see content | | ---------------------------------------------- | --------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | | REST API (screenshot, observe, input, session) | TLS to the API gateway; WSS to the host | The **API client** and the **rig**. The Glasswarp gateway **relays** these payloads transiently to fulfill the request. | | Live View (optional WebRTC) | DTLS-SRTP media | The **viewer you authorize** (typically the rig owner in Console). Glasswarp signaling handles offer/answer and ICE — not media frames. | ### What we store * **Stored:** account identity, API key hashes (not plaintext keys), session metadata and metering (start/end, minutes, action counts), audit rows, billing records. * **Not designed as a recording product:** screenshot JPEG bytes and input event payloads are handled to complete the API call. They are not written to a customer-facing screenshot archive. ### UIA text in observe `observe` (and `/targets`) can return structured UI Automation text alongside or instead of a JPEG: focused window title/role, click-target names/roles, and optional field values. That text is far easier to log, cache, and leak than pixels — treat it carefully in your own systems. | Field | Returned? | | ---------------------------- | -------------------------------------------------------------- | | Window title / role | Yes | | Target name / role / focus | Yes | | Edit / field **values** | Yes for ordinary fields (truncated) | | **Password / masked** fields | Role and name only — value always `[redacted]` (`masked=true`) | Never rely on the platform to scrub secrets from screenshots; pixels can still show masked dots or adjacent labels. Do not put passwords in prompts. Honest model for v1: the REST control plane is an **authenticated relay**, not application-layer end-to-end encryption. Glasswarp infrastructure can technically access those bytes while serving the request. Do not treat the screenshot/input API as “Glasswarp can never see your screen.” ### Who can watch Live View Live View is for **humans you authorize** (rig owner / console session), not a back-office feed for Glasswarp staff. Opening Live View is an explicit console action on a session you own. ### Longer-term privacy The product direction is a privacy model where **plaintext screenshots and input are not readable by Glasswarp** (for example WebRTC / DataChannel control, or application-layer E2E), matching Live View’s media encryption. Until that ships, use the table above — not marketing shorthand. ## What's yours vs ours
**Glasswarp provides** * Consent gate (per-rig API access) * Visible indicator * Owner kill switch (console session end + host tray **End API session…**; remote input blocked during tray confirms; Unpair disabled while a session is active) * Audit log + Live View * TLS / WSS in transit; WebRTC media encryption for Live View * Key scoping and rate limits
**You are responsible for** * What your agent decides to do * Prompts, guardrails, and human-in-the-loop (HITL) review in your app * Not exfiltrating or misusing screen contents * Windows licensing and physical/cloud host security
Glasswarp is the safe, observable I/O layer — not an AI-safety layer for your agent's decisions. Design your own review and approval flows on top. ## Next: safe agent sessions For a practical checklist (dedicated rigs, HITL, allowlists, budgets) — the decision-layer playbook that sits on top of these controls — see [Safe agent sessions](/guides/safe-agent-sessions). # Vision Source: https://docs.glasswarp.com/concepts/vision How your agent sees the screen — screenshots, observe, and dirty rects. Glasswarp's eyes are designed for fast agent loops: fresh, native-resolution frames with the option to fetch only what changed. ## Screenshot A high-fidelity, native-resolution JPEG from the host capture buffer — typically tens of milliseconds for capture + CPU JPEG encode — and it works **with or without** an active WebRTC Live View stream. Agent screenshots use the API capture path (DXGI Desktop Duplication by default, GDI BitBlt fallback); they are **not** re-encoded remote WebRTC frames and are **not** NVENC/GPU JPEG. ```python theme={null} frame = gw.screenshot(sid) with open("screen.jpg", "wb") as f: f.write(frame.jpeg) # Optional: downscale and/or crop to a region of interest. roi = gw.screenshot(sid, max_width=1280, x=100, y=200, w=800, h=600, quality=80) ``` ## Observe — one round trip `observe` returns everything an agent step needs in a single RTT: a JPEG, dirty regions when available, grounding targets (Set-of-Mark annotated when `mark=True`), `changed`, and `capture_mode` (`dxgi` | `gdi_fallback` | `unknown`; legacy hosts may report `gdi`). ```python theme={null} obs = gw.observe(sid, max_width=1280, mark=True) # obs.jpeg — frame (SoM-marked when targets present) # obs.dirty — DXGI dirty regions, or None when unavailable # obs.changed — False only when DXGI dirty was available and empty # obs.capture_mode — "dxgi" (preferred) or "gdi_fallback" (fallback) # obs.targets — click targets from host UIA (may be empty) ``` Use `observe` as the default "look" in your loop — it's the leanest way to get frame + change + grounding together. ## Dirty rects When capture is DXGI, poll only the regions that changed since your last call: ```python theme={null} dirty = gw.dirty_rects(sid) if dirty.get("available") is False or dirty is None: pass # assume changed — GDI fallback has no dirty metadata elif not dirty.get("rects"): pass # nothing changed — skip the model call ``` Skip a model call only when dirty is **available** and empty. If dirty is missing (`null` / `available: false`), **assume changed**. When dirty is available, crop screenshots to the changed region for cheaper loops — see [Build an agent loop](/guides/agent-loop). Capture is the **primary display only** today (first DXGI output / primary GDI screen). Multi-monitor selection is not yet exposed. Continue to [Grounding](/concepts/grounding). # Grounding: observe + Set-of-Mark Source: https://docs.glasswarp.com/examples/grounding Turn a frame into numbered targets a vision model can click by id. This example shows the grounding loop end to end: build targets, get a Set-of-Mark annotated frame, let a model pick an id, and click it. ## Build targets and annotate ```python theme={null} from glasswarp import GlasswarpClient, targets_from_grid gw = GlasswarpClient(api_key="gw_live_sk_...") session = gw.create_session(rig_id=rig.id, mode="desktop") sid = session.session_id # Grid targets over a board or canvas (or use gw.list_targets(sid) for UIA). targets = targets_from_grid(x0=100, y0=200, x1=900, y1=800, rows=8, cols=8) obs = gw.observe(sid, max_width=1280, mark=True, targets=targets) with open("marked.jpg", "wb") as f: f.write(obs.jpeg) # numbered marks drawn on the frame ``` ## Let the model choose, then act ```python theme={null} # Send obs.jpeg to your model; it returns a mark id, e.g. "12". target_id = your_model_pick(obs.jpeg) gw.click_target(sid, target_id, targets=targets) ``` ## ROI + dirty rects for tight loops ```python theme={null} rects = gw.dirty_rects(sid) if rects: r = rects[0] crop = gw.screenshot(sid, x=r.x, y=r.y, w=r.w, h=r.h) # feed only the changed region to your model ``` Set-of-Mark helpers (`som_annotate`, `targets_from_grid`) need the grounding extra: `pip install "glasswarp[grounding]"`. ## Reference agent loops * `minesweeper_solver_demo.py` — demo with CV + solver (no LLM) * `gemini_agent_loop.py` — SoM → Gemini → `click_target` → verify * `claude_agent_loop.py` — observe → Claude → act (skips idle frames by default) * `templates/observe_think_act/` — bare skeleton See [Minesweeper](/examples/minesweeper) and [Build an agent loop](/guides/agent-loop). # Minesweeper: agent loop + grounding Source: https://docs.glasswarp.com/examples/minesweeper An agent plays Google Minesweeper on a real Windows PC — observe → think → act → verify. A fast demo on a browser game. Glasswarp is eyes and hands; the solver or LLM is the brain (example code). ``` observe → think (CV+solver or Gemini) → click_target → verify → repeat ``` **Platform vs agent:** `observe` / `click_target` are Glasswarp. Board CV, `solve()`, and LLM prompts are example code under `sdk/python/examples/` — the pattern to copy for your own tasks. ## Prerequisites [Install the host](/get-started/install-host) → Console → Rigs → API access on. ```bash theme={null} pip install "glasswarp[grounding]" export GLASSWARP_API_KEY=gw_live_sk_... ``` ## Run (deterministic think — fastest) ```bash theme={null} pip install "glasswarp[demos]" export GLASSWARP_API_KEY=gw_live_sk_... glasswarp-demo minesweeper ``` Repo checkout: ```bash theme={null} python sdk/python/examples/minesweeper_solver_demo.py ``` Launches Chrome → Google Minesweeper (Medium) → plays to a win. Watch **Console → Sessions → Eye**. ## Run (LLM think — Gemini + SoM) ```bash theme={null} export GEMINI_API_KEY=... python sdk/python/examples/gemini_agent_loop.py ``` Same eyes/hands; Gemini picks a Set-of-Mark cell id each step. ## Platform vs agent | Step | Owner | | ----------------------------- | ------------------------- | | Observe (`observe`) | Platform | | Think (CV + solver, or model) | Your agent / example code | | Act (`click_target`) | Platform | | Verify (re-observe) | Your loop + platform eyes | Canonical shape: [Build an agent loop](/guides/agent-loop). Skeleton: `sdk/python/templates/observe_think_act/`. ## Tuning | Env var | Default | Purpose | | ----------------------- | ------- | ------------------------------- | | `DEMO_BOOTSTRAP_CHROME` | `1` | `0` if the game is already open | | `TOTAL_MINES` | `40` | Google Medium | | `CLICK_PAUSE_S` | `0.12` | Delay between clicks | Outcome-named starters. SoM + click\_target. # Mona Lisa: precise drags in Paint Source: https://docs.glasswarp.com/examples/mona-lisa An agent paints a ~40×60 / 18-color Mona Lisa mosaic in Paint and saves it to the Desktop. An agent turns a portrait into a paint-by-numbers plan (\~40 columns × \~60 rows, 18 quantized colors), paints it in **MS Paint** on a real Windows PC, and saves the JPEG to the Desktop — through the Glasswarp API. Paint is left open when the session ends so you can see the result. ``` launch_app(mspaint) → draw tile grid → fill colors (UIA Edit Colors) → Save As: address bar → shell:Desktop → short filename → Save → end_session(keep_apps=True) # Paint stays open ``` **Platform vs agent:** Glasswarp provides eyes and hands. The tile plan and Save As recipe are example / customer code. See [Safe agent sessions](/guides/safe-agent-sessions). ## Run ```bash theme={null} pip install "glasswarp[demos]" export GLASSWARP_API_KEY=gw_live_sk_... glasswarp-demo mona-lisa ``` Repo checkout: ```bash theme={null} cd sdk/python pip install "glasswarp[grounding]" python examples/paint_mona_lisa_demo.py ``` Bundled source: `examples/assets/mona_lisa.jpg`. ## Why it's hard Continuous drag paths break over laggy transports — if the strokes look right, the transport is right. * **Exact color** via Paint's Edit Colors dialog (UIA), not pixel guessing * **Native drags / fills** on a real Win32 canvas * **Fresh host screenshot** for verify — API capture JPEG (DXGI or GDI), not a re-encoded Live View / WebRTC frame ## Tuning | Env | Default | Purpose | | ---------------- | -------------------- | ---------------------------------------------- | | `MONA_COLS` | `40` | Mosaic columns (\~2400 tiles with aspect rows) | | `MONA_COLORS` | `18` | Palette size (Edit Colors visits) | | `MONA_SAVE_NAME` | `mona_glasswarp.jpg` | Desktop filename | | `PAINT_PATH` | `mspaint.exe` | App to launch | ## Robust Save As (optional) If Save As fails on your Paint build, the longer dialog-recovery script is: ```bash theme={null} python examples/paint_mona_lisa_demo_robust.py ``` Prefer the default showcase (`glasswarp-demo mona-lisa`, 40×18) for docs and demos. ## Related * [Paint wordmark](/examples/paint) * [Agent loop](/guides/agent-loop) * [Safe agent sessions](/guides/safe-agent-sessions) # Notepad: UIA targets + type Source: https://docs.glasswarp.com/examples/notepad An agent launches Notepad, finds the edit surface via UIA, and types. An agent **launches Notepad, finds the edit surface via UIA targets, and types** — no pixel guessing. ``` launch_app → observe / list_targets (UIA) → click_target → type_text ``` **Platform vs agent:** Glasswarp provides eyes (`observe` / `list_targets`) and hands (`click_target` / `type_text`). Choosing Notepad and what to type is the agent. Unlike pixel-only remote-desktop tools, the host returns sparse click targets from Windows UI Automation, and native input lands in the real Win32 control. ## Prerequisites See [Install the host](/get-started/install-host). ```bash theme={null} pip install glasswarp pillow export GLASSWARP_API_KEY=gw_live_sk_... ``` ## Run ```bash theme={null} python sdk/python/examples/notepad_uia_demo.py ``` | Env var | Default | Purpose | | -------------------- | --------------------------------- | -------------------------- | | `NOTEPAD_PATH` | `C:\Windows\System32\notepad.exe` | App to launch | | `DEMO_TEXT` | short hello line | Text typed into the editor | | `GLASSWARP_BASE_URL` | Fly signaling URL | API gateway | ## What you'll see ``` Rig: DESKTOP-… (…) Session: … → launch_app C:\Windows\System32\notepad.exe → observe jpeg … targets=N → UIA edit target id=… role='edit' … → typed into Notepad; saved notepad_final.jpg Result: ok ``` Artifacts: `notepad_observe.jpg` (SoM-marked when targets exist), `notepad_final.jpg`. Watch live via **Console → Sessions → Eye**. How UIA targets and Set-of-Mark make clicks reliable. # Paint: draw the wordmark Source: https://docs.glasswarp.com/examples/paint An agent draws a bold, hollow GLASSWARP wordmark in Paint on real Windows. The headline visual demo: an agent **launches Paint, themes the canvas, and drags a bold GLASSWARP wordmark** — on a real Windows PC, through the API. It uses native-resolution capture and precise drag input. ``` launch_app(mspaint) → maximize → theme canvas → drag wordmark → screenshot ``` **Platform vs agent:** Glasswarp supplies eyes (`screenshot` / Live View) and hands (`drag` / `mouse_down` + `mouse_up`). Choosing *what* to draw is the agent. ## Prerequisites See [Install the host](/get-started/install-host). The host must be rebuilt with `mouse_down` / `mouse_up` (required for drag). ```bash theme={null} pip install "glasswarp[grounding]" export GLASSWARP_API_KEY=gw_live_sk_... ``` ## Run ```bash theme={null} python sdk/python/examples/paint_mark_demo.py ``` ## What you'll see Paint opens and maximizes, discovers its canvas and color controls through UIA, fills the canvas dark `rgb(9, 13, 21)`, then strokes a geometric GLASSWARP outline in cyan `rgb(13, 196, 242)`. Letter counters stay hollow. Watch live via **Console → Sessions → Eye**. Artifact: `paint_mark_final.jpg`. ## Config `PAINT_PATH` defaults to `C:\Windows\System32\mspaint.exe`. The script derives wordmark size and placement from the native Paint canvas bounds. UIA targets + typing — structured clicks, no pixel guessing. Drag, click, type, and low-level primitives. # Templates Source: https://docs.glasswarp.com/examples/templates Outcome-named starters: observe → think → act → verify on a real Windows PC. Copy-paste starters under `sdk/python/templates/`. Each one is **example / customer code** — Glasswarp is eyes and hands; your model (or a few lines of Python) is the brain. ``` Observe → Think → Act → Verify ``` ## Catalog | Template | Shows | | -------------------- | ----------------------------------------- | | `observe_think_act/` | Minimal agent loop (no LLM required) | | `notepad_type/` | Notepad via UIA targets + native type | | `live_view_hitl/` | Open a session for human Live View / kill | ```bash theme={null} cd sdk/python export GLASSWARP_API_KEY=gw_live_sk_... python templates/observe_think_act/main.py python templates/notepad_type/main.py python templates/live_view_hitl/main.py ``` Longer examples live under `sdk/python/examples/` — [Mona Lisa](/examples/mona-lisa), [Minesweeper](/examples/minesweeper), Paint wordmark. Full catalog: `sdk/python/templates/README.md`. Unlike browser stacks that bundle runtime + Stagehand-style agent primitives + model routing, Glasswarp templates stay thin: session hygiene, observe/act, and a clear place to plug **your** planner. They use `examples/safe_session.py` (budgets, intent logs, allowlists) — see [Safe agent sessions](/guides/safe-agent-sessions). The canonical observe → think → act → verify shape. Continuous drag strokes in Paint — a stress test for input transport. # Your first session Source: https://docs.glasswarp.com/get-started/first-session Start a session, observe UIA targets, click by name, and clean up. With an [installed host](/get-started/install-host) and an [eligible PC](/get-started/pair-rig), you're ready to run a full session. ## Full example ```python theme={null} import os from glasswarp import GlasswarpClient gw = GlasswarpClient(api_key=os.environ["GLASSWARP_API_KEY"]) rig = next( (r for r in gw.list_rigs() if r.online and r.api_access_enabled), None, ) if rig is None: raise RuntimeError("No online PC with API access enabled") session = gw.create_session(rig_id=rig.id, mode="desktop") sid = session.session_id print("session:", sid) try: obs = gw.observe(sid) save = next((t for t in obs.targets if (t.name or "") == "Save"), None) if save is None: print("No target named Save — first few:", [ (t.role, t.name) for t in obs.targets[:8] ]) else: gw.click_target(sid, save) finally: gw.end_session(sid) ``` ## What happens The gateway checks ownership and consent, then the host starts a desktop session and shows the on-screen "API session active" indicator. One round-trip: JPEG plus UIA targets (names, roles, bounds). Native Win32 click at the named control's center — not a guessed pixel. The host runs `safety_restore` and tears the session down. Always end sessions — they meter usage. Wrap actions in `try/finally` and always call `end_session`. Sessions meter usage and hold the host's pipeline. `safety_restore` also runs automatically on disconnect, launch failure, and `END_SESSION`. ## Watch it live Open **Console → Sessions → Eye** to watch at 60fps while your code runs. See [Live View](/guides/live-view). Next: [ground your clicks](/concepts/grounding), [build an agent loop](/guides/agent-loop), and [safe agent sessions](/guides/safe-agent-sessions). # Install the host agent Source: https://docs.glasswarp.com/get-started/install-host Put Glasswarp on a Windows PC. The PC owner does not need a Glasswarp account. Glasswarp drives a **real Windows machine** — GPU, apps, and files stay there. The host agent is the black box that runs on that PC: capture, native input, on-screen session indicator, tray kill switch. This is a physical, one-time, owner-consent step. The person at the PC installs. The automator (Glasswarp account holder) pairs the tray code from another device. Nobody shares Windows passwords. ## Who does what | Step | Who | Login? | | -------------------- | --------------------------------------------- | --------------------------------------------------------------------------------- | | Download and install | Person at the Windows PC (you or your client) | **No** — [glasswarp.com/downloads](https://www.glasswarp.com/downloads) is public | | Paste pairing code | Automator (your Glasswarp account) | **Yes** — [glasswarp.com/pair](https://www.glasswarp.com/pair) | | Enable API access | Automator, with the PC owner's OK | Console → that PC | The client should **not** create a Glasswarp account. If they did, the PC would bind to *their* tenant and your API key would not see it. ## Requirements * Windows 10/11 (64-bit). * Current public host agent: **v0.2.21** (see [Downloads](https://www.glasswarp.com/downloads); installed agents auto-update when idle). * A GPU is recommended for Live View NVENC hardware encode. Without NVIDIA, Live View falls back to Media Foundation H.264, which may be unreliable in Chrome at high resolutions on non-NVIDIA GPUs. Agent observe/screenshot uses CPU JPEG and does not require NVENC. * **Primary display only** today — put the work you want seen on the primary screen. * Owner access to the machine. ## Install (on the Windows PC) On that PC, open [glasswarp.com/downloads](https://www.glasswarp.com/downloads) and get **`GlasswarpSetup.exe`**. You can email this URL to a client. Double-click **`GlasswarpSetup.exe`**. Accept the [Terms](https://www.glasswarp.com/terms) / [Privacy Policy](https://www.glasswarp.com/privacy), click **Install**, and approve the Windows Administrator prompt. The agent appears in the system tray (check the `^` overflow on Windows 11). Tray → **End API session…** is the on-machine kill switch. Tray → **Copy pairing code** (`ABCD-1234`). Send it to the automator. Do not sign in on this PC. ## Pair (automator, logged in) See [Pair a PC](/get-started/pair-rig). Paste the code at [glasswarp.com/pair](https://www.glasswarp.com/pair), then turn on **API access** for that PC (off by default). ## Verify From your laptop (not the client PC): ```python theme={null} import os from glasswarp import GlasswarpClient gw = GlasswarpClient(api_key=os.environ["GLASSWARP_API_KEY"]) for rig in gw.list_rigs(): print(rig.id, rig.online, rig.api_access_enabled) ``` You should see that machine with `online=True` and `api_access_enabled=True`. (`rig_id` is the API name for the PC.) If API access is off, sessions are refused. That is the per-PC consent guardrail — see [Safety and consent](/concepts/safety-and-consent). Next: [pair the PC](/get-started/pair-rig) and [start your first session](/get-started/first-session). # Install the SDK Source: https://docs.glasswarp.com/get-started/install-sdk Add the Glasswarp Python SDK to your project. The Glasswarp SDK is Python 3.9+. Any language can call the REST API directly — see the [API reference](/api-reference/overview) — but the SDK handles auth, retries, and the vision/input helpers for you. ## Install ```bash pip theme={null} python -m pip install glasswarp ``` ```bash uv theme={null} uv add glasswarp ``` ```bash poetry theme={null} poetry add glasswarp ``` ```bash pipenv theme={null} pipenv install glasswarp ``` The only core runtime dependency is `httpx`. Source: [github.com/glasswarp/python-sdk](https://github.com/glasswarp/python-sdk) (Apache-2.0). The Windows host agent and platform remain proprietary. ## Grounding extra For Set-of-Mark annotation and ROI image helpers (which use Pillow): ```bash pip theme={null} python -m pip install "glasswarp[grounding]" ``` ```bash uv theme={null} uv add "glasswarp[grounding]" ``` ```bash poetry theme={null} poetry add "glasswarp[grounding]" ``` Skip the grounding extra for the basic session and input loop. Add it only when you call `som_annotate` or crop screenshots client-side. ## From the monorepo Developing the SDK inside this repo: ```bash theme={null} python -m pip install -e sdk/python ``` Public installs use PyPI: [pypi.org/project/glasswarp](https://pypi.org/project/glasswarp/). ## Configure the client The client reads `GLASSWARP_API_KEY` from the environment: ```python theme={null} import os from glasswarp import GlasswarpClient gw = GlasswarpClient(api_key=os.environ["GLASSWARP_API_KEY"]) ``` | Parameter | Env var | Default | | ---------- | -------------------- | ------------------------------ | | `api_key` | `GLASSWARP_API_KEY` | required | | `base_url` | `GLASSWARP_BASE_URL` | `https://signal.glasswarp.com` | Next: [install the host agent](/get-started/install-host) on your Windows machine. # Connect via MCP Source: https://docs.glasswarp.com/get-started/mcp Give Claude, Cursor, or any MCP client eyes and hands on a Windows PC you or your client already own — no integration code. Glasswarp ships a **remote MCP server** so MCP-enabled assistants and IDEs get eyes and hands on a paired Windows PC without writing project code. Built for automation developers driving no-API Windows apps (and for agent builders). You bring the brain (the model and the prompts); Glasswarp provides vision, native input, and a safe session. Marketing overview: [glasswarp.com/mcp](https://www.glasswarp.com/mcp). Open-source server: [github.com/glasswarp/mcp-server](https://github.com/glasswarp/mcp-server). MCP sessions are ordinary metered Platform API sessions — same concurrency, active minutes, consent, indicator, kill switch, and audit as REST / SDK. ## Prerequisites The PC owner [installs the host](/get-started/install-host) with no Glasswarp account. You [pair the tray code](/get-started/pair-rig) while logged in and turn on **API access**. [Console → API Keys](https://www.glasswarp.com/console/keys). Same `gw_*` key authenticates REST and MCP. Copy it once when created. `rigs.list` only marks a rig **USABLE** when it is online **and** API-enabled. Start the Windows host agent before asking the assistant to act. ## Endpoint | | | | ------------ | --------------------------------------------------------- | | URL | `https://mcp.glasswarp.com/mcp` | | Transport | Streamable HTTP | | Auth | `Authorization: Bearer ` | | Platform API | `https://signal.glasswarp.com` (same sessions / metering) | ## Client config Use your **real** API key from the console — not placeholder text. ### Cursor (recommended) Add to Cursor One-click install uses `npx -y @glasswarp/mcp`. Set `GLASSWARP_API_KEY` to your console key when prompted. Current Cursor builds often fail remote Streamable HTTP with `fetch failed`. Bridge with [`mcp-remote`](https://www.npmjs.com/package/mcp-remote) over stdio, and use an **absolute** `npx` path for your OS (bare `npx` often breaks inside Cursor.app). ```json theme={null} { "mcpServers": { "glasswarp": { "command": "/opt/homebrew/bin/npx", "args": [ "-y", "mcp-remote", "https://mcp.glasswarp.com/mcp", "--header", "Authorization:${GLASSWARP_AUTH}" ], "env": { "PATH": "/opt/homebrew/bin:/usr/bin:/bin", "GLASSWARP_AUTH": "Bearer gw_live_sk_REPLACE_WITH_YOUR_KEY" } } } } ``` ```json theme={null} { "mcpServers": { "glasswarp": { "command": "/usr/local/bin/npx", "args": [ "-y", "mcp-remote", "https://mcp.glasswarp.com/mcp", "--header", "Authorization:${GLASSWARP_AUTH}" ], "env": { "PATH": "/usr/local/bin:/usr/bin:/bin", "GLASSWARP_AUTH": "Bearer gw_live_sk_REPLACE_WITH_YOUR_KEY" } } } } ``` ```json theme={null} { "mcpServers": { "glasswarp": { "command": "npx", "args": [ "-y", "mcp-remote", "https://mcp.glasswarp.com/mcp", "--header", "Authorization:${GLASSWARP_AUTH}" ], "env": { "GLASSWARP_AUTH": "Bearer gw_live_sk_REPLACE_WITH_YOUR_KEY" } } } } ``` If Cursor cannot find `npx`, use the full path from Node.js (often `C:\\Program Files\\nodejs\\npx.cmd`). ```json theme={null} { "mcpServers": { "glasswarp": { "command": "/usr/bin/npx", "args": [ "-y", "mcp-remote", "https://mcp.glasswarp.com/mcp", "--header", "Authorization:${GLASSWARP_AUTH}" ], "env": { "PATH": "/usr/bin:/bin", "GLASSWARP_AUTH": "Bearer gw_live_sk_REPLACE_WITH_YOUR_KEY" } } } } ``` If `npx` lives elsewhere (nvm, fnm), point `command` at that absolute path. Toggle the server off/on after saving. ### Direct remote URL ```json theme={null} { "mcpServers": { "glasswarp": { "url": "https://mcp.glasswarp.com/mcp", "headers": { "Authorization": "Bearer gw_live_sk_REPLACE_WITH_YOUR_KEY" } } } } ``` After reload, the client should list tools such as `rigs.list`, `session.start`, `screen.observe`, `input.click_target`, `input.type_text`, `input.send_actions`, `app.launch`, `demos.list`, `demos.get`, `session.live_view`, and `session.end`. ## First session Ask your assistant: > Open Notepad on my PC and type hello from MCP. Typical tool flow: `rigs.list` → `session.start` → `screen.observe` (text+targets; `image=true` if you need vision) → `input.send_actions` (or click/type) → verify from the batch result → `session.end` An invalid action is caught during validation **before any event is sent** and the response reports its index (nothing executed). The owner kill switch and session end abort an **in-flight** batch on agent **≥ 0.2.21** (`api_input_ack`): the call returns **`410 aborted_by_owner`** with `executed` / `aborted` / `kill_to_abort_ms` — only `executed` events landed. Older hosts are fire-and-forget; upgrade for guaranteed mid-batch abort. Always end the session when finished. Idle sessions auto-end after about **15 minutes**. ## What to expect from your assistant On connect, the server gives every MCP client behavioral guidance. In practice: your assistant handles quick tasks directly in chat — look at the screen, click, type, and batch predictable sequences. For longer multi-step work in a code-capable client, it will offer to write a small SDK agent that runs the loop at full speed instead of one chat turn per step — your choice either way. Chat-only clients simply keep working through MCP tools. Details and the pre-built demo catalog: [Ways to run agents](/guides/ways-to-run-agents). ## Live View (human in the loop) `session.live_view` returns a **console** deep link. The **rig owner** must be signed into Glasswarp to watch at \~60fps and intervene. API keys alone cannot open the console player — that preserves owner consent. See [Live View](/guides/live-view). ## Tools (v1) | Tool | Role | | --------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | | `rigs.list` | List paired rigs | | `session.start` / `session.end` | Session lifecycle | | `screen.observe` | Text + UIA targets by default; set `image=true` for a verification-grade JPEG (`max_width` 960 / quality \~60). JPEG omitted when `changed=false` | | `input.click_target` / `input.click_xy` | Mouse (prefer targets; `input.click_xy` uses **native** capture coords) | | `input.type_text` / `input.send_keys` / `input.drag` / `input.scroll` | Keyboard and pointer (prefer bundling into `input.send_actions`) | | `input.send_actions` | Up to 10 predictable actions in one call; `observe_after` defaults true with **text** verify (`observe_image=true` for JPEG) | | `app.launch` | Start an app on the host (`args` optional) | | `demos.list` / `demos.get` | Pre-built demo catalog (install + command; they do not run the demos) | | `session.live_view` | Owner console Live View URL | | `session.status` | Session status | **Prompts:** `best_practices`, `demo_mona_lisa`, `demo_minesweeper` ## Troubleshooting Bare `npx` is often broken in Cursor’s spawn environment. Use the absolute `npx` path for your platform from the Cursor section above (Apple Silicon Homebrew, Intel Homebrew, Windows Node.js, or Linux). Do not paste truncated keys with `…`. Use the full `gw_live_sk_…` value from the console. Prefer the stdio `mcp-remote` bridge on Cursor. Online=false means the Windows host agent is not connected. Start GlasswarpHost on that PC; keep **API access** enabled in Console → Rigs. If `signal.glasswarp.com` (or your configured API base URL) fails to resolve on your machine, flush your local DNS cache or try a public resolver (for example `1.1.1.1` or `8.8.8.8`). The MCP server reaches the Platform API from its own network — local DNS only affects tools you run on your machine. ## Building a product instead? Use the [Python SDK](/quickstart) and [Setup for Agents](/setup-for-agents). MCP is for assistants out of the box; the SDK is for products you ship. # Pair a PC Source: https://docs.glasswarp.com/get-started/pair-rig Bind a Windows machine to your Glasswarp account. The PC owner does not log in. A **PC** (API: `rig`) is a Windows machine running the host agent, paired to **your** account. Pairing is what lets your API key reach that machine. The person at the PC installs and reads you a tray code. **You** paste it while logged in. They should not create a Glasswarp account. ## Pair Send [glasswarp.com/downloads](https://www.glasswarp.com/downloads) to the owner, or install yourself on that machine. See [Install the host](/get-started/install-host). On your laptop, sign in and open [glasswarp.com/pair](https://www.glasswarp.com/pair). Enter the code (`ABCD-1234`). That binds the PC to **your** tenant. Open [Console → PCs](https://www.glasswarp.com/console). The machine should show **online**. Toggle **API access** on for that PC. Per-machine owner consent — off by default, required before any API session can route there. ## Eligibility An API session can start only when both are true: * `online` — the host agent is connected. * `api_access_enabled` — API access is on. ```python theme={null} rig = next( (r for r in gw.list_rigs() if r.online and r.api_access_enabled), None, ) if rig is None: raise RuntimeError("No online PC with API access enabled") ``` ## Keep the PC reachable Enable **Keep awake** in that PC's settings if sessions need to connect at any time. Windows stays awake while the physical monitor can rest. See [Keep a rig awake](/guides/keep-rig-awake). Next: [start your first session](/get-started/first-session). # Build an agent loop Source: https://docs.glasswarp.com/guides/agent-loop observe → think → act → verify after each meaningful step. Every Glasswarp integration is the same shape. Glasswarp owns **observe** and **act**; your model owns **think** and **verify**. Open-loop scripts (hardcoded coords, “always click 400,300”) break the moment scale or layout changes — fine as helpers *after* vision/UIA chose a region, not as the outer brain. ## The shape ``` Observe — screenshot / dirty / ROI / UIA targets Think — where is the thing? is it safe to click? did the last step work? Act — one action, or a short batch when the next screens are predictable Verify — look again after each meaningful step; correct or stop ``` Verify after each **meaningful step** — after a batch of predictable actions, or after a single action when the screen may change unpredictably (page loads, installers, network waits, anything that can pop a modal). You do not need a fresh observe between every click in a menu path or dialog you already planned. ```python theme={null} import os from glasswarp import GlasswarpClient gw = GlasswarpClient(api_key=os.environ["GLASSWARP_API_KEY"]) rig = next( (r for r in gw.list_rigs() if r.online and r.api_access_enabled), None, ) if rig is None: raise RuntimeError("No online rig with API access enabled") session = gw.create_session(rig_id=rig.id, mode="desktop") sid = session.session_id try: for step in range(50): obs = gw.observe(sid, max_width=1280, mark=True) # eyes action = decide(obs) # your brain if action is None: break act(gw, sid, action) # hands (one or a short batch) # verify after this meaningful step before the next decide finally: gw.end_session(sid) # always clean up ``` `decide` and `act` are yours: `decide` calls your model with `obs.jpeg` and `obs.targets`; `act` maps its output to `click_target`, `type_text`, or a batched `send_input` / MCP `send_actions` when the next few steps are known. After a meaningful step, **look again** — dialogs move, folders change, and “I clicked Desktop” is not the same as “Save As is on Desktop.” ## Batch when you can predict If the next 3–6 actions are predictable (menu paths, dialog fields, typing into a field you just focused), send them as one batch, then observe once to verify the whole sequence. Single-step when intermediate state is uncertain. If the verifying observe shows something unexpected, re-plan from there. MCP clients use `send_actions` (verification observe on by default — text and targets; a JPEG only when you set `observe_image=true`). SDK clients use `send_input` with an ordered `events` list — same host path. ## Make it cheap: skip idle frames ```python theme={null} for step in range(200): if not gw.dirty_rects(sid): continue # nothing changed — no model call obs = gw.observe(sid, mark=True) ... ``` The example loops skip idle frames by default. Empty `dirty_rects` means "don't spend a model call." ## Ground every action Prefer `click_target` over raw pixels so the loop survives layout shifts: ```python theme={null} obs = gw.observe(sid, mark=True) target_id = decide(obs) # model returns a mark id from *this* observe gw.click_target(sid, target_id, targets=obs.targets) ``` Target ids are valid for the observe that produced them. Do not persist them across sessions, and re-observe before acting when lists, file browsers, or tables may have changed. Scaffolded agents must re-resolve targets each loop — never hardcode ids. ## Keep task logic out of the transport Put prompts, CV, and planning in your own module. The Glasswarp client is a thin transport adapter — mixing task logic into it makes both harder to test. ## Reference loops The SDK ships runnable agent loops: * `templates/observe_think_act/` — bare agent-loop skeleton * `minesweeper_solver_demo.py` — demo with CV + deterministic solver * `gemini_agent_loop.py` — demo with Gemini + SoM → `click_target` * `claude_agent_loop.py` — observe → Claude → act (skips idle frames by default) * `paint_mona_lisa_demo.py` — full demo on a real app (place → paint → Save As) See [Templates](/examples/templates), [Minesweeper](/examples/minesweeper), and [Mona Lisa](/examples/mona-lisa). Before you turn autonomy up, read [Safe agent sessions](/guides/safe-agent-sessions) — budgets, human-in-the-loop review, and allowlists on top of platform consent and Live View. # Send input Source: https://docs.glasswarp.com/guides/input Native Win32 clicks, typing, keys, drags, and scrolls. All input is injected as **native Win32 events** on the real machine — not synthetic browser events. ## Click, type, keys ```python theme={null} gw.click(sid, x=500, y=400) # default left button gw.click(sid, x=500, y=400, button="right") gw.type_text(sid, "Hello from an agent") gw.key_press(sid, "Enter") gw.key_press(sid, "Ctrl+S") ``` ## Move and scroll ```python theme={null} gw.move_mouse(sid, x=640, y=360) gw.scroll(sid, dx=0, dy=-300) # scroll down ``` ## Drag Drag draws a continuous path — essential for canvases, sliders, and selection. ```python theme={null} gw.drag(sid, x1=200, y1=200, x2=600, y2=450) ``` For full control (e.g. multi-segment strokes), use the low-level primitives: ```python theme={null} gw.mouse_down(sid, x=200, y=200) gw.move_mouse(sid, x=400, y=300) gw.move_mouse(sid, x=600, y=450) gw.mouse_up(sid, x=600, y=450) ``` ## Batch events Send several events in one call to reduce round trips: ```python theme={null} gw.send_input(sid, events=[...]) ``` Events run in order on the host. Optional per-event `delay_ms` gives UIs time to settle between steps. Each event counts individually against the per-plan input rate limit (a 10-event batch uses 10 tokens of quota). On agent **≥ 0.2.21** (`api_input_ack`), the owner kill switch and session end abort an in-flight batch mid-sequence — the call returns **`410 aborted_by_owner`** with `executed` / `aborted` / `kill_to_abort_ms` counts (only `executed` events landed). Older hosts are fire-and-forget. MCP clients wrap the same endpoint as `send_actions` (max 10 actions, with a verification observe in the same turn — text and targets by default, JPEG only when `observe_image=true`) — see [Connect via MCP](/get-started/mcp). Prefer [grounded clicks](/concepts/grounding) (`click_target`) over raw coordinates whenever the app exposes UIA targets — they're far more robust to layout changes. Next: [Launch apps](/guides/launch-apps). # Keep a rig awake Source: https://docs.glasswarp.com/guides/keep-rig-awake Keep Windows reachable for agent and remote sessions while allowing the screen to rest. Agents can only reach a rig while Windows is awake and the host agent is connected. **Keep rig awake** prevents system sleep so the rig remains available for new sessions. ## What it does When enabled, the host asks Windows to keep the **system** awake while the Glasswarp agent is connected: * Windows does not enter system sleep from idleness. * The host remains connected and ready for an agent or remote session. * The physical monitor may still dim or turn off normally while no session is active. Keeping the system reachable does **not** keep the screen lit. * During an active API or streaming session, the host also asks Windows to keep the display awake. This is a software availability control. It does not power on a machine that is shut down or already asleep. ## Turn it on Go to **Console → Rigs** and use the **Keep awake** toggle (rig must be online). The host persists the preference, so it survives agent and machine restarts. ## When it is off The host keeps Windows ready while it is actively in use, then releases its system sleep block after the idle window (3 hours by default). Windows follows the machine's normal power settings after that and may sleep. Once Windows sleeps, Glasswarp cannot start a session until someone or something outside Glasswarp wakes the machine. The host environment variable `GLASSWARP_KEEP_ALIVE_IDLE_TIMEOUT` controls the idle window in seconds. Set `GLASSWARP_KEEP_ALIVE=0` to disable the keep-alive mechanism entirely. ## Recommended setup for an agent rig 1. Leave the Windows machine powered on. 2. Keep the Glasswarp host agent running and signed in. 3. Enable **Keep rig awake**. 4. Let Windows turn off the monitor normally when idle. This keeps the rig reachable without wasting power on an illuminated display. See also [Rigs and sessions](/concepts/rigs-and-sessions). # Launch apps Source: https://docs.glasswarp.com/guides/launch-apps Spawn and drive Windows applications from your session. Launch a Windows application on the rig and drive it with vision + input. ## Launch ```python theme={null} result = gw.launch_app(sid, path=r"C:\Windows\System32\notepad.exe") print(result) # {'ok': True, 'pid': 14180} ``` Pass arguments when needed: ```python theme={null} gw.launch_app(sid, path=r"C:\Windows\System32\mspaint.exe", args=[]) ``` ## Drive it After launching, use [observe](/guides/observe) to find targets and [input](/guides/input) to act: ```python theme={null} gw.launch_app(sid, path=r"C:\Windows\System32\notepad.exe") obs = gw.observe(sid, mark=True) edit = next((t for t in obs.targets if t.role == "edit"), None) if edit: gw.click_target(sid, edit.id, targets=obs.targets) gw.type_text(sid, "Driven through the Glasswarp API") ``` ## Clean up launched apps ```python theme={null} gw.kill_app(sid, pid=result["pid"]) # or kill all session-launched apps ``` Session-launched apps are cleaned up by `safety_restore` on session end, but kill long-running apps explicitly when your task is done. See the [Notepad example](/examples/notepad) and [Paint example](/examples/paint) for complete scripts. # Live View Source: https://docs.glasswarp.com/guides/live-view Watch any session at 60fps and keep humans in the loop. Live View is the human window into an API session — a real-time 60fps WebRTC feed in the Console. It's how owners supervise agents and how you debug loops. ## Open it Go to **Console → Sessions**, find your session by id, and click the **Eye** icon. You'll see the live feed plus telemetry (ping, delay, signal). ```python theme={null} session = gw.create_session(rig_id=rig.id, mode="desktop") print("Watch:", session.session_id) # Console → Sessions → Eye on that id ``` ## Programmatic stream To render the feed in your own surface, get WebRTC signaling credentials: ```python theme={null} creds = gw.stream(sid) # WebRTC signaling credentials for a live feed ``` ## The on-host indicator While the session is active, the host displays an **"API session active"** indicator on the physical machine. Live View and this indicator together mean an agent's actions are always observable — see [Safety and consent](/concepts/safety-and-consent). ## Color / HDR Live View is an **SDR** H.264 stream. If Windows **HDR / Advanced Color** is on, the DXGI capture path can look washed or dull even when screenshots look fine. | What happens | Detail | | ------------ | ------------------------------------------------------------ | | Auto-disable | API session start turns off primary-display HDR | | Restore | Session end, disconnect, and crash recovery restore it | | Console | Amber **HDR on** badge under **Rigs** if HDR is still active | | Opt-out | `CAPTURE_ALLOW_HDR=1` on the host (dev only) | If colors still look wrong after a session starts, turn off **Settings → System → Display → HDR** on the rig manually. See [Troubleshooting](/troubleshooting#live-view). Keep Live View open while developing an agent loop. Watching the real machine react in real time is the fastest way to catch mis-grounded clicks. For a full human-in-the-loop / allowlist / budget checklist, see [Safe agent sessions](/guides/safe-agent-sessions). # Use with n8n Source: https://docs.glasswarp.com/guides/n8n When an n8n workflow hits a Windows app with no connector, Glasswarp is that step — HTTP Request, not a community node. n8n stays the brain and the clock. Glasswarp is eyes and hands on a Windows PC you or your client already own. There is **no official Glasswarp n8n node** today. You call the same Platform API with **HTTP Request** nodes. This is the hop when the canvas reaches a desktop app that has no API — not a replacement for n8n, UiPath Orchestrator, or a job runner we host. Prerequisite: the PC owner [installs the host](/get-started/install-host) (no Glasswarp account on that machine). You [pair the tray code](/get-started/pair-rig) while logged in and turn on **API access**. Create an API key in [Console → API Keys](https://www.glasswarp.com/console/keys). ## What you get | n8n keeps | Glasswarp provides | | -------------------------------------------------- | ----------------------------------------------- | | Trigger, schedule, retries, the rest of the canvas | Session on a real Windows PC | | Your logic (IF, Code, LLM nodes) | Observe (text + click targets) and native input | | Credentials store | Kill switch, on-screen indicator, Live View | Sessions meter **wall-clock minutes** from start to end. Always `DELETE` the session — the importable workflow does. ## Import the smoke workflow The first workflow **does not click**. It lists a usable PC, starts a desktop session, observes **text and targets only** (`image=0` — no JPEG, so the HTTP node does not choke on a 4K frame), then ends the session. 1. Download [`n8n-observe-session.json`](https://www.glasswarp.com/n8n-observe-session.json). 2. In n8n: **⋯ → Import from File** (or Import from URL with that JSON link). 3. Create a **Header Auth** credential named **Glasswarp API**: * Name: `Authorization` * Value: `Bearer gw_live_sk_...` (the word `Bearer`, a space, then your key) 4. Attach that credential on every **HTTP Request** node (Import will prompt). 5. Confirm the host is **online** and API access is **on**. 6. **Test workflow**. Open **[Console → Sessions](https://www.glasswarp.com/console/sessions)** and click **Eye** while it runs — Live View is how you watch; this page does not add a “see the PC” button in Console. The last node must succeed. If observe fails, the template still attempts **End session** (`onError: continue` on observe). If **Start session** never ran, there is nothing to end. ### If import complains about the credential id Delete the dummy `GLASSWARP_HEADER_AUTH` credential reference and pick your Header Auth credential on each HTTP Request node. The name **Glasswarp API** is only a label. ## HTTP map (same API) Base URL: `https://signal.glasswarp.com`. Header: `Authorization: Bearer `. | Step | Method | Path | | -------- | -------- | ------------------------------------------------------ | | List PCs | `GET` | `/v1/rigs` | | Start | `POST` | `/v1/sessions` body `{ "rig_id", "mode": "desktop" }` | | Observe | `GET` | `/v1/sessions/{id}/observe?image=0` | | Input | `POST` | `/v1/sessions/{id}/input` body `{ "events": [ ... ] }` | | End | `DELETE` | `/v1/sessions/{id}` | Usable PC: `online === true` **and** `api_access_enabled === true`. Full contract: [API reference](/api-reference/overview). Default observe includes a JPEG (`jpeg_base64`). For n8n HTTP Request, start with `image=0` (text + targets). Request an image only when an LLM node must see pixels — then cap with `max_width=960&quality=60`. ## After the smoke: click a named control Hardcoding “click Export” is as brittle as a selector farm. For a **stable** dialog you have watched in Live View, resolve a target by `name` from `observe`, then `POST /input` with a `mouse_click` at the target center (`x + w/2`, `y + h/2`, native coordinates). For messy screens, put an **LLM node** on the observe `text.summary` / `targets` and let n8n decide — Glasswarp still does not own the brain. Always `DELETE` the session in a final node, including on the error path. ## Cloud vs self-hosted n8n Both work. The Windows host is the customer's PC; n8n only needs outbound HTTPS to `signal.glasswarp.com`. Do not put the Glasswarp host installer on the n8n server unless that server **is** the Windows PC you mean to drive. ## Related * [Pair a PC](/get-started/pair-rig) * [Ways to run agents](/guides/ways-to-run-agents) * [Observe](/guides/observe) * [Live View](/guides/live-view) * [Connect via MCP](/get-started/mcp) — Cursor / Claude, not n8n # Observe the screen Source: https://docs.glasswarp.com/guides/observe Capture frames, regions of interest, text summaries, and only what changed. ## One-shot screenshot ```python theme={null} frame = gw.screenshot(sid) with open("screen.jpg", "wb") as f: f.write(frame.jpeg) ``` Downscale for cheaper model calls, or crop to a region of interest: ```python theme={null} small = gw.screenshot(sid, max_width=1280, quality=80) roi = gw.screenshot(sid, x=100, y=200, w=800, h=600) ``` ## observe — frame + change + targets + text ```python theme={null} obs = gw.observe(sid, max_width=1280, mark=True) print(obs.text.summary if obs.text else "") print(obs.changed) # False only when DXGI dirty available + empty print(obs.capture_mode) # "dxgi" (preferred) or "gdi_fallback" print(obs.dirty) # None when dirty unavailable — assume changed print(len(obs.jpeg), "bytes") print(len(obs.targets), "targets") ``` `obs.text` is a compact structured summary — focused window title/role plus readable target lines like `[3] Filename (edit, focused)` — so your model can often decide without spending image tokens. Password and masked fields are redacted in that text (`masked`, value always `[redacted]`). See [Safety and consent](/concepts/safety-and-consent). Pass your own targets to get a Set-of-Mark annotated frame back: ```python theme={null} from glasswarp import targets_from_grid targets = targets_from_grid(x0=100, y0=200, x1=900, y1=800, rows=8, cols=8) obs = gw.observe(sid, mark=True, targets=targets) ``` ## Text-only verification (`image=False`) For simple checks — did the dialog close? is the field focused? — skip the JPEG: ```python theme={null} obs = gw.observe(sid, image=False) print(obs.text.summary if obs.text else "") print(obs.changed) # obs.jpeg is empty ``` This is much faster than a full frame. Request the image when you need to read or judge the screen visually. ## Cheap verification frames When you do need a JPEG for a quick verify, downscale harder: ```python theme={null} obs = gw.observe(sid, max_width=960, quality=60, mark=True) ``` ## Only look when something changed ```python theme={null} rects = gw.dirty_rects(sid) if rects: # Encode only the changed region when it is small: x, y, w, h = rects[0] frame = gw.screenshot(sid, x=x, y=y, w=w, h=h, max_width=960, quality=60) else: ... # nothing changed; skip the expensive call ``` Or use `observe` and check `obs.changed` / empty `obs.dirty["rects"]`. Pair dirty rects with ROI observe (`x,y,w,h`): encode only the changed region and skip full-frame model calls. ## Cadence You do not need a fresh observe between every click. Verify after each **meaningful step** — after a short batch of predictable actions, or after a single action when the next screen is uncertain. Prefer `image=False` for verification when text/targets are enough. Target ids from `observe` / `list_targets` are valid for that response only. Do not persist them across sessions, and re-observe before clicking when the screen may have changed (lists, file browsers, tables). See [Vision](/concepts/vision) for the concepts and [Agent loop](/guides/agent-loop) to put it together. # Safe agent sessions Source: https://docs.glasswarp.com/guides/safe-agent-sessions Simple checklist for running agents on real Windows PCs. Glasswarp gives an agent **eyes and hands** on a Windows PC you own. You supply the **brain**. Treat full desktop control as powerful — start small, watch it, then open the throttle. Glasswarp handles **session controls** (consent, on-screen indicator, kill switch, audit, Live View). Your agent code handles **what it’s allowed to do**. See also: [Safety and consent](/concepts/safety-and-consent) · [Live View](/guides/live-view) ## Checklist ### On the Glasswarp side * [ ] Use a **dedicated rig** when you can — not your everyday PC * [ ] Turn **API access** on only when you need it (Console → Rigs) * [ ] Scope API keys to **one rig** when possible * [ ] Keep **Live View** open on first runs; kill from Console if needed * [ ] Always **`end_session`** when finished ### In your agent code * [ ] **Ask a human** before delete, send, purchase, install, or security changes * [ ] **Allowlist** apps the agent may launch * [ ] Set a **max steps / max minutes** budget * [ ] Don’t put **passwords or secrets** in prompts * [ ] **Log** what the agent is about to do * [ ] Never aim the agent at the **Glasswarp tray** (unpair / end session) or Windows security UI ### Human-only on the host (platform-enforced) These are **not** agent API/MCP tools. The host also blocks remote input while the tray confirmation dialogs are open, so an agent with hands cannot click **Yes** for you. | Action | Where | Notes | | --------------------- | -------------------------------------------------------------- | ---------------------------------- | | **End API session** | Tray → **End API session…**, or Console → Sessions | Owner kill switch | | **Unpair this PC** | Tray (only when **no** API session) or Console → Rigs → Remove | End the session first, then unpair | | **API access on/off** | Console → Rigs | Consent gate | Agents **should** call `end_session` when their own work finishes — that is cleanup, not the owner kill switch. ## Simple split | Glasswarp | Your agent | | ---------------------------------------------------------- | --------------------------------------- | | Consent, indicator, kill, audit, Live View | Approvals, allowlists, budgets, prompts | | Owner tray/console controls (input blocked during confirm) | Do not automate Glasswarp chrome | You need both. ## Patterns **Watch first.** Open [Live View](/guides/live-view) (or the [`live_view_hitl`](/examples/templates) template) before full autonomy. **Batch only when intermediate screens are predictable.** Actions inside a batch execute without verification between them (MCP `send_actions` is capped at 10; an invalid action is rejected during validation **before anything is sent**, and the response names the failing action index — zero events reach the machine). On a host running agent **≥ 0.2.21** (with `api_input_ack`), the owner kill switch and session end abort an **in-flight** batch mid-sequence: the input call returns **`410 aborted_by_owner`** with `executed` / `aborted` / `kill_to_abort_ms` counts, and only the `executed` events landed. Older hosts are fire-and-forget (the session ends but the current batch may finish) — upgrade the host for guaranteed mid-batch abort. **Confirm risky acts.** ```python theme={null} def maybe_act(gw, sid, action): if action.get("risk") in ("delete", "send", "purchase", "install"): if not human_approved(action): return apply_action(gw, sid, action) ``` **Budget + always clean up.** ```python theme={null} MAX_STEPS = 40 session = gw.create_session(rig_id=rig.id, mode="desktop") sid = session.session_id try: for step in range(MAX_STEPS): obs = gw.observe(sid, max_width=1280, mark=True) action = decide(obs) if action is None or action.get("type") == "done": break maybe_act(gw, sid, action) finally: gw.end_session(sid) ``` ## Console | Action | Where | | ----------------- | ------------------ | | API access on/off | Console → Rigs | | Keys | Console → API Keys | | Watch / kill | Console → Sessions | ## In our examples Helpers live in `sdk/python/examples/safe_session.py` (budgets, intent logs, allowlist, optional `REQUIRE_CONFIRM=1`). Templates and demos use them. The longer demos (Paint / Minesweeper) run unattended by default — set `REQUIRE_CONFIRM=1` if you want prompts. ```bash theme={null} export REQUIRE_CONFIRM=1 export MAX_STEPS=40 export ALLOWED_APPS=notepad.exe,mspaint.exe,chrome.exe ``` ## Related * [Safety and consent](/concepts/safety-and-consent) * [Build an agent loop](/guides/agent-loop) * [Templates](/examples/templates) # Ways to run agents Source: https://docs.glasswarp.com/guides/ways-to-run-agents Three ways to drive a rig: n8n HTTP Request, chat-driven MCP, or a scaffolded SDK agent — plus pre-built demos. There are three ways to drive a rig. Pick the one that matches where the brain already lives. All three need the same prerequisite: an online rig with API access enabled by the owner — see [Pair a rig](/get-started/pair-rig). ## n8n (HTTP Request) Your canvas stays the scheduler and the brain. Glasswarp is one hop: when the flow hits a Windows app with no connector, start a session on that PC, observe or act, end the session, continue in n8n. There is **no official Glasswarp community node**. Use HTTP Request against the Platform API. Import the smoke workflow (text+targets observe, no click) from [Use with n8n](/guides/n8n). ## Chat-driven (MCP) Your assistant drives the PC step by step through MCP tools: it captures the screen, decides, clicks or types, and captures again. This is great for quick tasks and supervision — "what's on my screen?", "open Notepad and type hello", approving a dialog while you watch. The tradeoff: each *step* costs a model turn. Predictable sequences — menus, dialogs, typing into a field you just clicked — can be batched into one call (`send_actions`), so you are not paying a turn per keystroke. MCP defaults to text+targets observes (set `image=true` / `observe_image=true` only when you need vision). Adaptive loops where every move depends on new information — clearing a board, anything that repeats — still add up, and that is what the SDK path is for. Setup: [Connect via MCP](/get-started/mcp). ## Scaffolded SDK agents For multi-step work, a small Python script loops `observe → decide → act` at full speed. The decision logic lives inside the script — heuristics, computer vision, or an LLM call — so there's no chat round-trip per step. This is the right tool for games, workflows, and anything that repeats. If you're connected through a code-capable assistant, it will typically offer to write this script for you; you can also start from the [SDK quickstart](/quickstart) or the [agent loop guide](/guides/agent-loop). ```bash theme={null} pip install glasswarp export GLASSWARP_API_KEY=gw_live_sk_... python my_agent.py ``` ## Pre-built demos Some pre-built demo agents ship with the SDK — install and run, no code to write: ```bash theme={null} pip install "glasswarp[demos]" export GLASSWARP_API_KEY=gw_live_sk_... glasswarp-demo list glasswarp-demo minesweeper ``` | id | Command | | ------------- | ---------------------------- | | `minesweeper` | `glasswarp-demo minesweeper` | | `mona-lisa` | `glasswarp-demo mona-lisa` | | `paint` | `glasswarp-demo paint` | | `notepad` | `glasswarp-demo notepad` | MCP clients can browse the same catalog with the `list_demos` / `get_demo` tools, which return the install command and the exact command to run. ## Billing All paths use the same metered sessions: wall-clock minutes from session start to end, and idle sessions auto-end after about 15 minutes. See [pricing](https://www.glasswarp.com/pricing). ## Related * [Use with n8n](/guides/n8n) * [Connect via MCP](/get-started/mcp) * [Build an agent loop](/guides/agent-loop) * [Safe agent sessions](/guides/safe-agent-sessions) * [Python SDK](/quickstart) # Session webhooks Source: https://docs.glasswarp.com/guides/webhooks HTTPS callbacks when API sessions start and end — for ops alerts and agent orchestration. Register an HTTPS endpoint and Glasswarp will POST when a session starts or ends. Use this for **ops alerts and agent orchestration** — not for driving the PC (that's the API / MCP / SDK). Deliveries are **best-effort** and may be missed. For billing reconciliation, treat webhooks as a hint and reconcile against the [sessions API](/api-reference/overview) (source of truth). Retries are not implemented yet. ## Setup Webhook **management** (create / list / delete) uses your **console login JWT** — same auth as API Keys in the developer console. Platform API keys (`gw_*`) authenticate session/control calls; they **cannot** manage webhooks. 1. Open **Console → Webhooks**, or call `POST /v1/webhooks` with `Authorization: Bearer ` (the same JWT the console uses after you sign in). 2. Enter a public `https://` URL (private / loopback / link-local / cloud-metadata addresses are rejected; redirects are not followed). 3. Copy the **signing secret** shown once on create (`whsec_…`). It is stored server-side only and is **never** returned again on list/read — treat it like an API key. 4. Start a session — your endpoint should receive `session.started`, then `session.ended` when it closes (including idle timeout / owner kill). Max **5** webhooks per account. Events default to both lifecycle events. ## Payload ```json theme={null} { "type": "session.ended", "id": "delivery-id", "created_at": "2026-07-22T20:00:00.000Z", "data": { "session_id": "…", "rig_id": "…", "mode": "desktop", "status": "ended", "ended_reason": "client", "billed_minutes": 3 } } ``` ### `ended_reason` (on `session.ended`) | Value | Meaning | | -------------- | ----------------------------------------------------------- | | `client` | Caller ended the session (`DELETE` / SDK `end_session`) | | `owner_kill` | Rig owner killed the session (console or host tray) | | `idle_timeout` | No API activity for the session idle window (\~15m default) | | `max_age` | Session hit the maximum age cap | | `timeout` | Server-side timeout / orphan cleanup | | `host_gone` | Host disconnected while the session was active | | `unpaired` | Rig was unpaired mid-session | Idle timeout is still a `session.ended` event — there is no separate `session.reaped` event. Switch on `ended_reason=idle_timeout`. ## Headers | Header | Meaning | | ----------------------- | ---------------------------------------------------------- | | `Content-Type` | `application/json` | | `X-Glasswarp-Event` | `session.started` or `session.ended` | | `X-Glasswarp-Delivery` | Unique delivery id (dedupe) | | `X-Glasswarp-Timestamp` | Unix seconds when the delivery was signed | | `X-Glasswarp-Signature` | `sha256=` — HMAC-SHA256 of `"{timestamp}.{raw_body}"` | ## Verify the signature Reject deliveries whose timestamp is more than **5 minutes** from your clock (replay protection). ```python theme={null} import hmac, hashlib, time def verify(secret: str, body: bytes, timestamp: str, signature: str, tolerance=300) -> bool: ts = int(timestamp) if abs(int(time.time()) - ts) > tolerance: return False got = signature.removeprefix("sha256=") signed = f"{ts}.".encode() + body expect = hmac.new(secret.encode(), signed, hashlib.sha256).hexdigest() return hmac.compare_digest(got, expect) ``` ## Delivery behavior * \~5s timeout; response body is capped — a slow or huge reply cannot tie up the worker. * Failed deliveries update **last status** in the console. * Retries are not implemented yet — poll sessions if you need a durable record. ## Related * [Rigs and sessions](/concepts/rigs-and-sessions) * [Safe agent sessions](/guides/safe-agent-sessions) * [API reference](/api-reference/overview) # Glasswarp Source: https://docs.glasswarp.com/index The transport layer between your agent and Windows computers — eyes, hands, and a safe session. You bring the brain. For automators already doing no-API desktop work. # Give your agent eyes and hands on Windows Glasswarp is the **transport layer between your agent and Windows computers**. It gives any model real-time vision and native input on Windows machines you or your clients already own — including no-API EHR, ERP, insurance AMS, and other licensed desktops. **You bring the brain** — the model, prompts, and task logic. Built for automation developers and small shops already doing that work; vertical AI teams building on the client's PC are in scope. Pair a Windows PC once. Your agent sees the screen and controls keyboard and mouse through one API — on the real desktop and apps your users run. No rented cloud PC required. ## Start here Connect the SDK, start a session, capture a frame, and send input in under five minutes. When the canvas hits a Windows app with no connector — HTTP Request, not a Glasswarp node. Point Claude, Cursor, or any MCP client at Glasswarp — eyes and hands, no project code. Paste one prompt into your coding agent and let it wire Glasswarp into your project. Put the host agent on a Windows machine, pair it, and enable API access. Every `/v1/*` endpoint with an interactive request builder. ## The loop Every integration is the same shape: **see the screen, decide, act, repeat.** Glasswarp owns the see-and-act transport; your code owns the decide. ```mermaid theme={null} flowchart LR Model[Your model] --> GW[Glasswarp] GW --> Win[Windows PC] Win --> GW GW --> Model ``` ## How Glasswarp differs Typical computer-use sandboxes approximate a desktop (often Linux or a soft framebuffer). Glasswarp captures a **real Windows** machine with native input — your apps, your files, your GPU. Agent `observe` / screenshots use **DXGI Desktop Duplication by default (GDI fallback)** and are encoded as CPU JPEG; the GPU NVENC pipeline is used for **Live View** (the human watch stream), not the agent capture path. Details live in [Eyes and hands](/concepts/eyes-and-hands) and the FAQ on the marketing site. ## Next * [Quickstart](/quickstart) — first session in minutes * [Use with n8n](/guides/n8n) — HTTP Request hop from an existing canvas * [Connect via MCP](/get-started/mcp) — Claude / Cursor without writing code * [Concepts](/concepts/eyes-and-hands) — eyes, hands, and the session * [Examples](/examples/notepad) — agents on real Windows through the API # Quickstart Source: https://docs.glasswarp.com/quickstart Go from zero to a live Windows session in minutes. By the end of this guide, your code will start a session on a real Windows machine, capture a native-resolution frame, and send native input — all through the Glasswarp API. Using n8n? Start at [Use with n8n](/guides/n8n) (HTTP Request — no community node). Using Claude, Cursor, or another MCP client? See [Connect via MCP](/get-started/mcp). ## Prerequisites The PC owner installs from [Downloads](https://www.glasswarp.com/downloads) (no account). You pair the tray code and turn on **API access**. See [Install the host](/get-started/install-host). Create one in the [Console](https://www.glasswarp.com/console/keys). Keys look like `gw_live_sk_...`. The SDK is Python. REST works from any language — see the [API reference](/api-reference/overview). ## 1. Install the SDK ```bash pip theme={null} python -m pip install glasswarp ``` ```bash uv theme={null} uv add glasswarp ``` ```bash poetry theme={null} poetry add glasswarp ``` Add the grounding extra (`glasswarp[grounding]`) only when you need the Set-of-Mark and ROI image helpers, which pull in Pillow. ## 2. Set your API key ```bash theme={null} export GLASSWARP_API_KEY=gw_live_sk_... ``` The client reads `GLASSWARP_API_KEY` automatically. Never hardcode or commit it. ## 3. Run your first session ```python theme={null} import os from glasswarp import GlasswarpClient gw = GlasswarpClient(api_key=os.environ["GLASSWARP_API_KEY"]) session = gw.create_session(rig_id="office-pc", mode="desktop") sid = session.session_id try: obs = gw.observe(sid) save = next(t for t in obs.targets if t.name == "Save") gw.click_target(sid, save) finally: gw.end_session(sid) ``` `office-pc` is a stand-in. Use `gw.list_rigs()` to find an online PC with API access enabled — see [Pair a PC](/get-started/pair-rig). Prefer `click_target` over raw `x,y` (that path is PyAutoGUI-shaped and brittle). ## 4. Watch it live Open **Console → Sessions → Eye** to watch the session at 60fps while your code runs. You'll see the on-screen "API session active" indicator on the host — that is the owner-consent guardrail, always on. ## Next steps Click UIA targets instead of guessing pixels. observe → model → act, efficiently. Spawn and drive Windows applications. Every endpoint, with a request builder. # Setup for Agents Source: https://docs.glasswarp.com/setup-for-agents Paste one prompt into your coding agent and let it wire Glasswarp into your project. Glasswarp is built to be set up **by an agent, for an agent**. Paste the prompt below into your coding agent. It reads the public [`SKILL.md`](https://www.glasswarp.com/SKILL.md), inspects your project, installs the SDK, and wires up the eyes-and-hands loop — pausing for the human Windows-host step. Want eyes and hands inside Claude or Cursor with **no project code**? Use the remote MCP server instead — [Connect via MCP](/get-started/mcp). ## The prompt Copy this into your agent: ```text Setup for Agents theme={null} Set up Glasswarp in this project. Read https://www.glasswarp.com/SKILL.md and follow its setup instructions. Goal: give this project's agent eyes and hands on a real Windows machine through the Glasswarp API. Glasswarp supplies eyes and hands; this project supplies the model and task logic. Step 1 — Inspect and ask: - Inspect this codebase's language, framework, runtime, package manager, dependency files, entry points, tests, agent code, and secrets conventions. - Ask whether I want Glasswarp integrated here or a minimal Python quickstart. - Python 3.9+ is the supported SDK. For a non-Python project, explain the choice between a small Python worker and direct REST; do not silently add a runtime. Step 2 — Dependencies and key: - Install glasswarp from PyPI (https://pypi.org/project/glasswarp/) using the existing package manager (pip, uv, Poetry, or Pipenv) and update its dependency file. - Use the optional grounding extra only for image/ROI helpers. - Ask me to create an API key at https://www.glasswarp.com/console -> API Keys. Store it as GLASSWARP_API_KEY using the existing secrets convention. Never print, commit, or hardcode it. Step 3 — Human Windows setup: - A real Windows host is required. Direct the PC owner to https://www.glasswarp.com/downloads (no Glasswarp account on that machine). - Direct me (the automator) to https://www.glasswarp.com/pair to paste the tray code while logged in, then enable API access on that PC. - Pause and ask me to confirm an online, API-enabled PC exists. This is a normal owner-consent step; do not bypass it or request machine passwords. Step 4 — Integrate: - Match this project's conventions. - Implement list_rigs -> create_session -> observe -> click_target -> end_session. - Always end metered sessions in a finally block. - Keep prompts and task decisions in customer code, separate from the Glasswarp transport adapter. Step 5 — Verify: - Run the integration in the project's real environment. - Confirm an eligible rig, a successful session, a non-empty JPEG, a session visible in Console -> Sessions / Live View, and cleanup after failure. - Run the relevant formatter, type checker, and tests. - Diagnose and fix failures, then report dependencies installed, files changed, commands run, and checks passed. ``` ## What the agent will do It fetches [`SKILL.md`](https://www.glasswarp.com/SKILL.md) and detects your language, package manager, and conventions. It adds `glasswarp` with your package manager and stores `GLASSWARP_API_KEY` via your existing secrets convention. It stops and asks you to install the [host agent](/get-started/install-host), pair the rig, and enable API access — a physical, owner-consent step it can't do for you. It writes the integration in your style, runs it, and confirms a live session in the Console. The one step no agent can skip: a human installs the host agent on a real Windows machine and enables API access. That owner consent is by design — see [Safety and consent](/concepts/safety-and-consent) and [Safe agent sessions](/guides/safe-agent-sessions). Prefer to do it yourself? Follow the [Quickstart](/quickstart) instead. # Troubleshooting Source: https://docs.glasswarp.com/troubleshooting Common failures and how to fix them fast. ## Install Upgrade pip and retry: `python -m pip install -U pip glasswarp`. Package page: [pypi.org/project/glasswarp](https://pypi.org/project/glasswarp/). Inside the monorepo you can also use `python -m pip install -e sdk/python`. The grounding helpers need Pillow. Install the extra: `pip install "glasswarp[grounding]"`. ## Auth `GLASSWARP_API_KEY` is missing, expired, or invalid. Create a fresh key in [Console → API Keys](https://www.glasswarp.com/console/keys) and re-export it. Never hardcode or commit keys. ## MCP (Claude / Cursor) See [Connect via MCP](/get-started/mcp) for full setup. Use an absolute `npx` path for your OS with the `mcp-remote` stdio bridge (macOS Apple Silicon `/opt/homebrew/bin/npx`, Intel `/usr/local/bin/npx`, Windows Node.js `npx` / `npx.cmd`, Linux typically `/usr/bin/npx`). Bare `npx` often fails inside Cursor.app. Details: [Connect via MCP](/get-started/mcp). Usually the MCP server could not reach the Platform API, or the client sent a placeholder key. Confirm the Platform API health endpoint and that `GLASSWARP_AUTH` is a real `gw_*` key. If `signal.glasswarp.com` (or your configured API base URL) fails to resolve on your machine, flush local DNS or try a public resolver (`1.1.1.1` / `8.8.8.8`). The MCP server reaches the API from its own network — local DNS only affects tools on your machine. Host offline or API access off. Start the Windows agent; enable API access per-rig in the console. ## Rigs and sessions The host is offline, or API access is off. Open [Console → Rigs](https://www.glasswarp.com/console): confirm the rig is **online** and toggle **API access** on. This is the per-rig consent gate. Verify the host stayed online (check the Console), then retry once. If it recurs, restart the host agent on the Windows machine. Always `end_session` in a `finally` block. End stray sessions from **Console → Sessions**. `safety_restore` also runs on disconnect and `END_SESSION`. ## Live View Windows **HDR / Advanced Color** was on. DXGI capture into the SDR WebRTC stream washes colors; GDI screenshots can still look correct. Glasswarp **auto-disables HDR for the API session** and restores it when the session ends. If colors are still wrong: 1. Check **Console → Rigs** for an amber **HDR on** badge. 2. On the Windows machine: **Settings → System → Display → HDR → Off**. 3. Rebuild/restart the host agent so auto-disable is present. Dev opt-out only: `CAPTURE_ALLOW_HDR=1`. See [Live View](/guides/live-view). ## Input and vision Update the Windows host agent — drag primitives require a host build with `mouse_down` / `mouse_up`. Current public release: **v0.2.21** ([Downloads](https://www.glasswarp.com/downloads)). Host **v0.2.14+** merges popup HWNDs (menus, combo dropdowns) into the UIA snapshot. Confirm Console → Rigs shows `v0.2.14` or newer (auto-updates when idle), then reopen the menu and call `observe` again. Stop guessing pixels. Use [grounding](/concepts/grounding): prefer `list_targets` / `click_target`, or Set-of-Mark via `observe(mark=True)`. Use `observe()` (frame + dirty + targets) instead of a bare screenshot, and send a Set-of-Mark annotated frame so the model picks a target by id. ## Still stuck? Check the [API reference](/api-reference/overview) for exact request/response shapes, or watch the session live via [Live View](/guides/live-view) to see what the host is actually doing.