Vision

Glasswarp’s eyes are designed for fast agent loops: fresh, native-resolution frames with the option to fetch only what changed.

Screenshot

A high-fidelity, native-resolution JPEG from the host capture buffer — typically tens of milliseconds for capture + CPU JPEG encode — and it works with or without an active WebRTC Live View stream. Agent screenshots use the API capture path (DXGI Desktop Duplication by default, GDI BitBlt fallback); they are not re-encoded remote WebRTC frames and are not NVENC/GPU JPEG.

frame = gw.screenshot(sid)
with open("screen.jpg", "wb") as f:
    f.write(frame.jpeg)

# Optional: downscale and/or crop to a region of interest.
roi = gw.screenshot(sid, max_width=1280, x=100, y=200, w=800, h=600, quality=80)

Observe — one round trip

observe returns everything an agent step needs in a single RTT: a JPEG, dirty regions when available, grounding targets (Set-of-Mark annotated when mark=True), changed, and capture_mode (dxgi | gdi_fallback | unknown; legacy hosts may report gdi).

obs = gw.observe(sid, max_width=1280, mark=True)
# obs.jpeg          — frame (SoM-marked when targets present)
# obs.dirty         — DXGI dirty regions, or None when unavailable
# obs.changed       — False only when DXGI dirty was available and empty
# obs.capture_mode  — "dxgi" (preferred) or "gdi_fallback" (fallback)
# obs.targets       — click targets from host UIA (may be empty)

Use observe as the default “look” in your loop — it’s the leanest way to get frame + change + grounding together.

Dirty rects

When capture is DXGI, poll only the regions that changed since your last call:

dirty = gw.dirty_rects(sid)
if dirty.get("available") is False or dirty is None:
    pass  # assume changed — GDI fallback has no dirty metadata
elif not dirty.get("rects"):
    pass  # nothing changed — skip the model call

Skip a model call only when dirty is available and empty. If dirty is missing (null / available: false), assume changed. When dirty is available, crop screenshots to the changed region for cheaper loops — see Build an agent loop.

Capture is the primary display only today (first DXGI output / primary GDI screen). Multi-monitor selection is not yet exposed.

Continue to Grounding.

Get started

Setup

Concepts

Guides

Examples

Help

Screenshot

Observe — one round trip

Dirty rects

​Screenshot

​Observe — one round trip

​Dirty rects

Screenshot

Observe — one round trip

Dirty rects