Vision

Glasswarp’s eyes are designed for fast agent loops: fresh, native-resolution frames with the option to fetch only what changed.

Screenshot

A high-fidelity, native-resolution JPEG — fresh (~50 ms capture or less), and it works with or without an active WebRTC stream.

frame = gw.screenshot(sid)
with open("screen.jpg", "wb") as f:
    f.write(frame.jpeg)

# Optional: downscale and/or crop to a region of interest.
roi = gw.screenshot(sid, max_width=1280, x=100, y=200, w=800, h=600, quality=80)

Observe — one round trip

observe returns everything an agent step needs in a single RTT: a JPEG, the accumulated dirty regions, and grounding targets (Set-of-Mark annotated when mark=True).

obs = gw.observe(sid, max_width=1280, mark=True)
# obs.jpeg    — frame (SoM-marked when targets present)
# obs.dirty   — regions changed since last poll
# obs.targets — click targets from host UIA (may be empty)

Use observe as the default “look” in your loop — it’s the leanest way to get frame + change + grounding together.

Dirty rects (VP0)

For efficient loops, poll only the regions that changed since your last call:

rects = gw.dirty_rects(sid)   # accumulated DXGI dirty rects
if not rects:
    pass  # nothing changed — skip the model call

Skip a model call entirely when dirty_rects is empty, and crop screenshots to the changed region. This is the cheapest way to run a tight agent loop — see Build an agent loop.

Continue to Grounding.

Rigs and sessions Grounding

⌘I

Get started

Setup

Concepts

Guides

Examples

Help

Screenshot

Observe — one round trip

Dirty rects (VP0)

​Screenshot

​Observe — one round trip

​Dirty rects (VP0)

Screenshot

Observe — one round trip

Dirty rects (VP0)