> ## Documentation Index
> Fetch the complete documentation index at: https://docs.glasswarp.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Grounding: observe + Set-of-Mark

> Turn a frame into numbered targets a vision model can click by id.

This example shows the grounding loop end to end: build targets, get a
Set-of-Mark annotated frame, let a model pick an id, and click it.

## Build targets and annotate

```python theme={null}
from glasswarp import GlasswarpClient, targets_from_grid

gw = GlasswarpClient(api_key="gw_live_sk_...")
session = gw.create_session(rig_id=rig.id, mode="desktop")
sid = session.session_id

# Grid targets over a board or canvas (or use gw.list_targets(sid) for UIA).
targets = targets_from_grid(x0=100, y0=200, x1=900, y1=800, rows=8, cols=8)

obs = gw.observe(sid, max_width=1280, mark=True, targets=targets)
with open("marked.jpg", "wb") as f:
    f.write(obs.jpeg)   # numbered marks drawn on the frame
```

## Let the model choose, then act

```python theme={null}
# Send obs.jpeg to your model; it returns a mark id, e.g. "12".
target_id = your_model_pick(obs.jpeg)
gw.click_target(sid, target_id, targets=targets)
```

## ROI + dirty rects for tight loops

```python theme={null}
rects = gw.dirty_rects(sid)
if rects:
    r = rects[0]
    crop = gw.screenshot(sid, x=r.x, y=r.y, w=r.w, h=r.h)
    # feed only the changed region to your model
```

<Info>
  Set-of-Mark helpers (`som_annotate`, `targets_from_grid`) need the grounding
  extra: `pip install "glasswarp[grounding]"`.
</Info>

## Reference agent loops

* `gemini_agent_loop.py` — CV grid → SoM targets → Gemini → `click_target`
* `claude_agent_loop.py` — observe → Claude → act (VP0 on by default)

See [Build an agent loop](/guides/agent-loop) for the full pattern.