The shape
decide and act are yours: decide calls your model with obs.jpeg and
obs.targets; act maps its output to click_target, type_text, etc.
Make it cheap: skip idle frames
Ground every action
Preferclick_target over raw pixels so the loop survives layout shifts:
Keep task logic out of the transport
Put prompts, CV, and planning in your own module. The Glasswarp client is a
thin transport adapter — mixing task logic into it makes both harder to test.
Reference loops
The SDK ships runnable agent loops:gemini_agent_loop.py— CV grid → SoM targets → Gemini →click_targetclaude_agent_loop.py— observe → Claude → act (VP0 on by default)
