Hybrid Computer Use: A Unified Programmatic, Semantic, and Visual Interface for General Agents
4.40T1 sourcearXiv cs.HC
Source record
Published by arXiv cs.HC (T1 source). The original is at https://arxiv.org/abs/2610.41432.
Pipeline notes
The summary and note below are generated by the signal pipeline — they are Beyond Desk’s reading, not quotations from the source.
SummaryHybrid Computer Use gives agents one interface spanning code and shell commands, accessibility APIs, and GUI controls. It adds batched accessibility actions, compact observations, and diff-based effect reports, coordinates channel switching, and includes a skill for choosing channels. On OSWorld 2.0, evaluated agents improved efficiency and task success over comparison systems.
Why it mattersThe paper offers a concrete interface design and benchmark evidence for reducing screen interpretation and avoiding stale state when agents move between software channels.
Cited by
No citations on record.