DevicesWorld: Benchmarking Cross-Device Agents in Heterogeneous Environments
3.60T1 sourcearXiv cs.HC
arXiv cs.HCoriginal source ↗
Source record
Published by arXiv cs.HC (T1 source). The original is at https://arxiv.org/abs/2607.13465.
Pipeline notes
The summary and note below are generated by the signal pipeline — they are Beyond Desk’s reading, not quotations from the source.
SummaryDevicesWorld is a benchmark of 6,140 tasks spanning mobile, desktop, and IoT environments for evaluating LLM-based agents on cross-device collaborative operation. Five frontier agent systems tested achieve at most 12.5% success, failing on information acquisition, source/output device confusion, and premature termination.
Why it mattersReproducible cross-device agent benchmark with concrete failure modes; useful for anyone building or evaluating multi-device agentic systems.
Cited by
No citations on record.