TargetFinder: Detecting Widgets from Pixels on Desktop Interfaces
3.60T1 sourcearXiv cs.HC
Source record
Published by arXiv cs.HC (T1 source). The original is at https://arxiv.org/abs/2607.19907.
Pipeline notes
The summary and note below are generated by the signal pipeline — they are Beyond Desk’s reading, not quotations from the source.
SummaryTargetFinder is a computer vision system using fine-tuned YOLO models for real-time detection of GUI widgets across desktop platforms. Trained on 520 annotated screenshots (Windows, macOS, Ubuntu, web), it outperforms OmniParser and REMAUI baselines and enables system-wide deployment of target-aware pointing techniques like Bubble Cursor and Semantic Pointing. Dataset, models, and library are open-sourced.
Why it mattersReleases a cross-platform widget detection pipeline with dataset and models — a reusable building block for both accessibility pointing aids and agents that need to perceive arbitrary desktop GUIs.
Cited by
No citations on record.
