Papers
arxiv:2610.33645

PreView: Enhancing GUI Grounding with Preprocessed Views and Cues

Published on Oct 24
ยท Submitted by
Wenjun Hou
on Oct 29
Authors:

Abstract

PreView is an auxiliary preprocessor that enhances GUI grounding by providing localized zoomed views and visual-textual target cues to frozen grounding models, improving accuracy without retraining.

GUI grounding requires locating small controls in cluttered screenshots and identifying which control a natural-language instruction refers to. Cropping and enlargement can expose local detail, but the resulting image may still leave ambiguity about the intended target. We propose PreView, an auxiliary preprocessor that improves the inputs to an existing grounding model without further training that model. PreView prepares two complementary types of input features: task-relevant views for localization and candidate-specific cues for target understanding. A trained lightweight auxiliary model predicts and refines a center, crops the screenshot into a local view, and enlarges that view for zoomed-in grounding. It also prepares a visual cue (i.e., a red-circle marker with a location hint) and a textual cue describing the candidate's identity or function. The frozen grounding model uses the prepared inputs to predict the final click. Across five benchmarks, PreView improves the average grounding accuracy of Qwen3-VL-2B, OS-Atlas-4B, and Qwen3-VL-4B without further training these grounding models. Extensive analyses examine the contributions of views and cues, center refinement, and visual input budgets.

Community

Paper author Paper submitter

Hi everyone, author here! ๐Ÿ‘‹

Can we improve GUI grounding by better preparing what the grounding model sees?

We introduce PreView, an auxiliary preprocessor that combines task-relevant cropped and enlarged views with visual and textual cues about candidate targets. The idea is simple: zooming helps reveal small controls, while candidate-specific cues help disambiguate which control the instruction refers to.

A trained lightweight auxiliary model prepares these inputs, while the grounding model stays frozen. We evaluate PreView across five benchmarks and three grounding models.

Happy to discuss the approach and hear your feedback!

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2610.33645
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.33645 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.33645 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.33645 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.