UEyes: Understanding Visual Saliency across User Interface Types

Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI 2023), Hamburg, Germany.

Download Video(MP4, 57.7 MB)


While user interfaces (UIs) display elements such as images and text in a grid-based layout, UI types differ significantly in the number of elements and how they are displayed. For example, webpage designs rely heavily on images and text, whereas desktop UIs tend to feature numerous small images. To examine how such differences affect the way users look at UIs, we collected and analyzed a large eye-tracking-based dataset, UEyes (62 participants and 1,980 UI screenshots), covering four major UI types: webpage, desktop UI, mobile UI, and poster. We analyze its differences in biases related to such factors as color, location, and gaze direction. We also compare state-of-the-art predictive models and propose improvements for better capturing typical tendencies across UI types. Both the dataset and the models are publicly available.

Resources and Downloads


    We regenerated Table 2 in the paper after fixing some calibration error in the dataset. Sorry for the inconvenience.



BibTeX, 1 KB

	author = {Jiang, Yue and Leiva, Luis A. and Rezazadegan Tavakoli, Hamed and R. B. Houssel, Paul and Kylm\"{a}l\"{a}, Julia and Oulasvirta, Antti},
	title = {UEyes: Understanding Visual Saliency across User Interface Types},
	year = {2023},
	isbn = {9781450394215},
	publisher = {Association for Computing Machinery},
	address = {New York, NY, USA},
	url = {https://doi.org/10.1145/3544548.3581096},
	doi = {10.1145/3544548.3581096},
	booktitle = {Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems},
	articleno = {285},
	numpages = {21},
	keywords = {Computer Vision, Eye Tracking, Deep Learning, Interaction Design, Human Perception and Cognition},
	location = {Hamburg, Germany},
	series = {CHI '23}


For questions and clarifications, please get in touch with:
Yue Jiang yuenj.jiang@gmail.com