Yue Jiang
Email: yue.jiang [at] aalto.fi
CV (Updated in Mar. 2024) |
Linkedin |
Google Scholar
WeChat Blog: yuejiang_phd
I am a Ph.D. student in the Computational Behavioral Lab directed by Prof. Antti Oulasvirta, working on Computational User Interface Understanding and Eye Tracking on User Interfaces (Secondary advisor: Prof. Vikas Garg) at
Aalto University and the Finnish Center for Artificial Intelligence (FCAI), Finland ๐ซ๐ฎ.
I also collaborate closely with Prof. Wolfgang Stuerzlinger (Simon Fraser University, Canada ๐จ๐ฆ) and Prof. Christof Lutteroth (University of Bath, UK ๐ฌ๐ง) on our series of OR-Constraint (ORC) Adaptive GUI layout projects.
Previously, I worked on Human Motion Capture and the Reconstruction of Non-Rigid Deformations at the Max Planck Institute, Germany ๐ฉ๐ช. I got my Master's degree in Computer Graphics under Prof. Matthias Zwicker from the
University of Maryland, College Park, USA ๐บ๐ธ, and my Bachelor's degree in
Computer Science
Specialist with an additional
Mathematics
Major under Prof. Gerald Penn from the
University of Toronto, Canada ๐จ๐ฆ.
I am from Nanjing, a beautiful city in the east of China, which served as the capital of China ten times in the history of China ๐จ๐ณ.
I will be on the job market for tenure-track faculty positions or relevant industry research positions starting in 2025. Feel free to reach out if you are aware of any suitable openings!
|
Photo taken in Vancouver.
|
Highlights
-
Our paper 'Graph4GUI: Graph Neural Networks for Representing Graphical User Interfaces' was conditionally accepted by CHI2024.
-
I am visiting Prof. Jeffrey P. Bigham's BIG Lab at Carnegie Mellon University in 2024.
-
We are organizing our third Computational User Interface workshop, 'Computational Methodologies for Understanding, Automating, and Evaluating User Interfaces' at CHI2024. Welcome to participate!
-
I am serving as the Accessibility Co-Chair and Associate Chair (AC) for Late Breaking Works at CHI2024.
-
I am excited to receive the 2023 Meta Research PhD Fellowship! Yay! :-)
-
I enjoyed my internship with the UI Understanding team under Jeff Nichols at Apple AIML so much! I wish it could be longer!
-
I served as the Accessibility Co-Chair, Workshop Juror, and Associate Chair (AC) for Late Breaking Works at CHI2023.
-
We organized our second Computational User Interface workshop, 'The Future of Computational Approaches for Understanding and Adapting User Interfaces' at CHI2023.
-
I served as a PC member at IUI2023.
-
I received the 2022 Google Europe Students with Disabilities Scholarship.
-
I received Special Recognition for Outstanding Reviews for UIST2022 and CHI2021.
Full Paper Publications
|
Graph4GUI: Graph Neural Networks for Representing Graphical User Interfaces
Yue Jiang,
Changkong Zhou,
Vikas Garg*,
Antti Oulasvirta*
ACM CHI Conference on Human Factors in Computing Systems (CHI2024)
|
|
AXNav: Replaying Accessibility Tests from Natural Language
Maryam Taeb,
Amanda Swearngin,
Eldon Schoop,
Ruijia Cheng,
Yue Jiang,
Jeffrey Nichols
ACM CHI Conference on Human Factors in Computing Systems (CHI2024)
|
|
VisRecall++: Analysing and Predicting Visualisation Recallability from Gaze Bahaviour
Yao Wang,
Yue Jiang*(Corresponding Author),
Zhiming Hu,
Constantin Ruhdorfer,
Mihai Bรขce,
Andreas Bulling
ACM Symposium on Eye Tracking Research and Applications (ETRA2024)
|
|
Impact of Design Decisions in Scanpath Modeling
Parvin Emami,
Yue Jiang,
Zixin Guo,
Luis A. Leiva
ACM Symposium on Eye Tracking Research and Applications (ETRA2024)
|
|
UEyes: Understanding Visual Saliency across User Interface Types
Yue Jiang,
Luis A. Leiva,
Hamed R. Tavakoli,
Paul R. B. Houssel,
Julia Kylmala,
Antti Oulasvirta
ACM CHI Conference on Human Factors in Computing Systems (CHI2023)
|
|
HiFECap: Monocular High-Fidelity and Expressive Capture of Human Performances
Yue Jiang,
Marc Habermann,
Vladislav Golyanik,
Christian Theobalt
British Machine Vision Conference (BMVC2022)
|
|
Pretty Princess vs. Successful Leader: Gender Roles in Greeting Card Messages
Jiao Sun,
Tongshuang Wu,
Yue Jiang,
Ronil Awalegaonkar,
Xi Victoria Lin,
Diyi Yang
ACM CHI Conference on Human Factors in Computing Systems (CHI2022) (Best Paper Honorable Mention)
|
|
ReverseORC: Reverse Engineering of Resizable User Interface Layouts with OR-Constraints
Yue Jiang,
Wolfgang Stuerzlinger,
Christof Lutteroth
ACM CHI Conference on Human Factors in Computing Systems (CHI2021)
|
|
Vid2Doppler: Synthesizing Doppler Radar Data from Videos forTraining Privacy-Preserving Activity Recognition
Karan Ahuja,
Yue Jiang,
Mayank Goel,
Chris Harrison
ACM CHI Conference on Human Factors in Computing Systems (CHI2021)
|
|
โPositive Energyโ: Perceptions and Attitudes Towards COVID-19 Information on Social Media in China
Zhicong Lu,
Yue Jiang,
Chenxinran Shen,
Margaret C Jack,
Daniel Wigdor,
Mor Naaman,
ACM CHI Computer-Supported Cooperative Work and Social Computing (CSCW2021)
|
|
SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization
Yue Jiang,
Dantong Ji,
Zhizhong Han,
Matthias Zwicker
IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR2020) (Oral Top 5%)
|
|
ORCSolver: An Efficient Solver for Adaptive GUI Layout with OR-Constraints
Yue Jiang,
Wolfgang Stuerzlinger,
Matthias Zwicker,
Christof Lutteroth
ACM CHI Conference on Human Factors in Computing Systems (CHI2020)
|
|
The Government's Dividend: Complex Perceptions of Social Media Misinformation in China
Zhicong Lu,
Yue Jiang,
Cheng Lu,
Mor Naaman,
Daniel Wigdor
ACM CHI Conference on Human Factors in Computing Systems (CHI2020)
|
|
ORC Layout: Adaptive GUI Layout with OR-Constraints
Yue Jiang,
Ruofei Du,
Christof Lutteroth, Wolfgang Stuerzlinger
ACM CHI Conference on Human Factors in Computing Systems (CHI2019)
|
Workshop Papers, Posters, Extended Abstracts, and Demonstations
-
Computational Representations for Graphical User Interfaces
Yue Jiang
ACM CHI Conference on Human Factors in Computing Systems (CHI2024 Doctoral Consortium)
[PDF]
-
Computational Methodologies for Understanding, Automating, and Evaluating User Interfaces
Yue Jiang*,
Yuwen Lu*,
Tiffany Knearem,
Clara Kliman-Silver,
Christof Lutteroth,
Toby Jia-Jun Li,
Jeffrey Nichols,
Wolfgang Stuerzlinger
ACM CHI Conference on Human Factors in Computing Systems (CHI2024 Extended Abstract)
[PDF]
-
UEyes: An Eye-Tracking Dataset across User Interface Types
Yue Jiang,
Luis A. Leiva,
Hamed R. Tavakoli,
Paul R. B. Houssel,
Julia Kylmala,
Antti Oulasvirta
ACM CHI Conference on Human Factors in Computing Systems (CHI2023 Workshop Paper)
[PDF]
-
Computational Assistance for User Interface Design: Smarter Generation and Evaluation of Design Ideas
Lena Hegemann,
Yue Jiang,
Joon-Gi Shin,
Yi-Chi Liao,
Markku Laine,
Antti Oulasvirta
ACM CHI Conference on Human Factors in Computing Systems (CHI2023 Interactivity)
[PDF]
-
The Future of Computational Approaches for Understanding and Adapting User Interfaces
Yue Jiang,
Yuwen Lu,
Christof Lutteroth,
Toby Jia-Jun Li,
Jeffrey Nichols,
Wolfgang Stuerzlinger
ACM CHI Conference on Human Factors in Computing Systems (CHI2023 Extended Abstract)
[PDF]
-
Pervasive Eye Tracking and Mobile Eye-based Interaction (PETMEI)
Yao Wang,
Ludwig Sidenmark,
Teresa Hirzle,
Yue Jiang,
Andreas Bulling
ACM Symposium on Eye Tracking Research and Applications (ETRA2023 Extended Abstract)
[PDF]
-
Computational Approaches for Understanding, Generating, and Adapting User Interfaces
Yue Jiang*,
Yuwen Lu*,
Jeffrey Nichols,
Wolfgang Stuerzlinger,
Chun Yu,
Christof Lutteroth,
Yang Li,
Ranjitha Kumar,
Toby Jia-Jun Li
ACM CHI Conference on Human Factors in Computing Systems (CHI2022 Extended Abstract)
[PDF]
-
Disability Semantics for Academic Writing
Garreth Tigwell, Yue Jiang
AccessSIGCHI
-
Designer-in-the-Loop Layout Autocompletion with Graph Neural Networks
Yue Jiang,
Vikas Garg,
Antti Oulasvirta
Finnish Center for Artificial Intelligence AI Day Workshop, 2022
-
DocShop: Bringing Document Content to Life
Yue Jiang
Adobe Research Project Expo, 2020
Workshop Organization
|
Computational Methodologies for Understanding, Automating, and Evaluating User Interfaces
Yue Jiang*,
Yuwen Lu* (Equal Contribution),
Tiffany Knearem,
Clara Kliman-Silver,
Christof Lutteroth,
Toby Jia-Jun Li,
Jeffrey Nichols,
Wolfgang Stuerzlinger
ACM CHI Conference on Human Factors in Computing Systems (CHI2024)
|
|
The Future of Computational Approaches for Understanding and Adapting User Interfaces
Yue Jiang,
Yuwen Lu,
Christof Lutteroth,
Toby Jia-Jun Li,
Jeffrey Nichols,
Wolfgang Stuerzlinger
ACM CHI Conference on Human Factors in Computing Systems (CHI2023)
|
|
Pervasive Eye Tracking and Mobile Eye-based Interaction (PETMEI)
Yao Wang,
Ludwig Sidenmark,
Teresa Hirzle,
Yue Jiang,
Andreas Bulling
ACM Symposium on Eye Tracking Research and Applications (ETRA2023)
|
|
Computational Approaches for Understanding, Generating, and Adapting User Interfaces
Yue Jiang*,
Yuwen Lu*,
Jeffrey Nichols,
Wolfgang Stuerzlinger,
Chun Yu,
Christof Lutteroth,
Yang Li,
Ranjitha Kumar,
Toby Jia-Jun Li
ACM CHI Conference on Human Factors in Computing Systems (CHI2022)
|
Education
-
Honors Bachelor of Science in
Computer Science
Specialist &
Mathematics
Major (High Distinction) @
University of Toronto ๐จ๐ฆ
Sept. 2014 - Apr. 2017 (granted in 2018)
Supervisor:
Prof. Gerald Penn
|
Selected Professional Experience and Internship
|
Academic Service
-
Accessibility Co-Chair, CHI2023, CHI2024 FAQ
2023, 2024
-
Associate Chair (AC), CHI Late Breaking Works
2020, 2021, 2022, 2023, 2024
-
HCI Master Program Admission Committee Member, Aalto University, Finland
2024
-
Ph.D. Admission Committee Member, Finnish AI Center, Finland
2024
-
Program Committee Member, AI and HCI Workshop, ICML2023
2023
-
Workshop Juror, CHI2023
2023
-
Program Committee Member, IUI2023
2023
-
Session Chair, Human AI Collaboration, CHI2023
2023
-
Session Chair, Text Input and Textual Communication, CHI2023
2023
-
Co-Organizer & Poster Chair, HelsinCHI 2023 HelsinCHI 2023 Webpage
2023
-
Organizing Committee Member, European Lab for Learning & Intelligent Systems (ELLIS) Doctoral Symposium 2023
2023
-
Co-Organizer, Seminar of HCI for Chinese HCI Researchers around the World
2021 - Now
-
Committee Member, ACM SIGGRAPH Research Development Committee --- DEI and Accessibility
2021 - 2023
-
Ph.D. Admission Committee Member, Finnish AI Center, Finland
2023
-
Co-Host, DIS2022 Ask-Me-Anything Session
2022
-
Program Committee Member, IUI2021 Demos and Posters
2021
-
Session Chair, MobileHCI
2021
-
Ph.D. & Master Admission Committee Member, University of Maryland, College Park, USA
2019, 2020, 2021
-
Ph.D Student Panel Leader for Prospective Students, University of Maryland, College Park, USA
2019
-
Graduate Mentor for Technica Research Bootcamp, USA
2018, 2019
-
Department of Mathematics Ambassador, University of Toronto, Canada
2016
|
Selected Honors and Grants
-
Meta Research PhD Fellowship
2023
-
Google Europe Students with Disabilities Scholarship
2022
-
Special Recognitions for Outstanding Reviews for UIST2022
2022
-
Finnish Center for Artificial Intelligence (FCAI) Research Visit Funding (6 Months)
2023 - 2024
-
Special Recognitions for Outstanding Reviews for CHI2021
2021
-
ACM SIGCHI Gary Marsden Travel Award
2021
-
ACM SIGCHI Gary Marsden Travel Award (Virtual)
2021
-
Graduate Dean's Scholarship
, University of Maryland, College Park
2018 - 2019, 2019 - 2020
-
Adobe Research Gift Grant for Document Intelligent Research
Apr. 2020
-
Adobe Research Gift Grant for Document Intelligent Research
Sept. 2020
-
Jacob K. Goldhaber Travel Grant Award
2019
-
Department of Computer Science Travel Grant Award
, University of Maryland
2019
-
Grace Hopper Scholarship
2018
-
Dean's List Scholar
, University of Toronto
2015, 2016, 2017, 2018
-
New College Council In-Course Scholarship
, University of Toronto
2014 - 2015, 2015 - 2016
-
Award for Outstanding Achievement in Mathematical Expression and Reasoning for Computer Science
, University of Toronto
2015
-
National Olympiad in Biology in Provinces, First Prize
2012
-
China Adolescents Science and Technology Innovation Contest, First Prize
2011
|
Selected Teaching Experience
-
Guest Lecturer on the topic of Visual Rerception
Human Factors Engineering (Aalto University) ๐ซ๐ฎ
May 2023
-
Research Project Mentor,
Research Project in Human-Computer Interaction (Aalto University) ๐ซ๐ฎ
Jan. 2023 - May 2023
-
Guest Lecturer on the topic of Visual Saliency
Computational Design and Interaction (Aalto University) ๐ซ๐ฎ
Nov. 2022
-
Co-Instructor,
Seminar: Computing the User and their Interface (Saarland University) ๐ฉ๐ช
Oct. 2021 - Feb. 2022
-
Course Co-Supervisor,
Seminar: Computer Vision and Machine Learning for Computer Graphics (MPI) ๐ฉ๐ช
Apr. 2021 - Aug. 2021
-
Teaching Assistant,
CMSC740 Advanced Computer Graphics (UMD) ๐บ๐ธ
Jan. 2020 - May. 2020
-
Teaching Assistant,
CMSC427 Computer Graphics (UMD) ๐บ๐ธ
Aug. 2019 - Dec. 2019
-
Teaching Assistant,
CMSC427 Computer Graphics (UMD) ๐บ๐ธ
Aug. 2018 - Dec. 2018
-
Teaching Assistant,
CSC263 Data Structures and Analysis (UofT) ๐จ๐ฆ
Jan. 2017 - Apr. 2017
-
Teaching Assistant,
CSC263 Data Structures and Analysis (UofT) ๐จ๐ฆ
Sept. 2016 - Dec. 2016
-
Teaching Assistant,
CSC165: Mathematical Expression and Reasoning for Computer Science (UofT) ๐จ๐ฆ
Jan. 2016 - Apr. 2016
-
Peer Tutor,
Calculus, Linear Algebra I, Linear Algebra II (UofT) ๐จ๐ฆ
Sept. 2015 - Apr. 2016
-
Teaching Volunteer,
Elementary School and Junior High School Mathematics and Physics, Qinyuan, Shanxi, China ๐จ๐ณ
2014
|
My Name
My first name is a combination of my mother's last name and "the Moon", which has the same pronunciation as "the Moon" in Chinese. My last name is "river". So my full name is "the Moon over the river".
In China, "the Moon over the river" is widely used in poems to express the emotion that although we cannot live together with our family, we can still share the same Moon at night, and the Moon over the river can always send the best wishes to the ones we love!
I hope all the families can have more chances to reunite. If it cannot be realized, I hope the Moon over the river can help us send the best wishes.
|
Acknowledgment
I appreciate all the people who have helped or encouraged me throughout my life! I am an unlucky but also super lucky person. I may have gone through a little bit more ups and downs in my life than most people, but I always received countless support from so many people, no matter what situation I was in.
My goal is to be a good mentor in the future since some unforgettable professors and mentors always supported me and served as examples for me.
Prof. Wolfgang Stuerzlinger is the most important person in my academic journey. I am grateful for the opportunity he gave me to work on exciting HCI projects, even though we had never met in person until two years later.
He was the person who transformed me from a novice student to a researcher. He continuously taught me the skills to conduct high-quality research and has been an unwavering source of encouragement and support both academically and emotionally throughout these years.
It was also an unforgettable moment when Wolfgang squatted down to my level the first time we met, as he is tall.
Together with Prof. Christof Lutteroth, they made a significant impact on my academic journey. The second project I worked on with them remains my favorite project to date. I thoroughly enjoyed the topic and the experience of working with both of them. Their support and encouragement meant a great deal to me and made me feel happy every day. I was inspired by their hard work and dedication, especially considering I was just a remote student.
Prof. Faith Ellen was my idol during my undergrad years. She encouraged me to speak English, as I almost could not speak English when I went to Canada, and let me fall in love with data structure and computer theory. She taught me that persistence is much more important than intelligence.
Prof. Gerald Penn was my undergraduate advisor in the Mathematical Linguistics group. He has some illusion about me that he often told other people that he would rank me as the best undergraduate student he has ever worked with. I really thank him for overpraising me.
I was a data structure TA under Prof. Sam Toueg and Prof. Francois Pitt. I appreciate them giving me the chance to teach students data structure proofs for two hours every week. I broke all the chalks in the classroom during my first time teaching since I was too nervous. I still remember Sam and I worked past midnight to do the grading.
My graduate advisors, Prof. Antti Oulasvirta and Prof. Matthias Zwicker have been instrumental in advancing my research and teaching abilities. Through their guidance and support, they have helped me to make meaningful progress in these areas.
|
|