Yue Jiang
Email: yue.jiang [at] aalto.fi

CV (Updated in Mar. 2024) | Linkedin | Google Scholar
WeChat Blog: yuejiang_phd

I am a Ph.D. student in the Computational Behavioral Lab directed by Prof. Antti Oulasvirta, working on Computational User Interface Understanding and Eye Tracking on User Interfaces (Secondary advisor: Prof. Vikas Garg) at Aalto University and the Finnish Center for Artificial Intelligence (FCAI), Finland ๐Ÿ‡ซ๐Ÿ‡ฎ. I also collaborate closely with Prof. Wolfgang Stuerzlinger (Simon Fraser University, Canada ๐Ÿ‡จ๐Ÿ‡ฆ) and Prof. Christof Lutteroth (University of Bath, UK ๐Ÿ‡ฌ๐Ÿ‡ง) on our series of OR-Constraint (ORC) Adaptive GUI layout projects.
Previously, I worked on Human Motion Capture and the Reconstruction of Non-Rigid Deformations at the Max Planck Institute, Germany ๐Ÿ‡ฉ๐Ÿ‡ช. I got my Master's degree in Computer Graphics under Prof. Matthias Zwicker from the University of Maryland, College Park, USA ๐Ÿ‡บ๐Ÿ‡ธ, and my Bachelor's degree in Computer Science Specialist with an additional Mathematics Major under Prof. Gerald Penn from the University of Toronto, Canada ๐Ÿ‡จ๐Ÿ‡ฆ. I am from Nanjing, a beautiful city in the east of China, which served as the capital of China ten times in the history of China ๐Ÿ‡จ๐Ÿ‡ณ.
I will be on the job market for tenure-track faculty positions or relevant industry research positions starting in 2025. Feel free to reach out if you are aware of any suitable openings!

Photo taken in Vancouver.

Highlights

  • Our paper 'Graph4GUI: Graph Neural Networks for Representing Graphical User Interfaces' was conditionally accepted by CHI2024.
  • I am visiting Prof. Jeffrey P. Bigham's BIG Lab at Carnegie Mellon University in 2024.
  • We are organizing our third Computational User Interface workshop, 'Computational Methodologies for Understanding, Automating, and Evaluating User Interfaces' at CHI2024. Welcome to participate!
  • I am serving as the Accessibility Co-Chair and Associate Chair (AC) for Late Breaking Works at CHI2024.
  • I am excited to receive the 2023 Meta Research PhD Fellowship! Yay! :-)
  • I enjoyed my internship with the UI Understanding team under Jeff Nichols at Apple AIML so much! I wish it could be longer!
  • I served as the Accessibility Co-Chair, Workshop Juror, and Associate Chair (AC) for Late Breaking Works at CHI2023.
  • We organized our second Computational User Interface workshop, 'The Future of Computational Approaches for Understanding and Adapting User Interfaces' at CHI2023.
  • I served as a PC member at IUI2023.
  • I received the 2022 Google Europe Students with Disabilities Scholarship.
  • I received Special Recognition for Outstanding Reviews for UIST2022 and CHI2021.

Full Paper Publications

Graph4GUI: Graph Neural Networks for Representing Graphical User Interfaces
Yue Jiang, Changkong Zhou, Vikas Garg*, Antti Oulasvirta*
ACM CHI Conference on Human Factors in Computing Systems (CHI2024)
AXNav: Replaying Accessibility Tests from Natural Language
Maryam Taeb, Amanda Swearngin, Eldon Schoop, Ruijia Cheng, Yue Jiang, Jeffrey Nichols
ACM CHI Conference on Human Factors in Computing Systems (CHI2024)
VisRecall++: Analysing and Predicting Visualisation Recallability from Gaze Bahaviour
Yao Wang, Yue Jiang*(Corresponding Author), Zhiming Hu, Constantin Ruhdorfer, Mihai Bรขce, Andreas Bulling
ACM Symposium on Eye Tracking Research and Applications (ETRA2024)
Impact of Design Decisions in Scanpath Modeling
Parvin Emami, Yue Jiang, Zixin Guo, Luis A. Leiva
ACM Symposium on Eye Tracking Research and Applications (ETRA2024)
UEyes: Understanding Visual Saliency across User Interface Types
Yue Jiang, Luis A. Leiva, Hamed R. Tavakoli, Paul R. B. Houssel, Julia Kylmala, Antti Oulasvirta
ACM CHI Conference on Human Factors in Computing Systems (CHI2023)
HiFECap: Monocular High-Fidelity and Expressive Capture of Human Performances
Yue Jiang, Marc Habermann, Vladislav Golyanik, Christian Theobalt
British Machine Vision Conference (BMVC2022)
Pretty Princess vs. Successful Leader: Gender Roles in Greeting Card Messages
Jiao Sun, Tongshuang Wu, Yue Jiang, Ronil Awalegaonkar, Xi Victoria Lin, Diyi Yang
ACM CHI Conference on Human Factors in Computing Systems (CHI2022) (Best Paper Honorable Mention)
ReverseORC: Reverse Engineering of Resizable User Interface Layouts with OR-Constraints
Yue Jiang, Wolfgang Stuerzlinger, Christof Lutteroth
ACM CHI Conference on Human Factors in Computing Systems (CHI2021)
Vid2Doppler: Synthesizing Doppler Radar Data from Videos forTraining Privacy-Preserving Activity Recognition
Karan Ahuja, Yue Jiang, Mayank Goel, Chris Harrison
ACM CHI Conference on Human Factors in Computing Systems (CHI2021)
โ€œPositive Energyโ€: Perceptions and Attitudes Towards COVID-19 Information on Social Media in China
Zhicong Lu, Yue Jiang, Chenxinran Shen, Margaret C Jack, Daniel Wigdor, Mor Naaman,
ACM CHI Computer-Supported Cooperative Work and Social Computing (CSCW2021)
SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization
Yue Jiang, Dantong Ji, Zhizhong Han, Matthias Zwicker
IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR2020) (Oral Top 5%)
ORCSolver: An Efficient Solver for Adaptive GUI Layout with OR-Constraints
Yue Jiang, Wolfgang Stuerzlinger, Matthias Zwicker, Christof Lutteroth
ACM CHI Conference on Human Factors in Computing Systems (CHI2020)
The Government's Dividend: Complex Perceptions of Social Media Misinformation in China
Zhicong Lu, Yue Jiang, Cheng Lu, Mor Naaman, Daniel Wigdor
ACM CHI Conference on Human Factors in Computing Systems (CHI2020)
ORC Layout: Adaptive GUI Layout with OR-Constraints
Yue Jiang, Ruofei Du, Christof Lutteroth, Wolfgang Stuerzlinger
ACM CHI Conference on Human Factors in Computing Systems (CHI2019)

Workshop Papers, Posters, Extended Abstracts, and Demonstations

Workshop Organization

Computational Methodologies for Understanding, Automating, and Evaluating User Interfaces
Yue Jiang*, Yuwen Lu* (Equal Contribution), Tiffany Knearem, Clara Kliman-Silver, Christof Lutteroth, Toby Jia-Jun Li, Jeffrey Nichols, Wolfgang Stuerzlinger
ACM CHI Conference on Human Factors in Computing Systems (CHI2024)
The Future of Computational Approaches for Understanding and Adapting User Interfaces
Yue Jiang, Yuwen Lu, Christof Lutteroth, Toby Jia-Jun Li, Jeffrey Nichols, Wolfgang Stuerzlinger
ACM CHI Conference on Human Factors in Computing Systems (CHI2023)
Pervasive Eye Tracking and Mobile Eye-based Interaction (PETMEI)
Yao Wang, Ludwig Sidenmark, Teresa Hirzle, Yue Jiang, Andreas Bulling
ACM Symposium on Eye Tracking Research and Applications (ETRA2023)
Computational Approaches for Understanding, Generating, and Adapting User Interfaces
Yue Jiang*, Yuwen Lu*, Jeffrey Nichols, Wolfgang Stuerzlinger, Chun Yu, Christof Lutteroth, Yang Li, Ranjitha Kumar, Toby Jia-Jun Li
ACM CHI Conference on Human Factors in Computing Systems (CHI2022)

Education

  • Honors Bachelor of Science in Computer Science Specialist & Mathematics Major (High Distinction) @ University of Toronto ๐Ÿ‡จ๐Ÿ‡ฆ
    Sept. 2014 - Apr. 2017 (granted in 2018)
    Supervisor: Prof. Gerald Penn

Selected Professional Experience and Internship


Academic Service

  • Accessibility Co-Chair, CHI2023, CHI2024 FAQ 2023, 2024
  • Associate Chair (AC), CHI Late Breaking Works 2020, 2021, 2022, 2023, 2024
  • HCI Master Program Admission Committee Member, Aalto University, Finland 2024
  • Ph.D. Admission Committee Member, Finnish AI Center, Finland 2024
  • Program Committee Member, AI and HCI Workshop, ICML2023 2023
  • Workshop Juror, CHI2023 2023
  • Program Committee Member, IUI2023 2023
  • Session Chair, Human AI Collaboration, CHI2023 2023
  • Session Chair, Text Input and Textual Communication, CHI2023 2023
  • Co-Organizer & Poster Chair, HelsinCHI 2023 HelsinCHI 2023 Webpage 2023
  • Organizing Committee Member, European Lab for Learning & Intelligent Systems (ELLIS) Doctoral Symposium 2023 2023
  • Co-Organizer, Seminar of HCI for Chinese HCI Researchers around the World 2021 - Now
  • Committee Member, ACM SIGGRAPH Research Development Committee --- DEI and Accessibility 2021 - 2023
  • Ph.D. Admission Committee Member, Finnish AI Center, Finland 2023
  • Co-Host, DIS2022 Ask-Me-Anything Session 2022
  • Program Committee Member, IUI2021 Demos and Posters 2021
  • Session Chair, MobileHCI 2021
  • Ph.D. & Master Admission Committee Member, University of Maryland, College Park, USA 2019, 2020, 2021
  • Ph.D Student Panel Leader for Prospective Students, University of Maryland, College Park, USA 2019
  • Graduate Mentor for Technica Research Bootcamp, USA 2018, 2019
  • Department of Mathematics Ambassador, University of Toronto, Canada 2016


Selected Honors and Grants

  • Meta Research PhD Fellowship 2023
  • Google Europe Students with Disabilities Scholarship 2022
  • Special Recognitions for Outstanding Reviews for UIST2022 2022
  • Finnish Center for Artificial Intelligence (FCAI) Research Visit Funding (6 Months) 2023 - 2024
  • Special Recognitions for Outstanding Reviews for CHI2021 2021
  • ACM SIGCHI Gary Marsden Travel Award 2021
  • ACM SIGCHI Gary Marsden Travel Award (Virtual) 2021
  • Graduate Dean's Scholarship , University of Maryland, College Park 2018 - 2019, 2019 - 2020
  • Adobe Research Gift Grant for Document Intelligent Research Apr. 2020
  • Adobe Research Gift Grant for Document Intelligent Research Sept. 2020
  • Jacob K. Goldhaber Travel Grant Award 2019
  • Department of Computer Science Travel Grant Award , University of Maryland 2019
  • Grace Hopper Scholarship 2018
  • Dean's List Scholar , University of Toronto 2015, 2016, 2017, 2018
  • New College Council In-Course Scholarship , University of Toronto 2014 - 2015, 2015 - 2016
  • Award for Outstanding Achievement in Mathematical Expression and Reasoning for Computer Science , University of Toronto 2015
  • National Olympiad in Biology in Provinces, First Prize 2012
  • China Adolescents Science and Technology Innovation Contest, First Prize 2011


Selected Teaching Experience

  • Guest Lecturer on the topic of Visual Rerception Human Factors Engineering (Aalto University) ๐Ÿ‡ซ๐Ÿ‡ฎ May 2023
  • Research Project Mentor, Research Project in Human-Computer Interaction (Aalto University) ๐Ÿ‡ซ๐Ÿ‡ฎ Jan. 2023 - May 2023
  • Guest Lecturer on the topic of Visual Saliency Computational Design and Interaction (Aalto University) ๐Ÿ‡ซ๐Ÿ‡ฎ Nov. 2022
  • Co-Instructor, Seminar: Computing the User and their Interface (Saarland University) ๐Ÿ‡ฉ๐Ÿ‡ช Oct. 2021 - Feb. 2022
  • Course Co-Supervisor, Seminar: Computer Vision and Machine Learning for Computer Graphics (MPI) ๐Ÿ‡ฉ๐Ÿ‡ช Apr. 2021 - Aug. 2021
  • Teaching Assistant, CMSC740 Advanced Computer Graphics (UMD) ๐Ÿ‡บ๐Ÿ‡ธ Jan. 2020 - May. 2020
  • Teaching Assistant, CMSC427 Computer Graphics (UMD) ๐Ÿ‡บ๐Ÿ‡ธ Aug. 2019 - Dec. 2019
  • Teaching Assistant, CMSC427 Computer Graphics (UMD) ๐Ÿ‡บ๐Ÿ‡ธ Aug. 2018 - Dec. 2018
  • Teaching Assistant, CSC263 Data Structures and Analysis (UofT) ๐Ÿ‡จ๐Ÿ‡ฆ Jan. 2017 - Apr. 2017
  • Teaching Assistant, CSC263 Data Structures and Analysis (UofT) ๐Ÿ‡จ๐Ÿ‡ฆ Sept. 2016 - Dec. 2016
  • Teaching Assistant, CSC165: Mathematical Expression and Reasoning for Computer Science (UofT) ๐Ÿ‡จ๐Ÿ‡ฆ Jan. 2016 - Apr. 2016
  • Peer Tutor, Calculus, Linear Algebra I, Linear Algebra II (UofT) ๐Ÿ‡จ๐Ÿ‡ฆ Sept. 2015 - Apr. 2016
  • Teaching Volunteer, Elementary School and Junior High School Mathematics and Physics, Qinyuan, Shanxi, China ๐Ÿ‡จ๐Ÿ‡ณ 2014


My Name

    My first name is a combination of my mother's last name and "the Moon", which has the same pronunciation as "the Moon" in Chinese. My last name is "river". So my full name is "the Moon over the river".
    In China, "the Moon over the river" is widely used in poems to express the emotion that although we cannot live together with our family, we can still share the same Moon at night, and the Moon over the river can always send the best wishes to the ones we love!
    I hope all the families can have more chances to reunite. If it cannot be realized, I hope the Moon over the river can help us send the best wishes.


Acknowledgment

    I appreciate all the people who have helped or encouraged me throughout my life! I am an unlucky but also super lucky person. I may have gone through a little bit more ups and downs in my life than most people, but I always received countless support from so many people, no matter what situation I was in. My goal is to be a good mentor in the future since some unforgettable professors and mentors always supported me and served as examples for me.
    Prof. Wolfgang Stuerzlinger is the most important person in my academic journey. I am grateful for the opportunity he gave me to work on exciting HCI projects, even though we had never met in person until two years later. He was the person who transformed me from a novice student to a researcher. He continuously taught me the skills to conduct high-quality research and has been an unwavering source of encouragement and support both academically and emotionally throughout these years. It was also an unforgettable moment when Wolfgang squatted down to my level the first time we met, as he is tall. Together with Prof. Christof Lutteroth, they made a significant impact on my academic journey. The second project I worked on with them remains my favorite project to date. I thoroughly enjoyed the topic and the experience of working with both of them. Their support and encouragement meant a great deal to me and made me feel happy every day. I was inspired by their hard work and dedication, especially considering I was just a remote student.
    Prof. Faith Ellen was my idol during my undergrad years. She encouraged me to speak English, as I almost could not speak English when I went to Canada, and let me fall in love with data structure and computer theory. She taught me that persistence is much more important than intelligence.
    Prof. Gerald Penn was my undergraduate advisor in the Mathematical Linguistics group. He has some illusion about me that he often told other people that he would rank me as the best undergraduate student he has ever worked with. I really thank him for overpraising me.
    I was a data structure TA under Prof. Sam Toueg and Prof. Francois Pitt. I appreciate them giving me the chance to teach students data structure proofs for two hours every week. I broke all the chalks in the classroom during my first time teaching since I was too nervous. I still remember Sam and I worked past midnight to do the grading.
    My graduate advisors, Prof. Antti Oulasvirta and Prof. Matthias Zwicker have been instrumental in advancing my research and teaching abilities. Through their guidance and support, they have helped me to make meaningful progress in these areas.

Template: this.