Utilizing Tobii Eye-Tracking Technology to Discover Usability Problems on Pratt Institute’s Mobile Website
What This Project is About
Pratt Institute has recently redesigned its main website. Our Primary goal is to set up a consistent and embedded UX experience to set the stage for prospective undergraduate students within the mobile website. We used Tobii eye-tracking technology, which calculates data about the user, detecting the presence, attention, focus, and position of a person's eye. Our main findings tell us that certain experiences within site are not meeting user's expectations, especially the search experience and learning about programs, as well as the navigation guidance.
My Team
Daniel Meagher
Akshata Karekar
Katherin Aristizabal
Duration
3 Months
UX Researcher
MY ROLE
My role during this project was UX Researcher. To understand the website well, we designed six research questions that focused on different dimensions of the site. With my team of four, we focused on the Pratt mobile site. We each conducted eye-tracking with 2 participants that fit our user profile.
Problem
A little about Pratt Institute…
Pratt Institute is a college based in New York City. With opportunities in art, design, architecture, liberal arts and sciences, and information studies. Pratt offers nearly 50 undergraduate and graduate degree programs.
In the Summer of 2022, Pratt redesigned their main website. The problem and concern with the redesign are that prospective students might have issues finding information related to applying to programs, which will result in a drop in applications for next year.
Goals
1️⃣
How users navigate the site to find these information?
2️⃣
What elements are causing confusion or friction?
3️⃣
What information is being ignored by users?
4️⃣
Is homepage able to effectively direct users to the most important information?
Process
1/Client Kickoff with Pratt Communication Team
To learn more about what Pratt Institute’s goals were for our project with them, we had an initial meeting where each person from the groups we formed could ask them questions. This was crucial to understanding what they were trying to achieve with the information we would gather.
During this part of the process, we learned that Pratt was really interested in learning how prospective students perceive and interact with the content that is presented to them.
2/ Creating Research Plan & Design Tasks for our Study
We designed our moderator script and tasks for our eye-tracking study to gain insight into how users navigate the website and identify the issues they may encounter. During the eye-tracking study, we only allowed participants to think aloud and asked them questions during the Retrospective Think Aloud (RTA).
3/ How Did We Recruit Our 8 Participants?
Due to school rules of recruiting participants over the age of 18, we had constraints in recruitment.
We recruited participants currently in their undergraduate & graduate programs and work. We took a scenario-based approach where we asked the participant to take up a perspective of a prospective undergraduate student and perform the study.
We had to be careful when selecting certain participants because the eye-tracking software did not work well on individuals who wore glasses that were bi- or varifocal lenses. These lenses could have distorted the shape of the pupil when tracking eye movement.
4/ Moderate Tests and Collecting Data
After our tests were scheduled, we started conducting eye-tracking tests. Each participant had to complete the following:
-
Pre-test questionnaire: Gather data on their general knowledge of Pratt’s websites.
-
Tasks: Understand users' behaviors of how they navigate and find content
-
Retrospective Think Aloud: To have users explain their thoughts and feelings during task completion.
-
Post-test questionnaire: To understand the user's experience after completing the eye-tracking study.
-
System Usability Scale: To get a quick and reliable usability measure of the website's usable and learnable.
5/ Analyzing our Data
Collecting Qualitative Data
I devised the rainbow sheet to use. As a team, we each analyzed data from two participants and found specific themes we saw in the qualitative data by grouping the findings into specific themes with an affinity diagram.
Collecting Quantitative Data
In combination with the qualitative data, our team also used the outputs from Tobii (Gaze replays, heat maps, and gaze pots) to find out what people tended to look at and in which order they were looking at items on each page. To further find out more about the website’s usability, we recorded participants’ success per task, and the time it took them to complete each task.
6/ Providing Actionable Recommendations and Presenting It
After analyzing our data, we were able to develop several recommendations. Deliverables included;
-
Slide deck report to Pratt’s communication team.
-
Problem list included a list that described each usability problem with a short text and a severity rating.
-
Highlights reel, which was a cohesive and easy-to-digest collection of video excerpts from our user testing sessions
Results
Overall the participants of the eye-tracking study felt that the website was easy to learn but not easy to use.
1.1/ Information Is Not Where Users Expect
The Programs page lacks crucial information that participants wanted to see.
3 out of 8 participants said they expected to find information about course options, application deadlines, program requirements, and portfolio requirements on individual program pages.
"One thing that was missing for me is does this deadline apply to every program, including my program?“
Recommendation
1
2
Add the following categories to the submenu:
-
Curriculum
-
Program Requirements
-
Application Deadlines
Move the "Apply to Pratt" button and "Plan of Study" button up one level in the hierarchy
Hi-fidelity Mockup
1.2/ Abandoned Search Queries
When using the search tool, users struggled to realize that their search results were generating below the fold
“ I want to find results quick...”
Most participants viewed and interacted with filters. Only a few scrolled past the filter to view their results.
Recommendation
Strategic display of search filters.
Opt for overlay functionality for the filter, which will result in seeing search results and auto-suggestions above the fold.
Hi-fidelity Mockup
2.1/ Unclear Submenu Functions
Participants struggled to identify when the submenu navigated away or have more options. Lack of proper indication of the following:
1
Drops down to another set of items
2
Navigates to a new page
This caused users to continue scrolling in search of keywords that had been dropped down already.
Users scrolled through the entire page twice and failed to complete the task of finding “application requirements.” Because they did not realize that the results are underneath.
“I was looking for a keyword that would say requirements, but could not find it.”
Recommendation
Mockup
Mockup
2.2/Lack of menu hierarchy
The lack of hierarchy and similar menu options caused users to click through the different options before landing on the page that had the information they needed.
“... seemed disorganized in the sense of categorizing... I couldn't figure out exactly which side I should be looking at... forced me to click multiple things...”
Recommendation
Ultimately, further evidence from usability tests, like Card Sorts and A/B Testing, is needed for a solid recommendation.
Mockup
Want to See More?
Client Feedback and Future Plans
While there will be more work to do in the future, clients were satisfied and found our findings insightful.
"I think, is really interesting. And very helpful to kind of have some focused attention on mobile format. It is really super helpful." - Sarah Hromack, Director of Digital Communications
Reflections
-
Eye-tracking is an excellent way to evaluate and support usability findings. However, creating an intuitive, artificial environment for mobile users is challenging. In the future, I will probably rely more on eye-tracking studies on web pages.
-
When Conducting eye-tracking studies using Tobii, we had access to many data and had to be more selective on which piece of information would be helpful to tell our story.
-
Communicating effectively is key to success when working on complicated projects.