User Testing for Capital Project Website Discoverability and Accessibility
What This Project is About
The federal Monitor partnered with Pratt DX center to evaluate the beta version of the capital tracker project. Our project aims to identify the gaps that need to be fixed and ensure that interactions within the capital tracker are smooth and information is displayed in a user-friendly manner. We performed heuristic evaluations, Competitive analysis, and moderated user interviews to detect areas of improvement in the site. Our recommendations uncovered potential for layout improvements as well as problems with misinformation.
My Team
Jun Kok
Sacchit Vartak
& Me
Duration
3 Months
UX Researcher
MY ROLE
We all heavily collaborated on various aspects of the project. I took on the role of UX Researcher, which included Writing research questions, creating scripts, conducting moderated usability tests, Synthesizing testing data, and designing recommendations.
Problem
Who is NYCHA?
The New York City Housing Authority (NYCHA) is a public development corporation that provides public housing in New York City and is the largest public housing authority in North America.
NYCHA violated federal laws over the past few years by not disclosing unsafe living conditions for NYC residents. As a result, a federal monitor has been appointed to track their work and hopefully hold them accountable so that it doesn't happen again.
As part of this monitoring work, NYCHA was directed to develop a tool that communicates where and how it spends the money it receives from the city, New York state, and the US federal government for capital projects. However, they didn't do any research or evaluation with city residents prior to releasing the tool. So, the federal monitor partnered with pratt DX Center with the goal of gathering feedback from residents about the tool and its usability.
Process
🎬
Clients kickoff meeting
🧠
Brainstorming Research plan
📊
Collect and analyze data
📝
Develop recommendations
📺
Present finding to client
-
After our client kick-off meeting, we met with NYCHA chief compliance officer on a weekly basis to inform them of our process.
-
We conducted a competitive analysis
-
Heuristic evaluation
-
interviews
-
Survey
Setting up the usability study
For compensation reasons, we had to present our plan to the NYCHA team in order for them to approve a 25-dollar compensation for screening. And also for them to put us in touch with participants that would fall into our audience group. Due to challenges with finding the right participants, as a team, we decided to conduct quantitative research as well
Tasks for User Interviews
5 interviews with individuals that know NYCHA through work, residence, or community. The participants share their screens, follow a series of tasks, and provide feedback and opinions.
We asked a set of pre-test and post-test questions in order to get a better understanding of users’ experiences.
Sending out Usability Surveys
14 survey takers followed instructions to complete tasks, give feedback, and evaluate the usability of the NYCHA Capital Projects Tracker.
Results
What’s working?
1️⃣
All of our participants were able to locate the Capital Project Tracker from the home page.
2️⃣
93% of the participants were able to locate a project of their choice.
3️⃣
Participants used adjectives like organized, confusing, and informative to describe the home page.
What can be improved?
1️⃣
Only 43% of the participants rated their experience of finding the Capital Project Tracker as 5, which corresponds to easy.
2️⃣
36% of the participants rated their experience using the filter and map to locate the project as 3, meaning they have neutral experience while using them.
3️⃣
Only 36% of the participants think the Project Information page has enough details.
1. Layout Throughout the Site can be Improved
1.1 Homepage
Users are only greeted with the top nav menu and announcements when they get to the homepage.
There is unused “real estate,” which makes the user scroll more before getting to where they need to be.
Current Site
Recommendation
Adding a button that links to the tracker page on the top navigation menu can help users get to the page, without changing the drastically current design and layout of the homepage.
Hi-fidelity Mockup
1.2 Filter
Filter options are vague and difficult to use.
“Council district” is hard as not many people would know their council district number.
“Funding source” is too vague and doesn’t provide any substantial information.
Current Site
"Do I have to engage with all filters ?"
The tracker page takes up a lot of space on a single page.
Some users do not even notice the list of projects when searching for a project. The size of the map sometimes makes users unintentionally scroll the map instead of the page.
"...and I'm actually looking at it, it shows the district name on the map, so if I saw the map first, it would change my thought process."
Current Site
Recommendation
The layout of the elements should be rearranged: The map can be made smaller, the list put next to the map and the filters reduced, changed, and rearranged.
This will let the user utilize everything at once without navigating the page too much.
Hi-fidelity Mockup
1.3 Details Page
There is an repetition of information on the timeline of the project
This could lead to users spending more time than needed to find what they are looking for.
There is a lack of information users are interested in regarding the budget and contact.
"...We need to know how much it's going to cost to fund projects. To get to the point where we can commence the projects, if additional funding is needed, that's important.."
Current Site
Recommendation
1
The layout of elements on this page can be rearranged to reduce redundancy and streamline communication.
2
Detailed information on the budget source and contact information should be available for the users.
Hi-fidelity Mockup
1.4 Map
Currently, the map results cover all the buildings. This makes it hard for users to track and find out the exact locations where the projects are taking place.
For example, This project involves repairing and replacing mechanical systems at the development, such as HVAC, ventilation, and Plumbing. However, none of that is specified.
Current Site
"The map (Green dot) doesn't show the exact location where the projects are taking place..."
Recommendation
1
Color coding the dots at the development will allow users to differentiate if the project is in the exterior or interior.
2
When Zoom in, the map can specify which buildings the projects are taking place in.
Hi-fidelity Mockup
2. Misinformation
False information or lack of in regard to the projects
Some projects on Capital Tracker are not mentioned or placed as a standalone project where it’s actually two or three projects involved.
Moreover, the information provided about the funds is not all-inclusive of the info that is necessary.
In some cases, the progress bar in projects reflects the intended timeline, not the actual one.
Recommendation
Make an option for people to report information they think is false.
A potential solution could be that with a certain number of reports, the validity of the project would be flagged until the information is updated.
Hi-fidelity Mockup
Client Feedback and Future Plans
We recorded our presentation and shared it with our client.
" Thank you for the wonderful work you did on the NYCHA Capital Project Tracker Usability Study presentation." - Edna Wells Handy, NYCHA Chief Compliance Officer
Reflections
-
Using both qualitative and quantitative methods in a usability study can provide a more comprehensive understanding of usability issues and potential solutions, resulting in a better user experience.
-
To work with clients, set realistic timelines and goals, establish clear communication channels, and maintain a professional demeanor. By managing expectations effectively, you can build a positive working relationship and ensure a successful outcome.