Project 2
User Testing InReach's Mobile App and Desktop Website
Project Snapshot
Product: Tech non-profit beta mobile app and desktop website
Role: UX Research Intern
Responsibility: Collaborate with research lead in usability testing, data analysis, and reporting.
Methods: Remote Moderated Usability Testing
Tools: Miro, UserTesting, Asana
Context: Internship Project for InReach, Completed June 2023.
About
InReach is the world's first tech non-profit dedicated to connecting LGBTQ+ individuals and refugees facing persecution or discrimination with safe, verified resources. Available as both a mobile application and a desktop website, InReach offers a free and accessible platform for those in need of support.
Problem Statement
There has been a spike in demand as Anti-LGBTQ+ Legislation Grows. InReach wants to meet the communities need and FAST. Recognizing it is reliant on volunteers, InReach wants to offer more convenient, flexible volunteering opportunities for individuals who may be unable to make big time commitment.
Introduction
Local Community Review Program
This lead to the creation of the Local Community Review (LCR) Program. This program would provide a flexible volunteering opportunity for individuals with prior experience visiting LGBTQ+ service providers but who may not have a lot of time to volunteer
As part of this endeavor, the design team integrated innovative LCR features into both our app and website. Before launching these changes we want to ensure the design is user friendly, efficient, and offers a smooth user experience for the user.
My Role
As a UX Research Intern, my main responsibility was to gather user insights to refine these newly added LCR functionalities through remote usability testing the beta mobile application and website. By conducting usability tests early and throughout the development process, issues with the design, navigation, content, or functionality can be identified and rectified before the update is launched.
Looking at the Big Picture
To enhance my research effectiveness, I delved into InReach's vision, mission, and growth strategies. This thorough understanding of their rebranding and expansion plans allowed me to align my research with the company's wider goals, particularly in strengthening the LCR program's role in InReach's growth.
My analysis revealed that the LCR program's success, crucial for InReach, depends on:
Enhancing data accuracy on the site.
Expanding the range of resources listed.
Broadening geographical reach.
Given the program's remote nature, user experience is key to its success. To better understand our users, I immersed myself in LGBTQ+ issues, absorbing documentaries and interviews, and analyzing previous user data. This led to the creation of detailed user personas.
User Personas
Research Objectives
Evaluate User Navigation and Flow
Assess how intuitively users can navigate through the app and complete key LCR tasks without assistance.
Assess User Overall Satisfaction
Collect user feedback on their overall experience and satisfaction with application with a special focus on the added LCR functionalities.
Understand User Needs and Expectations
Explore user expectations regarding the functionalities and assess how well the current design meets these expectations.
Measure Task Completion Rate and Efficiency
Determine how effectively and quickly users can complete key LCR tasks within the app.
Methodology
Research Design
Cognitive Walkthrough
I like to start with a cognitive walk through before a usability test for various reasons. The insights gained from a cognitive walkthrough can inform the creation of more realistic and relevant task scenarios for the usability test. It helps ensure that the tasks align closely with real-world user needs and challenges. In addition, performing a cognitive walkthrough helps me understand the user's journey and context better. This understanding is crucial in designing tasks that are meaningful and representative of actual user behavior. The best part? It requires little time and effort and there’s no need for recruitment.
User Interviews
To deeply understand our user’s we included a brief interview before the usability session. This interview povides valuable context about the user’s background which helps me interpret their behavior during the usability test more accuratley.
Usability Test
The interview was followed by a 45-60 minute remotely moderated usability test where users performed a series of tasks on the beta version app or website.
Participants
Recruitment
We used UserTesting’s scheduling feature, and aimed for obtain 12 participants to run two pilot sessions and five sessions on the mobile application and five sessions on the website.
Challenge: No-Shows
Due to participant no-shows, we couldn't meet our initial goal and had to adapt quickly to meet our deadline. This involved incorporating data from two pilot sessions and conducting a combination of website and mobile app sessions for our report.
For future projects, I recommend recruiting more participants in advance, sending reminders through multiple channels, and building early rapport with them to reduce no-shows.
I also learned that usability testing across different platforms, like websites and mobile apps, can be tricky. There's a risk of inaccurately applying findings from one platform to another, potentially leading to less effective improvements.
Demographics
Ethical Considerations
Prior to conducting interviews, participants were presented with a consent form outlining the study's purpose, their rights, including privacy and data use
Analysis
Uncovering Patterns
Thematic Analysis
In the data analysis phase, I transformed our collected data into actionable insights.
I meticulously extracted key ideas, observations, and quotes from our note-taking sheets. These were then visualized as individual sticky notes on a Miro whiteboard, allowing for a dynamic and collaborative analysis.
Together with the research lead, we categorized these notes into distinct themes, ensuring a structured and comprehensive understanding of our findings. This process was not only about organizing data, but also about uncovering patterns and narratives that would inform our design decisions.
Task Scoring Analysis
A task scoring analysis is a method used to evaluate and quantify the performance of users while they complete specific tasks, typically in a user experience (UX) or usability study. The process involves
Establishing Scoring Criteria:
Scoring Performance: After the user testing, each participant’s performance on each task is scored based on the criteria.
The scores are analyzed to identify patterns or trends. This analysis can reveal areas where the product performs well and where it might be causing confusion or difficulties for users.
Procedure
Materials
Discussion Guide
Working closely with the UX Research Lead, I applied my knowledge of psychological principles to co-develop the discussion guide that was to be used during our testing sessions. This collaboration enabled us to design a guide that effectively captures valid and unbiased data. To encourage genuine interactions with our platform during a session, I thoughtfully aligned the guide's tasks to mirror a typical volunteer's experience on our platform.
The discussion guide was made up of two parts.
Part 1: Open ended background questions to bring context to the session
Part 2: Task Based Usability Questions, on the participant’s background.
Notetaking Table
To help with consistency and data collection I drafted a note taking excel table with slots to include click-paths, behavioral observations, stand out quotes, a task success rating, and other notes.
Testing Sessions
The testing sessions were ran using UserTesting and were remotley moderated by myself and the research lead. We alternated between being a notetaker and moderator. The sessions ran as follows:
Background Interview
Task Based Questions
SUS Survey
Challenge: Last Minute Stakeholder Request
Stakeholders requested feedback on the InReach Onboarding Manual and asked us to include this aspect in our ongoing research. I promptly adjusted the discussion guide, restructured the timing of the sessions, and updated the note-taking template to accommodate this new request. However, sessions had already commenced, not all participants had the opportunity to provide feedback on the manual. This situation presented a unique challenge, allowing me to showcase my adaptability and my capability to uphold the quality of the research even under pressure.
Findings
Finding No1
A Struggle to Search Organization by Name
Most participants attempted to search up organization names by using the landing page search bar. Many assumed this was not a functionality.
“I wish I could search by name of place… but I’m stuck because it searches specific location not name of organization.”
-P6 (mobile)
Finding No2
Questioning Filter Application
Multiple participants thought the website didn not update when the filter option was chosen because the change was not visible to them.
Finding No 3
Suggesting a New Organization
A few participants said they would send an email to InReach to suggest a new organization. They did not notice the “suggest an organization” function. Others expected to use the “look up an organization” bar to add an organization they wanted to suggest.
Finding No4
Safety Button
Most users said the Saftey Exit was a “great feature”. One user believed it blended into the background and suggested it should be a button that follows you as you navigate the site [Sticky Button].
Finding No5
Location vs Service Type
Most users said the Saftey Exit was a “great feature”. One user believed it blended into the background and suggested it should be a button that follows you as you navigate the site [Sticky Button].
“Many services now are virtual doesn't make a lot of sense for it to be location first I would think you should be able to search by, maybe I don't care where service is located but I care about legal services. Search by the specific resource you need not just by location on the first page.”
-P1 (mobile)
Finding No6
Reviews
Participants emphasized the importance of reviews. They said without them they would be skeptical to recommend the organization to their loved one. After not seeing any reviews, they believed the site was either “too new” or “too outdated”.
Recommendations
Recommendation No1
Combine Search Bars
Consider merging the results from different kinds of search types into one.
IMPACT
May boost users’ confidence and efficiency in finding resources and service providers as well as clean the UI to reduce confusion.
Recommendation No2
New Filtering IA
Combine or bring together Service Type and “More Filters”. Use proximity principles and nested IA. This could mean organizing the filters in a way where selecting a primary filter (like “Service Type”) could lead to revealing more specific, related filters (under “More Filters”).
IMPACT
This helps in avoiding clutter and making the interface more user-friendly. Help users recognize and apply filters easily when they are more accessible.
Recommendation No3
Signify Filtering Application
When users select filters, consider displaying service tags with their filtering criteria in specific location, all of which can be unchecked.
IMPACT
This might help users understand that their criteria were successfully applied for the location they were looking at.
Recommendation No4
Navigate Map in Application
Allow users to easily plan their trip in the app by using a map with a zoom in and out feature. A quick fix could be linking the map with another navigation service like Google maps.
IMPACT
Users won’t need to leave the app. They could estimate distance, scan the safety of the area, get navigation direction with one click.
Recommendation No5
Incentivize Reviews
All participants mentioned the importance of reviews in the app. Consider a strategic approach to gather more trustworthy reviews.
IMPACT
Users will be more confident trusting the services when social proof is in place.
Recommendation No6
Offer More Sharing Options
Consider adding a modal with the most common easy to share info online to match the participant’s expectations.
IMPACT
User will have the ability to share resource in various convenient ways without the need to leave the app.
Recommendation No7
App/Web Walkthrough
Offer a training mode when users log in or use website or app for the first time post launch
IMPACT
User will have the ability to share resource in various convenient ways without the need to leave the app.
