User testing was done in person to record the user's screen and body language.
Venmo Usability
Testing Report
Or in other words, finding Venmo's calculator
Background
Venmo is an American mobile payment service founded in 2009 and owned by PayPal since 2012 (Venmo.com, 2021). Venmo was aimed at friends and family who wish to split bills, e.g. for movies, dinner, rent, or event tickets and etc. (DBpedia.org, 2021) The App enables you to pay and request money from your friends. At its core, Venmo provides a social way to pay your friends when you owe them money and don't want to deal with cash.
Research plan
Venmo was aimed at friends and family who wish to split bills, e.g. for movies, dinner, rent, or event tickets etc. (Brock, 2022). As splitting bills is its signature feature, I would like to explore the journey users take when they decide to pay/request and split a payment, in order to understand better the challenges and needs they face in these circumstances.
I have recently learned Venmo has a built-in calculator to allow easy bill splitting. Even though I have been using Venmo for years, I have never noticed it and always used a separate calculating app to do the splitting math. I decided to conduct a series of usability tests to discover if there is a pattern and if users are not aware of that feature, and why.
I was set to test out my hypothesis: Users are missing the calculator feature.
Research Plan
As splitting bills is its signature feature, I would like to explore the journey users take when they decide to pay/request and split a payment, in order to understand better the challenges and needs they face in these circumstances and do discover if they use the in-app calculator to do so.
Objectives
Explore the in-app calculator usage when splitting a bill and understand if and what causes users to miss out on the app's calculator. Additionally, I would like to learn about any pain points users encounter during their process.
Participants
I conducted a series of five moderated usability tests. This method will enable me to collect both qualitative and quantitative data to deepen my understanding of the digital app’s bill-splitting option.
Tasks
After screening participants for ones how used Venmo before and feel comfortable with online money transferring, I creates a task-based script that included 8 questions.
Table 1: Task completion
Task Completion Metric
Paying and requesting money had a 100% completion rate.
The lowest completion rate was for the fourth task: “Confirm payment.”
Four out of five of the users searched for their payment history (feed) to confirm payment in the ‘me’ section.
For complex tasks which required the use of a calculator, there was a 100% success rate.
Table 2: time on task
Time on task
Tasks that required calculating an amount using a calculator (tasks #3 and #6) took participants much longer to complete those two tasks (151.4 seconds & 108.4 seconds on average).
Four out of five users were unaware that the payment screen had a built-in calculator.
Three out of five users calculated the amount of money to split via a separate calculator app.
Table 3: Efficiency
Efficiency
Although task success was high, from watching the participant's actions and testimonials I suspected efficiency levels were low. Table 3 shows average efficiency rates per task. Efficiency levels were lowest for tasks #3 and #4 (13% & 11%). I ascribe that to (a) the difficulty of participants to conduct calculations since they were not aware of the app’s calculation feature (see ‘time on task’ metric 3.2) and the (b) tendency of participants to go to the app’s ‘Me’ section for payment history details (see ‘task completion’ metric 3.1).
Recommendations
Below are the main usability problems found by examining each of the three metrics. To classify each problem to a severity level, I considered first the level of importance and then the commonality of the relevant app action.
To determine the importance of app actions I relied on users’ testimonials that clearly showed ‘paying & requesting payments’ as the most important app use for them. To determine the commonality of app actions I relied on my own experience using the app, and how common was each action for me.
Transaction history
Users wrongfully looked for their payment history under the ‘Me’ section and felt disoriented when they got there. Viewing transaction history required much scrolling down and was easily missed (figure 1). Users relied on the app’s notification via email/text and could not access the notification icon through the main page. I classify this problem as a Severity 4 problem since it relates to the most critical app actions (pay & request) and from my experience, it is the most common app action.
Recommendations:
Apply the notification icon on all pages, not just on the ‘Me’ page.
Have a clear title of past transactions on the main page and the ‘Me’ section, with filters to filter payment/requests/completed.
Reduce/decrease the size of distracting elements before payment history.
In-app calculator
For an app that takes pride in its splitting bill option, users performed very poorly. Users missed the in-app calculator and had to use a native app calculator, resulting in a less efficient and frustrating flow. I classify this problem as a Severity 3 problem because it is high on importance but medium on commonality.
Recommendation:
Redesign the built-in calculator interface to have more contrast and resemble a native app calculator (have the actions on the right instead of the left, see the image below)
Two actions
Users were unaware of the ability to perform two actions at once (pay/request from two people at once). I classify this problem as a Severity 2 problem since it involves an important action with low commonality.
Recommendation:
Rethink the payment bar interface. Consider having a clear ‘+’ icon to add people to the payment bar, instead of the comma, which is small and unnoticeable. (see figure 5)
