Venmo Usability
Testing Report

Or in other words, finding Venmo's calculator

Background

Venmo is an American mobile payment service founded in 2009 and owned by PayPal since 2012 (Venmo.com, 2021). Venmo was aimed at friends and family who wish to split bills, e.g. for movies, dinner, rent, or event tickets and etc. (DBpedia.org, 2021) The App enables you to pay and request money from your friends. At its core, Venmo provides a social way to pay your friends when you owe them money and don't want to deal with cash.

Research plan

Venmo was aimed at friends and family who wish to split bills, e.g. for movies, dinner, rent, or event tickets etc. (Brock, 2022). As splitting bills is its signature feature, I would like to explore the journey users take when they decide to pay/request and split a payment, in order to understand better the challenges and needs they face in these circumstances. 

I have recently learned Venmo has a built-in calculator to allow easy bill splitting. Even though I have been using Venmo for years, I have never noticed it and always used a separate calculating app to do the splitting math. I decided to conduct a series of usability tests to discover if there is a pattern and if users are not aware of that feature, and why.

I was set to test out my hypothesis: Users are missing the calculator feature.

Research Plan

As splitting bills is its signature feature, I would like to explore the journey users take when they decide to pay/request and split a payment, in order to understand better the challenges and needs they face in these circumstances and do discover if they use the in-app calculator to do so.

Objectives

Explore the in-app calculator usage when splitting a bill and understand if and what causes users to miss out on the app's calculator. Additionally, I would like to learn about any pain points users encounter during their process.

Participants

I conducted a series of five moderated usability tests. This method will enable me to collect both qualitative and quantitative data to deepen my understanding of the digital app’s bill-splitting option. 

Venmo Heuristic Evaluation

User testing was done in person to record the user's screen and body language.

Tasks

After screening participants for ones how used Venmo before and feel comfortable with online money transferring, I creates a task-based script that included 8 questions.

Venmo Heuristic Evaluation

Table 1: Task completion

Task Completion Metric

  • Paying and requesting money had a 100% completion rate.

  • The lowest completion rate was for the fourth task: “Confirm payment.”

  •  Four out of five of the users searched for their payment history (feed) to confirm payment in the ‘me’ section.

  • For complex tasks which required the use of a calculator, there was a 100% success rate. 

Venmo Heuristic Evaluation

Table 2: time on task

Time on task

  • Tasks that required calculating an amount using a calculator (tasks #3 and #6) took participants much longer to complete those two tasks (151.4 seconds & 108.4 seconds on average).

  • Four out of five users were unaware that the payment screen had a built-in calculator.

  • Three out of five users calculated the amount of money to split via a separate calculator app.

Venmo Heuristic Evaluation

Table 3: Efficiency

Efficiency

Although task success was high, from watching the participant's actions and testimonials I suspected efficiency levels were low. Table 3 shows average efficiency rates per task. Efficiency levels were lowest for tasks #3 and #4 (13% & 11%). I ascribe that to (a) the difficulty of participants to conduct calculations since they were not aware of the app’s calculation feature (see ‘time on task’ metric 3.2) and the (b) tendency of participants to go to the app’s ‘Me’ section for payment history details (see ‘task completion’ metric 3.1).

Recommendations

Below are the main usability problems found by examining each of the three metrics. To classify each problem to a severity level, I considered first the level of importance and then the commonality of the relevant app action. 

To determine the importance of app actions I relied on users’ testimonials that clearly showed ‘paying & requesting payments’ as the most important app use for them. To determine the commonality of app actions I relied on my own experience using the app, and how common was each action for me.

Transaction history

Users wrongfully looked for their payment history under the ‘Me’ section and felt disoriented when they got there. Viewing transaction history required much scrolling down and was easily missed (figure 1). Users relied on the app’s notification via email/text and could not access the notification icon through the main page. I classify this problem as a Severity 4 problem since it relates to the most critical app actions (pay & request) and from my experience, it is the most common app action.

Recommendations:

  • Apply the notification icon on all pages, not just on the ‘Me’ page.

  • Have a clear title of past transactions on the main page and the ‘Me’ section, with filters to filter payment/requests/completed.

  • Reduce/decrease the size of distracting elements before payment history.

In-app calculator

For an app that takes pride in its splitting bill option, users performed very poorly. Users missed the in-app calculator and had to use a native app calculator, resulting in a less efficient and frustrating flow. I classify this problem as a Severity 3 problem because it is high on importance but medium on commonality. 

Recommendation:
Redesign the built-in calculator interface to have more contrast and resemble a native app calculator (have the actions on the right instead of the left, see the image below)

Two actions

Users were unaware of the ability to perform two actions at once (pay/request from two people at once). I classify this problem as a Severity 2 problem since it involves an important action with low commonality.

Recommendation:
Rethink the payment bar interface. Consider having a clear ‘+’ icon to add people to the payment bar, instead of the comma, which is small and unnoticeable. (see figure 5)