An Evaluation of Best Buys

Project Overview

Best Buys is a white label product that works with news publishers, to create content around reviewing products and services. The goal of the product is to provide end-users with enough information for them to use our affiliated links to purchase the product. The commission generated from the affiliated links is then split between Global Savings Group (GSG) and the publisher.

During the start of my role as a Product Designer for GSG, I got stuck in research to understand a product user base. Building empathy between yourself and the end-user gives me a greater understanding to help support my design decisions.

The Challenge/Problem

Due to Best Buys being a reasonably new product, it hasn't had the opportunity to be studied as a whole. Previously research has only been done at a feature level. While this approach works for the majority when you are trying to build out a product road map, not taking a step back and seeing the product as a whole can limit your insights for improvement.

What we wanted to find out

To understand how Best Buys is being interpreted by our users, I planned out a user interview study that focuses on four key parts;

• Exploration - This is a stage within a users purchase journey that focuses on the gathering information to purchase a product or service. With Best Buys, we wanted to understand how the interface, content and reading experience affected the "Exploration" stage.
• Memory Recall - Is our layout optimise for users to recall information. As users would use Best Buys to find information to purchase a product, we wanted to find out if we had an optimal layout for end-users to rememeber key information.
• Key Information - Another stage a user goes through during the purchase journey is the "Evaluation" stage. This is when users have narrowed down what product/service they potentially want to buy and are now trying to decide which to purchase. We want to find out if there is a piece of common information that a user needs to purchase a product/service.
• Emotional Impact - We additionally wanted to investigate what users felt pre and post-purchase to help improve our current and future features.

Testing Plan

The testing plan detailed the session into two groups. The first (Group A) focused on users who were are the start of the purchase journey. Users who are in the Exploration stage, and were mainly going onto multiple websites to gain as much info as possible. The second group (Group B) focused on users who were towards the end of the purchase journey and were about to buy a product and were doing some final reading before purchasing.  

Group A

Group A had their session broken into three parts.  

User Interview (40 Minutes) - The moderator will interview the participant, asking questions about them and potentially their buying habits.

Browsing Task (10 Minutes) - The users will be asked to select an article of interest. They will be questioned on why they picked the article, what they expect before reading and their thoughts after reading.  

Memory Test (10 Minutes) - The user will be asked to go through an article. Users will then be asked to answer questions about the article. We will test them on what they were able to remember about the content.

Group B

While Group B had there session broken into two sections.

User Interview (40 Minutes) - The moderator will interview the participant about a recent purchase that required research around it, including their motivations behind it and how they found the whole process.

Pre-Purchase Research Task (20 Minutes) - The user will have one task around gathering information from Best Buys to get to a point of purchase. A second stage interview will occur, asking about how they found the process, what, if anything, was lacking in regards to information and if they could get to the stage of purchase from Best Buys itself.

Recruiting Users

For each group, 5 separate users where recruited via UserInterviews.com. The participants were chosen by narrowing down on key demographics from age, income, devices and if they use our or our competitor's offering. With the latter being the most important. We wanted to speak to users who uses already naturally use review article websites.

Running the User Interviews

The interviews were run over two weeks, with each session taking up to an hour to record. The sessions were recorded remotely on Zoom. In each session, I was the moderator for the session, with another colleague taking notes.

‍

Analysing

Once all tests have been run, I then uploaded each recording into DoveTail. This was our research repository for GSG, which had the useful feature of creating a transcript of the recording.

I then took the transcript and then ran a thematic analysis to pull out key informations and to see key patterns between each participant. With each interview section being 40 minutes, you can imagine a lot of useful insights were generated.

‍

Furthermore, I took Groups A browsing tasks and analysed if what the user thought before clicking on an article matched with their thoughts post reading the article. This discussion could also be analysed with thematic analysis. While the memory test, was marked to see if they got the answer right after reading the article.

With Group B, I took the user's thoughts about how they found navigating the website to find information and again analysed what they said by pulling key quotes and assigning a theme.

‍

Results

I won’t detail everything that was discovered during this research session, however three key results that where discovered is the following;

‍

Detail of Content/Language Used - With Best Buys being a content-led product, content will be the first thing users talk about. Multiple users (if not all), mentioned the level of details being provided was not enough. This greatly impacted the users in the "Exploration" stage as when trying to gather data to support their purchase journey, the information from Best Buy was too basic. A powerful quote that summarises this pain point comes from participant 6, " I haven't learned much from this description because, because there's so much text I'm skimming it and what I'm seeing when I'm skimming, it is not the technical details I'm looking for. It's marketing speak, which lowers my level of trust in the article in general."

Hard to Compare Products - Due to the length of articles and how much text was written, users found it difficult to compare products within the same articles. Again this affected users in the "Exploration" stage as it was hard for users to be guided to a product that would fit there needs.

Memory Recall - We tested what a user could re-call after reading an article. Users struggle to recall basic information, from what the article was called to fundamental specifications about products recommended. This was mainly due to a pain point of having too much text. Users would be daunted by the amount of information on a page and start to skim reading. The issue would come when they skip out on key parts of the article. Making it harder to remember what was written.

To see how these issues where resolved, why not read the following case studies?