Measuring task success

This is part 5 in a series of articles about measuring Key Experience Indicators (KEIs). In this series I go deeper into the Google HEART framework for large-scale data analysis. The framework was put in place to help choose and define appropriate metrics that reflect both the quality of user experience and the goals of your product. Each article in the series discusses one of the HEART dimensions — Happiness, Engagement, Adoption, Retention, and Task success. Enjoy and use it!

What is task success?

Task success goes back to the definition of traditional usability and include two primary components — user effectiveness and efficiency. When measuring task success, there has to be a known task assigned to users. This means that looking at analytics data or analyzing usage logs is not going to be helpful here since you have no idea what users want to accomplish. When you create a situation where you know what the user is trying to do, then KEIs that inform you about time-on-task (efficiency) and task completion rates (effectiveness) are extremely helpful. You can create these situations using user research tools such as UserZoom.

Why measure task success?

Task success metrics give you priceless indications about the users’ experience. A lot of individuals and companies speak highly about user experience yet don’t bother measuring their users’ effectiveness and efficiency. When you and your team become aware of whether or not your users achieve their goals when they use the product, you have the luxury of changing it for the better.

Key mistakes in measuring task success

Only measuring during moderated usability tests. Moderated usability tests are a great way to lean about strengths and weaknesses of your design or product. There’s nothing more useful to acquire that knowldge than what happens during the interaction between a moderator and a research participant. However, if your goal is to reach large-scale data analysis and measure overall task success, that’s not something that can be achieved in moderated usability tests. The reason is that the small number of participants in these tests are not scalable. To scale, you need hundereds of participants using your product in an unmoderated (yet controlled)…