Measuring user engagement

This is part 2 in a series of articles about measuring Key Experience Indicators (KEIs). In this series I go deeper into the Google HEART framework for large-scale data analysis. The framework was put in place to help choose and define appropriate metrics that reflect both the quality of user experience and the goals of your product. Each article in the series discusses one of the HEART dimensions — Happiness, Engagement, Adoption, Retention, and Task success. Enjoy and use it!

What is engagement?

In the context of products and services, engagement is the level of user involvement with a product. The term normally refers to the frequency, intensity, or depth of interaction between user and product, feature, or service over a given period of time. User engagement is an unbiased behavioral measurement and is therefore trustworthy, valid, and reliable.

Why measure engagement?

Understanding people’s engagement behavior toward a thing (feature, service, process, etc.) is revealing how much that thing is meeting a true human need and how much people are hooked into using it. Different engagement metrics can reveal not just frequency of usage, but also depth and volume of usage that can be more indicative of great value, product, and user experience.

Key mistakes in measuring engagement

Only measuring overall engagement. While overall product engagement is an extremely useful metric to track, it is a business rather than experience metric. Compare, “Our overall 7-day active went down from 64% in June to 48% last month” to “7-day active for transacting with us remained flat at 23% while time between submitting product reviews per user went down 27% last month compared to June.” The former is interesting yet not extremely actionable, while the latter is specific, clear, and driving action.

Reporting total count. When you report a total count of users who use the product, that number grows usally due to having more users, not more usage. Although many people and product teams are interested in this number, it is a vanity metric that doesn’t say much about engagement, product health, or growth. It is generally more useful to report engagement metrics as an average per user, rather than…