Jan 27, 2018
Continuous user research in 11.6 seconds
Amazon releases new production code once every 11.6 seconds.
15 years ago I worked in a software technology company that released its primary product once a year on a CD that was sent to customers worldwide over snail mail. User research has traditionally been “released” in a similar manner following these steps:
- Initiation: A researcher or someone else becomes interested in research.
- Planning: The researcher comes up with research questions, methodology, participant criteria, schedule, and discussion guide.
- Recruiting: The researcher or a recruiter finds participants for the research study and schedules with them.
- Data collection: The researcher (and usually a couple of other team members or client representatives) observe research participants in their environment (home, office, street, car, etc.) and/or interview them.
- Synthesis & analysis: The researcher (alone or with others) reaches conclusions about what was found and what he or she recommends as next steps.
- Reporting: The researcher prepares a report that usually takes the form of a document or slide deck, and shares it with the team (or client) in a formal presentation.
- Follow-up: The researcher keeps in touch with the team (or client) to make sure recommendations are implemented in the product or service.
Okay, so what’s the problem with that?
I call these “dedicated studies.” These are research studies that are put in place to answer very specific research questions such as, “Why are our customers adding items to the cart, but not checking out?” or “Which products or features are the least understood?” The challenge is that these studies will only and always answer the questions they are designed to answer. They will almost never answer questions you didn’t know you should ask, reveal hidden truths, and help uncover opportunities to innovate.
Another challenge with traditional, dedicated research studies introduces itself when somebody important in the organization suddenly asks, “What do we know about [x]?” If there was no previous dedicated study done on x, there is no answer to the question, and a researcher would go through the lengthy seven-step process described above.
The way we do research has fundamentally changed
Amazon calls it continuous delivery. Can your company continuously deliver research once every 11.6 seconds?
Continuous user research is fast-rhythm research that is open-ended in nature. It’s not dedicated to any specific topic and it’s not research that anyone asks for.
For example, in each of the last three companies I worked (Check Point Software Technologies, Google, and WeWork), my teams were interviewing customers once a week, every two-weeks, or monthly with guided, but very open-ended, conversations. We usually asked customers to show and tell us how they used the product, what was working well for them, and what challenges they were facing. They controlled the conversation. They decided what they wanted to talk about. Not us. Not the team. Not our stakeholders. This is user-centered research.
Why is that so powerful?
Continuous user research brings precious value:
- Immediate answers to most research questions.
- Quick answers to specific questions (by directing continuous interviews to include specific discussion topics).
- Increased buy-in for research among stakeholders and executives (from academic and slow to practical and nimble).
Can you do it?
Yes, your company can definitely deliver research once every 11.6 seconds. Even faster. You may even have some of the makings of continuous research already.
- An open definition of the user. In some organizations, Marketing works off customer segments, growth teams use Salesforce accounts, Product groups define cohorts, and UX develops personas. Continuous research relies on a transparent distributed definition of who the user is. Incongruities between the concept of the user between functional units creates inefficiencies and barriers to continuous research. If you ask people from different parts of the organization, “Who is the user?” and they give equivalent answers or point you to a single source of truth, you are on the right track.
- Staying out of the building. Steve Blank (serial entrepreneur and educator) is known for saying that one of the keys to startup success is getting out of the building. For continuous research, you want people out of the building and in the field at all times. Research should never sleep. Spending a day as a new hire working “the front lines” or listening to support calls every once in a while is not sufficient. Continuous research relies on trained specialists who know how to see people and make their experience visible through high-quality, impactful user research.
- Distributed access to insights. Rather than reports living in folders or meetings, insights need to be structured, atomized, and accessible for self-discovery. Several companies have versions of this such as Mailchimp’s database, Nasdaq’s Mosaiq, and WeWork’s Polaris, or some kind of agreement or system for storage on Dropbox or Drive. Whatever the system you use, understanding what the organization knows about a certain aspect of the user experience should be as easy as finding a product on Amazon. The quality of this system will determine the efficacy of continuous research in impacting change.
Do you implement continuous research in your organization? Would it be beneficial?
If you liked this article, check out:
Benjamin Gadbaw and I created the atomic research approach and Polaris product and framework.
Related articles
Key Experience Indicators: How to decide what to measure?
The three most popular questions about Atomic Research
Foundations of atomic research
Measuring the WeWork Member Experience
Democratizing UX
Deciding what to work on
The atomic unit of a research insight
