Knowing that product data analysis is very important, but it is quite likely that at some point in our professional careers we have made mistakes. And it’s normal. We are humans.
In this post, we have identified 5 common errors and, with them, possible ways to solve them. We are going to break down all these points, intending to create a product data analysis strategy that serves both researchers in their tasks and product managers when making decisions.
Not measuring actual engagement
When we work on a product, there are many metrics at our fingertips. The number of new users or product downloads are two of them that receive a lot of attention in the analysis of product data. However, these two metrics essentially serve to know the degree of adoption, but not the actual use of our product. If we want to measure engagement, we must look at other metrics that refer to the actual use of our product. What are those metrics? Those that talk about frequency (number of pages per session) or actions (who advance in a conversion funnel) will be much more useful to measure those interactions with the product.
Forget qualitative data
If looking at the correct metrics is important, choosing our sources of the information correctly is no less so. Talking with those who use our product is essential and without that feedback, we will not be able to know what our strengths and weaknesses are. For this reason, it is important to keep communication channels open: support, surveys or in-depth interviews will help us understand how the product is used. Is it more expensive than setting up a dashboard? Of course. But the data that we can obtain from those conversations can be fundamental to understanding the real why of everything we see in those metric panels. Feedback is very useful, even after our product is launched. For example, some industrial equipment has manuals that should be easy for every user to understand. We can take the Atlas Copco GA 37 VSD Compressor as an example. This manual is easy for users to understand and of course their positive feedback is very useful for future product development.
Let ourselves be carried away by the feedback
Although feedback is important, it is no less important to know how to differentiate it. In this sense, there are different strategies to be able to use it and take advantage of the best of everything that those who use our product tell us. The first thing to keep in mind is that, just as we are not users of our product, those who provide us with the most opinions and feature requests may not represent our ideal audience. All this information will be valuable, obviously, but it can be subject to biases of different kinds (they come from another application, it is not the main use of the product). For this reason, in our research strategy, we must look for formulas to activate those who do not expressly offer their opinion, but whose use of the product is more in line with the standards.
Only look at metrics when they drop
When a critical business metric drops, it’s probably too late. We can anticipate a possible decrease in metrics if we relate it to others. A clear example is when there are increases in subscription cancellation rates or churn rates. In addition to external factors such as new competitors, we can anticipate those cancellations through metrics such as logins or usage rates of some features. It is so important to look at the metrics that allow us to correctly understand the usage rates of our product, as we mentioned at the beginning since on many occasions they will allow us to anticipate adverse circumstances.
Measure too much data
Not by measuring much, we will better understand how our product behaves. Looking at too much data can lead us to not be able to make decisions or reach wrong conclusions.
We always have to determine what we want to measure and design a research strategy in which we access the data that allows us to understand behaviors and draw the appropriate conclusions. This also affects product managers, since measuring too many parameters can make them lose focus on what is important, even affecting the product roadmap.
These five errors are easily correctable: with the proper planning of both research and product strategies, we will be able to analyze the data we need so that all our data analysis is correct and, with it, obtain actionable conclusions to take our product.