Episode 5: A Day in the Life of a Product Manager Analyzing Data to Make Decisions
+ 6 Steps to Start Analyzing Data
If you are new to product management or involved in day-to-day operations, knowing how to analyze data to make informed decisions is crucial.
When organizations claim to be "data-driven" or use data for decision-making, what does that actually mean?
Today, I want to shed light on a typical scenario a product manager might encounter.
Example
Imagine you are a product manager at a pet supplies eCommerce company, focusing on delivering exceptional value to your customers and business. Noticing that competitors offer push notifications for order statuses, you begin to wonder if introducing a similar feature could enhance customer experience.
First step:
Understand the current experience. How many pages does it take to get order status information? How many clicks or taps? How many people currently visit the site for this purpose? Also, consider how many of those visiting for order status end up shopping for additional products.
Findings:
10,000 orders a month
1,000 unique customers go through the existing order status flow
500 of these customers proceed to create a cart and make another purchase
7 total pages to navigate to
8 steps
45 total taps (or clicks if on the website)
You notice the current process involves numerous pages, taps, and steps, which customers have indicated they find cumbersome.
Next, conduct a Harvey Ball analysis of the competition to understand the competitive landscape better.
Harvey Ball Analysis
Key Findings:
Our process requires the most steps in the industry to find order status.
All competitors have order status accessible directly from the homepage.
All competitors, except one, send push notifications when an order is delivered, but not for earlier statuses.
This analysis reveals clear opportunities for improvement.
What should this product manager prioritize first?
To decide, analyze aspects of the order status retrieval flow, such as app visits with successful authentication, navigation to the order status list, and specific order pages, and the percentage of visits that result in further shopping actions.
Let’s build a formula to discover how this work could impact our king KPI of Revenue
What aspects of the order status retrieval flow can we identify?
A.Visits to the app with successful authentication per month
B.Visits with successful navigation to the order status list page
C.Visits with successful navigation to the specific order page to view status
D.% of visits that go on to specific order page
E.% visits that viewed order status, added items to cart, and visited cart
F.% visits to cart that go on to purchase (Cart Conversion)
G.Revenue
H.AOV
Remember these variables; will use them below!
Assuming static numbers for average order value (AOV), cart conversion, and app visits, you can calculate the impact of viewing a specific order page and the subsequent shopping actions.
Now that we have some of the formula’s, time to get those numbers out of your analytic system (Adobe, Google Analytics, FullStory, etc)
Hold On! This is NOT incremental revenue, more about this in a moment
Incremental Revenue – the additional revenue generated by introducing new products or iterations to existing products. This additional revenue is above and beyond the revenue that would have been generated if no changes had been made.
To get incremental revenue, we need to get a sense of how much we can impact visits to the order status page.
If we launch push notifications for order status, how will this impact the visits that view a specific order?
Slice and dice the data:
What else could we look at?
1.Are people also checking order status on the website?
Yes! An additional 2,000 customers check status on the website; they don’t have the app.
2.Could we market the new push notifications there to drive more people to the app?
Yes! Previous marketing promotions to drive app downloads have driven 25% of customers from web to app.
3.How does AOV differ from desktop to app?
AOV is $50 more on app than desktop.
It’s important to define the KPIs, collect and analyze the data, visualize the data, draw insights and conclusions, and make data-driven decisions that can improve the performance of the product.
By driving more customers to the mobile app by marketing the push notification feature, plus the higher AOV, you can generate $8,000 incremental revenue from this feature per month by enjoying the higher AOV from customers that switched from desktop to mobile.
In addition, for existing customers, you are having them enter the app more often by sending them valuable push notifications. This in turn would generate $15,000 revenue from the feature per month.
A total of $23,000 of incremental revenue
By analyzing additional data points, like order status checks on the desktop and potential app conversion rates, you can forecast the impact of new features like push notifications on customer engagement and revenue.
Defining KPIs, collecting and analyzing data, visualizing findings, drawing insights, and making data-driven decisions are critical steps in enhancing product performance.
Prioritizing actions that offer the highest business return or customer experience improvement, while considering effort levels, ensures agile and effective product management.
Now that you have a sense of the business impact this feature could have; you now need to put together a pitch deck (templates, examples, and approaches about how to do this in the Product Protégé Guide: The Art of Product Management for eCommerce and Beyond) so you can get stakeholder feedback and then ultimately the level of effort to build this from your engineering and UX teams.
Ultimately that will get you a sense of the return on investment for this feature that you can then compare against the other features on your backlog so you can properly prioritize the features!
Here are 6 Steps to Start Analyzing Data
As a product person, finding trends in data (metrics, king KPIs, etc., for example, for an e-commerce app: Revenue, Site Conversion, Average Order Value, Visits, and Overall Engagement is crucial for making data-driven decisions that can improve the performance of the product.
Step 1 – Define KPIs:
Start by defining the primary KPIs for the product, such as Revenue, Site Conversion, Average Order Value, and Visits we’ve discussed. These KPIs should be measurable and aligned with the product’s goals. Then, define second- and third-level KPIs that are relevant to the primary KPIs. For example, if the primary KPI is Revenue, the second-level KPI could be the Number of Visits to Cart Without an Order, and the third-level KPI could be the Out-of-Stock scenarios for users in a given timeframe
Step 2 – Collect Data:
Once the KPIs are defined, collect data for each. This can be done using tools such as Adobe, Google Analytics or analytics built into the platform. Have a high standard for the data ensuring its accuracy, completeness, and reliability. Don’t have the data? Work with development team and/or analytics team to see what options you have to start tagging the data.
Step 3 – Analyze the Data:
Use data analysis tools to analyze the data and identify trends in the KPIs. Look for patterns and correlations between the KPIs. For example, if the site conversion rate is decreasing, check to see if there is a correlation with the number of visits or the average order value. Check with stakeholders who may have a different perspective on the situation.
Step 4 – Visualize the Data:
Create visualizations such as charts and graphs to help communicate the trends in the data. This will make it easier to identify patterns and insights.
Step 5 – Draw Insights and Conclusions:
Use the data and visualizations to draw insights and conclusions about the performance of the product. For example, if the revenue is increasing but the site conversion rate is decreasing, it may indicate that the product is attracting more visitors, but the user experience is not optimized for conversion.
Step 6 – Make Data-Driven Decisions:
Use the insights and conclusions drawn from the data to make data-driven decisions that can improve the performance of the product. For example, if the site conversion rate is decreasing, it may be necessary to optimize the user experience, such as improving the checkout process or simplifying the navigation prior to creating new features. Biggest return on investment for the business and/or return on experience for the customer should be prioritized to the top as long as the level of effort makes sense in the context of agility.
Enjoy examples like this where we deep dive into the tactical work of a product manager? Please let me know! Your feedback is a gift and can help ensure I’m giving you the most valuable content weekly.
Until next week!
