For many organizations, Product Analytics still means something fairly specific.
You have a website or an app. You track what people do on it. You measure conversion, engagement, retention, feature usage, funnels and drop-off. You give product managers the data they need to decide what to build next.
That model made sense when the product was mostly the digital interface.
I am not sure it makes as much sense anymore.
As AI becomes part of products, the product is increasingly built from many different sources of information. A customer’s experience may depend on what they did on the website, what they bought in a store, what they asked customer service, what they received by email, what the company knows about them, what an AI model predicts about them and what another system decides to show them.
The website is still part of the product.
But it is no longer necessarily where the product lives.
And that creates a bigger question:
If the product increasingly brings different data sources together to create an experience, why would Product Analytics still be organized around one data source?
We may be organizing Product Analytics around the wrong thing
Think about how many organizations are structured today.
Web analytics sits somewhere.
Product analytics sits somewhere else.
Marketing analytics sits somewhere else.
Customer analytics sits somewhere else.
Retail or store analytics sits somewhere else.
CRM data sits somewhere else.
Data science sits somewhere else.
Each function can become very good at understanding its own part of the organization.
But the customer does not experience the organization in parts.
A customer does not think:
“I am now interacting with the website analytics part of the company.”
They experience one company.
They may see an advertisement, visit the website, open an email, talk to customer service, visit a store, make a purchase and then receive a recommendation from an AI system.
From the customer’s perspective, these are not six different data domains.
They are six parts of the same experience.
This creates an interesting mismatch.
The organization may be structured around the sources of data while the product is increasingly created from the relationships between those sources.
That distinction becomes much more important with AI.
AI changes what we mean by “the product”
Before AI, it was relatively easy to point at a product.
It was the website.
Or the mobile application.
Or the physical product.
Or the software.
Analytics could therefore follow the thing fairly easily.
But imagine a product where an AI system decides what a customer sees based on their previous behavior.
Now the product is partly the interface.
It is partly the recommendation model.
It is partly the customer data used by the model.
It is partly the business rules.
It is partly the feedback loop created by previous interactions.
And it may even depend on information coming from outside the original product experience.
The product has become less like a thing and more like a system.
That changes Product Analytics.
If the product is a system, Product Analytics cannot only study the surface of the system.
It needs to understand what is happening underneath it.
The website may become the visible part of a much larger product
Consider a simple example.
A customer visits an ecommerce website.
They see a personalized recommendation.
The recommendation is based on previous purchases, browsing behavior, customer profile information, inventory availability and an AI model.
The customer clicks on the recommendation and buys the product.
Where did the product experience happen?
On the website?
Partly.
But if you only analyze the website, you might conclude that the recommendation performed well.
You know that the customer saw it.
You know that they clicked.
You know that they purchased.
But you may not know why the recommendation was made.
You may not know whether the customer had already been considering the product.
You may not know whether the recommendation worked differently for new and existing customers.
You may not know whether inventory availability influenced the recommendation.
You may not know whether the model is systematically recommending products to people who would have purchased them anyway.
The website analytics can tell you what happened at the interface.
It cannot necessarily explain the product.
And this is where the role of Product Analytics starts to become much more interesting.
Product Analytics should follow the product question, not the data source
This may be the more useful way to think about the function.
Instead of asking:
“Who owns website analytics?”
Ask:
“Who understands how the product behaves across the different systems that create the customer experience?”
That is a different question.
The first question is about ownership of a data source.
The second is about understanding a system.
And the second question becomes much more important as products become more intelligent.
If I want to know whether personalization is working, I may need website behavior, purchase history, customer attributes, marketing interactions, inventory information and model outputs.
If I want to understand why a customer churned, I may need product usage, transactions, customer service interactions, pricing changes, communications and perhaps qualitative feedback.
If I want to know whether an AI assistant is improving the customer experience, I may need to connect conversations with downstream behavior, purchases, satisfaction and repeat usage.
The question determines the data.
Not the other way around.
This does not mean Product Analytics should own all analytics
There is an important distinction here.
If Product Analytics becomes responsible for every piece of data in the company, it quickly becomes an impossible team.
That is not the point.
Marketing should still understand marketing.
Finance should still understand finance.
Operations should still understand operations.
Customer service should still understand customer service.
And domain teams should continue to own the data and knowledge that belong specifically to their domain.
The problem appears when the organization needs to understand something that crosses those boundaries.
That is where a different capability is needed.
Product Analytics could become the function responsible for understanding the connections between the domains when those connections shape the product experience.
That is a very different role from being the team that owns website dashboards.
The team would be organized around questions, not channels
This also changes how the team itself might be designed.
Today, you might naturally create teams such as:
- Web Analytics
- Marketing Analytics
- Customer Analytics
- Product Analytics
- Data Science
But perhaps the future organization needs another dimension.
For example:
Customer journey and behavior
Understanding how people move across the product and the organization.
Product experience
Understanding whether changes to the product actually improve outcomes.
Experimentation and learning
Understanding what happens when the organization changes something and whether the change caused the outcome.
AI and decision analytics
Understanding how models, recommendations and automated decisions affect customers and the business.
Cross-domain product intelligence
Connecting the different data domains when the answer cannot be found inside one of them.
The important point is that these are capabilities, not data sources.
The team is organized around the questions the organization needs to answer.
There is a deeper reason for this
There is something else happening here that is easy to miss.
When data was separated, organizations could afford to think of analytics as a collection of specialists.
Someone understood web traffic.
Someone understood customers.
Someone understood marketing.
Someone understood sales.
Someone understood operations.
The organization could then combine those perspectives when necessary.
But AI changes the economics of combining information.
AI can increasingly bring information from different sources together and reason across it.
That means the scarce capability may no longer be simply the ability to analyze one dataset.
It may be the ability to understand which datasets need to be connected in the first place.
That is a different skill.
It is closer to understanding the system than understanding the table.
The more connected the product becomes, the less useful isolated analytics becomes
This creates an interesting contradiction.
The organization may invest more heavily in analytics at exactly the moment when analytics teams become more fragmented.
Every domain gets better data.
Every team gets better dashboards.
Every function gets better AI tools.
And yet the organization may still have difficulty answering a relatively simple question:
“What is actually happening to the customer?”
Because the answer is distributed across all of those systems.
You can have excellent analytics everywhere and still have poor understanding of the product.
That may be one of the biggest analytics problems organizations will face in an AI environment.
More analytics does not necessarily create a better understanding of the product.
Sometimes it creates more pieces of understanding that nobody is responsible for putting together.
The real product analytics problem may become the relationship between data
This is where I think the role becomes particularly interesting.
We have spent a lot of time thinking about data quality.
Is the website data accurate?
Is the customer data accurate?
Is the transaction data accurate?
Are the AI model outputs accurate?
Those questions are still important.
But there is another question:
Do we understand how these things relate to each other?
A customer viewed a product.
Then they received an email.
Then they visited a store.
Then they spoke to customer service.
Then an AI model recommended something.
Then they purchased.
Each individual system may record its part correctly.
But the organization may still not understand the sequence.
And without the sequence, it is difficult to understand causality.
This is why simply putting all the data into a warehouse does not automatically solve the problem.
Having the data is not the same as understanding the relationships between the data.
A large repository can give the organization enormous memory without giving it better understanding.
This also changes what success looks like for Product Analytics
If Product Analytics is primarily website analytics, success can be relatively easy to measure.
How many dashboards?
How many users?
How quickly can we answer requests?
How accurate are our reports?
How many product decisions use our analysis?
But if the function is supposed to help the organization understand the product as a system, the measures need to change.
A more interesting set of questions would be:
What questions can we answer today that we could not answer before?
What relationships between customer, product and business behavior do we now understand?
What assumptions have we disproven?
Which experiments have changed what we believe?
What questions did Product Analytics cause the organization to start asking?
And perhaps the most important one:
If the Product Analytics team disappeared tomorrow, what would the organization no longer know how to see?
That last question changes the definition of the function.
The value is no longer only in the analysis it produces.
It is in the understanding it leaves behind.
Maybe Product Analytics becomes a horizontal capability
I do not think the answer is necessarily to create one enormous centralized Product Analytics team.
It may actually be the opposite.
The organization could have analysts embedded in different domains while maintaining a horizontal Product Analytics capability that connects them.
The domain teams understand their part of the system.
The horizontal function understands the relationships between the parts.
Data Engineering makes the data available.
Data Science builds models.
Product teams make changes.
But Product Analytics helps answer a different question:
How do all of these things come together to create the product experience and its outcomes?
That is closer to being the organization’s product intelligence function.
And it is a much bigger role than website analytics.
The unit of analysis is changing
Maybe that is ultimately the shift we need to make.
We have traditionally asked:
What happened on the website?
Then:
What happened in the product?
Increasingly, we may need to ask:
What happened in the system that creates the product experience?
That system can include the website.
It can include the app.
It can include stores.
It can include customer service.
It can include marketing.
It can include transactions.
It can include operational data.
And increasingly, it can include AI models making decisions inside the experience.
The product is becoming the result of all of these things interacting.
So perhaps Product Analytics should not be defined by the place where the customer clicks.
It should be defined by the product question we are trying to understand.
And that leads to a much more interesting definition:
Product Analytics is not the analytics of a product surface. It is the capability to understand how the different parts of an organization come together to create, change and influence the product experience.
That is why I think the role of Product Analytics is likely to become more important, not less, in an AI organization.
AI can connect more data.
AI can produce more analysis.
AI can answer more questions.
But someone still needs to understand which pieces belong together, what the organization is actually trying to learn, and what the answer means for the product.
The more intelligent the product becomes, the less useful it may be to have analytics teams organized around the individual systems that happen to produce its data.
The product is becoming a system.
Perhaps the analytics function needs to become one too.
Leave a Reply