People are seeing observability as a separate subject from product making. I see it as two side of the same "does this thing work" coin. To a product person, does it work mean are people using the tool to solve their problem. To engineer, it means the tool is solving the problem correctly. The former is answered by various analytics tool while the latter is answered by QA and observability tools.
IMO this is doubling effort. Given all the logging,metrics and traces, can we not tell how good the product is performing? Given we know how a feature is used by users, all the clicks and APIs, can we not guaranteed future changes not breaking a user journey? I'm at the moment prototyping a tool to capture my thinking in product and software development. Having OtEL is making my idea so much easier to realise
One major issue to reconcile is the granularity of data and retention. For engineers incentivized to fix problems, they need extremely high granularity, which is expensive to store long-term. Product Managers typically care more about aggregations that they want to track over a long period of time. In theory you could source these concerns from the same place, but it's challenging to do it, since you'd need to process, store, and query that data in different places. This doesn't make it impossible, and I think people should try it - but it may be too hard for a lot of organizations.
I served both role as PM and engineer. Having high level metrics is great when things are going smooth. You will need to drill down, zoom in into user journey when things don't work as expected. E.g. increased drop-off, 404s, 400s. Another aspect is causality, given a result, what caused it? Is it because a deployment, is it because a marketing campaign, or is it just seasonality. As a product person, they should want the ability to zoom in into data. Right now the only option is through communication.
As pointed out by sister comment, developer need to know how their code is impacting their business. I would even argue developers are inherently a product person. You are always building for someone, be it another developer, internal users of you company, or general public. You have to know how people are using your tool. Hence as a developer you should want to be able to zoom out from whatever observability platform is currently offering.
Guess what, developer thinks product managers are stupid because all they know is either looking at data and measuring the wrong thing, or chasing for status update. Product managers thinks developer are nerds that only know punching on keyboard and not talking to the users. They are both right and wrong. But we create the chasm ourselves. There are alternative ways of building things.
> People are seeing observability as a separate subject from product making. I see it as two side of the same "does this thing work" coin.
I have noticed the same thing with performance and UX. Most SaaS vendors only care about client / frontend UX and neglect the backend aspect of it. After a few years of intense development, they own good looking software that is also mindbogglingly slow and in some extreme cases unusable.
Mixing up these two concerns is a bad idea. Your product manager now has to convince engineers to implement the metrics they want. Data that is irrelevant to engineers now needs dragging along to places inside the system so that events are useful to both engineers who car about data and latency and product managers who care about individual user behaviour.
So while there is indeed overlap, the two types of metrics involve different people, different teams, different concerns, different data, different privacy concerns, and different reliability/uptime concerns.
All of your arguments are great, for why you need to treat observability as a core product feature and tie it to core product features.
Engineers need to be serving the orgs goals, and one of the best ways to do that is to give them incentive and tooling to be thinking about the same things the product owner is. Have them build metrics for business metrics. Those are the things you should be alerting on, and to the extent you alert on technical pieces it should be in the service of those business metrics; don't alert on latency because it's an arbitrary goal, alert on latency because you can see how it affects user experience.
Structure follows strategy. There are various reasons product manager exist but the role doesn't exist unless the team grows to a certain scale. All founders serve as a product manager some point in their start up journey because they need to know if the thing the built works. There are companies wanting to spend energy on building things that work and skipping a layer of communication is one way of doing it.
That's also how compiler speed up a program sometime, by inline expansion or by loop unrolling. It doesn't work all the time. But as an option.
Right now there seem to be an orthodox in product development
IMO this is doubling effort. Given all the logging,metrics and traces, can we not tell how good the product is performing? Given we know how a feature is used by users, all the clicks and APIs, can we not guaranteed future changes not breaking a user journey? I'm at the moment prototyping a tool to capture my thinking in product and software development. Having OtEL is making my idea so much easier to realise