Measuring Whether Organizational-Health Insights Drive Action for B2B Customers
Company: eBay
Role: Product Manager
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Other
A B2B product gives its customers data insights about the health of their own organization. How would you measure whether those insights are successful?
```hint Seeing is not using
An insight that is opened but never changes a decision has not done its job. Trace the path from someone at the customer looking at an insight to their organization doing something differently, and ask what you could observe at each step.
```
### Clarifying Questions
- What does "organization health" cover here: financial results, operations, workforce and engagement, or a mix the customer configures?
- How are the insights delivered: a dashboard people log into, scheduled reports and alerts, or panels embedded in a product the customer already uses?
- Are the insights sold as a paid product of their own, or included to make a broader platform more valuable?
- Who at the customer is expected to act on them: executives, line-of-business managers, or analysts who brief both?
### What a Strong Answer Covers
- Why the customer buys the insights and why the provider offers them, with the buyer distinguished from the day-to-day users
- A chain of metrics from data setup and reach through engagement and customer action to a change in the organization's health
- A north-star metric that captures insights being acted on, measured at the account level from product data
- Trust and quality signals (data accuracy, freshness, load time) alongside account outcomes such as renewal and expansion
- How to attribute an improvement in a customer's health to the insights rather than to everything else going on
### Follow-up Questions
- Usage of the insights is rising, but renewals among the accounts that use them are falling. How would you find out why?
- How would you show a customer's executive team that the insights improved their organization's health?
- An insight is accurate but delivers news an executive does not want to hear, and they rate it poorly. How do you keep that from distorting your metrics?
Overview: Asks how to measure the success of data insights about organizational health that a B2B product gives its customers. Tests separating buyers from daily users, building metrics from trusted data and adoption through customer action to renewal, and attributing changes in a customer's health to the insights.