Build a Self-Service Market Data Discovery and Support Layer
Company: Jain Global
Role: Data Engineer
Category: Software Engineering Fundamentals
Difficulty: medium
Interview Round: Technical Screen
## Build a Self-Service Market Data Discovery and Support Layer
An analyst asks where to find Bloomberg company identifiers and the associated figures needed for an analysis. The organization has vendor feeds and several internal access paths, but no mature data catalog. Design both the immediate support process and an incremental self-service layer that serves analysts and developers without encouraging direct use of unvalidated raw vendor data.
### Constraints & Assumptions
- Available interfaces may include Excel, SQL, APIs, and an internal data platform.
- Vendor licensing and user entitlements can differ by dataset and use case.
- Refreshes can fail or arrive late, and consumers need actionable status.
- Lineage must connect an exposed figure to its vendor source and internal transformations.
### Clarifying Questions to Ask
- Which Bloomberg identifier namespace and which business figures does the analyst mean?
- What as-of date, frequency, and level of aggregation are required?
- Is the work interactive exploration, a repeatable report, or an application dependency?
- May the requested data be stored, transformed, exported, or shared with the intended audience?
### Part 1 — Resolve the Immediate Discovery Request
Explain how you would locate the identifier mapping and figures when no reliable catalog exists. Include the evidence you would gather before recommending a dataset.
#### What This Part Should Cover
- Clarification of identifier namespace, field semantics, grain, and freshness.
- Search across schemas, ingestion jobs, query history, code, owners, and vendor documentation.
- Validation with sample keys and lineage rather than choosing a table by name.
- A durable record of the discovered owner, contract, and access path.
```hint Follow one figure end to end
Trace a sample company identifier and figure backward to its delivery and forward to a known consumer before certifying the dataset.
```
### Part 2 — Match Interfaces to Consumer Personas
Choose appropriate access paths for an analyst doing exploration and a developer building a maintained integration. Explain when Excel, SQL, an API, or the platform is the right interface and when it is not.
#### What This Part Should Cover
- Persona-specific workflows without creating different definitions of the data.
- Stable semantic contracts shared across interfaces.
- Guardrails for large extracts, automation, credentials, and schema evolution.
- A migration path from ad hoc exploration to a production dependency.
```hint Share semantics, vary the interface
Excel, SQL, and API users can consume one governed product without receiving identical interaction patterns.
```
### Part 3 — Communicate Refresh Failures
Design monitoring and alerts for failed or late refreshes. Show what a consumer sees and what the owning team receives.
#### What This Part Should Cover
- Dataset-specific expected schedules and freshness states.
- Alerts with affected partitions, last good version, impact, owner, and runbook.
- Suppression or aggregation of duplicate symptoms from one root failure.
- A machine-readable status that downstream jobs can enforce.
```hint Report the usable version
A useful refresh alert says which version remains valid and which consumers are affected, not only that a job failed.
```
### Part 4 — Enforce Licensing, Entitlements, and Lineage
Explain how self-service users discover and request governed products while raw vendor deliveries remain restricted. Include how licensing, entitlement decisions, and lineage are presented.
#### What This Part Should Cover
- Entitlements enforced consistently across Excel, SQL, API, and platform access.
- License metadata that describes permitted uses without exposing confidential contract text.
- Certified products separated from raw or quarantined feeds.
- Field-level or dataset-level lineage from vendor delivery to exposed figure.
```hint Separate discovery from authorization
A user may be allowed to learn that a dataset exists without being allowed to read its values.
```
### What a Strong Answer Covers
- A practical support workflow that produces evidence and improves future discovery.
- One governed data product with multiple appropriate interfaces, not competing copies.
- Useful freshness communication and a clear ownership model.
- Self-service that accelerates valid use while preserving licensing and quality controls.
### Follow-up Questions
1. How would you respond if the only working copy is an analyst-owned spreadsheet?
2. What should an API return while the latest refresh is quarantined but the previous version is valid?
3. How can a user inspect lineage without receiving access to the raw vendor payload?
4. Which signals would tell you that a discovered dataset is not safe for production use?
Quick Answer: Design a self-service market-data discovery layer that helps analysts find company identifiers and figures while serving Excel, SQL, API, and platform users safely. Explore catalog gaps, refresh alerts, licensing, entitlements, lineage, and controls around raw vendor data.