Ranking Content for Instagram Users: Objective, Signals and Evaluation

Quick Overview

A machine learning product question asking how to design the model that ranks content on an Instagram surface such as the feed. It tests defining a ranking objective from several user actions, signals and training labels, the multi-stage pipeline, evaluation with experiments and guardrails, and ranking risks.

Ranking Content for Instagram Users: Objective, Signals and Evaluation

Company: Datadog

Role: Product Manager

Category: Machine Learning

Difficulty: medium

Interview Round: Other

You are a product manager working with the machine learning team at Instagram. Design a model that ranks content for Instagram users: what it should optimize, which signals and data it learns from, and how you would evaluate and launch it. ```hint Define a good ranking first Before picking features or an architecture, decide what a good ordering means for the viewer, for creators and for Instagram, and which user actions can stand in for it. ``` ### Clarifying Questions - Which surface is in scope: the main feed, Stories, Reels or Explore? Each draws candidates from a different pool, from accounts the user follows to accounts they have never seen. - Should the model optimize long-term engagement, time spent, user satisfaction or creator outcomes, and which integrity rules are fixed constraints? - How deep should the answer go on model architecture compared with the product decisions around it? - Is there a latency or compute budget for ranking one feed request? ### What a Strong Answer Covers - A chosen surface and an objective that combines several predicted actions, with reasoning for the weights - The ranking pipeline: candidate generation, a light first-stage ranker, a heavy ranker and a re-ranking pass for diversity, freshness and integrity - Signals about the viewer, the post, the author and their relationship, and how training labels are built - Offline metrics, online experiments, and long-term and creator-side guardrails - Risks such as engagement bait, feedback loops, cold start for new users and creators, and serving latency ### Follow-up Questions - Time spent rose in your experiment but survey satisfaction fell. What do you ship, and how do you change the objective? - How would you make sure a new creator's first posts get a fair chance to be seen? - What changes if the surface is Explore, where the viewer follows none of the candidate authors?

Overview: A machine learning product question asking how to design the model that ranks content on an Instagram surface such as the feed. It tests defining a ranking objective from several user actions, signals and training labels, the multi-stage pipeline, evaluation with experiments and guardrails, and ranking risks.

Datadog logo
Datadog
Aug 30, 2025
mediumProduct ManagerOtherMachine Learning
0
0

You are a product manager working with the machine learning team at Instagram. Design a model that ranks content for Instagram users: what it should optimize, which signals and data it learns from, and how you would evaluate and launch it.

Clarifying Questions Guidance

  • Which surface is in scope: the main feed, Stories, Reels or Explore? Each draws candidates from a different pool, from accounts the user follows to accounts they have never seen.
  • Should the model optimize long-term engagement, time spent, user satisfaction or creator outcomes, and which integrity rules are fixed constraints?
  • How deep should the answer go on model architecture compared with the product decisions around it?
  • Is there a latency or compute budget for ranking one feed request?

What a Strong Answer Covers Guidance

  • A chosen surface and an objective that combines several predicted actions, with reasoning for the weights
  • The ranking pipeline: candidate generation, a light first-stage ranker, a heavy ranker and a re-ranking pass for diversity, freshness and integrity
  • Signals about the viewer, the post, the author and their relationship, and how training labels are built
  • Offline metrics, online experiments, and long-term and creator-side guardrails
  • Risks such as engagement bait, feedback loops, cold start for new users and creators, and serving latency

Follow-up Questions Guidance

  • Time spent rose in your experiment but survey satisfaction fell. What do you ship, and how do you change the objective?
  • How would you make sure a new creator's first posts get a fair chance to be seen?
  • What changes if the surface is Explore, where the viewer follows none of the candidate authors?