PracHub
QuestionsLearningGuidesInterview Prep
|Home/Analytics & Experimentation/Attentive

Investigate SMS delivery-rate drop at Attentive

Last updated: Mar 29, 2026

Quick Overview

This question evaluates diagnostic analytics and incident‑triage competencies, including delivery‑metric definition, carrier error‑code interpretation, hypothesis prioritization, experiment design, and hierarchical statistical attribution relevant to messaging systems.

  • hard
  • Attentive
  • Analytics & Experimentation
  • Data Scientist

Investigate SMS delivery-rate drop at Attentive

Company: Attentive

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Technical Screen

Attentive’s US SMS delivery rate (Delivered/Attempted) drops by 6.5 percentage points starting March 10, 20:00–22:00 ET, more pronounced on two major carriers; CA/UK unaffected. A new link-shortener domain and updated 10DLC registrations rolled out March 9. Build a triage and validation plan: 1) Define exact delivery, acceptance, and failure metrics using carrier error codes; include guardrails (CTR, unsubscribe, complaint rate). 2) List the precise slices you’ll run: by carrier, sender type (short code/toll-free/10DLC), campaign/template, message length/character set, URL domain, vertical (SHAFT-sensitive), send-time, client, region, and new-vs-existing senders. 3) Describe artifact checks (duplicate sends, clock skew/timezone, logging gaps, retry policy changes, throughput throttles, MPS caps, carrier queue backoffs). 4) Prioritize hypotheses and how you’d test each: (a) carrier filtering—use error-code mix shift, acceptance vs delivery delta; (b) link-domain reputation—A/B old vs new domain with stratification by carrier and sender; (c) 10DLC registration/brand-score issues—audit TCR status and vetting outcomes; (d) content/template changes—token-level diff and spam-score models. 5) Specify the experiment design (randomization unit, sample sizing/MDE for delivery-rate uplift, sequential monitoring, holdouts), and the statistical model you’d use for attribution (hierarchical logistic regression controlling for carrier×sender×template×hour). 6) Provide an action and rollback plan, escalation criteria to carriers, and what evidence would be sufficient to declare the incident resolved.

Quick Answer: This question evaluates diagnostic analytics and incident‑triage competencies, including delivery‑metric definition, carrier error‑code interpretation, hypothesis prioritization, experiment design, and hierarchical statistical attribution relevant to messaging systems.

Related Interview Questions

  • Diagnose March Uber ride-volume drop - Attentive (hard)
  • Interpret Results and Address Multiple Testing Concerns - Attentive (medium)
  • Apply Multiple Testing Corrections for Valid Results Analysis - Attentive (medium)
|Home/Analytics & Experimentation/Attentive

Investigate SMS delivery-rate drop at Attentive

Attentive logo
Attentive
Oct 13, 2025, 9:49 PM
hardData ScientistTechnical ScreenAnalytics & Experimentation
2
0

Triage and Validation Plan: US SMS Delivery-Rate Drop

Context

On March 10 from 20:00–22:00 ET, the US SMS delivery rate (Delivered/Attempted) dropped by 6.5 percentage points, with larger impacts on two major US carriers. Canada and the UK are unaffected. On March 9, a new link‑shortener domain was rolled out and 10DLC registrations were updated.

Build a triage and validation plan that:

  1. Defines delivery, acceptance, and failure metrics using carrier error codes; and specifies guardrails (CTR, unsubscribe, complaint rate).
  2. Lists the precise analysis slices: by carrier, sender type (short code/toll‑free/10DLC), campaign/template, message length/character set, URL domain, vertical (SHAFT‑sensitive), send‑time, client, region, and new‑vs‑existing senders.
  3. Describes artifact checks: duplicate sends, clock skew/timezone, logging gaps, retry policy changes, throughput throttles, MPS caps, carrier queue backoffs.
  4. Prioritizes hypotheses and how to test each: (a) carrier filtering—use error‑code mix shift and acceptance vs delivery delta; (b) link‑domain reputation—A/B old vs new domain stratified by carrier and sender; (c) 10DLC registration/brand‑score issues—audit TCR status and vetting outcomes; (d) content/template changes—token‑level diff and spam‑score models.
  5. Specifies the experiment design: randomization unit, sample sizing/MDE for delivery‑rate uplift, sequential monitoring, holdouts; and the statistical model for attribution (hierarchical logistic regression controlling for carrier×sender×template×hour).
  6. Provides an action and rollback plan, escalation criteria to carriers, and evidence sufficient to declare the incident resolved.
Loading comments...

Browse More Questions

More Analytics & Experimentation•More Attentive•More Data Scientist•Attentive Data Scientist•Attentive Analytics & Experimentation•Data Scientist Analytics & Experimentation

Write your answer

Your first approved answer each day earns 20 XP.

Sign in to write your answer.
PracHub

Master your tech interviews with 9,000+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • AI Coding Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.