PracHub
QuestionsPremiumCoachesLearningGuidesInterview Prep
|Home/Analytics & Experimentation/Attentive

Investigate SMS delivery-rate drop at Attentive

Last updated: Mar 29, 2026

Quick Overview

This question evaluates diagnostic analytics and incident‑triage competencies, including delivery‑metric definition, carrier error‑code interpretation, hypothesis prioritization, experiment design, and hierarchical statistical attribution relevant to messaging systems.

  • hard
  • Attentive
  • Analytics & Experimentation
  • Data Scientist

Investigate SMS delivery-rate drop at Attentive

Company: Attentive

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Technical Screen

Attentive’s US SMS delivery rate (Delivered/Attempted) drops by 6.5 percentage points starting March 10, 20:00–22:00 ET, more pronounced on two major carriers; CA/UK unaffected. A new link-shortener domain and updated 10DLC registrations rolled out March 9. Build a triage and validation plan: 1) Define exact delivery, acceptance, and failure metrics using carrier error codes; include guardrails (CTR, unsubscribe, complaint rate). 2) List the precise slices you’ll run: by carrier, sender type (short code/toll-free/10DLC), campaign/template, message length/character set, URL domain, vertical (SHAFT-sensitive), send-time, client, region, and new-vs-existing senders. 3) Describe artifact checks (duplicate sends, clock skew/timezone, logging gaps, retry policy changes, throughput throttles, MPS caps, carrier queue backoffs). 4) Prioritize hypotheses and how you’d test each: (a) carrier filtering—use error-code mix shift, acceptance vs delivery delta; (b) link-domain reputation—A/B old vs new domain with stratification by carrier and sender; (c) 10DLC registration/brand-score issues—audit TCR status and vetting outcomes; (d) content/template changes—token-level diff and spam-score models. 5) Specify the experiment design (randomization unit, sample sizing/MDE for delivery-rate uplift, sequential monitoring, holdouts), and the statistical model you’d use for attribution (hierarchical logistic regression controlling for carrier×sender×template×hour). 6) Provide an action and rollback plan, escalation criteria to carriers, and what evidence would be sufficient to declare the incident resolved.

Quick Answer: This question evaluates diagnostic analytics and incident‑triage competencies, including delivery‑metric definition, carrier error‑code interpretation, hypothesis prioritization, experiment design, and hierarchical statistical attribution relevant to messaging systems.

Related Interview Questions

  • Diagnose March Uber ride-volume drop - Attentive (hard)
  • Interpret Results and Address Multiple Testing Concerns - Attentive (medium)
  • Apply Multiple Testing Corrections for Valid Results Analysis - Attentive (medium)
Attentive logo
Attentive
Oct 13, 2025, 9:49 PM
Data Scientist
Technical Screen
Analytics & Experimentation
2
0

Triage and Validation Plan: US SMS Delivery-Rate Drop

Context

On March 10 from 20:00–22:00 ET, the US SMS delivery rate (Delivered/Attempted) dropped by 6.5 percentage points, with larger impacts on two major US carriers. Canada and the UK are unaffected. On March 9, a new link‑shortener domain was rolled out and 10DLC registrations were updated.

Build a triage and validation plan that:

  1. Defines delivery, acceptance, and failure metrics using carrier error codes; and specifies guardrails (CTR, unsubscribe, complaint rate).
  2. Lists the precise analysis slices: by carrier, sender type (short code/toll‑free/10DLC), campaign/template, message length/character set, URL domain, vertical (SHAFT‑sensitive), send‑time, client, region, and new‑vs‑existing senders.
  3. Describes artifact checks: duplicate sends, clock skew/timezone, logging gaps, retry policy changes, throughput throttles, MPS caps, carrier queue backoffs.
  4. Prioritizes hypotheses and how to test each: (a) carrier filtering—use error‑code mix shift and acceptance vs delivery delta; (b) link‑domain reputation—A/B old vs new domain stratified by carrier and sender; (c) 10DLC registration/brand‑score issues—audit TCR status and vetting outcomes; (d) content/template changes—token‑level diff and spam‑score models.
  5. Specifies the experiment design: randomization unit, sample sizing/MDE for delivery‑rate uplift, sequential monitoring, holdouts; and the statistical model for attribution (hierarchical logistic regression controlling for carrier×sender×template×hour).
  6. Provides an action and rollback plan, escalation criteria to carriers, and evidence sufficient to declare the incident resolved.

Solution

Show

Submit Your Answer to Earn 20XP

Sign in to leave a comment

Loading comments...

Browse More Questions

More Analytics & Experimentation•More Attentive•More Data Scientist•Attentive Data Scientist•Attentive Analytics & Experimentation•Data Scientist Analytics & Experimentation
PracHub

Master your tech interviews with 8,000+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.