Calculate Probability of Honest and Relevant Chatbot Answers
Company: Meta
Role: Data Scientist
Category: Statistics & Math
Difficulty: easy
Interview Round: Onsite
##### Scenario
Evaluating a customer-service chatbot: P(honest answer)=0.7, P(relevant answer)=0.8.
##### Question
What is the probability that an answer is both honest and relevant if the two events are independent? Given logs of 1,000 answers, how many would you expect to be neither honest nor relevant? Describe how you would run a hypothesis test to compare two LLMs’ relevance rates at α=0.05.
##### Hints
Basic probability rules; two-proportion z-test setup and interpretation.
Quick Answer: This question evaluates understanding of basic probability (independence and joint event probabilities), expectation calculations for sample counts, and the design and interpretation of hypothesis tests for comparing models.