Mercor Machine Learning Engineer Interview Experience — Probability Phone Screen on PII Detection

I'm sharing my Mercor MLE phone screen, the probability round. I don't think anyone has posted this one here yet. Good luck to everyone landing offers soon.

Part 1

We anonymize customer datasets by replacing every real name with a fake one. The first step is finding them: a detector goes through a document token by token, flagging whatever looks like PII.

Take a document with 100 PII tokens and 100,000 tokens that aren't. The detector catches all 100, and it also flags 1% of the rest.

Question 1. Pick one of its flags at random. How likely is it to be real PII?

Question 2. We add a second detector, just as good as the first, and only replace a token when both flag it. Assume the two make their mistakes independently. Pick a token flagged by both at random. How likely is it to be real PII?

Part 2

Documents in a delivery vary enormously in length. To check how much PII we missed, we pick a token at random and review whatever document it came from.

Question 1. Is that document longer than, the same length as, or shorter than the average document in the delivery?

Part 3

Assume finding the names is solved: every name that needs replacing has been located, and each real name always gets the same fake, so one company never turns into three.

We have a list of V fake names. For each of the n real names in a dataset, the pipeline picks one from the list uniformly at random without checking whether that fake has been used already. Define a collision as any pair of real names that end up sharing the same fake name.

Question. On average, how many collisions should we expect in a dataset of n names?