Product improvement questions

A product improvement question in a PM interview ("How would you improve X?") asks you to find the one problem worth fixing in a product that already exists. The tempting answer is a list of features you'd add. Each might be a good idea, and the interviewer still has no way of knowing what you'd ship first, or why.

We'll use "How would you improve a public library's e-book app?" Picture a generic one: you sign in with your library card, search the collection, borrow a book for a few weeks, and join a waiting line when every copy is out. A design question starts from a blank page. Here the product exists and people are already getting stuck in it, so I start by following one of them.

Walk one reader through the app

Pick one reader on one real errand and follow them, writing down each point where they could give up. Fixes wait. My reader is Maya, who borrows print books and has just heard a friend rave about a novel. This is my picture of a typical app, so in the room I'd check it with the interviewer.

1. Signing in. The app asks for her card number and PIN, and the card is in a drawer at home. She could give up here.

2. Searching. She finds the novel: 10 copies, 120 people ahead of her, a wait of about 36 weeks. She could buy it elsewhere, or forget about it.

3. Looking for something else. Books she could read tonight sit behind a filter she doesn't notice. She closes the app.

4. Months later. An alert says her book is ready and she has three days to borrow it. She's halfway through something else, misses the window and drops to the back of the line.

5. Reading. Loans last 21 days. On a long book, the loan ends at page 300 and the book vanishes from her phone.

6. Finishing. On a short book, she reaches the last page on day 12 and closes the app. Nothing asks her to return it, so her copy sits unread for 9 more days.

Step 6 is easy to miss, because Maya doesn't give up at all. Someone in line behind her pays for it. Keep it on your list anyway.

For this step alone, take Lyft Product Improvement: Rider Journey From Ride Confirmation to Trip End, write the path, mark where someone could give up, then stop.

Who pays for this app, and what would they call better?

Maya pays nothing. The library pays, usually with money from its city or county, and the library decides whether to keep the app. So ask what the library would count as better, since that decides which of those six steps matters most.

Start with how it buys books. Many publishers license e-books to libraries for a set time, often two years, or for a set number of loans, such as 26, after which the library has to buy again. A licensed copy generally goes to one reader at a time, which is why the line exists. A shorter line usually means a bigger bill.

So I'd guess a library director wants more cardholders reading books they actually wanted, for the same licence money. Heavy readers cost money here, since every loan draws on a licence the library paid for. Subscription services meet this too: Scribd, which paid publishers when subscribers read a set share of a book, cut back its romance catalog in 2015 because its heaviest readers cost too much. So find out how the payer's costs grow with use before you rank anything.

"Readers use this app, but the library pays for it. I'll assume the library counts it as better when more cardholders get books they wanted for the same licence money, so I'll favor fixes that get more out of licences it has already bought. If the library has a different goal this year, such as new cardholders, tell me, because that changes my pick."

For either question below, write who pays and what they'd count as better, in two sentences, and stop.

How to pick one problem

Now score what you wrote down. My table has four columns, each from 1 to 3, added up. The scores are judgment calls, and that's useful: the interviewer can disagree with one cell instead of your whole answer.

Long waits win. That may look odd, since the app can't create copies, which is why it gets 2 on control and not 3. But look at what a wait is made of. 120 people ÷ 10 copies = 12 rounds of loans, and 12 × 21 days = 252 days, which is 36 weeks. Each round lasts as long as a copy stays out, including the days after its reader finished. Step 6 is part of the wait.

Be ready to defend the pick against the one you passed over:

"Sign-in stops people cold, but only new readers hit it, and the fix depends on the library's card system. If this year's goal were new cardholders, its library gain would rise to 3, while waits and the missed window would matter less, falling to 2 and 1. That gives waits 10, sign-in 8 and the window 8. Waits still win, and the tie means my table can't pick second place, so I'd let the goal decide: sign-in, since every new cardholder meets it first."

Practice this on TikTok Improvement Roadmap: Ranking Existing Features and Planning the Fixes or Improve Zoom Meetings: Pick a User Segment, Prioritize a Pain Point, Define Success. Build your own scoring table, with your own columns, and stop before any fix.

Options at very different sizes

For the chosen problem, offer fixes that differ a lot in size, so the interviewer sees you know what each one costs. The costs below are my rough guesses.

I'd start with the return prompt, because of these three it's the only one that shortens the line without new licence money, which is what I assumed the library wants. It gives the person waiting nothing they can see on day one, and I'd say so. The shelf comes next. Pay-per-loan barely reaches this problem, since the books with the longest lines are rarely offered that way, though its titles could stock the shelf, because each one can be borrowed at once.

The cheapest way to learn whether it works

Begin with data the app may already have. I'm assuming it records what page each reader is on, and I'd check that first, since library apps may limit what they record about reading. If it does, you can find out what day people reach the last page. If they get there on day 20, there's no slack, and the idea dies at no cost.

Suppose it shows (my assumptions) that 60% of loans get finished, on day 12 on average, and nearly everyone keeps the book until day 21. Now run a small test: show the prompt to a random half of readers for five weeks, and compare only loans that began in the first two weeks. Each of those has had time to end, while a shorter window would cut off loans still running, which are the long ones. If half of finishers tap it:

• 60% × 50% = 30% of loans end 9 days early.

• 30% × 9 days = 2.7 days off the average loan, so 21 − 2.7 = 18.3 days.

• 12 rounds × 18.3 days ≈ 220 days, down from 252. That's about a month shorter.

Buying the same cut would take 120 × 21 ÷ 220 ≈ 11.5 copies, about 15% more than the 10 you have, on every title with a line.

Say the number that would change your mind before you run the test, so you can't explain away a poor result afterward:

"If fewer than 1 in 5 finishers tap it, I'd drop this."

Here's why that line. 60% × 20% = 12% of loans, and 12% × 9 days ≈ 1.1 days, so a loan averages about 19.9 days. 12 rounds × 19.9 days ≈ 239 days, which takes 13 days off an eight-month wait. I wouldn't put another prompt in front of readers for that.

Then name what could go wrong, using a guardrail, a number that warns you of harm elsewhere:

• Licences run out sooner. A copy capped at a set number of loans reaches that cap faster when loans get shorter, so the library buys again sooner. I'd watch monthly licence spending. If most popular titles are licensed that way, the case for the prompt gets much weaker.

• Accidental returns. Count readers who tap return and rejoin the line for the same book within two days.

There's one more catch. Both test groups share the same lines, so the test can't show you shorter waits directly. It shows shorter loans, and you work out the wait from there. The metrics lesson covers this kind of shared-pool trap.

On Google Photos Annual Recap: Diagnose Weak Spots and Prioritize Improvements, write the one number that would change your mind before you think about anything else.

When the prompt says "pick any product"

"Pick any product and tell me how you'd improve it" lets you choose your own ground, so have a choice ready. It should be something you've used this month and seen fail, for you or someone nearby, and whose payer you can name. Smaller is often better, because on a huge product "improve it" drifts into company strategy.

I'd steer away from two picks. Your favorite product tempts you to defend it, which is what the favorite product question asks for. The interviewer's own product is risky unless you've used it for a while, since they know the reasons behind what looks broken and you don't.

Open with the reason in one line, then run the same walk:

"I'll take my city's bus app, because I got stuck in it on Sunday. Let me walk through what happened, then work out who pays for it."

The question below leaves the choice to you. Pick in ten seconds, say why, and walk one traveler through it.