Decide When Automated Tests Are Necessary
Company: Coinbase
Role: Software Engineer
Category: Behavioral & Leadership
Difficulty: medium
Interview Round: Onsite
## Question
What is your testing philosophy? Explain when automated tests are mandatory, when a short-lived experiment or one-time script may justify lighter testing, and how you decide that a test suite covers the important behavior without relying on a coverage percentage alone.
### Constraints & Assumptions
- Address code used by others or running in production, prototypes, and one-time operational scripts.
- Include risk, reversibility, observability, and cost of failure in the decision.
- Distinguish unit, integration, contract, property, and end-to-end tests by purpose.
- “No tests” never means “no validation.”
### Clarifying Questions to Ask
- Is the question about a specific codebase or general principles?
- Should I discuss legacy code, release gates, or test maintenance?
- Are manual verification and observability in scope?
```hint Test the contract and dangerous branches
Coverage reports where execution went, not whether assertions captured the right behavior. Start with invariants, boundaries, and failure modes.
```
### What a Strong Answer Covers
- Risk-based expectations for production, shared libraries, and prototypes.
- A test portfolio mapped to failure boundaries.
- Boundary, error, concurrency, and regression cases.
- Treatment of flaky, slow, and low-value tests.
- Concrete validation for the rare case where durable automated tests are not worth keeping.
### Follow-up Questions
1. How do you test a failure that is difficult to reproduce?
2. When should a flaky test be quarantined versus fixed immediately?
3. What makes an end-to-end test worth its maintenance cost?
4. How do you add tests around legacy code safely?
Quick Answer: Explain a testing philosophy that distinguishes mandatory automation from justified lighter testing and evaluates meaningful behavioral coverage beyond a percentage target.