Decide When Automated Tests Are Necessary

Quick Overview

Explain a testing philosophy that distinguishes mandatory automation from justified lighter testing and evaluates meaningful behavioral coverage beyond a percentage target.

Decide When Automated Tests Are Necessary

Company: Coinbase

Role: Software Engineer

Category: Behavioral & Leadership

Difficulty: medium

Interview Round: Onsite

## Question What is your testing philosophy? Explain when automated tests are mandatory, when a short-lived experiment or one-time script may justify lighter testing, and how you decide that a test suite covers the important behavior without relying on a coverage percentage alone. ### Constraints & Assumptions - Address code used by others or running in production, prototypes, and one-time operational scripts. - Include risk, reversibility, observability, and cost of failure in the decision. - Distinguish unit, integration, contract, property, and end-to-end tests by purpose. - “No tests” never means “no validation.” ### Clarifying Questions to Ask - Is the question about a specific codebase or general principles? - Should I discuss legacy code, release gates, or test maintenance? - Are manual verification and observability in scope? ```hint Test the contract and dangerous branches Coverage reports where execution went, not whether assertions captured the right behavior. Start with invariants, boundaries, and failure modes. ``` ### What a Strong Answer Covers - Risk-based expectations for production, shared libraries, and prototypes. - A test portfolio mapped to failure boundaries. - Boundary, error, concurrency, and regression cases. - Treatment of flaky, slow, and low-value tests. - Concrete validation for the rare case where durable automated tests are not worth keeping. ### Follow-up Questions 1. How do you test a failure that is difficult to reproduce? 2. When should a flaky test be quarantined versus fixed immediately? 3. What makes an end-to-end test worth its maintenance cost? 4. How do you add tests around legacy code safely?

Quick Answer: Explain a testing philosophy that distinguishes mandatory automation from justified lighter testing and evaluates meaningful behavioral coverage beyond a percentage target.

|Home/Behavioral & Leadership/Coinbase
Coinbase logo
Coinbase
May 12, 2026, 12:00 AM
mediumSoftware EngineerOnsiteBehavioral & Leadership
0
0

Question

What is your testing philosophy? Explain when automated tests are mandatory, when a short-lived experiment or one-time script may justify lighter testing, and how you decide that a test suite covers the important behavior without relying on a coverage percentage alone.

Constraints & Assumptions

  • Address code used by others or running in production, prototypes, and one-time operational scripts.
  • Include risk, reversibility, observability, and cost of failure in the decision.
  • Distinguish unit, integration, contract, property, and end-to-end tests by purpose.
  • “No tests” never means “no validation.”

Clarifying Questions to Ask Guidance

  • Is the question about a specific codebase or general principles?
  • Should I discuss legacy code, release gates, or test maintenance?
  • Are manual verification and observability in scope?

What a Strong Answer Covers Guidance

  • Risk-based expectations for production, shared libraries, and prototypes.
  • A test portfolio mapped to failure boundaries.
  • Boundary, error, concurrency, and regression cases.
  • Treatment of flaky, slow, and low-value tests.
  • Concrete validation for the rare case where durable automated tests are not worth keeping.

Follow-up Questions Guidance

  1. How do you test a failure that is difficult to reproduce?
  2. When should a flaky test be quarantined versus fixed immediately?
  3. What makes an end-to-end test worth its maintenance cost?
  4. How do you add tests around legacy code safely?
Loading comments...