Testing a New Feature: Planning Test Cases and Handling Flaky Tests

You own quality for a team, and a new feature is about to enter testing. Explain how you test it and how you plan its test cases, then explain how you deal with flaky tests in the team's automated suite. If no specific feature is given, pick a concrete one and use it throughout your answer. ### Clarifying Questions - What kind of product and feature is it (a web UI, an API, software running on network devices), and is there a written requirement or only a verbal description? - What test infrastructure already exists: developer unit tests, an automated API or UI suite, CI on every change? - What is the release cadence, and how costly is a defect that reaches customers? ### Part 1 — Testing a new feature and planning its test cases Walk through how you test the feature, from the moment you see its requirements until it ships: what you do before any code exists, how you derive and prioritize test cases, which levels of testing you use, and how you decide it is ready to release. ```hint Start before the code Think about what you can review, and which questions you can raise, when only the requirement or the design exists. ``` ```hint Not every case is equal You cannot run every combination of inputs and states. Think about how you choose which cases to write first and which ones to automate. ``` #### What This Part Should Cover - Requirement analysis, with testability questions raised early - Systematic test-design techniques, including negative cases, boundaries and state changes - Prioritization by risk, and the choice of test level for each case - Entry and exit criteria, and how release readiness is decided ### Part 2 — Handling flaky tests Some automated tests pass and fail intermittently without any code change. How do you find them, diagnose them, fix them, and keep them from eroding the team's trust in the suite? ```hint Prove it is flaky Before fixing anything, consider how you would show that a failure is intermittent rather than a real regression. ``` ```hint What differs between two runs List what a test depends on that can be different between two runs of exactly the same code. ``` #### Clarifying Questions for this Part - How large is the suite, how often does it run, and does CI already retry failed tests automatically? - Who owns a failing test today: QA, or the developers whose code it covers? #### What This Part Should Cover - Detection and confirmation of flakiness using repeated runs and failure history - The main root-cause categories and how to diagnose each - Fixes that remove the nondeterminism rather than hide it - A process for quarantine, ownership and measurement ### What a Strong Answer Covers - A risk-based plan rather than an exhaustive list of cases - A deliberate split between automated checks and exploratory testing - Collaboration with developers and product on requirements and on ownership of flaky tests - Measurable signals, such as escaped defects and flake rate, used to improve the process ### Follow-up Questions - The release date is fixed and you only have time for a third of your planned cases. Which do you run? - A test fails once in about 200 runs. Is it worth fixing, and how do you decide? - How would you test a feature whose behavior depends on time, such as a task that becomes overdue after its deadline? - How do you know whether your test suite is effective, not just large?