DataAnnotation · Software Engineer
Updated · 2026-10-02

DataAnnotation Software Engineer
Interview Guide

THE 60-SECOND BRIEF

DataAnnotation hires Software Engineers; this guide collects what candidates report about the process.

This guide is scoped to a Software Engineer candidate at DataAnnotation.

DataAnnotation candidates report 4 rounds over 3-5 weeks. The stages below are what candidates describe, not a published process.

PythonSQLEvaluation of AI model outputs (response ranking)

25 min read

Browse Software Engineer questions

See the practice prompts

11Practice promptsAcross five skill areas

DataAnnotation hires Software Engineers; this guide collects what candidates report about the process.

01

Profile Creation

reported

Submit background details, programming languages, and portfolio links.

What to demonstrate

  • Submit background details, programming languages, and portfolio links
  • Depth in Python

How to prepare

  • Work Python until you can explain it without notes
  • Work SQL until you can explain it without notes
DataAnnotation Software Engineer candidate reports ↗
02

Starter Coding Assessment

reported

Complete an untimed coding assessment in your local environment.

What to demonstrate

  • Complete an untimed coding assessment in your local environment
  • Depth in Python

How to prepare

  • Work Python until you can explain it without notes
  • Work SQL until you can explain it without notes
DataAnnotation Software Engineer candidate reports ↗
03

Specialized Onboarding Modules

reported

Access paid onboarding modules including coding qualifications and training.

What to demonstrate

  • Access paid onboarding modules including coding qualifications and training
  • Depth in Python

How to prepare

  • Work Python until you can explain it without notes
  • Work SQL until you can explain it without notes
DataAnnotation Software Engineer candidate reports ↗
04

Advanced Qualifications

reported

Pass advanced qualifications to unlock access to high-paying projects.

What to demonstrate

  • Pass advanced qualifications to unlock access to high-paying projects
  • Depth in Python

How to prepare

  • Work Python until you can explain it without notes
  • Work SQL until you can explain it without notes
DataAnnotation Software Engineer candidate reports ↗

PracHub editorial advice for the preparation topics above.

01

Prioritize Written Explanations

Never rush your written justifications. A correct piece of code with a vague, single-sentence explanation will often be rejected. Explain why you chose a specific data structure and how your logic processes the input step-by-step.

02

Double-Check Your Output

Treat the output specifications like a strict contract. If a problem asks you to output a parsed secret message, verify that there are no trailing whitespaces, incorrect line breaks, or missing characters.

03

Read Instructions Completely Before Starting

Qualification tests often contain detailed instructions outlining prohibited approaches, required libraries, or formatting constraints. Read the document twice before writing a single line of code.

04

DataAnnotation enforces a strict zero-tolerance policy regarding AI-assisted plagiarism

Submitting AI-generated responses for your evaluations or code assessments will result in immediate termination from the platform.

Choose a category, try a prompt, then open its approach, worked solution or follow-up when you need it.

8 technical prompts0 include a worked solution

Diff a projection against the primary without per-row point reads

hard
reconciliationrange hashingthrottling

The listing projection has drifted and some rows show a stale version. The primary holds 40,000,000 resource rows across 12,000 tenants while serving 1,200 writes and 14,000 reads per second. The obvious repair, reading each resource row and comparing its version against the projection, is correct and would eventually finish. Explain precisely why it is unacceptable here, then give a diff that finds the differing rows, state its complexity, and make it safe to run against a live primary. Replication lag is usually under 100 ms and is not bounded.

Approach
  1. Quantify the naive cost rather than calling it slow: 40,000,000 point reads at even 0.5 ms each is over five hours serialised, and the only lever is concurrency, which is exactly what you cannot spend. The primary's pool is sized for the write path, and 40,000,000 random reads evict the buffer cache that sustains the 85 percent cache hit rate, so the audit degrades the system it is auditing.
  2. Replace random access with one ordered pass per side. Both sides can be read in (tenant_id, resource_id) order, which is a sequential scan on each and a merge join in O(n) time and O(1) memory. For a dense diff that is the whole answer, and it reads the primary once instead of 40,000,000 times.
  3. For the expected sparse case, compare range hashes instead of rows: partition the key space, compute per range an order-independent aggregate over hash(resource_id, version), compare aggregates, and descend only into ranges that differ. With d differing rows and branching factor B, at most d ranges mismatch per level, so the drill-down examines O(d log_B(n/d)) ranges and reads full rows only in mismatching leaves.
  4. Aggregate with a sum modulo 2^64 or a multiset hash, never XOR. XOR is order-independent but self-cancelling, so two rows wrong in the same way, or a row duplicated on one side, leave the range aggregate matching and the range is declared clean.
Follow-up
  • The diff reports 900 stale rows. How do you decide between patching those rows and rebuilding the projection from resource_revision?
  • Same job, but the projection lives in a search index that cannot be scanned in key order. What changes?

Merge partitioned event streams into one ordered feed with bounded lateness

hard
k-way mergewatermarksout-of-order streams

The read-model service consumes 64 log partitions carrying about 4,000 events per second in total. Each partition is ordered within itself, but partitions drift by up to 30 seconds, and the activity feed must present a tenant's events in occurred_at order. Produce the merge. State its complexity, the buffer it requires in events and in bytes, what happens when one partition is idle, and what you do with an event that arrives after you have already emitted its position. Payloads average 1 KB.

Approach
  1. Merge with a min-heap over the 64 partition heads keyed on (occurred_at, event_id): O(log P) per event and O(n log P) overall. The tie-break on event_id is what makes the output deterministic when two partitions carry the same millisecond, which matters because the feed is paginated and a non-deterministic order reorders pages under the reader.
  2. Emitting the heap head is only correct once every partition has produced everything up to that timestamp, so the emit condition is a watermark: the minimum across partitions of the highest occurred_at seen, less the allowed lateness. Events are held until the watermark passes them, which is what turns individually ordered streams into a jointly ordered one.
  3. Size the buffer from the lateness rather than guessing: 4,000 events per second times 30 seconds is 120,000 buffered events, and at 1 KB each about 120 MB of heap. That number is the real price of the ordering guarantee and belongs in front of whoever asked for it.
  4. Handle the idle partition explicitly, because it fails the feed rather than corrupting it: a partition with no traffic never advances its own maximum, so the watermark freezes and output stops entirely. Either every partition emits a periodic idle marker carrying the broker's current time, or the watermark falls back to wall clock for a partition silent beyond a threshold.
Follow-up
  • The lateness budget is raised to five minutes. What is the new buffer, and what besides memory changes?
  • The consumer restarts. Where does it resume from, and what does the feed look like for the first 30 seconds?

Archive a resource graph without breaking live references or recursing

medium
graph traversaltopological ordertenant isolation

Resources reference other resources within a tenant; for the largest tenant the reference table holds up to 2,000,000 nodes and 8,000,000 edges. Archiving a resource must archive everything reachable from it that nothing outside the set still references, refuse when a live external referrer exists, and terminate when references form cycles, which they legitimately do. Produce the archive order and the refusal list, targeting O(V+E). Say what stops the traversal crossing a tenant boundary, and why recursion is the wrong control structure at this size.

Approach
  1. Load the subgraph with the tenant predicate on both endpoints of the edge, not only on the side you started from. Scoping the left table alone is the classic cross-tenant leak: one mis-entered edge then pulls another tenant's resources into the traversal and, worse, into the archive.
  2. Traverse iteratively with an explicit stack. A 2,000,000-node graph can hold a chain deep enough to exhaust a native stack in the low tens of thousands of frames, and that failure is a process crash rather than an error you can return.
  3. Treat cycles as data rather than corruption: compute strongly connected components with Tarjan in O(V+E) using its own explicit stack, then condense. The condensation is a DAG, so a topological order over it gives the archive order, and every member of a component archives in one transaction because no order within a cycle is valid.
  4. Decide refusals with reverse edges. A candidate is archivable only if every in-edge originates inside the candidate set, so build the transpose or count in-degrees restricted to the visited set, and emit each blocked resource with the id of the external referrer, which is the only part of the answer an operator can act on.
Follow-up
  • The graph is read in one query and the archive writes a minute later. What can change in between, and how do you make the write safe?
  • The candidate set is 400,000 resources. Is that one transaction, and if not, what does a half-finished archive look like to a reader?

Built from the rounds and topics DataAnnotation candidates report.

Small steps. Visible outcomes.0 / 7 completed
ONE WEEK · YOUR PACE

Prepare, practise & reflect

One practical outcome each day. Spend longer where you need it.

0 / 7 done
01Map the DataAnnotation loop
  • Write out the reported sequence: Profile Creation, Starter Coding Assessment, Specialized Onboarding Modules, Advanced Qualifications.
  • For each round, write one sentence on what it is judging, from the description above, and mark the one you are least ready for.

Deliverable: A one-page map of the 4 reported rounds, with the weakest marked.

02Work Python
  • Spend the session on Python, which DataAnnotation candidates report being tested on.
  • Write one worked example in Python and time yourself on it.

Deliverable: One timed worked example in Python.

03Work SQL
  • Spend the session on SQL, which DataAnnotation candidates report being tested on.
  • Write one worked example in SQL and time yourself on it.

Deliverable: One timed worked example in SQL.

04Work Evaluation of AI model outputs (response ranking)
  • Spend the session on Evaluation of AI model outputs (response ranking), which DataAnnotation candidates report being tested on.
  • Write one worked example in Evaluation of AI model outputs (response ranking) and time yourself on it.

Deliverable: One timed worked example in Evaluation of AI model outputs (response ranking).

05Consolidate
  • Re-work the problem you got wrong earliest in the week, from scratch, without looking at your previous attempt.

Deliverable: A second, cleaner solution to the problem you got wrong first.

06Rehearse your own examples
  • Prepare three examples from your own work where you made the decision, each with the outcome you can quantify.

Deliverable: Three examples written out, each with a number attached.

07Dry run for DataAnnotation
  • Run one full mock under time, then write down the two questions you most want to ask your interviewers.

Deliverable: A completed timed mock and two questions to ask.

Expand any day for tasks and deliverables. Your progress is saved on this device.

Behavioural rounds judge the decision you made and what it cost.

Turn a code review disagreement into a decision

easy
code reviewoptimistic concurrencycommunication

A colleague's change updates a row with UPDATE resource SET version = version + 1 WHERE resource_id = $1 AND version = $2 and treats an affected-row count of zero as a successful no-op. You read that as a silently lost update; they think returning 200 is friendlier to clients than returning a conflict. Describe how you have handled a review disagreement of this shape: what goes in the comment, when you leave the thread, and who decides. Then write the comment you would leave here, in under 80 words.

Approach
  1. Sort the disagreement before writing anything. A silently discarded write is a correctness claim about data; the choice between 409 and 412 is taste. Only the first justifies blocking a merge, and saying which one you are doing is most of the value of the comment.
  2. Make the claim reproducible in the comment itself with an interleaving rather than a principle: A reads version 7, B reads version 7, B commits version 8, A's predicate matches zero rows, A is told it succeeded and A's edit is gone.
  3. Offer the alternative with its cost attached: return 409 carrying the current version and the revision that won, so the client can re-read and re-apply. Note that automatic retry is not the fix, because a retry re-reads the winner's state and reapplies an intent formed against data that no longer exists.
  4. Apply an escalation rule you can state: two round trips on the thread, then a call, and the service's owner decides rather than the reviewer. A reviewer who cannot be overruled is a bottleneck with extra steps.
Follow-up
  • Where would you put the test that fails if someone reintroduces the swallowed zero rowcount?
  • The author says clients cannot handle a 409. How do you check whether that is true?

Unblock an engineer without taking the keyboard

easy
mentoringleasesat-least-once

A teammate has spent two days on a job handler that occasionally writes duplicate rows. They are certain the queue is delivering twice by mistake. You suspect a lease expiring under a slow handler, so the job is running concurrently with itself. Describe how you have unblocked someone in this position: what you asked before offering a hypothesis, what you showed them rather than told them, and what you left them owning. Then say what you would do if their theory turned out to be the right one.

Approach
  1. Ask before diagnosing, and ask for things answerable from data they already have: the attempt count on the job rows that produced duplicates, the handler's observed duration against its lease expiry, and whether the duplicate rows share a natural key that a unique constraint could have caught.
  2. Teach the shape rather than the answer. A lease cannot distinguish a dead worker from a slow one, so a handler that outruns its lease is running twice by design, and deploys deliver the other half by killing handlers mid-run on every rollout. Both of their candidate theories produce identical duplicate rows, which is why the evidence has to come from timings rather than from argument.
  3. Hand over a checklist they execute: a natural key on every write the handler performs so the second copy collides rather than appends, the record of intent written before any external effect, a lease heartbeat while running, and the metric that shows it working.
  4. Keep ownership with them deliberately. Pair on the first write, then step back; if you finish it yourself you have closed one ticket and left the same person stuck on the next redelivery.
Follow-up
  • How would you distinguish a genuine double-delivery from a lease expiry using only the data already stored?
  • Their handler calls an external endpoint before recording that it did. What do you tell them to change first?

Estimate work you have never done and defend the range

hard
estimationbackfillsexpand-contract

You are asked to estimate a change you have never attempted: add a column to a 100-million-row table, populate it, move reads across, and drop the old shape. Give a range with the assumptions that generate it, including batch size, the signal your backfill throttles on, and wall-clock hours, and name the three unknowns that would move the number most. Then describe a real estimate you gave under comparable ignorance: how you expressed its uncertainty, what you committed to, and how wrong you turned out to be.

Approach
  1. Decompose into independently deployable steps before estimating anything: add the column nullable, write both shapes, backfill in batches, verify, move reads, stop writing the old shape, drop it. That is four deploys spread over days, and the calendar estimate is dominated by them rather than by the loop's runtime.
  2. Do the arithmetic aloud for the part that has arithmetic in it: batch size times number of batches times per-batch duration, at a write rate the primary can absorb alongside roughly 1.2k writes per second of production traffic. The loop is throttled by replication lag and lock waits, not by how fast it can issue statements.
  3. Price the schema step by its lock rather than its statement duration. In PostgreSQL an ALTER TABLE taking ACCESS EXCLUSIVE waits for every open transaction on that table while later queries queue behind it, so a millisecond change issued during a thirty-second analytics query stalls that table for thirty seconds. Adding a nullable column with a non-volatile default avoids a rewrite from version 11; a new index wants CREATE INDEX CONCURRENTLY, which cannot run inside a transaction block and leaves an invalid index behind if it fails.
  4. Express the answer as a range whose endpoints each trace to a stated assumption, then name the cheapest experiment that collapses it, which is almost always running one real batch against the real table and multiplying.
Follow-up
  • How do you verify the backfill genuinely finished, given rows written by production traffic while it ran?
  • Where does the backfill resume from after a worker is killed mid-batch, and what makes that resume point trustworthy?
  • 01

    A colleague's change updates a row with UPDATE resource SET version = version + 1 WHERE resource_id = $1 AND version = $2 and treats an affected-row count of zero as a successful no-op. You read that as a silently lost update; they think returning 200 is friendlier to clients than returning a conflict. Describe how you have handled a review disagreement of this shape: what goes in the comment, when you leave the thread, and who decides. Then write the comment you would leave here, in under 80 words.

  • 02

    A teammate has spent two days on a job handler that occasionally writes duplicate rows. They are certain the queue is delivering twice by mistake. You suspect a lease expiring under a slow handler, so the job is running concurrently with itself. Describe how you have unblocked someone in this position: what you asked before offering a hypothesis, what you showed them rather than told them, and what you left them owning. Then say what you would do if their theory turned out to be the right one.

  • 03

    You are asked to estimate a change you have never attempted: add a column to a 100-million-row table, populate it, move reads across, and drop the old shape. Give a range with the assumptions that generate it, including batch size, the signal your backfill throttles on, and wall-clock hours, and name the three unknowns that would move the number most. Then describe a real estimate you gave under comparable ignorance: how you expressed its uncertainty, what you committed to, and how wrong you turned out to be.

PracHub preparation framework ↗
How difficult are the initial coding assessments?

The initial starter coding assessment is generally considered approachable for anyone with basic computer science knowledge (equivalent to an introductory algorithms course). However, later qualifications—such as the agentic coding qual and paid training—are significantly more demanding and require true software engineering experience.

DataAnnotation Software Engineer candidate reports ↗
Can I use external libraries or tools during the coding test?

Yes. Unless explicitly forbidden by a specific problem prompt, you are allowed to use standard libraries and common tools in your local environment. However, using AI tools like ChatGPT or Copilot to solve the test for you is strictly prohibited and leads to immediate disqualification.

DataAnnotation Software Engineer candidate reports ↗
What happens if my code output is off by one character?

Precision is vital. Assessments rely on automated verification scripts to validate output formats. If your program produces an extra space, missing newline, or incorrect character casing, the automated test case will fail, which often leads to rejection. Always double-check your output against sample specifications.

DataAnnotation Software Engineer candidate reports ↗
How long does it take to hear back after submitting an assessment?

Review timelines vary. Some candidates receive approval within 8 to 72 hours, while others may wait up to two weeks depending on current project demand and applicant volume. If accepted, new project tasks and qualifications will automatically appear on your platform dashboard.

DataAnnotation Software Engineer candidate reports ↗
Is this role fully remote and flexible?

Yes. All work is 100% remote and flexible. You choose when and how much you work, logging your time directly through the platform dashboard as you complete project tasks.

DataAnnotation Software Engineer candidate reports ↗
What topics does DataAnnotation test in interviews?

DataAnnotation interviews most often cover Python, Supervised Learning, Natural Language Processing (NLP), AI Training (Model Training), and Machine Learning. The exact emphasis depends on the specific role you apply for.

DataAnnotation Software Engineer candidate reports ↗
Sources & methodology 3 sources ↗

No official company page is cited. Rounds and questions come from candidate reports and PracHub editorial material; each source shows the date it was read.