Extract W-2 PDF Data and Evaluate It Against Reference JSON

You are given a repository containing W-2 PDF documents and corresponding JSON files that represent the data in those documents. Describe how you would implement an AI-assisted program that extracts the data from each form or table and measures its correctness against the reference JSON. Explain your approach to inspecting the files, defining the output contract, extracting values, matching predictions to references, and reporting extraction errors. The actual repository and schema are not supplied here, so state which details you would discover from them rather than inventing field names or assuming every PDF has the same layout. AI tools may help you develop the program. Explain how you would review and test their output, and how you would keep the reference answers from leaking into the extraction path. ### What a Strong Answer Covers - A distinction between PDFs with usable text layers and scanned pages, including a fallback for missing or unreliable text. - An explicit mapping between documents, individual forms or repeated records, and the reference JSON structure. - Schema-aware normalization that preserves meaningful distinctions such as an absent value versus zero. - Reproducible field-level and document-level correctness measures, with failures and unexpected extracted fields visible in the report. - A development workflow in which AI-generated changes are inspected and evaluated against a stable comparison contract. ```hint Separate prediction from evaluation The reference JSON tells you how to score the output. It must not become an input used to fill values that the extractor could not read from the PDF. ``` ### Follow-up Questions - How would you compare outputs if one PDF contains several forms and their order differs from the reference JSON? - How would you diagnose a high overall accuracy score that hides consistently missed fields?