Quick Overview

This question evaluates skills in scalable data ingestion, robust data cleaning and normalization, memory-efficient joins and aggregations, and handling heterogeneous file encodings and delimiters within SQL/Python workflows; Domain: Data Manipulation (SQL/Python).

Merge four CSVs locally, robustly and efficiently

Company: Capital One

Role: Data Scientist

Category: Data Manipulation (SQL/Python)

Difficulty: medium

Interview Round: Technical Screen

You receive four CSV files that must be merged locally on a laptop with 8 GB RAM, without relying on cloud services: - products.csv: product_id, category, product_name - purchases.csv: purchase_id, product_id, price, stars - ads_events.csv: date (YYYY-MM-DD), user_id, ad_id, watch_seconds, clicks - categories.csv: category, category_display_name Constraints and quirks: files may use different delimiters (comma/semicolon), encodings (UTF-8/Windows-1252), and may contain duplicate header rows and stray BOMs; purchases.csv can exceed 10 million rows. Task: Produce two outputs—(A) category_min_price.csv containing, for every category in products.csv, the lowest price among purchases with stars > 4 (or 0 if none), and (B) weekly_watchtime.csv containing, for every (ISOYear, ISOWeek, category), the total watch_seconds from ads_events.csv after joining ads_events to products via AdID→product_id is not available; instead, a lookup file ad_product_map.csv is provided at runtime with columns ad_id, product_id. Describe and then implement (in pseudocode or Python) a robust merge pipeline that: detects and normalizes encodings and delimiters; streams large files in chunks; de-duplicates by natural keys; handles missing/NULL stars; computes ISO weeks correctly (state the formula or library call); avoids out-of-memory errors (e.g., via chunked pandas + on-disk store, or SQLite/DuckDB with CREATE TABLE AS SELECT); validates row counts and key coverage; and writes both outputs sorted and reproducible. Include exact join keys, data types you will enforce, and how you would unit test the correctness on a 1000-row synthetic sample.

Overview: This question evaluates skills in scalable data ingestion, robust data cleaning and normalization, memory-efficient joins and aggregations, and handling heterogeneous file encodings and delimiters within SQL/Python workflows; Domain: Data Manipulation (SQL/Python).

Minimum 5-star price per category (include categories with no qualifying purchases)

You are given product and purchase data loaded into SQL tables. Create a result that contains **one row per category that appears in `products`**, with the **lowest purchase price** among purchases where **stars > 4**. Rules: - Join `purchases` to `products` using `product_id`. - Only consider purchases with `stars > 4`. - `stars` can be NULL; NULL should not count as > 4. - If a category has **no qualifying purchases**, return **0.00** for that category. - Output columns: `category`, `min_price` - Sort by `category` ascending.

Tables

products(product_id INT, category VARCHAR(50), product_name VARCHAR(200))

purchases(purchase_id BIGINT, product_id INT, price DECIMAL(10,2), stars INT)

ads_events(event_date DATE, user_id VARCHAR(50), ad_id VARCHAR(50), watch_seconds INT, clicks INT)

ad_product_map(ad_id VARCHAR(50), product_id INT)

categories(category VARCHAR(50), category_display_name VARCHAR(100))

Hints

  1. A LEFT JOIN from products to purchases ensures categories with no purchases still appear.
  2. Use an aggregate with a condition (FILTER or CASE) and COALESCE to return 0.00 when there are no qualifying rows.

Weekly (ISO year/week) watch time by category via ad-to-product mapping

You are given ad event data and a runtime mapping from `ad_id` to `product_id`. Compute total watch time per **(ISO year, ISO week, category)**. Rules: - Join `ads_events` to `ad_product_map` on `ad_id`. - Join to `products` on `product_id` to get `category`. - Exclude ad events that do not have a match in `ad_product_map`. - Use ISO week definitions (Monday-based weeks; week 1 is the week with the year's first Thursday). In PostgreSQL, you can use `EXTRACT(ISOYEAR FROM event_date)` and `EXTRACT(WEEK FROM event_date)`. - Output columns: `iso_year`, `iso_week`, `category`, `total_watch_seconds` - Sort by `iso_year`, `iso_week`, `category` ascending.

Tables

products(product_id INT, category VARCHAR(50), product_name VARCHAR(200))

purchases(purchase_id BIGINT, product_id INT, price DECIMAL(10,2), stars INT)

ads_events(event_date DATE, user_id VARCHAR(50), ad_id VARCHAR(50), watch_seconds INT, clicks INT)

ad_product_map(ad_id VARCHAR(50), product_id INT)

categories(category VARCHAR(50), category_display_name VARCHAR(100))

Hints

  1. Use an inner join to the mapping table to drop unmapped ad_ids.
  2. In PostgreSQL, EXTRACT(ISOYEAR ...) and EXTRACT(WEEK ...) produce ISO week-based groupings.

Loading coding console...