Citadel Data Scientist Interview Experience — Temperature Models, Interpolation, and Streaming Regression

First question

Background

You are given daily temperature data for P towns and daily temperature data for New York City over the same N days. Write functions to answer several questions.

Part One — five questions

Q1: Which place, either one of the towns or New York City, has the greatest temperature variation, measured by standard deviation?

Q2: When Town2's temperature is between 90 and 100 degrees, what is the median New York City temperature? Round the result to the nearest integer.

Q3: Use each town's temperature separately in a simple linear regression with an intercept to predict New York City temperature. Add the absolute values of the P regression coefficients, excluding the intercept, and round the result to the nearest integer.

Q4: Which single town predicts New York City temperature best? Choose the one with the lowest MSE after linear regression.

Q5: Which pair of towns predicts New York City temperature best? Again use the lowest MSE, this time from a joint regression using two towns.

Part Two — one question

Q6: Find five towns whose temperatures work best together to predict New York City temperature. The prompt warns that exhaustively checking every combination will not finish in time, so an approximate method such as a greedy approach is enough. The result is scored by MSE.

Function requirements

Write two functions, q1_q5(df) and q6(df), and have each return a list. The input is a pandas DataFrame whose columns are the place names, including NYC.

Second question

You are given a collection of points and must implement piecewise linear interpolation yourself.

Given n points (x_knots[i], y_knots[i]), the points arrive in an arbitrary order and are not guaranteed to be sorted. After sorting by x-coordinate, connect each pair of adjacent points with a straight line. That polyline defines the function LI(x).

Write a function linear_interpolate(n, x_knots, y_knots, x_input) that returns LI(x_input), the y-value on the polyline for the given x.

If x_input lies outside the range of all the points, extend the line through the two nearest boundary points and use it for extrapolation.

You cannot use an existing interpolation library; you have to write the logic yourself.

Third question

Part 1: basic version

You are given two sets of asset-return data, dfx and dfy. Each column xi corresponds to a column yi. For each pair, run a univariate linear regression without an intercept and calculate the slope beta. Because there is no intercept, the formula is simply beta = sum(x*y) / sum(x^2). Calculate one beta for every xi, yi pair and return a list of length N.

Part 2: advanced version

The data now arrives in M batches instead of all at once. Calculate the beta using only batch 0, then using batches 0 and 1, then using batches 0, 1, and 2, and continue until the calculation uses all M batches.

In other words, at every step, refit the regression using all historical data from the beginning through the current batch. Output M results, where each result is a list of N beta values.