Lease Renewal Prediction and Non-Renewal Reason Categorization Take-Home

This take-home gives you 24 hours of work, followed by a 30-minute presentation that serves as a research deep-dive on your approach. The objective is to build a model that predicts the likelihood that a resident renews their lease. One task is a supervised learning problem and one calls for unsupervised learning. You have two sources of data, and both are split into a training portion and a test portion: 1. JSON files containing 1,000 conversations with tenants who are considering renewing their leases. 2. A SQLite database with three tables: | Table | Contents | |---|---| | `residents` | Resident information | | `maintenance_history` | Maintenance work orders submitted by each resident | | `renewal_offers` | Renewal offers made to each unit | ### Constraints and Clarifications - No schema beyond the one-line table summaries is provided. You must discover the columns, the join keys and where the renewal outcome lives by inspecting the data. - The prediction file has one row per resident in the test set, with three mandatory columns, `resident_id`, `unit_number` and `renewal_decision`, plus `probability_of_renewal` if your model produces probabilities. The graders run their own evaluation on the test set. - Task 3 must use the gte-base sentence-transformer model. - Deliverables: the prediction CSV; a Jupyter notebook with clean, well-documented code, which is submitted and used during the deep-dive; and a short presentation of the high-level approach and results. ### Clarifying Questions - Is the renewal outcome recorded per resident or per unit, and can one unit hold several residents with different outcomes? - How do conversations link to residents or units, and does every resident have one? - Were the conversations recorded before the renewal decision, or can they contain the decision itself? - Should `renewal_decision` be a 0/1 flag or a text label, and do false alarms and missed non-renewals cost the same? ### Part 1 — Build the renewal prediction model Build a model that uses the available data to predict whether or not a unit or resident renews. Discuss the preprocessing steps you apply. ```hint Fix the grain first Decide what one training row represents before building features, given that work orders are recorded per resident, offers per unit, and conversations as free text. ``` #### What This Part Should Cover - Joining the three tables and the conversations at a stated grain, with a located and verified label - Preprocessing and features fit on the training split only - A leakage review of every feature source, especially the conversations - A justified model choice for a small dataset that mixes tables and text ### Part 2 — Evaluate against a random baseline and produce test predictions Identify a metric that demonstrates your model's performance compared with a random baseline. Then output a CSV with a prediction for each resident in the test dataset, in the required format. ```hint What random scores Work out what a random predictor is expected to score on your chosen metric, and whether that value depends on how many residents renew. ``` #### What This Part Should Cover - A metric whose random-baseline value is known, and why it fits this decision - An evaluation protocol on the training split that estimates test performance honestly, with uncertainty - How the decision threshold and the probability column are produced and checked ### Part 3 — Categorize the reasons tenants do not renew Using the gte-base sentence-transformer model, train a model to categorize the reasons tenants do not renew their leases. Analyze the reasons tenants give and come up with categories that cover their range. Discuss how you defined the categories and any patterns or insights you observed in the data. ```hint No labels yet The data contains no reason labels. Think about what the embedding space lets you find before there is anything to train a classifier on. ``` #### Clarifying Questions for this Part - Should the categorizer cover only tenants who did not renew, or every conversation? - Can one tenant give more than one reason? #### What This Part Should Cover - Which text is embedded, and at what granularity - A repeatable method for deriving categories, and the human judgment used to name and edit them - How the trained categorizer is validated when no labels exist at the start - Insights that connect the categories to the structured data ### Part 4 — Prepare the research deep-dive Prepare a short presentation of the high-level approach and results, and have the notebook ready, since it will be used during the 30-minute session. ```hint Expect live questions The reviewers will have the notebook open. Think about which decisions they are most likely to challenge, and where in the notebook the evidence for each one sits. ``` #### What This Part Should Cover - A storyline from objective to results to recommendations that fits a short slot - Honest limitations and next steps - A notebook that reruns cleanly and mirrors the presentation ### What a Strong Answer Covers - Leakage-aware features and validation splits that respect how residents and units are grouped - Comparison with random and naive baselines, with uncertainty rather than a single number - Categories grounded in the actual conversations, with counts and examples - Prioritization within 24 hours: a complete, valid submission early and refinements later - Findings translated into actions a property manager could take ### Follow-up Questions - How would the model change if it had to predict renewal months before any offer or conversation exists? - How would you check that the reason categories are stable across clustering runs and choices of the number of clusters? - If the graders' test-set score came back well below your cross-validated score, what would you investigate first? - How could the reason categories feed back into the renewal model, and what leakage would that risk?