Design a **text-only personalization and memory system** for an AI chatbot. The chatbot should use a user's previous conversations, stated preferences, and feedback to generate more personalized responses in future sessions. The system should support retrieval-augmented generation (RAG) or a similar approach; the exact implementation is up to you. Walk through how you would store, retrieve, reconcile, and maintain user memory, and how the system improves over time. The question is broken into the parts below. Treat them as one coherent design — your components should be consistent across parts. ### Constraints & Assumptions - **Text only** — no images, audio, or other modalities. - Memory is **per-user** and persists across sessions; the system is multi-tenant. - The chatbot is interactive, so end-to-end latency must stay low (interactive chat budget, roughly a second or two for the full turn, of which retrieval is only a slice). - A user's raw history can grow without bound over months or years of use. - A generative LLM produces the final response; you control everything around it (storage, retrieval, prompt assembly, post-processing). - Privacy matters: users may view, edit, or delete their data. ### Clarifying Questions to Ask - What is the latency budget for a chat turn, and what share of it may retrieval consume? - What is the expected scale — number of users and average/max memories or history length per user? - Are there explicit personalization controls (settings) the user sets, or is everything inferred from conversation? - What are the privacy and data-retention requirements (right to delete, regional rules, sensitive-data handling)? - Is the underlying LLM fixed, or can we fine-tune / change models? What is its context window? - How do we measure "better personalization" — is there a target metric (satisfaction, correction rate, task success)? ### Part 1 — Storing user memory How would you store memory derived from past conversations? Distinguish raw conversation logs from durable, queryable memory. Define what a "memory" is, its schema, and how memories are created from conversations. ```hint What is the unit of memory? Before designing a schema, decide what a single "memory" actually is. Is the raw message the durable memory, or is it raw material for something more compact? Consider what role the full transcript still plays once you have that more compact form. ``` ```hint Let the later parts pressure-test your schema Sketch the fields, then check them against the work Parts 3–6 demand: reconciling contradictions, expiring stale facts, and weighting by trust and freshness. Which attributes does a record need so those operations don't require re-reading the original conversation? ``` #### What This Part Should Cover - **Two-layer separation**: raw conversation log (audit / reprocessing tier) vs durable, extracted memory — not "store every message and inject it." - **Concrete schema**: the fields a memory record needs, and *why* each exists (provenance, recency, confidence, status, TTL, type). - **Memory typing**: a sensible taxonomy (preference, profile fact, goal/constraint, recent context) and how type drives later policy. - **Creation pipeline**: an asynchronous extraction job that filters for memory-worthiness, normalizes, classifies, and de-duplicates — kept off the response path. ### Part 2 — Retrieving the right memories For a new user query, how do you select the right memories to inject into the prompt? Cover candidate generation, ranking, and how memories enter the LLM context. ```hint Where does pure similarity fall short? Think about a memory like "prefers concise answers." Would a vector search keyed on the current query reliably surface it? If not, what second source of candidates covers the always-relevant facts — and how do you then narrow a large candidate pool down to the few that belong in the prompt? ``` ```hint Treat the context window as adversarial space Once a memory is in the prompt, what stops a maliciously crafted one from reading like an instruction to the model? Consider how the prompt distinguishes "things to know about the user" from "things to do." ``` #### What This Part Should Cover - **Hybrid candidate generation**: semantic (ANN) retrieval *plus* a structured fetch of always-relevant facts that vector search alone would miss. - **Ranking**: a multi-signal reranker (relevance, recency, confidence, type importance) that reduces a large candidate pool to a few snippets. - **Prompt assembly & context hygiene**: a fixed injection budget and a clear delimiter that marks memories as data, not instructions. - **Selectivity**: deciding when a query even needs personalization, and excluding superseded / uncertain memories. ### Part 3 — Handling conflicting information The user's history contains contradictions — e.g. an old preference ("prefers Java") contradicted by a newer one ("now prefers Python"). How does the system detect and resolve such conflicts so the prompt doesn't contain both? ```hint Detection needs more than text matching Two contradictory memories rarely share wording. What would let you recognize that they describe the *same thing* about the user, so you can compare them at all? ``` ```hint Which memory wins, and what happens to the loser? When two memories disagree, enumerate the dimensions that could decide a winner — and ask whether all sources of a memory deserve equal weight. Then decide the loser's fate: is deleting it safe, or does something downstream (audit, retrieval) need it to stick around in a marked state? And what should happen when no dimension clearly breaks the tie? ``` #### What This Part Should Cover - **Detection on the same attribute**: entity/attribute matching (not free-text equality) to recognize that two memories describe the same thing. - **Resolution policy**: an explicit ordering over recency, confidence, and explicit-correction priority, with a clear winner per attribute. - **Loser handling**: versioning / supersession rather than destructive overwrite, so audit and reprocessing still work. - **Tie-breaking**: a defined fallback (e.g. ask the user) when no dimension cleanly decides, plus the invariant that the prompt never holds two contradictory facts about one attribute. ### Part 4 — Reducing retrieval latency If every chatbot response performs retrieval (and possibly summarization), how do you keep the system fast? ```hint Separate the work that must happen per turn from the work that doesn't Of everything memory-related — extraction, consolidation, summarization, lookup — which steps truly need to run inside the response path, and which could run ahead of time or in the background? Pushing work off the critical path is the biggest lever here. ``` ```hint Make the per-turn read cheap, and sometimes skip it For the work that does stay on the path, think about how to avoid a fresh multi-record search every turn (precomputation, caching, approximate search). And ask: does *every* query even need personalization at all? ``` #### What This Part Should Cover - **Critical-path discipline**: which work is moved off the response path (async extraction/consolidation) vs what must stay (a cheap read). - **Precomputation**: a per-user precomputed summary that serves the common case without a fresh multi-record search. - **Cheap reads**: two-stage retrieval, approximate (ANN) search, and caching to keep the per-turn read sublinear and bounded. - **Selective retrieval**: skipping personalization for queries that don't need it, and tying choices back to the stated latency budget. ### Part 5 — Managing a very large history As a user's raw history grows indefinitely, how should the system manage and summarize it so storage and prompt size stay bounded? ```hint Not all memory is accessed the same way Some memory is needed every turn, some occasionally, some almost never. If you grouped memory by how often and how urgently it's read, what would the groups be — and what different storage and freshness policy would each one get? ``` ```hint What makes the two curves flat? Two things must stay bounded as history grows: total storage and per-turn prompt size. For each, name a mechanism that actively shrinks or caps it over time rather than letting it accumulate (think summarization, deduplication, expiry, and a hard injection budget). ``` #### What This Part Should Cover - **Tiering by access pattern**: distinct tiers (recent context, session summaries, canonical long-term memory, cold archive) with different storage and freshness policies. - **Active shrinking mechanisms**: summarization, distillation / dedup, and TTL / expiry that cap the curve rather than letting it accumulate. - **Bounding both curves**: a separately named mechanism for storage growth *and* for per-turn prompt size (a hard injection budget). - **Cost awareness**: pushing cold data to cheap storage while keeping the hot path small. ### Part 6 — Feedback loop How would you design a feedback loop so personalization quality improves over time? Cover the signals collected, how they update memory and ranking, and the safety guardrails. ```hint Two things can learn here, not one Distinguish the signals you can collect (some users state explicitly, some you infer from behavior) from what they should improve. Personalization quality is part *which memories you trust* and part *which ones you surface* — does your loop close on both? ``` ```hint Why is a single thumbs-down dangerous? A response can be rated badly for reasons that have nothing to do with memory. What guardrails stop one noisy signal from rewriting a user's profile — and why is extra caution warranted for sensitive memories? ``` #### What This Part Should Cover - **Signal taxonomy**: both explicit signals (ratings, corrections, edits) and implicit ones (acceptance, engagement, follow-ups). - **Two learning targets**: updating *which memories you trust* (confidence) AND *which you surface* (the reranker / extraction) — the loop closes on both. - **Anti-overfitting guardrails**: requiring multiple consistent signals before large updates, and treating one thumbs-down as weak evidence. - **Sensitive-data caution**: stricter thresholds or human review before feedback rewrites sensitive memories. ### What a Strong Answer Covers These dimensions span all parts; an interviewer looks for them threaded through the whole design, not in any single section: - **Coherent architecture**: the components in Parts 1–6 fit together (one memory schema, one retrieval path, one extraction pipeline) rather than reading as six disconnected answers. - **Privacy & safety throughout**: user view/edit/delete, multi-tenant isolation, encryption / access control, and prompt-injection mitigation that appear consistently, not only when asked. - **Observability**: the metrics that prove the system works — retrieval precision/recall, stale-memory leakage rate, latency percentiles, correction rate, satisfaction. - **Trade-off awareness**: latency vs relevance, recall vs prompt budget, and freshness vs stability, stated explicitly rather than assumed away. ### Follow-up Questions - A retrieved memory is **wrong or harmful** (poisoned by an adversarial prompt, or simply mis-extracted). How do you detect this and prevent it from steering responses? - How would you **evaluate** the personalization system offline before shipping a change to extraction or ranking — what dataset and metrics would you use? - The user invokes **"delete my data"** (GDPR-style). Walk through what must be removed across logs, canonical memory, vector index, summaries, and caches, and how you verify completeness. - How would your design change if memory had to be **shared across multiple agents/products** (e.g. one assistant in several apps) rather than a single chatbot?