Design a Rule Management Service for AI Agents with Drafts, Publishing and Rollback

Read the full interview experience this question came from →

Quick Overview

Design a rule management system that lets customers configure behavioral rules for AI agents, with rule editing, saved drafts, publishing and rollback to earlier deployments. The question focuses on the data model, APIs and data flow, including how an external renderer always reads one consistent set of active rules.

Design a Rule Management Service for AI Agents with Drafts, Publishing and Rollback

Company: Decagon

Role: Software Engineer

Category: System Design

Difficulty: medium

Interview Round: Technical Screen

Design a rule management system that lets customers configure behavioral rules for their AI agents, such as "Be polite", "Keep responses under three sentences" or "Avoid political discussions". Customers can create, update and delete rules, save drafts, publish a configuration, and roll back to a previous deployment. An external Agent Renderer, which you treat as a black box, retrieves the active rules and executes them. The interviewer set scalability aside at the start: the round is about the data model, the APIs and the data flow between the customer's editor, your service and the Agent Renderer, which makes it closer to object-oriented design than to large-scale system design. ```hint Edited versus served Decide which records change while a customer is editing and which records the Agent Renderer reads, and whether anything the renderer has already read can change afterwards. ``` ```hint What a rollback restores List everything a rollback has to bring back (the rules' text, which rules exist, their order, whether each one is enabled), then ask which record would let you restore all of it in one step. ``` ### Constraints and Clarifications - Scalability is explicitly out of scope; correct versioning, drafts, publishing and rollback are in scope. - You own only the interface through which the Agent Renderer fetches the active rules, not how it interprets or enforces them. - Not given: whether rules belong to one agent or are shared across a customer's agents, whether a draft covers one rule or the whole configuration, whether rules are ordered, and how soon a publish or rollback must reach the renderer. Ask, or state your assumptions. ### Clarifying Questions - Does a customer run several agents, each with its own rule set, and can one rule be shared by several agents? - Is a draft a working copy of the whole configuration or of a single rule, and can several people edit the same draft at once? - Does publishing take the whole draft at once, or can individual rules be published? - Does a rollback make an earlier configuration active again, and what happens to the draft the customer is working on? - Do rules have an order or priority, and is the service expected to detect rules that contradict each other? - How does the Agent Renderer get the rules: on every conversation, from a cache, or by push? How quickly must it see a publish or a rollback? - If a customer deletes a rule, should configurations that were already published still contain it? ### What a Strong Answer Covers - Settles the scope of a rule set, the unit of drafting and publishing, and what a rollback restores, before modelling - A data model that separates the mutable draft from immutable published versions and records which version is active, with a deployment history - APIs for rule editing, drafts, publishing and rollback, with validation, safe retries and protection against concurrent edits - A narrow read API for the Agent Renderer that always returns one consistent configuration with a version identifier - The data flow from an edit to the draft, to a published version, to the renderer, and how a publish or rollback becomes visible - Deletion semantics, audit history, and failure cases such as a publish interrupted halfway ### Follow-up Questions - Two people edit the same draft at the same time. How does your API detect the conflict, and what does each of them see? - A customer wants to try a new configuration on a small share of conversations before publishing it everywhere. What changes in the model and in the renderer's API? - The Agent Renderer caches rules. How does it learn about a rollback, and what does it serve in the meantime? - A customer deletes a rule, publishes, and later rolls back to a deployment that still contained the rule. What should the renderer and the draft show?

Overview: Design a rule management system that lets customers configure behavioral rules for AI agents, with rule editing, saved drafts, publishing and rollback to earlier deployments. The question focuses on the data model, APIs and data flow, including how an external renderer always reads one consistent set of active rules.

Read the full Decagon Software Engineer interview experience this question came from

Decagon logo
Decagon
Oct 14, 2026
mediumSoftware EngineerTechnical ScreenSystem Design
0
0

Design a rule management system that lets customers configure behavioral rules for their AI agents, such as "Be polite", "Keep responses under three sentences" or "Avoid political discussions". Customers can create, update and delete rules, save drafts, publish a configuration, and roll back to a previous deployment. An external Agent Renderer, which you treat as a black box, retrieves the active rules and executes them.

The interviewer set scalability aside at the start: the round is about the data model, the APIs and the data flow between the customer's editor, your service and the Agent Renderer, which makes it closer to object-oriented design than to large-scale system design.

Constraints and Clarifications

  • Scalability is explicitly out of scope; correct versioning, drafts, publishing and rollback are in scope.
  • You own only the interface through which the Agent Renderer fetches the active rules, not how it interprets or enforces them.
  • Not given: whether rules belong to one agent or are shared across a customer's agents, whether a draft covers one rule or the whole configuration, whether rules are ordered, and how soon a publish or rollback must reach the renderer. Ask, or state your assumptions.

Clarifying Questions Guidance

  • Does a customer run several agents, each with its own rule set, and can one rule be shared by several agents?
  • Is a draft a working copy of the whole configuration or of a single rule, and can several people edit the same draft at once?
  • Does publishing take the whole draft at once, or can individual rules be published?
  • Does a rollback make an earlier configuration active again, and what happens to the draft the customer is working on?
  • Do rules have an order or priority, and is the service expected to detect rules that contradict each other?
  • How does the Agent Renderer get the rules: on every conversation, from a cache, or by push? How quickly must it see a publish or a rollback?
  • If a customer deletes a rule, should configurations that were already published still contain it?

What a Strong Answer Covers Guidance

  • Settles the scope of a rule set, the unit of drafting and publishing, and what a rollback restores, before modelling
  • A data model that separates the mutable draft from immutable published versions and records which version is active, with a deployment history
  • APIs for rule editing, drafts, publishing and rollback, with validation, safe retries and protection against concurrent edits
  • A narrow read API for the Agent Renderer that always returns one consistent configuration with a version identifier
  • The data flow from an edit to the draft, to a published version, to the renderer, and how a publish or rollback becomes visible
  • Deletion semantics, audit history, and failure cases such as a publish interrupted halfway

Follow-up Questions Guidance

  • Two people edit the same draft at the same time. How does your API detect the conflict, and what does each of them see?
  • A customer wants to try a new configuration on a small share of conversations before publishing it everywhere. What changes in the model and in the renderer's API?
  • The Agent Renderer caches rules. How does it learn about a rollback, and what does it serve in the meantime?
  • A customer deletes a rule, publishes, and later rolls back to a deployment that still contained the rule. What should the renderer and the draft show?