Design content moderation for a posting app

Scenario

You are given a small demo “post creation” web app similar to a social network composer:

• Frontend: a page where users type a post and submit.
• Backend: Node.js service. Many features are currently mocked/stubbed .

You must add a policy-violating content detection module.

Requirements

1. Detect potentially violating content in user-submitted posts (text is sufficient; you may mention how you’d extend to images/links).
2. Provide a user experience during composition/submission:
   • Show a warning/prompt when content is suspected to violate policy.
   • Optionally mask violating spans (e.g., replace characters with * ).
3. The solution should be extensible (easy to add new categories/rules/models) and safe to evolve.
4. Discuss and/or implement (high-level is fine):
   • API contracts between frontend and backend
   • data model or response schema (e.g., violations, spans, severity, suggested edits)
   • latency and failure behavior (what happens if detection is slow/unavailable?)
   • logging/metrics for monitoring quality and abuse

Interview follow-up

After you present your approach, expect two additional feature requests (e.g., add a new violation type, add an override/appeal flow, add an admin configuration toggle) and explain how your design supports implementing them quickly.