Design webhook, POI, chat, CI/CD, payments

You are asked to design several large scale backend systems. For each one, describe:

• Core requirements and assumptions
• High level architecture and main components
• Data model and storage choices
• How you will meet scale, reliability, and latency requirements
• How you will handle failures, retries, and consistency

1. Design a webhook delivery service

Design a webhook service that allows users to register callbacks for specific events.

Functional requirements

• Clients can register a subscription by providing:
  • eventId (string or numeric)
  • callback URL (HTTP endpoint)
• The mapping eventId to callback URL is unique: each eventId triggers exactly one callback URL.
• When an event with a given eventId is triggered, the system must send an HTTP request to the registered callback URL containing a payload with event data.
• The system should provide visibility into delivery status (success, failure, retry attempts) per event.

Non functional requirements

• Scale: up to 1 billion events per day.
  • Rough ballpark: about 11.6k events per second on average, with traffic spikes.
• High availability and durability of events.
• Delivery guarantees: at least once delivery is acceptable; exactly once can be achieved with idempotency at the receiver.
• Low but not real time latency: it is acceptable if callbacks are delivered within a few seconds.
• Multitenant and secure: only authorized clients can create or manage their subscriptions.

Design the full system: APIs, storage, event ingestion, dispatch workers, retry logic, monitoring, and how you would scale this to 1B events per day.

2. Design a places of interest search service (Yelp or Foursquare like)

Design a service that manages and searches places of interest around the world.

Data model

• Each place has at least:
  • id
  • name
  • location: latitude and longitude
  • place type: for example restaurant, hotel, park, etc.
  • Optional metadata like rating, address, opening hours.

Functional requirements

• Support registering, updating, and removing places.
• Support search queries:
  • Given a user location (latitude, longitude) and a place type, find the N nearest places.
  • Optionally support a radius or maximum distance.
  • Sort primarily by distance; you can mention additional ranking signals like rating.

Non functional requirements

• Data scale: on the order of hundreds of millions of places globally.
• High read throughput with low latency search (for example p95 under 100 ms for queries).
• Updates to places do not need to be visible in real time; eventual consistency is acceptable.

Design the system, including:

• How you store place data.
• How you index geospatial locations to support efficient nearby search.
• How you handle filtering by place type and other attributes.
• How you shard and replicate data globally.
• How you handle hotspots (for example city centers) and caching.

3. Design a Slack like chat system

Design a messaging system similar to Slack with support for individual and group chats.

Functional requirements

• Users can send messages to:
  • Another individual user (one to one chat).
  • A group chat with multiple members.
• Users can:
  • Create new group chats.
  • Add or remove users from group chats (assume permissions can be simplified).
• The system delivers notifications to recipients when new messages arrive.
• Messages can contain rich media such as images; large media content should be stored efficiently.
• Users can delete a message they previously sent.
  • Clarify semantics: delete for everyone in the conversation, or only hide for the deleting user.

Non functional requirements

• Support a large number of concurrent users and active conversations.
• Low latency message delivery (near real time for online users).
• Message history should be stored and queryable (for example loading chat history when opening a conversation).
• High availability and graceful handling of clients that disconnect and reconnect.

Design the system, including:

• Overall architecture and main services.
• How clients maintain real time connections.
• Data model for users, conversations, membership, and messages.
• Storage of message content versus media attachments.
• How to deliver messages reliably and in order per conversation.
• How to implement message deletion.

4. Design a distributed CI or CD workflow system (GitHub Actions like)

Design a continuous integration and continuous delivery workflow engine similar to GitHub Actions.

Scenario

• The system is integrated with a source code hosting service.
• When repository events happen (for example git push, pull request opened, tag created), the system should automatically trigger predefined workflows.
• Workflow definitions are stored as configuration files (for example YAML) in the repository itself.

Functional requirements

• Detect repository events (for example a git push) and map them to workflows that should run.
• For each trigger, create a workflow run that may contain multiple jobs and steps.
• Schedule jobs to run on a distributed pool of workers (for example container based runners or VMs).
• Manage the lifecycle of each workflow run:
  • Queueing, execution, retries, timeouts, cancellation.
  • Collection and storage of logs and artifacts.
• Report status back to the source control system for display (for example success, failure, in progress).

Non functional requirements

• Multi tenant system: many organizations and repositories.
• Scalable to tens or hundreds of thousands of concurrent workflow runs.
• Fair scheduling across projects and organizations with rate limiting.
• Strong isolation between workflows for security.

Design the architecture, including:

• Event ingestion from the git service.
• How to find and parse the workflow configuration from the repository commit that triggered it.
• The orchestration layer that manages workflow state and dependencies between jobs.
• The worker or runner infrastructure that executes jobs.
• Storage for workflow runs, logs, and artifacts.
• How you scale, ensure reliability, and handle failures.

5. Design a payment processor service

Design a simplified but realistic payment processing system.

Functional requirements

• Merchants integrate with your service via an API to process payments.
• For a payment request, the system should:
  • Receive a payment initiation request from the merchant (for example charge a card or digital wallet).
  • Forward the request to an appropriate downstream payment gateway or processor (for example card network, bank, third party provider).
  • Receive the response from the downstream processor.
  • Return a result to the merchant with a clear status such as authorized, declined, or error.
• Support related operations such as refund or capture can be mentioned.

Non functional requirements

• Strong security and compliance, including safe handling of sensitive payment data.
• High availability and low latency for the synchronous part of payment authorization.
• Exactly once or at least once behavior that avoids double charging the customer, even under retries.
• Auditability: all payment events must be logged and traceable; a ledger or transaction history must be maintained.

Design the overall architecture, including:

• External API design for merchants and how you handle authentication and idempotency.
• Core services involved in routing requests to different payment processors.
• Data model for payments, merchants, and transaction history.
• How you integrate with external payment gateways and handle their failures.
• How you maintain a reliable ledger, support reconciliation, and ensure consistency.
• How you address security and compliance concerns at a high level.