How do I approach System Design interview questions?

System Design questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master system design interviews.

What difficulty level is this interview question?

This is a hard difficulty System Design question, commonly asked during Technical Screen rounds at OpenAI.

What role is this question designed for?

This question is commonly asked for Software Engineer candidates at OpenAI during technical interviews.

Design a GPU credit allocation service | OpenAI Interview Question

Q: Design a GPU credit allocation service

This question evaluates system design proficiency in distributed resource accounting, real-time quota enforcement, billing models, idempotency, and observability for multi-tenant GPU platforms, and falls under the System Design category.

Design a Multi‑Tenant GPU Credit Allocation Service

Context

You are designing a multi-tenant platform where organizations run GPU jobs. Each organization receives monthly GPU credits that are consumed as jobs run. The system must expose APIs to issue, transfer, and spend credits; enforce budgets and rate limits in real time; and integrate with a job scheduler to admit or reject workloads based on available credits.

Assume credits are the unit of spend (e.g., 1 credit = $0.01 of GPU time) and that different GPU types have different prices (credits per GPU-minute). Jobs may be submitted by users to projects within an org. The platform should support prepaid and postpaid billing models.

Requirements

Design a service that:

Defines APIs to:
- Issue and expire monthly credits.
- Transfer credits between org, project, and user accounts.
- Reserve, spend, and refund credits tied to workloads.
- Query balances, budgets, and audit logs.
Enforces budgets, quotas, and rate limits in real time under concurrency.
Integrates with a job scheduler to admit/reject jobs based on available credits and quotas.
Supports prepaid vs. postpaid models and per-user/project quotas.
Ensures idempotency, prevents overspend, and provides audit logging/reporting.
Addresses data model, consistency model, failure recovery, scaling (partitioning, caching), and observability/alerting.

Make minimal reasonable assumptions where unspecified, and call them out.

Context

Requirements

Design a service that:

Defines APIs to:

Issue and expire monthly credits.
Transfer credits between org, project, and user accounts.
Reserve, spend, and refund credits tied to workloads.
Query balances, budgets, and audit logs.

Enforces budgets, quotas, and rate limits in real time under concurrency.

Integrates with a job scheduler to admit/reject jobs based on available credits and quotas.

Supports prepaid vs. postpaid models and per-user/project quotas.

Ensures idempotency, prevents overspend, and provides audit logging/reporting.

Addresses data model, consistency model, failure recovery, scaling (partitioning, caching), and observability/alerting.

Make minimal reasonable assumptions where unspecified, and call them out.

Design a GPU credit allocation service

Quick Overview

Design a Multi‑Tenant GPU Credit Allocation Service

Context

Requirements

Solution

Comments (0)

Design a GPU credit allocation service

Quick Overview

Design a Multi‑Tenant GPU Credit Allocation Service

Context

Requirements

Solution

Comments (0)