Snowflake Software Engineer Interview Experience — A Quota System Design That Ran Out of Time

Interview process

Both sides introduced ourselves. The interviewer asked about one of my projects — I don't quite remember what exactly they asked. Then system design. Then I asked the interviewer some questions.

System design

Question: design a Quota System that's shared by multiple upstream services. For example, Google Drive and Google Photos share 100GB of space, and each user has one global quota.

Functional requirements:

• Able to consume or release quota
• (optional) able to query the current quota

Non-functional requirements:

• Quota can never be overdrawn at any point in time (as opposed to "can be briefly overdrawn as long as it's not overdrawn in the end")
• The system must be highly consistent and highly available
• (optional) response time should be as fast as possible

Scale: millions-level DAU and QPS (because the upstream services' QPS is at the millions level).

Interview process requirements: during the interview you're allowed to look up any materials, including using AI, except for pasting the question straight into AI and reading out the answer.

My clumsy answer

I'm new to system design, I haven't been studying it for very long, so I didn't answer particularly well. Also, in the first step of asking clarifying questions, I probably didn't think it through very thoroughly, and later the interviewer kept asking "what would you do if XX situation came up," which made me feel like that was probably the first point where I lost points. And in the end I only finished the high level design before time ran out — I didn't get to the detailed design, which was probably a big point loss too.

What I still remember covering includes:

• DB: chose an RDBMS, since the system needs to strictly guarantee ACID, and quota data can be represented relationally.
• Data sync: the write DB uses a fairly strict ack mechanism (every write requires all replicas to ack before returning), and reads can come from any replica (but that's slow).
• To satisfy non-functional requirement 1, consuming quota is done synchronously rather than going through an MQ asynchronously, to prevent the case where consuming quota fails (e.g. insufficient balance) but the upstream service still goes ahead and operates anyway.
• Quota consumption flow: whenever an upstream request wants to consume quota, it first calls the consume API. The consume API first tries to reserve the quota — if the balance is insufficient or something else goes wrong, it returns an error, otherwise that portion of quota moves to a Reserved state and it returns success. After the upstream service gets a success response, it finishes its own operation (e.g. Google Drive keeps uploading the file), and once that operation is done it sends a message to the Quota service's MQ. The Quota service then asynchronously moves that quota from Reserved to Occupied. Quota in the Reserved state can be stored in some storage medium with a TTL, so that if no message is received from the upstream service within a certain window, that quota moves back from Reserved to Available.
• Quota release flow: either calling the release API synchronously or asynchronously works. The upside of synchronous is that the released quota can be used immediately, the downside is longer response time; async is the opposite trade-off.
• Cache: I mentioned you could use a write-through or similar caching strategy, so that the cache either has no info for a user's quota or has accurate info. But the interviewer said that given the complexity of maintaining a cache, we could skip it for now.
• Sharding: could shard based on the user's geographic location.
• A load balancer, and using either master-slave or master-master failover for each server, both work.

Interview result

I wasn't told the individual round result, but the overall result was a rejection. It's quite possible this system design round failing led directly to the overall rejection. It's also possible that because I had another offer deadline, I pushed them too hard for a decision, so as a non-top-choice candidate they just decided to reject me first. So going forward I might rather accept the other offer first, and if this one turns out better, back out of the first one then — instead of pushing this hard.