Deploying an image captioning system

Deploy a captioning model behind an image-upload service. Estimate retained data and measured inference capacity, then define how preprocessing, optional context, generation, and output review work together.

The preceding lesson described BLIP-2’s visual connection and training stages. Serving uses the resulting checkpoint to generate captions; it does not repeat pretraining or update model weights during each request.

The request carries an image while the result is short text. Upload validation and decoding can therefore matter as much as response transfer. The main quality risk is a fluent statement unsupported by the image, especially when optional metadata or style instructions add information.

Use the resource estimation principles to keep three budgets separate:

• Storage estimation

• Inference server requirements

• Network bandwidth estimation

Start with the data retained after a request finishes.

Storage estimation

Assume 100 million daily active users, ten requests each per day, and separately 100 million stored profiles. Use decimal byte units and state whether original images are retained.