Evaluation metrics for generative AI systems

Compare automatic metrics with human judgments for generated text and images. Work through overlap and probability calculations, then identify what each score measures and which errors it can miss.

Evaluating a generation system starts with the task: what should the output contain, and what would make it unusable? A translation should preserve meaning; a product image should preserve the requested objects and attributes. Fluency or visual appeal alone cannot establish either requirement.

An evaluation process supports three decisions:

• Acceptance: Check the output against task requirements, including factual accuracy, relevance, and format. A passing score is evidence under those checks, not a guarantee that the output has no errors.

• Comparison: Test candidate models or system changes on the same representative cases. Keep the data, metric implementation, and scoring procedure consistent.

• Failure analysis: Examine recurring defects, including harmful content, bias, missing constraints, and low diversity. Use the findings to choose a change and test it again.

Evaluation can reveal mode collapse, in which a generator produces a narrow range of outputs. Measuring diversity does not itself prevent that failure; the training or sampling process must address the cause.

This lesson groups methods by who or what assigns the score:

1. Automatic metrics: A defined computation scores outputs. The scale and preferred direction depend on the metric: lower FID and perplexity are preferred, whereas higher BLEU is preferred under a fixed evaluation setup.

2. Human evaluation: Reviewers judge outputs using a rubric, a rating scale, or a direct comparison. A one-to-five scale is one design choice, not a universal standard.

For each method below, identify its inputs, its score direction, and a plausible failure it would overlook. This is more useful than memorizing a list of metric names.

Automatic metrics

Automatic scoring can make repeated comparisons cheaper and more consistent. People still choose the dataset, references, encoders, and acceptance criteria. The resulting number measures that particular procedure, not quality in every sense.

Inception score

The inception score, or IS, uses a pretrained classifier such as Inception v3. It favors confident class predictions for individual generated images and a varied class distribution across the collection. A high score therefore reflects the classifier’s view of recognizable class diversity; it does not directly compare the collection with real images or establish that every image is realistic.

The inception score is calculated using the following steps:

1. Obtain a class-probability distribution for each generated image.

2. Average those distributions across the image collection to obtain the marginal class distribution.

3. Calculate the KL divergence from each image’s distribution to that marginal distribution.

4. Average those divergences across images, then take the exponential. The reference implementation makes this averaging step explicit.

Follow how the per-image distributions contribute to the collection-level score in the figures:

Note: A classifier-based diversity measure could be designed for text using topic probabilities. That would be a separately specified adaptation, not the original image inception score. Its usefulness would depend on whether the classifier’s categories match the task.

Fréchet inception distance

FID compares statistics of feature embeddings from two image collections, usually generated images and a real reference set. Both collections pass through the same feature encoder. A lower score means their estimated feature distributions are closer; the encoder and sample selection determine which differences the comparison can detect.

FID can be calculated by:

1. Extract feature vectors for each image in both collections.

2. Estimate a mean vector and covariance matrix for each collection.

3. Compare the two fitted Gaussian distributions using the Fréchet-distance calculation. The original implementation uses a covariance-matrix term as well as the squared distance between means.

The scalar expression $d(X,Y)=({\mu }_x-{\mu }_y{)}^2+({\sigma }_x-{\sigma }_y{)}^2$ is the one-dimensional Gaussian special case, with standard deviations. It is not the general multivariate FID formula, nor a distance computed from just one generated image and one reference image. The following diagram shows the multivariate calculation, with a feature vector for each image and collection-level statistics.

BLEU score

BLEU compares candidate and reference text through modified n-gram precision and a brevity penalty. Candidate counts are clipped to the counts available in the references, so repeating a matching word cannot earn unlimited credit. BLEU was designed for machine-translation comparison across a corpus; a single-sentence example is useful for learning the arithmetic but can disagree with a person’s judgment of meaning.

Compare the reference sentence, "The library has a large collection of useful books," with the candidate, "The library has a small collection of useful books." Treat each word as one token, ignore the final period, and calculate precision for one-, two-, and three-word sequences.

Let’s calculate the BLEU score for the example above.

We use the following formula:

Here, $\text{N}$ is the largest n-gram order included. The brevity penalty lowers the score for an output shorter than its reference, while the geometric average combines the modified precisions. Where $w_n=\frac{1}{N}$. So, we get: Here, $r$ is the reference length and $c$ is the candidate length. Both contain nine tokens in this example, so $\frac{r}{c}$ = 1 and $e^{1-1}=1$. The brevity penalty is therefore 1.

The score is about 0.725, reflecting substantial word-sequence overlap. Yet changing “large” to “small” reverses a meaningful claim. This is a concrete reason to pair overlap metrics with a check of task meaning rather than treating a high score as factual correctness.

ROUGE score

ROUGE is a family of reference-overlap measures developed for summarization evaluation. Recall asks how much reference content is represented in the candidate, while precision and F-measures can account for extra candidate content. These are text-overlap measures; they do not independently verify the summary’s claims.

ROUGE-N compares n-gram counts. ROUGE-L uses a longest common subsequence, which preserves order without requiring every matched word to be adjacent. ROUGE-S compares in-order word pairs with gaps; a maximum gap can be configured, but one intervening word is not its general definition. Report the variant and whether the result is precision, recall, or an F-score.

Here is an example of a ROUGE-1 score (a variant of ROUGE-N):

For this ROUGE-1 example, combine precision and recall into an F1 score:

Where:

• $P$ is the precision metric.

• $R$ is the recall metric.

Note: Here, $Length$ counts tokens under the example’s splitting rule. For instance, [The, cat, is, on, the, parrot] contains six tokens. Real evaluations must apply the same tokenization to references and candidates.

Let’s calculate these values for our example:

The example’s F1 score is about 0.77 because five tokens match a seven-token candidate and a six-token reference. That describes overlap under the stated counting rule; it does not establish that the two sentences express the same claim.

The F1-score is the harmonic mean of precision and recall: $F_1=2PR/(P+R)$. It is high when both precision and recall are high.

Perplexity score

Perplexity summarizes the probabilities a language model assigns to an observed token sequence. It is the exponential of average negative log-likelihood. Lower perplexity means the model assigned higher probability to those tokens under the available context; it is not a direct measure of truth, coherence, or calibrated confidence.

• Compare perplexity on the same data with a compatible tokenization and context policy. The Transformers explanation shows why tokenization and fixed context windows affect the result.

• There is no universal threshold of 20 that separates a confident model from an uncertain one. Interpret the value relative to the evaluation setup and suitable baselines.

Consider a hypothetical word-level model with vocabulary ["Hello", "Cat", "My", "Dog", "name", "is","Edward"] and the sentence “My name is Edward.” The conditional probabilities below refer to those exact word tokens.

• The probability of the first word: $P(\text{“My”})$

• The probability of the second word given the first word is “My”: $P(\text{“name”}|\text{“My”})$

• The probability of the third word given the first string is “My name”: $P(\text{“is”}|\text{“My name”})$

• The probability of the fourth word given the first string is “My name is”: $P(\text{“Edward”}|\text{“My name is”})$

Assign hypothetical conditional probabilities of 0.3, 0.5, 0.9, and 0.7 to the four words:

The perplexity $\text{PPL}(x)$ here will be $\frac{1}{\text{PNorm}(x)}$. Here, $\text{PNorm}(x)$ names the geometric mean token probability for a sentence $x$ with word count $n$. It is not a probability distribution normalized across sentences:

The geometric mean token probability is approximately 0.5544, so perplexity is about 1.80. This calculation describes the assigned probabilities; it does not verify the sentence’s coherence.

CLIP score

CLIP uses jointly trained image and text encoders to compare the two modalities in a shared representation. CLIP-based scores use this relationship as a proxy for image-text compatibility. The published CLIPScore method was evaluated for image captioning without requiring a reference caption.

The basic comparison has two steps:

1. Encode: Apply the corresponding CLIP encoders to the image and its description.

2. Compare: Normalize the feature vectors and calculate their cosine similarity. The result reflects the encoder’s learned representation, which can miss a wrong count, attribute, or relationship.

Cosine similarity lies between -1 and 1, but those endpoints are geometric relationships between vectors, not calibrated judgments of meaning. Published CLIPScore applies a nonnegative clipping and scaling rule; it is not simply a universal normalization to a zero-to-one quality scale.

• Specify the encoder, preprocessing, and score transformation before comparing results.

• Choose thresholds using examples from the target task. Values such as 0.6 or 0.5 do not provide universal strong- and weak-alignment boundaries.

Note: A CLIP-based evaluation can be adapted to other multimodal tasks, but the sampling and aggregation rules must be defined. A frame-level image-text comparison, for example, does not by itself measure a video’s temporal consistency.

Automatic scores leave task judgments that the following human-review methods can address.

Human evaluation

Human reviewers can assess dimensions that automatic metrics miss, such as whether a story follows its prompt or a caption is misleading. Their judgments still need an explicit rubric and representative examples. The methods below differ in whether reviewers give an overall score, separate dimension scores, or a relative preference.

Mean opinion score

A mean opinion score, or MOS, averages reviewers’ ratings on a shared scale. For example, a study might use one to five with descriptions of what each value means. Report the rating procedure and variation across reviewers alongside the mean; the average can hide disagreement.

Task-specific quality evaluation

Task-specific quality evaluation uses separate rubric dimensions. For a writing task, these might include:

• Fluency: Determines the grammatical correctness and natural flow of text.

• Relevance: Determines how well the generated output aligns with the input prompt or context.

• Creativity: Determines the originality and novelty of the content, especially in tasks like story generation or art creation.

Consider the hypothetical prompt “Write a brief story about a space explorer.”

• Model A generates: “Captain Nova landed on distant planet, marveling at its blue vegetation.”

• Model B generates: “The explorer bought a telescope to look at the stars.”

A reviewer could score the three dimensions separately:

The illustrative rubric can use a one-to-five scale for each dimension. Keep the dimensions separate when an overall average would hide a serious defect, such as accurate grammar paired with a response that ignores the prompt.

Pairwise comparison

In a pairwise comparison, a reviewer sees two outputs for the same input and chooses the better one under a stated criterion. This directly supports a relative model comparison, but it does not establish that either output meets an absolute acceptance standard. Allow a tie or a “both unacceptable” judgment when the study’s contract requires it.

Pairwise comparison can capture human preferences that automated metrics miss. Use representative prompts and consistent evaluation instructions; a larger sample alone does not eliminate evaluator bias.

Conclusion

A useful evaluation plan names the failure each check should catch. BLEU and ROUGE measure reference overlap; perplexity measures assigned token probabilities; IS, FID, and CLIP-based scores inspect different properties of generated images or image-text pairs. Human ratings add task-specific judgments, with their own cost and variability.

Before choosing a model, write down one case where the preferred automatic score could improve while the task result gets worse. The “large” versus “small” example already supplies one. Include such cases in the evaluation set, then combine metric results with a rubric that tests the actual requirement. Agreement between several scores is useful evidence only when their checks cover the failures that matter.