Metrics

Evaluate mention recognition, entity resolution, and the complete linking pipeline separately. Define exact-match correctness before counting errors, then state whether you aggregate counts across documents or average document-level scores.

In the previous lesson, we talked about various applications of named entity linking system and how it can be used as a component in bigger tasks/systems such as a virtual assistant system. Therefore, we require metrics that:

1. Compare different entity linking models based on their performance.

   This will be catered to by offline metrics.

2. Measure the performance of the bigger task when a particular model for entity linking is used.

   This will be catered to by online metrics.