What Is Named Entity Recognition and How Is It Used to Train AI?

If you are considering text annotation or AI evaluation work, understanding NER in depth gives you a significant advantage. This guide covers what NER is, how it works, what annotation tasks it generates, and where specialist domain knowledge commands the highest rates.

---

What named entity recognition actually does

NER is the task of identifying and classifying named entities within a text. An entity is a real-world object that can be referred to by name: a person, an organisation, a location, a date, a quantity, a chemical compound, a legal instrument, a medical condition.

Given a sentence, an NER system identifies each entity mention and classifies it into a predefined category.

A basic example:

"The European Medicines Agency approved the use of Pembrolizumab for unresectable stage III non-small cell lung cancer in March 2024."

An NER system trained on biomedical text would identify:

• European Medicines Agency → Regulatory body
• Pembrolizumab → Drug / therapeutic agent
• unresectable stage III non-small cell lung cancer → Disease / indication
• March 2024 → Date

This seems simple in an isolated sentence. In practice, NER operates on complex documents where entity mentions are ambiguous, entities refer to each other across long distances in the text, and the same word can be an entity in one context and not in another.

---

Why NER matters for AI applications

NER is a prerequisite for a wide range of downstream AI capabilities.

Information extraction Before an AI can answer a question like "which drugs were approved for lung cancer indications in 2024?" it needs to have extracted the relevant entities and their relationships from a large body of text. NER is the first step in that extraction pipeline.

Document search and retrieval Legal, medical, and financial document search systems use NER to index documents by the entities they contain. A contract management AI that can retrieve all contracts mentioning a specific counterparty by name needs NER to have identified and indexed those entity mentions.

Knowledge graph construction Knowledge graphs represent relationships between entities: Drug A treats Disease B, Company X acquired Company Y, Regulation Z applies to Industry W. Building these graphs from unstructured text requires first identifying the entities, which is NER, and then identifying the relationships between them, which is a related task called relation extraction.

Automated summarisation Summarisation AI systems use entity recognition to ensure that key entities, the people, organisations, and events most central to a document, are preserved in a summary. Without entity awareness, a summarisation system might produce an accurate summary that omits the name of the defendant in a court judgment or the drug being evaluated in a clinical trial.

---

The annotation task: how NER training data is created

NER models learn from labeled text where human annotators have already identified and classified every entity mention. This is the annotation task.

How it works in practice

An annotator is given a document and an entity schema: a defined set of entity types they should identify, with clear definitions and examples for each. They read through the text and mark each entity mention, applying the correct label.

The complexity varies dramatically with the domain and the schema.

A general news schema might have half a dozen entity types: Person, Organisation, Location, Date, Product, Event. Most entity mentions are unambiguous. "The Prime Minister" in a UK news context refers clearly to a Person. "Heathrow Airport" is clearly a Location.

A clinical NER schema for annotating discharge summaries might have thirty or more entity types: Drug (generic name), Drug (brand name), Dosage, Route of administration, Frequency, Disease, Symptom, Body site, Procedure, Laboratory test, Laboratory value, Negation (to capture "no fever" as distinct from "fever"). Deciding whether "500mg twice daily" is a Dosage, a Frequency, or both, and how to handle the relationship between the drug mention and the dosage mention nearby in the text, requires genuine clinical familiarity with how medical language works.

A legal NER schema for contract analysis might require identifying: Parties, Defined terms, Obligations, Rights, Conditions, Dates, Monetary values, Governing law clauses, Jurisdiction references, Termination triggers. These categories require understanding how legal documents are structured and what legal significance each entity type carries.

---

Where specialist knowledge makes the difference

General NER annotation is relatively accessible. Specialist NER annotation is where domain expertise commands a significant premium, because the entity types are only meaningful to someone who understands the domain.

Clinical and biomedical NER

Clinical NER annotation is among the most complex and best-paid text annotation work available. The reasons:

Disambiguation is technically demanding. Consider the word "MS" in a clinical document. It might refer to multiple sclerosis (a disease), morphine sulfate (a drug), or a manuscript (in a research context). Correct entity classification requires reading the surrounding text with genuine clinical understanding.

Negation and speculation are clinically significant. "The patient denied chest pain" and "the patient reported chest pain" contain the same entity (chest pain) but opposite clinical meanings. NER schemas for clinical text typically include negation and uncertainty markers, which non-clinical annotators often apply incorrectly.

Anatomy and pharmacology vocabulary is large and precise. Recognising that "atorvastatin" and "Lipitor" refer to the same drug (one generic, one brand name), or that "myocardial infarction" and "heart attack" are the same condition in different register, requires pharmaceutical and clinical knowledge that takes years to develop.

Legal NER

Legal NER requires understanding how legal documents use language deliberately and precisely. Entity types that seem similar to non-lawyers are technically distinct. An "obligation" (a party must do something) is different from a "right" (a party may do something) is different from a "condition" (something that must be true for another provision to apply). Misclassifying these entity types in a contract analysis AI training dataset produces a model that cannot reliably distinguish mandatory from optional provisions, which is a significant failure for a commercial legal application.

Financial NER

Financial NER annotation appears in several contexts: earnings call transcription analysis, regulatory filing review, analyst report summarisation, and financial news monitoring. Entity types include not just company names and monetary figures but more complex categories like financial instruments (is "the Series B preferred stock" one entity or several?), forward-looking statements (which have specific regulatory meaning), risk factors, and accounting line items.

Scientific literature NER

Annotating scientific papers for NER involves entity types like chemical compounds, gene and protein names, species, experimental methods, measurement units, and statistical results. The annotation schemas for scientific literature NER are often developed in collaboration with domain experts specifically because the entity types require specialist understanding to apply consistently.

---

Relation extraction: the step after NER

NER identifies that "Methotrexate" is a Drug and "rheumatoid arthritis" is a Disease. It does not establish that Methotrexate is used to treat rheumatoid arthritis.

Relation extraction is the annotation task that captures relationships between entities. It is typically performed after NER, using the entity annotations as a foundation.

Common relation types in biomedical text:

Treats

Causes

Contraindicated in

Inhibits

Biomarker of

In legal text:

Bound by

Entitled to

Effective from

Governed by

Relation extraction annotation requires more judgment than NER because relationships are not always explicitly stated. "The drug is approved for use in patients with advanced colorectal cancer" contains an implicit indication relationship between the drug entity and the disease entity. An annotator needs to recognise that relationship even though the text does not use a word like "treats" or "indicated for."

---

Quality standards in NER annotation

NER annotation quality is measured primarily through inter-annotator agreement: the degree to which different annotators make the same entity classification decisions on the same text.

Agreement is typically calculated as an F1 score comparing two annotators' outputs. F1 combines precision (were the same spans selected?) and recall (did both annotators find the same entities?).

Clinical NER projects targeting F1 above 0.9 require not just skilled annotators but careful schema design, extensive guideline documentation, and regular calibration sessions where disagreements are discussed and resolved.

The investment in reaching this level of agreement is justified by what the data is used for. A clinical NER model trained on annotation with 0.75 inter-annotator agreement will have lower precision and recall than one trained on annotation at 0.90. In a clinical context, that difference in model performance has direct patient safety implications.

---

Frequently asked questions

Is NER annotation work available remotely? Yes. NER annotation is entirely text-based and is typically completed through a web interface. All major annotation platforms offer NER projects for remote contributors.

How is NER different from text classification? Text classification assigns a single label to an entire document or sentence ("this review is positive"). NER assigns labels to specific spans of text within a document, identifying individual mentions rather than classifying the whole. A single document can contain dozens or hundreds of NER annotations.

Can I do NER annotation without reading the entire document? Technically yes, but good NER annotators read the surrounding context for each entity mention rather than labeling in isolation. The correct entity classification often depends on context that appears sentences earlier or later in the text.

What tools are used for NER annotation? Most platforms have proprietary NER annotation interfaces that allow annotators to highlight text spans and apply labels. Common standalone tools include Prodigy, BRAT, Doccano, and Label Studio. The specific tool matters less than the underlying skill of reading carefully and applying a schema consistently.

Do NER annotation projects require previous annotation experience? For general NER, detailed guidelines and a qualification task are usually sufficient to get started. For specialist clinical, legal, or financial NER, a combination of domain knowledge and annotation experience is typically required. Platforms often assess both separately.

---

Summary

Named entity recognition is a foundational AI capability with applications across medicine, law, finance, science, and media. The annotation work that trains NER models ranges from accessible general tasks to highly specialist clinical and legal annotation that requires genuine domain expertise.

The pay premium in NER annotation mirrors the complexity of the domain. General NER annotation is competitive and moderately paid. Clinical NER annotation that requires understanding medical language, negation, and clinical context is among the best-compensated text annotation work available. The combination of linguistic precision and subject matter expertise is what makes it rare and valuable.