Enricher

Add an enricher that summarises and tags a document, a page or a chunk.

An enricher says what a text says and what it is about: an Enrichment carrying a summary, prose in the text’s own language, and tags, LabelScore entries whose vocabulary is yours rather than a taxonomy’s. It is the third return shape beside the classifier and the entity extractor: a classifier says what a text is, an extractor what is named in it, an enricher what it says. There are three, one per unit: DocumentEnricher, PageEnricher and ChunkEnricher, and one class may serve all three. Package and install it as the overview describes.

from typing import Any

from indx_interfaces import (
    CapabilityDescriptor,
    CapabilityId,
    Device,
    DocumentEnricher,
    EnricherId,
    Enrichment,
    LabelScore,
    PageEnricher,
    excerpt,
)


class HeadlineEnricher:
    # What a request names in `enrichment.document_ids` or `enrichment.page_ids`.
    # IDs are one namespace across all eight annotation ports.
    id = EnricherId("acme-headline")
    device = Device.CPU

    def enrich(self, text: str) -> Enrichment:
        # You are handed the whole unit; bound it yourself if your engine needs it.
        head = excerpt(text, 200).strip()
        if not head:
            # Nothing to say is an answer: an empty enrichment lets the next
            # enabled enricher speak.
            return Enrichment()
        # A part you cannot give is left out, not filled with a blank.
        return Enrichment(
            summary=head.splitlines()[0],
            tags=(LabelScore(label="invoice", confidence=0.6),) if "Invoice" in head else (),
        )


class Provider:
    def descriptors(self) -> tuple[CapabilityDescriptor, ...]:
        # This distribution annotates what other capabilities read; it reads nothing.
        return ()

    def create(self, capability_id: CapabilityId) -> Any:
        raise ValueError(f"this provider declares no capabilities, so not {capability_id}")

    def __init__(self) -> None:
        # One object through two hooks: the registry advertises the ID once and
        # would refuse a second instance as a duplicate.
        self._enricher = HeadlineEnricher()

    def document_enrichers(self) -> tuple[DocumentEnricher, ...]:
        return (self._enricher,)

    def page_enrichers(self) -> tuple[PageEnricher, ...]:
        return (self._enricher,)

Each part is won separately. A request enables enrichers in order, and the first with a summary wins the summary for that unit while the first with tags wins the tags. Give the part you can defend and leave the other out; a second enricher fills it rather than replacing yours. A blank summary is refused by the model, so leave the field None instead.

A caller enables yours with {"enrichment": {"document_ids": ["acme-headline"]}} on encode, or names it under page_ids. Installed IDs are advertised on the capability snapshot outside its content hash, so installing yours moves no plan. Raising means “mine, and broken”: the executor logs it, skips that unit, and asks the next enricher rather than failing an encode the run already paid to read. device and cost_usd are read off your class with defaults, exactly as for a classifier. Declare Device.EXTERNAL for a hosted model, and a request carrying data_residency is refused before the source is fetched rather than silently sending the text off-box.

For chunks instead

Declare through chunk_enrichers(), and a request names your ID in enrichment.chunk_ids. Everything else is identical. Two consequences: chunk granularity is required, so naming a chunk enricher in a request that did not ask for CHUNK is a 422 before the fetch, and you will be called once per chunk rather than once per document, so a model that costs a call per invocation costs a great many more.

What the executor does with your answer

A document enricher’s answer is written to the document block’s metadata under enrichment, as your Enrichment serialises with only the parts you gave. A page enricher’s answer goes to each page block under the same key. A chunk enricher’s answers ride the document block under chunk_enrichment, keyed by chunk block ID, because chunk blocks carry no metadata. The trace names you on every unit you won a part on.

enrichment and chunk_enrichment are reserved: a caller supplying either on EncodeRequest is refused rather than overwritten.

Checklist

Write the summary in the text’s language, as prose. Never a blank.
Leave a part out rather than returning an empty one.
Rank your tags and report a confidence you can defend; nothing calibrates it.
Advertise nothing when you cannot run, without your extra or your model, rather than an enricher that raises.
Keep module scope cheap; the engine import belongs inside enrich(), and a model behind the first call.

Enricher

For chunks instead

What the executor does with your answer

Checklist

Reference

Developer