API contract
The full HTTP, CLI, and Python contract, including the typed error envelope.
The HTTP API is a transport over the same public Pydantic contracts used by the Python facade. The contract is synchronous, and stateless except for the documents a deployment opts into keeping (Kept documents).
Operations
Section titled “Operations”| Operation | Purpose |
|---|---|
GET /health |
Process liveness and package version |
GET /v1/capabilities |
Current versioned capability and embedding-space snapshot |
POST /v1/plan |
Inspect processing decisions without executing them |
POST /v1/encode |
Execute an optional supplied plan or create one first |
POST /v1/embed |
Encode a text or image query into a selected embedding space |
POST /v1/documents |
Encode a document and keep it for the caller until it expires |
GET /v1/documents |
List the caller’s kept documents |
GET /v1/documents/{digest} |
Read one kept document with its whole result |
GET /v1/documents/{digest}/source |
Download the file a kept document was read from |
DELETE /v1/documents/{digest} |
Forget a kept document now |
POST /v1/search |
Search the caller’s kept chunks by words and by meaning |
Inputs
Section titled “Inputs”Planning and encoding accept either:
application/jsonwith a URI source or base64 inline bytes; ormultipart/form-datawith a binaryfileand a JSONrequestform part. That part isPlanRequestPart/EncodeRequestPart/EmbedRequestPartin the schema — the JSON request without the field the upload supplies, since the file is the source. It is a named schema rather than an opaque string so a generated client can build it.
Query embedding accepts inline text, a URI/inline image in JSON, or an image in the same multipart form. Both transports validate into the same domain model.
Capabilities
Section titled “Capabilities”GET /v1/capabilities returns the current snapshot: the capabilities and
embedding spaces this installation can route against, plus the routing policy
version they belong to.
The snapshot ID is content-addressed, so the same inventory always has the same
identity and a caller can detect that its environment moved. An optional
snapshot_id query parameter re-requests a specific retained snapshot instead
of the current one; an ID that is not retained returns 404 rather than
silently answering with a different inventory.
Capability availability follows what is installed. A provider distribution advertises its capabilities through an entry point, so installing one is what makes its capability appear here, and a provider that fails to load is omitted rather than taking the endpoint down. Snapshot IDs are stable for a given install, not across releases.
Planning
Section titled “Planning”A successful plan contains:
- deterministic
plan_idand caller/request ID; - source content digest;
- capability snapshot and policy versions;
- observed public decision reasons and signature matches;
- document default, page overrides, optional region overrides;
- ordered fallbacks and estimated cost/latency/quality; and
- an explicit ready or unsatisfied status.
The response is a PlanResult: that plan under plan, and beside it components
naming the distributions that produced it – the loader that fetched the bytes,
the sniffer that named the media type and the observer that produced the
evidence, in the shape the encode trace uses. Only the plan is hashed and
bound; the components are diagnostics, and a plan supplied back to encode is
the plan field alone.
Private raw preflight features are not part of the wire contract.
signature_detection asks the installed parsers whether they recognize the
document. A match reports the signature, the scope it covers, the evidence it
matched on, and the capability it nominates; that capability then leads the
route for those scopes while the generic ladder stays behind it as fallbacks, so
a wrong signature costs one cheap attempt rather than the document. confidence
is the share of independent signals matched, a signal counting in proportion when
it is one (the share of a process chart’s symbols that carry a label), not a
calibrated probability – nothing in this release has been measured against a
labelled corpus. A match whose nominated
capability is not installed is still reported, because the caller is entitled to
know their document was recognized even when the thing that recognized it is
gone; it simply does not change the route.
A media type only a parser declares, planned with signature_detection unset,
comes back unsatisfied naming the parser and the flag. Nomination is a parser’s
only door, and the silent alternative – routing every page to manual review –
would read exactly like an install carrying no parser at all.
Encoding
Section titled “Encoding”POST /v1/encode accepts an optional plan. If present, its source digest, media
type and snapshot must match the request. The executor may use the selected route or its
declared fallbacks only; it cannot silently replan.
It also accepts metadata, a free-form JSON object of the caller’s own labels –
an owner, a tenant, a sensitivity classification. It travels with the source and
comes back on the document block, and indx enforces none of it: who may read a
vector afterwards is the index’s question rather than the router’s. The field is
on EncodeRequest alone. plan_id is the hash of every field of the plan it is
decided into, so a label on PlanRequest would give two identical documents two
plans for a value that routes nothing. Every key the executor writes is refused
rather than accepted: languages from an installed language detector,
classification and chunk_classification from the classifiers a request
enabled, entities from the entity extractors it enabled, enrichment and
chunk_enrichment from the enrichers it enabled, lanes from all three, and a
label silently replaced
is a label the caller believes travelled and did not.
And it accepts classification, extraction and enrichment, all absent by
default, all naming installed implementations by ID per unit: classification
and enrichment take document_ids, page_ids and chunk_ids, extraction
takes page_ids and chunk_ids, and each list is asked in the order given. Nothing bounds how much
of the document an implementation is shown – each is handed the full text of its
unit and applies its own bound if it needs one, so there is no sample field and
no deployment default behind it
(ADR-0033).
Three refusals, all 422, all before the source is fetched. An ID no installed
provider declares is unknown_classifier, unknown_extractor or
unknown_enricher, naming the ones that are. One that would hand the text to an external service is
data_residency when the request carries that constraint. And a chunk_ids
entry in a request that did not ask for CHUNK granularity is
granularity_required, because there would be no chunks to annotate and an empty
answer is indistinguishable from no opinion. The IDs a caller may name are on the
capability snapshot, outside its content hash; they are one namespace across all
eight ports, so no two may share a string
(DocumentClassifier,
classifiers,
entity extractors,
enrichers).
chunking names the chunkers to ask first: {"chunkers": [{"id": "pdf", "version": 2}]}, the
version optional. It is a preference and not a replacement
(ADR-0068):
the named chunkers lead, in order, and the installation’s own order still serves every page they
did not claim, so naming a PDF chunker costs a slide deck nothing. An ID without a version means
the version this installation asks by default, which the snapshot marks default on chunkers,
and the newest where none is. A version not marked default runs only for the requests that name
it. The refusals are 422 invalid_chunker, before the fetch like the others: unknown_chunker,
unknown_chunker_version, each naming what is installed, and granularity_required for a
request that names chunkers and asked for no chunks. The trace’s chunker components say which
version cut each page. Beside default, each advertised pair carries media_types, the types
the chunker cuts (empty means any), and summary, one phrase saying what a chunk is under it;
both are for a caller choosing between versions for a document, and neither gates a request.
The response returns the plan, a flat addressable block tree, embedding-space
descriptors, execution trace, and usage. Blocks represent document, page,
region, and chunk scopes and link through id / parent_id.
How many chunks a page yields is installation-dependent: boundaries are drawn
by installed chunker distributions (Chunker), first answer
per page wins, and indx-chunker-page is the floor that keeps one chunk per
readable page. An install carrying no chunker returns no chunk blocks and no
document vectors, the way a stock install resolves no URI.
A chunk block carries either a bbox or a bbox_reason, never both and never
neither. bbox_reason is one of no_geometry (the chunker knows no positions:
worksheets, Word sections, text and email; a page read by recognition gets a
rectangle per recognized line instead), hidden (the source
marks the shape hidden; its text is read, nothing is drawn) or unresolved (the
format has positions and this shape’s could not be resolved). PDF pages read by
the text layer and PowerPoint slides carry boxes; every other chunk today says
no_geometry. Document and page blocks carry neither field.
A chunk block may also carry a layout, when its chunker can tell what the chunk is
(ADR-0070):
a role of paragraph or table, and for a table its rows, columns, header_rows and
cells, each cell naming a 1-based row and column, its spans, its text and a bbox where the
format has positions. The chunk’s text is still the plain rendering, a table’s rows on lines and its
cells tab-separated, so nothing that reads only text changes. Absent means the chunker did not say.
The document block’s provenance names what the source was read as and where it
came from – media_type, plus source_uri for a URI source or filename for
an inline one that supplied a name – beside the plan bindings every block
carries. Provenance rather than metadata, because the glossary reserves this dict
for where a block came from and metadata for what was extracted out of it; an
exported row has to say where it came from without the plan beside it, and a
digest is not something anyone searches for.
Its metadata carries the caller’s labels and, where a language detector is
installed, languages: {"language": "ja", "confidence": 0.98} entries, highest
first, on each readable page block and, averaged over the pages by how much text
each score was computed over, on the document block. A stock install detects
nothing and carries no key at all
(LanguageDetector).
Where the request enabled a classifier, the document block’s metadata also
carries classification: {"document_type": [{"label": "invoice", "confidence": 0.9}]},
a facet per key, labels highest first, the first enabled classifier with an
opinion on a facet winning it. Facet names are the classifier’s own, and a
document nothing could read carries no key. A page classifier writes to each page
block instead, and a chunk classifier to the document block’s chunk_classification,
keyed by chunk block ID, because chunk blocks carry no metadata.
Where the request enabled an entity extractor, the document block’s metadata
carries entities: the spans found, keyed by the block each was found in, each a
surface text with a [start, end) character range into that block’s own text
and a confidence; and beside them a count per (label, NFKC-normalized surface form) with the raw surface forms kept. A count is not an identity – nothing here
resolves coreference
(entity extractors).
Where the request enabled an enricher, the document block’s metadata carries
enrichment: {"summary": "...", "tags": [{"label": "billing", "confidence": 0.9}]},
only the parts answered, the first enabled enricher with an opinion on a part
winning it. A page enricher writes to each page block instead, and a chunk
enricher to the document block’s chunk_enrichment, keyed by chunk block ID
(enrichers).
Those keys hold the merge, what each lane won. Every lane’s whole answer, the
answers that lost included, rides beside it under lanes: {lane ID: {...}},
where {...} holds the keys above that lane answered, in the same shape, with
entities counts over that lane’s spans alone. It sits on the document block and,
for page classifiers and page enrichers, on each page block. A lane with no
opinion has no entry, and a block no lane answered has no lanes key. Every lane
a request names is asked and billed, even one whose every facet an earlier lane
already won
(ADR-0079).
Fields a specialized capability extracted ride on metadata of the block that
stated them – an invoice’s number, dates, and amounts under an invoice key.
Money stays a digit string rather than a number, because money in a float is a
rounding waiting to happen, and dates stay the strings the document printed,
because normalizing 5/8/2026 means deciding whether it is May or August and
nothing here knows the issuer’s locale.
A slide the process-chart signature recognized carries process_chart on its page
block, the same way: {"title", "nodes": [{"id", "kind", "symbol", "role", "label", "bbox"}], "edges": [{"id", "kind", "from", "to", "arrow", "path"}]}. A node’s kind is
symbol (with symbol in triangle, ellipse, diamond, double_triangle and role in
material, operation, inspection, assembly), picture, container or text; an
edge’s is connection (a connector between two symbols, arrow saying whether the file drew
one, path its two ends) or contains (a container and a node whose centre it encloses).
IDs are shape: plus the shape’s own cNvPr id, stable within a file; bbox and path
use Block.bbox’s normalized convention, so a node is overlaid with the arithmetic a chunk
uses. The signature is page-scoped: the match names the slide, and process-chart-parser
leads only that page’s route. Fields absent from a node or edge are omitted, not null.
A block whose scope could not be read carries unreadable or failed with a
reason rather than being omitted, and the trace records every capability that
was attempted, including the ones that failed and what stopped them. That is
what makes a fallback visible: a trace showing only the capability that finally
worked would hide the reason fallbacks exist at all.
The trace’s components names what took part without a capability ID to be
named by: {"role": "chunker", "distribution": "indx-chunker-pdf", "id": "pdf", "version": 1, "pages": [1, 2]}.
A chunker states its id and version, the pair the snapshot advertises on chunkers
outside its content hash; no other role carries a version.
Roles are loader, sniffer, chunker, language_detector and classifier;
distribution is the installed package that declared the component, pages are
the pages a chunker cut or a detector answered, and a classifier repeats the id
the request named beside the facets it won. An inline source of a declared type
names no loader and no sniffer, because nothing fetched or recognized it. The
observer is absent by construction – execution never observes – and belongs to
the plan response once that has a wrapper to carry it.
Two outcomes deliberately are not errors. A plan whose constraints nothing
installed can satisfy comes back 200 with a failed document block and a failed
trace, because nothing conflicts and the request was valid – flattening the
caller’s constraints into an error string would lose the part they can act on. A
route whose every declared capability failed is a 503: the routing ladder ends
in manual review by construction, so exhausting one means an installation missing
its terminal fallback, or a caller who set constraints.fallbacks_allowed to
false. Together with the constraints.capability_ids allowlist, that flag is
how a request names its reader (ADR-0063). An allowlist that leaves a page nothing
to read it, or names an ID the snapshot does not hold, is an unsatisfied plan whose
line starts with capability_ids:.
Each vector records its embedding_space_id and embedder_id. An embedding
space declares vector dimension, distance metric, normalization, and compatible
document/query embedders. Text and image embedders share a space ID only when
their vectors are directly comparable.
Query embedding
Section titled “Query embedding”POST /v1/embed requires an embedding-space ID. It chooses that space’s
query-role embedder for the input modality and returns vectors plus the exact
public embedder configuration used. Provider credentials and private endpoints
never appear in the descriptor.
A space that is not in the snapshot, or one with no query-role embedder for the
input’s modality – an image into a text-only space – is a 422 with
type: invalid_embedding_space and a code naming which. An embedder that is
advertised but cannot run, or fails, is a 503: embedders have no fallback
ladder, because a caller who named a space asked for exactly that one.
Every install carries default-text, a 256-dimension feature-hashed character
trigram space. It is a lexical baseline – two strings that share character
grams land near each other and nothing else does – offered so that vectors
exist without a model, not as retrieval quality. uv sync --all-packages --extra fastembed adds minilm-multilingual (384-d, text) and clip-vit-b32
(512-d, text and image queries in one comparable space) through fastembed, and
--extra hosted a hosted text space, hosted-text, configured with
INDX_EMBED_MODEL (a
LiteLLM model id) and INDX_EMBED_DIMENSION, optionally INDX_EMBED_API_KEY,
INDX_EMBED_API_BASE, INDX_EMBED_TIMEOUT_SECONDS and INDX_EMBED_RETRIES.
The dimension is declared rather than discovered because a space has to state
it before any vector exists and a hosted provider only answers for a fee. The
hosted embedder’s device follows the vision lane’s rule: a model LiteLLM routes
to this box is gpu, a hosted one is external, and a plan request naming that
space under data_residency is refused the way the hosted vision lane is. What
a hosted embedder reports charging joins cost_usd on the trace and the usage.
Document vectors are produced by POST /v1/encode for every space named in
embedding_space_ids, one per chunk block; page and document blocks carry none.
A chunk is usually text, and sometimes a rendered page. A page no capability
could read as text has no boundary to cut, and used to leave a page block
carrying its reason and nothing retrievable; indx-chunker-pdf renders it into
one image chunk instead. A space is selected by what a run produces rather than
by naming a text embedder, and what a run produces is
DOCUMENT_EMBEDDING_MODALITIES – (text, image), in that order, so a page
that was read never also pays for a render. A space whose only document-role
embedder reads images is therefore routable: it answers for exactly the pages
that were rendered, and a document may carry text vectors and image vectors in
one space. What still comes back unsatisfied is a space declaring no document
lane at all, saying it embeds no document input this run produces rather than
reporting a missing text embedder it in fact has. Query lanes remain
unconstrained by any of it.
Kept documents
Section titled “Kept documents”A deployment that names a record store and a vector index keeps what its callers
encode, each caller’s apart, until a TTL runs out
(ADR-0071).
One that names neither keeps nothing, and every route below answers
404 store_disabled.
POST /v1/documents takes the same body as POST /v1/encode, JSON or
multipart, runs it, and returns 201 with the StoredDocument: a summary
(digest, name, media type, page and chunk counts, the spaces its vectors are in,
and when it was created, updated and expires) beside the whole EncodeResult.
name is a query parameter and defaults to the uploaded file’s name. Within a
caller a document is its source digest, so keeping the same bytes again replaces
it and restarts its TTL.
A deployment that also names a source store keeps what the document was read from, under the same caller and until the same expiry, so that a document gathered into a dataset can be read again (ADR-0081). Unset, the source is discarded after the read, as before.
GET /v1/documents/{digest}/source returns the original bytes to the caller who kept them
(ADR-0084),
under the media type they were kept with. The answer is always
Content-Disposition: attachment with the file’s name in filename*, and
X-Content-Type-Options: nosniff: a browser saves an uploaded file and never renders it. A
kept document’s summary carries source_kept, true when the deployment stored its bytes; it is
false for a document kept from a URI. Another caller’s document, one never kept, a URI source
and a deployment that keeps no source are all 404 source_not_kept.
The caller is the indx_owner cookie: set on the first keep, refreshed on every
one, HttpOnly, SameSite=Lax, scoped to /v1, and Secure unless
INDX_STORE_COOKIE_SECURE=false. It is a random bearer token, and the server
keeps only its SHA-256. A caller with no cookie has kept nothing. Another
caller’s document is 404 document_not_found, the same as one never kept, and
DELETE answers 204 either way, so nothing tells a caller what someone else
kept.
POST /v1/search searches every chunk the caller kept by the words it contains,
including chunks kept without embeddings. Naming embedding_space_id adds a
second leg: the query is embedded in that space and compared with the chunks
that have a vector in it. Every chunk the words found comes first, and chunks found only
by meaning follow: a space resembles by shared letters or by sense, and a resemblance is not
as good an answer as a chunk that says what was asked. Within each group the legs are fused
by reciprocal rank, so a hit’s score orders the response and means nothing on its own. A
chunk of nothing but punctuation is never offered on resemblance alone. text_score (BM25)
and vector_score (the space’s metric, higher nearer) are each leg’s own, and
lanes says which legs found it. A space with no text query embedder is the
422 POST /v1/embed gives.
| Variable | Default | Meaning |
|---|---|---|
INDX_DOCUMENT_STORE |
unset | The indx.document_stores entry point to keep records in: memory, sqlite or postgres |
INDX_VECTOR_INDEX |
unset | The indx.vector_indexes entry point to keep chunks in; set with the one above or not at all |
INDX_SOURCE_STORE |
unset | The indx.source_stores entry point to keep a kept document’s source in: memory, sqlite or postgres. Unset, no source is retained |
INDX_STORE_TTL_SECONDS |
86400 |
How long a document is kept after its last keep |
INDX_STORE_MAX_DOCUMENTS_PER_OWNER |
20 |
A caller’s ceiling; keeping one more drops their oldest |
INDX_STORE_COOKIE_SECURE |
true |
Whether the owner cookie is Secure; false only where the server is reached over plain HTTP |
INDX_STORE_MEMORY_MAX_DOCUMENTS |
512 |
indx-store-memory: documents kept per process across every caller, least recently kept dropped first |
INDX_STORE_MEMORY_MAX_BYTES |
268435456 |
indx-store-memory: what each backend holds, counted as JSON length, least recently kept dropped first; a document over it alone is 413 input_too_large. Python holds 3 to 6 times this. The source store drops only sources that expire, and refuses with the same 413 when the sources datasets hold fill it |
INDX_STORE_MEMORY_PRESSURE |
0.8 |
indx-store-memory: the share of its memory limit (the cgroup’s, else the machine’s) at which the process drops the oldest tenth of its kept documents on each keep; 0 turns it off |
INDX_STORE_SQLITE_PATH |
unset | indx-store-sqlite: the file both ports keep their tables in, so what was kept survives a restart; naming sqlite without it fails at startup with store_not_configured |
INDX_STORE_POSTGRES_URL |
unset | indx-store-postgres: the libpq connection string of the database every port keeps its tables in, shared by every instance in front of it; naming postgres without it fails at startup with store_not_configured |
INDX_STORE_POSTGRES_SCHEMA_NAME |
indx |
indx-store-postgres: the Postgres schema the tables live in, created when missing, so one database can hold several installations |
The two backends are read when the server starts, because a backend holds what
it kept. The rest are read per request. indx-store-memory keeps everything in
the server process, so it is lost on restart and is not shared between two
instances. It suits a deployment that runs one instance.
Document types
Section titled “Document types”A deployment that names a schema store keeps the installation’s document types
(ADR-0075).
One that names none answers every route below with 404 store_disabled. Types
belong to the installation, not to a caller: every caller reads the same ones,
and they never expire.
GET /v1/types lists them by key, with writable: whether this deployment takes
writes at all. GET /v1/types/{id} reads one, and a key it does not hold is
404 type_not_found. PUT /v1/types/{id} creates or replaces one from a
DocumentTypeSpec (label, fields, signature) and keeps when it was first
made. DELETE /v1/types/{id} answers 204 whether or not the type was there.
A write needs Authorization: Bearer <INDX_WORKSPACE_TOKEN>. A missing or wrong
token is 401 unauthorized, and a deployment that set no token is read-only:
403 workspace_read_only. signature names the installed parser whose format
signature recognizes the type, such as invoice-parser; any other capability is
422 invalid_type.
| Variable | Default | Meaning |
|---|---|---|
INDX_SCHEMA_STORE |
unset | The indx.schema_stores entry point to keep document types in: memory, sqlite or postgres |
INDX_WORKSPACE_TOKEN |
unset | The operator’s bearer for every write to the types and labels; unset, they are read-only |
Labels
Section titled “Labels”A deployment that names a label store as well keeps reviewers’ verdicts on what
the lanes answered
(ADR-0076).
One that names none answers every route below with 404 store_disabled. Like
types, labels belong to the installation and never expire, and each copies the
answer it judged, so it outlives the kept document it was read from.
GET /v1/labels lists them by digest then ID, narrowed by digest and
type_id when given, with writable. PUT /v1/labels keeps a LabelSpec: the
document’s digest and document_name, a target (an entity with its block,
page, box, label, range and surface, or a facet with its unit, block, page,
facet and label), a verdict (approved, corrected or rejected), the
correction a corrected verdict needs, and optionally type_id, field and
reviewer. The label’s ID is derived from the digest and the answer judged, so
a second verdict on the same answer replaces the first. A type_id or field
the workspace does not hold is 422 invalid_label. DELETE /v1/labels/{id}
answers 204 whether or not the label was there. Writes need the same token as
types.
| Variable | Default | Meaning |
|---|---|---|
INDX_LABEL_STORE |
unset | The indx.label_stores entry point to keep labels in: memory, sqlite or postgres |
Datasets
Section titled “Datasets”A deployment that names a dataset store gathers reviewed documents into datasets,
one purpose each: training or evaluation
(ADR-0077). One that names none
answers every route below with 404 store_disabled. A dataset’s members are
documents, each a digest with its name copied; its examples are the labels those
documents hold now, so a dataset is a live view.
GET /v1/datasets lists them by key, each with examples, the count of labels its
documents hold, readable, the count of its documents whose source the installation
kept, and writable. GET /v1/datasets/{id} reads one, and a key it does
not hold is 404 dataset_not_found. GET /v1/datasets/{id}/labels lists its
examples. PUT /v1/datasets/{id} creates or replaces one from a DatasetSpec
(label, purpose, type_id, documents) and keeps when it was first made; a
type_id the workspace does not hold, or a document already in a dataset of the other
purpose for the same type, is 422 invalid_dataset. DELETE /v1/datasets/{id}
answers 204 and keeps the documents’ labels. Writes need the same token as types.
A PUT copies each added document’s source from what the caller kept (the
indx_owner cookie sent with the request) to the installation, where it does not
expire. A document the caller did not keep, or kept where no source store is named,
joins unreadable. A source no dataset names any longer is dropped.
| Variable | Default | Meaning |
|---|---|---|
INDX_DATASET_STORE |
unset | The indx.dataset_stores entry point to keep datasets in: memory, sqlite or postgres |
Routers
Section titled “Routers”A deployment that names a router store keeps the installation’s routers
(ADR-0078): each version a branch per document type, a default branch and
optionally a recognizer. One that names none answers every route below with
404 store_disabled. A version never changes once published.
GET /v1/routers lists the latest version of each router, with writable.
GET /v1/routers/{id}/versions lists one router’s versions, oldest first, and
GET /v1/routers/{id}/versions/{version} reads one; a router or version the
workspace does not hold is 404 router_not_found. PUT /v1/routers/{id} publishes a
RouterSpec (label, branches, default, recognizer) as the next version; restoring
an old version is publishing its spec again. A branch for a type the workspace does not
hold, or naming a reader, lane, chunker, space or recognizer this installation does not
have, is 422 invalid_router. DELETE /v1/routers/{id} drops every version and answers
204. Writes need the same token as types.
/v1/plan, /v1/encode and /v1/documents take router: {id, version?}; without a
version it is the latest. The server resolves it before planning, adding each type’s
parser and the digests a person approved to a type on the recognizer’s facet. The plan
names the version (router) and lists branches, typed_pages and recognizer; the
trace lists each page’s branch in branches. A request that names a router names no
chunkers, lanes or spaces of its own (422), and a supplied plan decided under another
version is 409 plan_conflict with param="plan.router".
| Variable | Default | Meaning |
|---|---|---|
INDX_ROUTER_STORE |
unset | The indx.router_stores entry point to keep routers in: memory, sqlite or postgres |
A version a promotion published carries origin, the experiment and the candidate it came
from; every other version’s origin is null, a restored one included.
Experiments
Section titled “Experiments”A deployment that names an experiment store, beside a workspace, scores candidate routes
on an evaluation dataset and promotes one
(ADR-0082,
ADR-0083).
One that names none answers every route below with 404 store_disabled.
PUT /v1/experiments/{id} creates one from an ExperimentSpec (label, dataset_id,
router_id, candidates). A candidate is a key, a label and a router branch that
names no type. The server freezes the dataset’s type and documents, the labels those
documents hold now, and the router’s latest version, and puts production first: the
branch the router has for the type. An experiment never changes, so a key already taken is
409 experiment_exists. A dataset that is not an evaluation set of one type, a document
with no stored source, a dataset with no verdict to score, or a candidate naming what is
not installed is 422 invalid_experiment.
PUT /v1/experiments/{id}/run starts or resumes the run in the background and answers at
once, or 404 store_disabled on a deployment that names no job runner; DELETE on the same path stops it before the next document. Each candidate reads
each document as the router version it would become, with the document typed before
reading. GET /v1/experiments/{id} returns the ExperimentReport:
stateisrunning,stoppedorcomplete, derived from the cells stored and never kept;cellsanddonecount them.candidatessums each candidate’s stored cells:documentsscored andfailed,expectedandright,pagesandneeds_person,cost_usdandlatency_msas the results reported them, androws, one per field (assigned) or unassigned entity label or facet.resultslists every cell, a failed one with itserror.promotionnames the router version a candidate was published as, once one was.
An entity verdict is right when the page’s merged entities hold its label with the same surface after NFKC folding, a facet verdict when its top label matches. Approved expects the answer, corrected the correction, rejected its absence. Offsets and boxes are not compared.
PUT /v1/experiments/{id}/promotion with {"candidate": key} publishes the router’s next
version with the type’s branch replaced, and returns it. It is 409 experiment_incomplete
unless the candidate scored every document, and 409 experiment_stale when the router has
moved past the version the run measured. GET /v1/experiments lists experiments by key
with their state, and DELETE /v1/experiments/{id} stops the run, drops the cells and
answers 204. Writes need the same token as types.
| Variable | Default | Meaning |
|---|---|---|
INDX_EXPERIMENT_STORE |
unset | The indx.experiment_stores entry point to keep experiments and their cells in: memory, sqlite or postgres |
INDX_JOB_RUNNER |
unset | The indx.job_runners entry point that runs an experiment in the background: memory, one worker thread in the server’s process, whose runs do not outlive it. Unset, an experiment is kept and read and cannot be run |
Errors
Section titled “Errors”All failures use one typed envelope containing type, code, message,
optional param, and optional request_id. That includes the transport-level
ones the router raises before an operation is reached: an unknown path answers
404 and a wrong method 405, both with type http_error and code
not_found or method_not_allowed. They belong to no operation, so they are
not listed in the per-operation responses of the OpenAPI document.
| Status | Meaning |
|---|---|
401 |
A write to the document types without the operator’s bearer token (unauthorized) |
403 |
A write to the document types on a deployment that set no token (workspace_read_only) |
404 |
Pinned capability snapshot ID is not retained (GET /v1/capabilities), the caller has no kept document with that digest (document_not_found), the workspace holds no type with that key (type_not_found), the workspace holds no dataset with that key (dataset_not_found), no such router or version (router_not_found) or no experiment with that key (experiment_not_found), or the deployment keeps no documents, types, labels, datasets, routers or experiments (store_disabled) |
409 |
Supplied plan does not match the source or capability snapshot; or an experiment’s key is taken (experiment_exists), its candidate has not scored every document (experiment_incomplete), or its router has moved (experiment_stale) |
413 |
Request exceeds the server-advertised input limit |
415 |
Content type is unsupported, no installed loader resolves the source URI scheme (unsupported_source), or no installed capability reads its media type |
422 |
Request validation failed, the source did not resolve to the expected bytes (source_unreadable), needs a password this server was not given (source_encrypted), names a destination a loader’s guard refuses to fetch (source_forbidden), asks for a block granularity nothing installed produces (unsupported_granularity), an embedding space cannot serve the request as asked, a document type names a signature no installed parser has (invalid_type), a label names a type or field the workspace does not hold (invalid_label), a dataset names an unknown type or a document that would both teach and grade one type (invalid_dataset), or a router names a type or an ID this installation does not hold (invalid_router) |
500 |
An error the server did not type; the message is generic and the X-Request-ID header locates the log line |
501 |
Contract exists but its application implementation has not landed |
503 |
A declared capability is temporarily unavailable |
504 |
The request exceeded INDX_REQUEST_TIMEOUT_SECONDS, or the caller disconnected before the work finished |
Every response carries an X-Request-ID header: the value the caller sent, or
one the server generated. It is the correlation ID on every log line the
request produced, and it is distinct from the envelope’s request_id, which is
the caller’s own identifier from the request body and is echoed whenever the
body carried one – including on validation failures. param is the dotted
location of the field that failed, as pydantic reports it; a member of a
discriminated union carries its tag as a segment (source.uri.uri for an empty
uri in a {"type": "uri"} source), and body means the body was not JSON.
INDX_REQUEST_TIMEOUT_SECONDS bounds the time a request may spend in
planning, execution, or embedding; unset or 0 means unbounded. The deadline is
checked once the source is loaded and then between capability attempts, never
inside one, so a source fetch or a capability mid-call finishes before the
504 is answered. Timeouts and disconnects stop the work; they never fall
back to another route.
The input limit is advertised as limits.max_input_bytes on the capability
snapshot and configured per deployment with INDX_MAX_INPUT_BYTES. It bounds
decoded source bytes, not the encoded request body, and is deliberately excluded
from the snapshot’s content hash so raising it does not invalidate existing plans.
resolvable is excluded for the same reason and is not the same kind of thing.
It reports what this installation can reach – resolvable.schemes – and what
it can look at – resolvable.observable_media_types – and both grow by
installing a distribution rather than by changing a setting. Gaining either
widens what can be planned next without changing what any outstanding plan
selected, so hashing them would answer every plan in flight with a 409 for a
reason unrelated to routing.
Planning estimates start from per-capability-kind defaults that a deployment
corrects with INDX_ROUTING_ECONOMICS, a path to a JSON object keyed by
capability ID or kind, whose values may set quality, cost_usd_per_page, and
latency_ms_per_page. An ID wins over its kind. quality is between 0 and 1,
the other two are at or above zero, and latency_ms_per_page is a whole number
of milliseconds. The numbers are private routing
input, not part of the wire contract, but they move estimates and can change
which route a constrained request receives, so a malformed file is refused
rather than ignored, an unknown field and a fractional latency included.
The variable is operator-side because a capability that could price itself would
be buying its way into a route, and the deployment paying the bill is the party
with a reason to be honest. What that costs is real and is not hidden: a hosted
OCR charging $0.05 per page is admitted against maximum_cost_usd on the OCR
default of $0.0005 and publishes that as estimates.cost_usd, until someone
writes the real figure into this file. Correct a specific install by capability
ID; correct a whole class of work by kind.
Recognition engines ship behind extras, so a default install is a router and a
native-extraction lane with no ONNX runtime, no model weights, and no
credentials. uv sync --all-packages --extra ocr adds PP-OCR through
onnxruntime; --extra vlm adds a vision lane through LiteLLM. What an install
does not carry is advertised as unavailable with the reason, and the routing
ladder descends past it rather than failing the request.
The vision lane is configured with INDX_VLM_MODEL (a LiteLLM model id),
optionally INDX_VLM_API_KEY and INDX_VLM_API_BASE, and
INDX_VLM_TIMEOUT_SECONDS / INDX_VLM_RETRIES / INDX_VLM_DPI. The OCR lane
takes INDX_OCR_DPI and reports PP-OCR’s own per-line scores as the page’s
confidence, weighted by the length of the line each was given for. It scores
recognition and not recall: text the detector never boxed is absent from the
number rather than counted against it. Both raster lanes also take INDX_OCR_MAX_PIXELS and INDX_VLM_MAX_PIXELS (default 178956970, Pillow’s own decompression-bomb ceiling): a page that would rasterize to more pixels than that at the lane’s DPI is refused before the bitmap exists, and returned as unread so the plan’s next rung gets it.
INDX_TEXT_ENCODING belongs to the text lane and is a comma-separated ordered
list, default utf-8-sig. Each name is tried strictly and the first that
decodes wins; a source matching none of them fails rather than being mangled
into whichever was tried last, and an encoding Python does not know names the
variable in its message rather than reporting the source as bad. It exists
because Japanese public-sector CSV is overwhelmingly Shift_JIS — 気象庁 and
東京都オープンデータ both serve CP932 — so a deployment reading those sets
utf-8-sig,cp932. UTF-8 leads because it is the strong test: arbitrary bytes
rarely decode as valid UTF-8, while CP932 decodes almost anything, so ordering
is what stops a UTF-8 file being read as mojibake. The default is utf-8-sig
rather than utf-8 because it is the same codec for a file with no byte-order
mark and strips one for a file that has it.
INDX_CHUNK_DPI (default 120) is the third of these and belongs to
indx-chunker-pdf, which renders a page no capability could read as text into a
single image chunk so it has a vector at all. It is read per call like the other
two. A value that is not a positive integer raises rather than falling back to
the default, but the raise does not fail the encode: the executor skips the
chunker for that document with a logged warning and the page floor chunks it
instead, so the misconfiguration shows in the log and in coarser chunks.
INDX_CHUNK_LAYOUT_MAX_CHARS (default 2000) belongs to indx-chunker-layout, version 2 of the
pdf chunker: a paragraph that would run past it is closed at the next line break. It is a
ceiling and not a target, and like INDX_CHUNK_DPI a value that is not a positive integer raises,
the executor skips the chunker for that document, and version 1 cuts it instead.
INDX_CHUNK_GRID_MAX_ROWS (default 50) belongs to indx-chunker-grid: the body rows one table
chunk carries before the table continues in the next, its header rows repeated. A value that is
not a positive integer raises, and the executor skips the chunker for that document.
INDX_CHUNKERS is the deployment’s own preference, in the spelling the CLI uses:
pdf@2,page. It means what chunking means on a request and sits behind it: the
request’s names are asked first, then these, then the installed order, so a
deployment moves to a new chunker version without waiting for it to become
everyone’s default. Unset, nothing changes. An entry nothing installed can serve
is the operator’s error and not the caller’s: the facade refuses to build, naming the
variable, so the server fails to boot and the CLI fails to start, and the variable is
read again before every chunked encode so a retune needs no restart. Neither path
answers a 422 to a request that named nothing.
INDX_LANGUAGE_CANDIDATES, INDX_LANGUAGE_MINIMUM_CONFIDENCE (default 0.05)
and INDX_LANGUAGE_LOW_ACCURACY belong to indx-language-lingua, behind the
lang extra. The candidate list is ISO 639-1 codes, comma separated, and unset
means all seventy-five: honest and expensive, since every language loaded costs
memory and every language considered is one more way to be confidently wrong
about a short page. The floor is not optional decoration – the engine returns a
value for every language it was built with, so without one a page would carry
seventy-five entries of which seventy-four are noise.
INDX_NER_PATTERNS_GAZETTEER points indx-ner-patterns at a JSON object of
{label: [surface, ...]}, matched literally and longest first beside the shipped
patterns, so a deployment’s customer list or product names come back as spans
under the label it chose. Unset means the patterns alone.
INDX_NER_VIBRATO_DICTIONARY and INDX_NER_VIBRATO_CACHE belong to indx-ner-vibrato,
behind the ja extra. The first names a compiled vibrato dictionary, system.dic or
system.dic.zst, replacing the IPADIC release the lane otherwise downloads once into the
second (default ~/.cache/indx-ner-vibrato) and verifies by digest on every read. Without
the extra the distribution advertises no extractor at all.
INDX_NER_ONNX_MODEL (default Xenova/bert-base-multilingual-cased-ner-hrl),
INDX_NER_ONNX_MODEL_FILE (default onnx/model_quantized.onnx),
INDX_NER_ONNX_MAX_TOKENS (default 512), INDX_NER_ONNX_WINDOW_CHARACTERS
(default 400, how much text the model is shown at once, cut at whitespace) and
INDX_NER_ONNX_MINIMUM_CONFIDENCE (default 0.5, the mean token score a span
needs) belong to indx-ner-onnx, behind the ner extra. The weights download on
first use into the Hugging Face cache and never during discovery; without the
extra the distribution advertises no extractor at all.
INDX_NER_LLM_MODEL, INDX_NER_LLM_API_KEY, INDX_NER_LLM_API_BASE,
INDX_NER_LLM_TIMEOUT_SECONDS (default 60), INDX_NER_LLM_RETRIES (default
1), INDX_NER_LLM_MAX_CHARACTERS (default 6000) and INDX_NER_LLM_LABELS
(default person,organization,location,date,money,invoice_number, the labels
the model is asked for) belong to indx-ner-llm, behind the llm extra: its own
prefix, so a deployment may extract with a different model than it classifies
with. Without a model the distribution advertises no extractor at all. The
model answers surfaces and the lane locates each in the text; the device rule
is the LLM classifier’s.
INDX_ENRICH_EXTRACTIVE_SENTENCES (default 3, how many sentences make the
summary) and INDX_ENRICH_EXTRACTIVE_MAX_CHARACTERS (default 600, the ceiling
on it, cut at a whitespace boundary) belong to indx-enrich-extractive, the
enricher a default install carries. Wrapped lines are rejoined and a sentence
ends at a terminator in either language; the sentences chosen are the ones that
carry most of the text’s own vocabulary, by term frequency with a redundancy
penalty, returned verbatim in document order. A text with nothing to rank, a
form of short fields, gets its leading lines. It coins no tags.
INDX_ENRICH_LLM_MODEL, INDX_ENRICH_LLM_API_KEY, INDX_ENRICH_LLM_API_BASE,
INDX_ENRICH_LLM_TIMEOUT_SECONDS (default 60), INDX_ENRICH_LLM_RETRIES
(default 1), INDX_ENRICH_LLM_MAX_CHARACTERS (default 6000),
INDX_ENRICH_LLM_SUMMARY_SENTENCES (default 2, how long a summary the model is
asked for) and INDX_ENRICH_LLM_TAGS (default 5, how many tags at most) belong
to indx-enrich-llm, behind the llm extra: its own prefix, so a deployment may
summarise with a different model than it classifies with. One call answers both
parts. Without a model the distribution advertises no enricher at all, and the
device rule is the LLM classifier’s.
INDX_CLASSIFIER_WORDS_LABELS points indx-classifier-words at a JSON taxonomy
replacing the shipped one: facets of labels, each with names in ja and en, the
words that vote for it, and a threshold – an integer count of distinct words or
a fraction of the list. Unset means the shipped document-type, business-unit and
industry tables for Japanese and English business documents. The confidence is
the share of the list that matched, a ratio and not a calibrated probability.
INDX_CLASSIFIER_ZEROSHOT_MODEL (default the Xenova ONNX mirror of
MoritzLaurer/mDeBERTa-v3-base-xnli-multilingual-nli-2mil7),
INDX_CLASSIFIER_ZEROSHOT_MODEL_FILE (default onnx/model_quantized.onnx),
INDX_CLASSIFIER_ZEROSHOT_HYPOTHESIS (default This document is a {}.),
INDX_CLASSIFIER_ZEROSHOT_MINIMUM_CONFIDENCE (default 0.5),
INDX_CLASSIFIER_ZEROSHOT_MAX_TOKENS (default 512) and
INDX_CLASSIFIER_ZEROSHOT_LABELS belong to indx-classifier-zeroshot, behind the
zeroshot extra. Each label becomes one hypothesis against the text, and the
entailment probability is the label’s confidence; the token window is the
lane’s own bound, so the model reads the head of the document. The weights
download on first use into the Hugging Face cache and never during discovery.
INDX_CLASSIFIER_LLM_MODEL, INDX_CLASSIFIER_LLM_API_KEY,
INDX_CLASSIFIER_LLM_API_BASE, INDX_CLASSIFIER_LLM_TIMEOUT_SECONDS (default
60), INDX_CLASSIFIER_LLM_RETRIES (default 1),
INDX_CLASSIFIER_LLM_MAX_CHARACTERS (default 6000, the head of the document
the model is shown, since the contract bounds nothing) and
INDX_CLASSIFIER_LLM_LABELS belong to indx-classifier-llm, behind the llm extra. Without a model the
distribution advertises no classifier at all rather than a broken one, the
hosted-embedding rule. The classifier’s device follows the rule below: a model
LiteLLM routes to this box is gpu, a hosted one external, and only the
former may be enabled by a request carrying data_residency. What the vendor
charged reaches usage.cost_usd.
None of these appear on the wire, but the model id decides one thing that does:
a model LiteLLM routes to this box (ollama/…, or an api_base on localhost) is
declared as a gpu device and a hosted one as external, so a request carrying
data_residency keeps the local vision lane and is correctly refused the hosted
one. Changing the configured model therefore changes the capability snapshot ID,
which is the point: the residency guarantee a plan was decided under is no longer
the same guarantee.
INDX_VALIDATION_MIN_CONFIDENCE is the confidence a deployment refuses output
below, and it defaults to 0.0 – off. A page a capability scored under the
floor is treated as unread: it descends to the plan’s next capability, its output
is discarded, and the trace records the measured value against the floor. Raising
it therefore buys accuracy with money and latency, one rung at a time, which is
why nothing here picks a number for you: no threshold in this repository has been
measured against a labelled corpus.
The floor itself stays off the wire; what it is compared against does not.
trace.events[].quality is what a capability claimed about the pages that one
event covers – a page-scoped event carries that page’s own score, a
document-scoped one the mean over the group it stands for – and
actuals.quality is the mean over the document. Both are self-reports on exactly
the same footing as actuals.cost_usd, not measurements against ground truth,
and null when nothing reported one.
Sources arrive either as inline base64, which always works, or from a URI whose
scheme an installed loader resolves. The four indx ships – file, data,
http, https – come from indx-loader-file and indx-loader-http and are
installed distributions like any other, so the resolvable set is a property of
the installation rather than of indx: GET /v1/capabilities reports it as
resolvable.schemes, and a stock pip install indx resolves none of them. A
source’s media type is then recognized from its bytes before any declared value
is trusted — by the same installed observers, which is what makes recognition
installation-dependent too: an install carrying none believes the declared
Content-Type, then the filename.
There are three refusals here, all installation-dependent, and they are
deliberately distinguishable. A scheme no installed loader resolves is a 415
with code unsupported_source, and it comes first, because nothing else can
happen until the bytes arrive. A media type no installed capability reads is
a 415 without that code. A media type no installed observer can look at is a
422 with code source_unreadable, from preflight. The same source can
therefore be refused on one install and planned on another. GET /v1/capabilities lists what is readable under capabilities, and what is
reachable and observable under resolvable.
A loader may also refuse a destination it could have fetched: 422 with code
source_forbidden. indx-loader-http declines loopback, private, link-local,
reserved and multicast addresses unless INDX_LOADER_HTTP_ALLOW_PRIVATE_HOSTS
is set, re-checking every redirect hop and refusing a redirect into a scheme it
does not speak; INDX_LOADER_HTTP_TIMEOUT_SECONDS bounds the fetch.
indx-loader-file reads any path this process can until INDX_LOADER_FILE_ROOTS
names the directories it may read, after which anything resolving outside them is
refused. In both cases the message names the variable that would permit it and
the server log carries what was actually resolved – the response does not,
because telling a caller which internal address a hostname resolves to is what an
SSRF probe is asking for.
Above both loaders sits INDX_URI_SCHEMES, read by indx-source where the scheme
is dispatched. Unset, every scheme an installed loader declares is resolvable; set,
only the schemes listed are; and none disables URI sources altogether, which is
how a public host without authentication runs. A scheme that is installed but not
enabled is the same 415 unsupported_source as one no loader claims, with the
message naming the variable, and resolvable.schemes on the snapshot advertises
the enabled set rather than the installed one, so a client learns before its first
request whether the URI lane exists at all. Inline sources are never affected:
bytes handed over were never fetched. The roots keep their meaning under it as a
second layer – a deployment that later enables file still reads only the
directories it named.
indx-loader-s3 resolves s3://bucket/key, one object per URI, through boto3
behind the s3 extra; without the extra the scheme is still advertised and a
fetch is a 422 source_unreadable telling the operator to install
indx-loader-s3[s3]. Credentials come from INDX_LOADER_S3_ACCESS_KEY_ID and
INDX_LOADER_S3_SECRET_ACCESS_KEY (with INDX_LOADER_S3_SESSION_TOKEN when a
role’s temporary keys carry one), and when those are unset boto3’s own chain
applies – AWS_* variables, a profile, an instance role. That second layer is
deliberate, so a deployment on EC2 or EKS uses its role without copying
secrets, and it is why INDX_LOADER_S3_BUCKETS exists: a comma-separated
allowlist, unset meaning any bucket the way INDX_LOADER_FILE_ROOTS does, and
once set a URI naming any other bucket is 422 source_forbidden with the
variable in the message and the configured list in the log.
INDX_LOADER_S3_ENDPOINT_URL points the loader at an S3-compatible server and
forces path-style addressing; INDX_LOADER_S3_REGION is passed through when
set; INDX_LOADER_S3_TIMEOUT_SECONDS bounds the connect and the read (default
30). A ?versionId= query is refused rather than dropped.
indx-loader-gs resolves gs://bucket/object the same way, through
google-cloud-storage behind the gs extra, and without it a fetch is the same
422 naming indx-loader-gs[gs]. INDX_LOADER_GS_CREDENTIALS_JSON is a
service-account key as the JSON itself, never a path, and wins when set; unset,
Application Default Credentials apply, so a deployment on Cloud Run or GKE uses
its service account without a copied key (ADR-0066). INDX_LOADER_GS_BUCKETS
is the allowlist that second layer needs, with INDX_LOADER_S3_BUCKETS’s rules.
INDX_LOADER_GS_ENDPOINT_URL points the loader at an emulator and, with no key
set, makes it anonymous rather than sending the process identity’s token to
whatever the endpoint names. INDX_LOADER_GS_TIMEOUT_SECONDS bounds each
request and the retries around it (default 30). A ?generation= query is
refused rather than dropped.
indx-loader-az resolves az://container/blob through azure-storage-blob
behind the az extra, and without it a fetch is the same 422 naming
indx-loader-az[az]. The storage account is configuration and never part of the
URI, so one deployment reads one account (ADR-0066).
INDX_LOADER_AZ_CONNECTION_STRING wins when set and is the whole story: it
carries the account, its key or SAS, and for an emulator the endpoint. Unset,
INDX_LOADER_AZ_ACCOUNT names the account and DefaultAzureCredential applies,
so a deployment on Container Apps or AKS uses its managed identity without a
copied key; with neither set a fetch is a 422 naming both variables.
INDX_LOADER_AZ_CONTAINERS is the allowlist that second layer needs, with
INDX_LOADER_S3_BUCKETS’s rules. INDX_LOADER_AZ_TIMEOUT_SECONDS bounds the
connect and the read (default 30), and the SDK’s retries are off so that it
does. A ?snapshot= or ?versionid= query is refused rather than dropped.