Skip to content

Index Document API

Edit page

The Index API adds or updates a typed JSON document in a specific index, making it searchable. The following example inserts the JSON document into the “twitter” index with an ID of 1:

PUT /twitter/tweet/1
{
"user": "Kronuz",
"post_date": "2019-03-22T14:35:26",
"message": "Trying out Xapiand"
}

The result of the above index operation is a 204 No Content HTTP response code with no body. The document is searchable right away (reads see the writes of the node). To have the operation answer the indexed document, pass the echo query param (or start the node with --echo): the answer is then 200 OK with

{
"user": "Kronuz",
"post_date": "2019-03-22T14:35:26",
"message": "Trying out Xapiand",
"_id": 1,
"_version": 1
}

(the comments query param adds #docid and #shard, see Get). See what a write answers.

The index operation automatically creates an index if it does not already exist. The index operation also creates a dynamic schema for the index if one does not already exist. By default, new fields and objects will automatically be added to the schema definition if needed. Check out the Schemas section for more information on schema definitions, and the indexes create API for information about updating document schema manually.

The index operation can be executed without specifying the ID. In such a case, an ID will be generated automatically. Here is an example (note the POST used instead of PUT):

POST /twitter/tweet/
{
"user": "Yosef",
"post_date": "2019-03-22T14:46:10",
"message": "Also trying out Xapiand!"
}

The result of the above index operation is a 201 Created HTTP response code with the new document’s URL in the Location header; with echo it is 200 OK with the same header and the document, with its _id and _version. The ID is minted from the type of the index’s _id field, which the first document sets:

  • On a new index, a UUID (version 6 by default, --legacy-ids mints version 1).
  • Where the IDs are integers, as in the examples here, an integer.
  • Where the IDs are keyword or text, a random string.

Sources: src/server/search_views.cc:938-985; src/database/schema.cc:2713-2717, 2745-2870; src/opts.cc:235-238.

The ID in the path takes the type of the index’s _id field. Where the IDs are integers, any integer is an ID, 0 and negative numbers included, and the document answers it as a number; anything else, such as 1.5 or abc, answers 400 (see Exact Values). Where the IDs are keywords, 0 and 00 are two IDs, each answered as a string.

PUT /idsapi/numbers/7
{
"n": "seven"
}
PUT /idsapi/numbers/0
{
"n": "zero"
}
GET /idsapi/numbers/0
PUT /idsapi/names/a
{
"n": "a"
}
PUT /idsapi/names/0
{
"n": "zero"
}
GET /idsapi/names/0

Sources: docs/mechanisms/document-ids-and-versions.md; has_document_id (src/database/handler.h: any number is an ID); tests/test_node.cc (test_document_ids_are_themselves).

We can execute script the same way scripts work in the Update API and as explained in the Scripting section.

By passing the commit query param to the request, you can ensure the operation returns once the document changes have been committed to disk.

Index operations can be made conditional and only be performed if the last current document version matches the passed version in the query param (or the _version in the body). If a mismatch is detected, the operation will result in a 409 Conflict HTTP response code. See Versioning for more details.

A write is directed to one of the index’s shards by its ID (an index has 5 shards by default, see Create Index):

  • An ID goes to one of 1,024 fixed buckets: an integer ID k from 1 to 2^63 - 1 to bucket (k - 1) mod 1024, any other ID to the FNV-1a hash of its term modulo 1,024. Bucket b belongs to shard b mod n of the n shards.
  • A document that exists is written in the shard that holds it; a new one in the shard its bucket belongs to.
  • A POST to an index whose IDs are integers goes to the least used shard. It issues the smallest integer ID above every integer ID in the index whose bucket it owns, so new IDs never take an ID that was written before. With one shard they run 1, 2, 3, and so on, up to 2^63 - 1 (2^53 for float IDs); past the last one, a POST fails.

An ID is only an ID: the document numbers each shard keeps are its own, in the order documents arrive, and never derive from an ID.

Source: docs/mechanisms/document-ids-and-versions.md.