Options
Edit pagexapiand --help lists every option of the node, with its type and its allowed
values. An option takes its value as --port 8880 or --port=8880, and an
option that does not exist is an error: the node prints The following arguments were not expected and exits.
Verbosity
Section titled “Verbosity”Verbosity of the servers logs can be set by using the -v, --verbose or
--verbosity options (-v can be repeated, and adds to --verbosity).
very-very-very verbose output is usually enabled with -vvvv or
--verbosity=4. This mode also enables --human, --echo, --pretty and
--comments options by default.
Each of these four options has a --no- form that turns it off even at verbosity 4
(--no-echo, --no-pretty, --no-human, --no-comments), and each can be set
for a single request with the query parameter of the same name: ?echo,
?pretty, ?human and ?comments turn it on, ?echo=false and so on turn it
off. The request wins over the node.
Echo makes Xapiand return newly created or edited objects as part of the
response (200 OK with the document, its _id and its _version). Without it,
a write with an ID in the URL answers 204 No Content and a POST without an ID
answers 201 Created with the document’s path in the Location header, both
without a body (see What a write answers).
Returning a body can be enabled with the --echo option, with ?echo on the
request, or by using a verbosity level higher or equal to 4.
Pretty
Section titled “Pretty”Pretty makes Xapiand return pretty (formatted) JSON output as responses. This
option can be enabled with the --pretty option, with ?pretty on the request,
or by using a verbosity level higher or equal to 4.
PUT /options/demo/1
{ "title": "Options"}GET /options/demo/1?pretty=trueGET /options/demo/1?pretty=falseHuman makes Xapiand return humanized numbers for size and times in the output
of the responses ("took": "39.889ms" instead of seconds as a number). This
option can be enabled with the --human option, with ?human on the request, by
setting output to be “pretty” or by using a verbosity level higher or equal to 4.
SEARCH /options/demo/?human=trueSEARCH /options/demo/?human=falseComments
Section titled “Comments”Comments add the fields that start with # to the documents of a response:
#docid and #shard to a document, and #rank, #weight and #percent to the
hits of a search. They can be enabled with the --comments option, with
?comments on the request, or by using a verbosity level higher or equal to 4.
GET /options/demo/1?comments=trueConsole
Section titled “Console”--colorsets when the console output is colored:auto(the default),always,never,truecolor,256or16(on,off,24bit,256color,ansiandstackedare accepted too).--no-coloris the same as--color=never, and an explicit--colorwins.--iterm2sets marks, tabs, title, badges and growl notifications in iTerm2.--logtakes one or more ofepoch,iso8601,timeless,seconds,milliseconds,microseconds,thread-namesandlocations, which change the time stamp of each log line and add the thread name and the source location.-L,--logfilewrites the log to a file.
Strict
Section titled “Strict”Schemas are normally automatically created by default, guessing the type of new
fields being indexed. The --strict option disables this
Dynamic Typing
feature and forces the user to specify a type for all new fields, making the
request fail with 412 Precondition Failed if the datatype is missing.
The same switch is a property of one index’s schema, _strict, which a node
without --strict honors for that index alone:
PUT /options/strict/
{ "_schema": { "_strict": true, "name": { "_type": "keyword" } }}PUT /options/strict/1
{ "name": "typed", "nickname": "untyped"}Node and Data
Section titled “Node and Data”-D,--databasesets the path to the root of the node, where all index data lives (<prefix>/var/db/xapiandby default, where<prefix>is the install prefix). The node refuses a directory that holds data from1.0.0-alpha.10or earlier, which it does not read, and exits with status 65; see Upgrading.--forceuses the path as the root of the node without checking that its.xapianddirectory is empty or holds only the node’s own files; the node exits with status 65 when that check fails.--namesets the node name.--shardssets the default number of shards per index (5).--portsets the HTTP REST API port (8880).--bind-addresssets the address the node listens on (all addresses by default).-d,--detachruns the process in the background. It writes the log to<prefix>/var/log/xapiand.logand the process ID to<prefix>/var/run/xapiand.pidunless-L,--logfileand-P,--pidfilesay otherwise.-P,--pidfilesaves the process ID in a file.--uidand--gidset the user and the group the node runs as. A node started as root needs--uid.--processorssets how many processors the node uses (all by default). It runs one HTTP loop per processor, and sizes the fsync threads (half the processors, at least 1 and at most 10).
Identifiers
Section titled “Identifiers”--uuidselects how generated document IDs and UUIDs are written:vanilla(the standard text form),encoded(base59) or, on builds that enable them,guidandurn;compactshortens the form, andpartitionpartitions UUID index paths. Without it the node usesencodedandcompact. A UUID in an index path is written in that form, so every spelling of one UUID (upper case,{…},urn:uuid:…,~…) names one index, and the path INFO reports names it too. In the default mode a version 1 UUID is written by the v6 codec and its text changes (see Upgrading). Withpartition, a UUID’s index lives two directories down, under its third and second groups (4526/10c3/5759b016-10c3-4526-…).--legacy-idsmints and decodes UUIDs with the legacy v1 codec only (pure v1, no v6), for a deployment that must stay byte-for-byte compatible with pre-1.0 ids.
Administrative Commands
Section titled “Administrative Commands”--admin-commands enables the FLUSH and QUIT commands, which a node answers
only at the root URL (FLUSH /). FLUSH commits and closes every open index and
answers when they are closed; QUIT asks the node to shut down (see
Stopping the node).
Without the option both answer 405 Method Not Allowed, as on this node:
FLUSH /The API has no authentication, so enable the option only on a node that untrusted clients cannot reach.
Capacity and Caches
Section titled “Capacity and Caches”--max-clientsis the most client connections the node holds open (1000).--max-filesis the most files the node opens. By default the node computes what its clients and indexes need, and raises the process limit to it; it lowers--max-clientsand--database-pool-sizewhen the limit is too low.--full-flush-ordering(off) makes macOS order commits with a full flush (F_FULLFSYNC) instead of the barrier, whose order Apple guarantees only on internal SSDs. Where the barrier is not guaranteed the node logs one warning per volume naming this switch. Linux and FreeBSD already flush in full.--database-pool-sizeis the most indexes (shards) the pool keeps open. By default it holds what 1% of physical RAM covers at about 40 KiB per open index (about 4,200 on 16 GiB), within the open-files limit above; a size you give wins.--schema-pool-sizeis the most schemas the node keeps (100), and--schema-versions-sizethe most versions of schema it keeps in its cache (100).--scripts-cache-sizeis the cache size for compiled scripts (100).--resolver-cache-sizeis the cache size for resolved index paths (100).--wal-writer-cache-sizeis the cache size of the write-ahead log writers (100).
Searching many indexes
Section titled “Searching many indexes”A search that names several indexes (a list, a wildcard, _all; see Multiple Indexes)
reads them in batches: it holds at most one batch of indexes open at once, searches it, merges its best hits into the
result and moves on to the next. A search over thousands of indexes therefore needs only a batch’s worth of open
indexes, however wide it is. A batch holds whole indexes (all the shards of an index are read together), so an index
with more shards than the batch is a batch of its own.
--concurrent-search-indexessets the batch size: the most shards one search holds open at once. By default it is a loop’s share of the open indexes the database pool holds (--database-pool-sizeover the HTTP loops, at least 1), so one wide search on every loop at once holds about what the pool does.--search-walk-queue(8, at least 0) and--search-walk-wait(10 seconds) bound the lookup of index names for patterns: the node reads its data directory for one search at a time, a search that finds--search-walk-queueothers waiting answers429, and one that waits--search-walk-waitseconds for its turn answers503; both carryRetry-After: 1. Searches by name, writes and everything else are not held by it.--max-search-indexes(100,000, at least 1) is a cap on a runaway pattern. A search that names more indexes, counting the ones a wildcard expands to, answers400with the limit in the message. So does one whose patterns read more than that many directory entries to find their indexes, whether the entries match or not (*,-*matches nothing and reads the whole tree, and the directories of nested indexes count as well): the same number bounds the names and the work. A pattern goes only into the directories it can still match below (logs/2026/*readslogsandlogs/2026), a**goes everywhere. A data directory with more entries in one directory than the cap cannot be walked by a wildcard until the node’s index names are built. The first pattern starts that build (one walk of the data directory, in the background), and from then on a pattern reads index names instead of directory entries: each name it reads under its prefix counts against the cap, never more than the walk would read, sologs-2026*is served on a node with millions of indexes. Matching the names has a budget too (4,096 steps per entry the cap allows, so patterns that backtrack over long names answer400rather than hold a thread). A search names at most 32 patterns (wildcards and-exclusions together).
Each index is read as its batch starts, so a search does not see all its indexes at one moment; each index is whole, and
a write acknowledged before the search began is in it. A relevance weight is read against the statistics of the index
the document is in (its document count, its term frequencies), as Elasticsearch reads a shard’s: a document weighs the
same in a search of one index and in a search of a thousand, and the hits, their order, their weights, the totals and
the aggregations do not depend on the batch size. The xapiand_search_pinned_indexes_peak metric reports the most
indexes searches have held open at once.
Write-Ahead Log
Section titled “Write-Ahead Log”--writerssets the number of WAL writer threads (half the processors, at least 1 and at most 10).--wal-writer-cache-sizesets the size of the writers’ cache (100). Both tune the WAL writer pool.
Dump and Restore
Section titled “Dump and Restore”--dump INDEX writes the documents of an index and --restore INDEX reads them
into one, without starting the HTTP server: the node uses the data directory
(-D) alone. They read and write MessagePack, on stdout and stdin unless
-o, --out (with --dump) or -i, --in (with --restore) name a file,
and they log the SHA-256 of the stream. --dump and --restore cannot be used
together, and --shards sets the shard count of the restored index. The
DUMP and RESTORE requests do the same over HTTP; see
Backups.
xapiand -D /var/db/xapiand --dump twitter/user --out users.mpxapiand -D /var/db/new --restore twitter/user --in users.mpMonitoring
Section titled “Monitoring”GET /:metrics answers the node’s metrics in the Prometheus text format, as
xapiand_* series:
GET /:metricsxapiand_reactor_lag_seconds reports how late each event loop runs. The probes
behind it wake every 1 ms while a loop serves requests, and XAPIAND_LAG_PROBE=0
in the environment starts the node without them.
Stopping the node
Section titled “Stopping the node”SIGINT or SIGTERM stops the node. The stop has two waits, and the node exits when the second ends:
- The clients. The node waits for the requests in flight. With clients that keep sending, this has taken up to about 45 s.
- The rest, at most 35 s in all. In this order, each step starting when the one before it has ended:
- The HTTP service stops: it stops accepting, a read ends at its next checkpoint, an admitted write is let finish (one that waits for its index’s turn answers
503after 30 s), the connections of what is left are cut after a grace of 15 s (a write that finishes after that applies, but its client is not answered), and the loops stop one grace later. - The autocommits that were waiting are dropped; closing an index commits what it has pending.
- Every index closes.
- The write-ahead logs flush.
- The fsync threads finish.
- The HTTP service stops: it stops accepting, a read ends at its next checkpoint, an admitted write is let finish (one that waits for its index’s turn answers
What is still running when the 35 s end (steps 1 to 3 share them) is logged and abandoned: the requests in flight per loop, and the indexes whose turn or references are held. The node then flushes the
write-ahead logs and exits. Without abandoned work the status is 0; with it the status is 75 (EX_TEMPFAIL) and the log lists what was abandoned. An index that cannot close within the 35 s (for example on a hung
disk) counts, whether or not a request is in flight: the node logs Shutdown abandoned the indexes that are still in use and exits 75. The next start replays the log. After a stop, a write answered with a 2xx
is in the log; one that was not answered was never promised. A crash is different: a write is answered when its log line is queued, not when it is synced, so a crash can lose writes answered since the last commit.
A second SIGINT or SIGTERM during the second wait ends it at once: the node logs what is unfinished, flushes the logs and exits 75. The longest stop is therefore the first wait plus 35 s, plus the
flush of the logs; set a supervisor’s stop timeout above that (for example 120 s). A node that cannot set itself up (a port that is taken) logs why and exits with status 70.
--tls-certand--tls-key(PEM files, given together) serve the REST API over TLS on--port, in place of plain HTTP. The key file must be readable by its owner only (mode0600); the certificate must be valid now, and Xapiand warns when it expires within 14 days. TLS 1.2 and 1.3 only, forward-secret ciphers only, no session resumption.--tls-max-connectionscaps the TLS connections held at once (1024 by default; it must fit the open-files limit). Over TLS, a connection idle 75 seconds, headers not done in 30 seconds, or a request or reply that moves no data for 60 seconds is closed.
HTTPS protects traffic between clients and the node. It does not authenticate clients: anyone who can reach the port can use the API.
Environment
Section titled “Environment”XAPIAND_DATA_PATHis the directory that holds the node’smime.typesand, optionally, astopwords/directory of lists that replace the compiled-in ones (see Stop Words). Without it the node looks beside the executable (../share/xapiand, or the source tree for a build that is not installed) and then in the install prefix.XAPIAND_LAG_PROBE=0starts the node without the event-loop lag probes (see Monitoring).- The Docker image reads
XAPIAND_DATABASE,XAPIAND_NAME,XAPIAND_UIDandXAPIAND_GID(see Installation).
Removed Options
Section titled “Removed Options”These options do not exist any more, and the node rejects them as unknown arguments. The cluster went away with its options, until it returns, and the thread counts went away with the pools they sized.
- Cluster (among others):
--solo,--cluster,--discovery-port,--discovery-group,--discovery-interface,--replica-port,--primary-node,--matchers,--flush-threshold,--replicas,--replicators,--remote-clients,--remote-servers,--replication-clients,--replication-servers,--discoverersand--database-stall-time. - Threads:
--committers,--fsynchers,--http-servers,--http-clients,--bulk-preparersand--bulk-indexers. Use--processors. --max-database-readers: an index is opened once and shared by every request.
Build-Conditional Options
Section titled “Build-Conditional Options”A build configured with -DRANDOM_ERRORS=ON adds --random-errors-net,
--random-errors-io and --random-errors-db, which inject errors with the
probability (0 to 1) they are given. A build with -DENGINE_CENSUS=ON writes the
engine facilities each request uses to the file XAPIAND_ENGINE_CENSUS names,
and one with -DCRASH_POINTS=ON kills the process at the commit or replay point
XAPIAND_STRATA_CRASH=point:n names. They are for testing; see
Building.
For the complete, authoritative list, run xapiand --help.