Skip to content

ChangeLog

Edit page

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.



  • ICU now links on Apple Silicon. The Homebrew prefix was hardcoded to the Intel path, so find_package(ICU) failed and ICU was silently compiled out of the arm64 build (and its bottle), losing Unicode word-boundary handling despite the formula depending on icu4c.
  • The Homebrew formula is head-only and never goes stale: the bottling workflow injects the released tag’s url and a fresh source checksum instead of a pinned default. Asio is fetched by the build rather than declared as a formula dependency.
  • Stale contrib: the 2018 FreeBSD ports skeletons, the Django-Haystack backend, and a broken example script.

  • Renamed “indices” to “indexes” throughout: the reserved metadata database is now .xapiand/indexes, and the reference guide follows. A clean break, with no fallback to the old name.
  • The Python client (xapiand-py) was rewritten as a fully async (asyncio / aiohttp) library and bumped to 1.0.0. There is no synchronous compatibility layer.

Second alpha: response-path efficiency, a crash-safe data store, a dependency-free test harness, and the full release/CI pipeline. The documentation site moved to Astro/Starlight.

  • Zero-copy response logging: the access log reads the response straight from the writer’s buffer instead of copying the body and headers a second time per request. Requests and responses are logged through Kronuz/http-log.
  • The data store writes a crash-safe v2 volume format (still reads v1).
  • Documentation migrated from Jekyll to Astro/Starlight, with an Architecture section, D2 diagrams, a rewritten Lua scripting guide, and OS auto-detection on the quick-start.
  • macOS Homebrew bottles (arm64 + x86_64 via Rosetta), FreeBSD packages, multi-arch Docker images, and RPM/DEB packages, all built on tag push.
  • ASAN + ThreadSanitizer CI, with the doc-driven E2E run under both.
  • A request watchdog that flags requests taking too long, and harness/ci_status.sh for one-glance CI status of a ref.
  • The splash banner shows the build’s git short-hash and version.
  • Aggregations fail clearly on reserved-but-unimplemented types instead of silently returning nothing.
  • FreeBSD build fixes (<utility> include, __builtin_exp10 lowering); the version now derives from lightweight git tags.
  • Bumped the cuuid JS dependency to close a uuid advisory (GHSA-w5hq-g745-h8pq).
  • GoogleTest: the unit tests run on a dependency-free harness and the legacy test tree was pruned (many suites revived: serialise, endpoint, fieldparser, geospatial HTM, and more).

First 1.0 milestone (alpha): a broad modernization of the engine, the runtime, and the build, on top of a de-vendored library tree. Six years and 151 commits after 0.40.0. Benchmarked native-vs-native against Elasticsearch and against 0.40.0; see the Benchmarks page.

  • Migrated the vendored Xapian engine from the 1.5.0-dev snapshot to 2.0.0 (rebuilt as clean per-feature patches on pristine 2.0.0).
  • Rebuilt the runtime on an Asio-based reactor and retired libev entirely: the HTTP server, the Xapian remote-backend protocol, the replication protocol, and the discovery/cluster plane (Bus + Raft) all run on the reactor now.
  • Broke the monolith into a tree of standalone libraries consumed via FetchContent (msgpack, storage, traceback, logger, threadpool, scheduler, datetime, io, fs, system, htm, soundex, and ~20 more), replacing the vendored copies.
  • Moved to C++20: fmtstd::format, tclap → CLI11, and the ChaiScript scripting engine → Lua (sol2).
  • Distributed search now runs through one native multi-database Enquire that overlaps the shard fan-out, replacing the serial shard-by-shard loop and the hand-rolled MSet merge.
  • Data-store volumes and inline document data are written with Zstandard.
  • Remote protocol synced to Xapian 2.0.0 (v47), as a non-blocking two-phase match the reactor can hold.
  • Apple Silicon / AArch64 build support.
  • Native multi-database overlapped shard fan-out (see Changed) and a read-only remote-database stats cache that eliminates the per-query MSG_UPDATE storm.
  • --color=<mode> log-color control and --discovery-interface selection.
  • RESTORE streams its request body concurrently (O(buffer) memory).
  • Content negotiation through the framework (http::AcceptCache).
  • A full test + benchmark harness: differential E2E capture/check, multi-node cluster liveness checks, a load driver, a native Elasticsearch comparison, and a Remote-protocol proxy/decoder.
  • Double Metaphone sort key.
  • Distributed query_id made unique per shard, so shards co-located on one node no longer collide in the process-global pending-query map.
  • A single SIGINT drains in-flight HTTP requests and shuts down the remote and replication layers gracefully.
  • Data paths (stopwords, mime.types) resolve relative to the executable.
  • Restored relaxed-JSON parsing, the xapiand_http_requests_summary metric, and the guarantee that an auto-generated _id is never left empty.
  • libev, entirely.
  • The vendored copies of ~30 libraries (now external Kronuz-family deps).
  • The hand-rolled per-shard fan-out and manual MSet merge, and Xapiand’s bespoke HTTP parser / Request / Response classes (now the http:: library).

  • Added limits to caches without it
  • Added missing Z to represent UTC on datetimes values
  • Fix update outdated cache schemas from local and foreign schemas
  • Fix update cache of index settings when the schema is updated
  • Increase node lifespan and heartbeat timeout

  • Serialization the get_mset steps due the concurrency produce a segmentation fault

  • Search offsets when merging results

  • Raft election timeouts incresed by 2x

  • Added --primary-node option to fix a primary node
  • Doing GET /.nodes only reads nodes internally

  • Improve QueryDSL handling of wildcards and fuzzy

  • Allow specifying partial using ** at the end
  • Badly placed Wildcards
  • Partials for keywords

  • Send errors in script as 400
  • Notify of database changes during periodic cleanups
  • Limit shutdown retries to exit
  • Prevent queries ending with to match all
  • Fix scripts in strict mode

  • Using stemmer during queries

  • No longer logging replication operations by default (it now requires --log replicas)
  • Uncommon race condition while destroying clients
  • Replicas no longer send db updated messages
  • Always replicate databases from primary

  • Adds default operator for queries parsed with the boolean parser
  • Discovery protocol callback executed by appropriate thread
  • Fixes race condition during shutdown

  • BREAKING: Foreign schemas no longer accept selectors
  • BREAKING: Global string fields indexed as text datatype
  • QueryDSL using _and or _or also enable booleans
  • Fix foreign schemas URIs
  • Fixes issue #19
  • Fixes issue #20

  • BREAKING: UPDATE method no longer receives upsert query param
  • BREAKING: Added UPSERT method
  • BREAKING: Removed MERGE and STORE methods
  • Restore always commits at the end
  • Restoring of documents in a clustered index
  • Commit in databases with remote shards (remote wasn’t receiving the commit)
  • Better handling of database flags (when changed)
  • Uncompressed UUIDs representation was buggy

  • BREAKING: UPDATE and PATCH don’t create new objects, object must exist (or use upsert query param)
  • BREAKING: STORE removed (use PUT or UPDATE)
  • Updated xapian-core to latest 1.5 (git@00f69cf3928b44a756ddafcde248610f72babf62)
  • Schemas not saving default schema as metadata some times
  • Remote protocol matcher, for remote queries, now holds list for all pending queries

  • BREAKING: Errors returned with status and message (no hash #)
  • Fixed _meta producing errors when passed to POST, PUT or UPDATE

  • Added support for _meta inside schemas (to allow user to add custom meta data associated with it)
  • BREAKING: Schema declaration no longer needs nested _schema -> schema node; it now only needs _schama
  • BREAKING: Strict mode requires newly created indexes to declare _settings with at least number_of_shards and number_of_replicas.
  • BREAKING: Foreign schemas no longer use _endpoint, it was renamed to _foreign.
  • Aggregations (min/max) which have no elements could result in errors.
  • Fix parser query params to qdsl for not operator

  • BREAKING: Clustering: Added primary shard promotions
  • Updated xapian-core to latest 1.5 (git@db790e9e12bb9b3ebeaf916ac0acdea9a7ab0dd1)
  • Python client updated
  • Quarantine (invalid) WAL and database files/directories when appropriate
  • Only nodes marked as primary for shards get the replica, others simply erase (or quarantine) stalled databases.
  • Unused server sockets are now fully closed during shutdown
  • BREAKING: Clustering: Increased overall cluster stability
  • Remote protocol: Race conditions for some used database objects after checkin
  • Refactored node shutdown process
  • Schemas LRU properly detect changes to schemas now checking only inside “schema” and “description”
  • WAL had a bug which could lead to skipped slots (it now checks revision before writing lines and recovers after errors)
  • Many solo mode (and CLUSTERING=off) fixes.

  • BREAKING: Geo: Renamed height to altitude
  • BREAKING: Geo: Renamed lng to lon
  • Schema with nested arrays and objects

  • Support for ‘float’ as an alternative name to ‘floating’ datatype
  • Undefined behavior in UUID, serializer and term generator
  • Error when requesting invalid types

  • Performance improvements with restore by launching parallel DocIndexers
  • Queries using glob wildcards
  • Nested arrays and objects in arrays
  • Restoring for GCC compiled Xapiand

  • BREAKING: Added _version to document data
  • BREAKING: Non-hashed prefixes for fields
  • Indexing optimizations (_id and _version values)
  • RESTORE now doesn’t check _version
  • Concurrency issues
  • Issues with document version checks
  • Fuzzy Queries fixed

  • Enabled Honey backend
  • Issues when launching multiple nodes simultaneously
  • Retry database initialization during node launch

  • Fix compiler warning about DropAction/GetAction “changing meaning”

  • Default do check_at_least one when using aggregations
  • Updated xapian-core, fixes issues with leaking sockets
  • BREAKING: Added patches for xapian-core to handle parallelized queries
  • Fix issues when starting servers simultaneously

  • Fix issue with aggregations returning invalid results at times

  • BREAKING: Remote Protocol issue which provoked messaged to get out of sync
  • Issues schemas not being saved (unexpected “_id” errors)
  • Detection of remote schema updates
  • Certain cluster remote queries

  • Experimental parsing of YAML
  • Postman collection for testing documentation requests
  • Require CMake 3.12 (again)
  • Docker image using Alpine 3.9, GCC 8 and ICU
  • Check for requests with invalid body
  • Compilation with cluster disabled

  • Casts

  • Throttling of commits (exception was being thrown)

  • Compilation using cmake 3.5 (use static libraries instead of object libraries)
  • Regression which slowed down bulk indexing

  • GCC 7 and GCC 8 support (and libstdc++)
  • Refactored pfh module
  • Using ENUM() with minimal perfect hash functions (improves performance)
  • Upgraded xapian-core to latest 1.5 (git@84ccecdfb8938b4011aac0bf9539c139204acd22)

  • Colorized INFO
  • Tracebacks with normalized address
  • Require CMake 3.12
  • Use nanosleep instead of usleep and libev checking for EINTR
  • Fix warning about available files under OS X

  • Added CJK NGRAM and CJK words as parameters to ‘text’ datatype
  • Enable fuzzy searches in queries: use ''. E.g. “unserten3” would expand to “uncertain”
  • Ignore XAPIAN_* environment variables
  • INFO signal showing busy threads callstacks, when tracebacks are enabled
  • All exceptions now contain tracebacks, when tracebacks are enabled
  • Hash for resolving cluster node for indexes skipping slashes
  • Fix ranges in query
  • Fix restoring from an empty request
  • Formatting applied to saved dates

  • BREAKING: Remove support for \xHH in json
  • Fixed errors during heavy writes/reads
  • Fixed race condition in restore indexer

  • Fixed logging for verbose request/responses

  • The ?echo query param returns the new/updated object
  • BREAKING: By default, write/update operations no longer return the object (see ?echo)
  • BREAKING: Removed :schema, schemas are now added/edited by using the Create Index API.
  • Verbosity level 4 or higher (-vvvv) turns on --echo, --pretty and --comments
  • Replication protocol now properly closing connections on errors

  • Set metadata
  • Added PUT index
  • BREAKING: Removed :touch
  • HTTP Mappings
  • Schemas

  • Selectors were not working
  • Mime Types

  • Added —pretty (and —no-pretty) options (for default ?pretty)
  • Verbosity also toggles “prettiness”
  • HTTP method mapping + override
  • TravisCI builds, thanks to Vitold S. (@vit1251)
  • Fixed _version checks

  • Sharding support added
    • By default there are now five shards per index
    • Shards are put inside subdirectories with the shard number prefixed with .__
  • BREAKING: All schemas are now foreign by default, and get put inside the databases indexes
  • BREAKING: Databases indexes directories work like shards and thus were renamed from .xapiand/<node_name> to .xapiand/index/.__<node_idx>

  • BREAKING: Cluster directory renamed from .cluster to .xapiand;
  • BREAKING: Indexes directories renamed from .index/<node_idx> to .xapiand/<node_name>
  • BREAKING: Upgraded xapian-core to latest 1.5 (master)

  • Fix regression introduced in 0.12.1 during cluster bootstrapping

  • Fix requests resulting in Bad Gateway with stalled endpoints
  • Renamed options for logging from --log-* to --log <setting>

  • Aggregations with _aggs in different order
  • JSON serialization of Infinite and NaN
  • Fixed race condition during bulk indexing
  • BREAKING: Fields in responses renamed and nested differently: + query.hits moved to hits + query.matches_estimated renamed to count + query.total_count renamed to doc_count
  • MERGE method is deprecated and will be removed, use UPDATE instead
  • Default UUID mode is now encoded + compact

  • Fix Docker build
  • Added support to routing key hints in the API, by using routing query param

  • Cluster replication is now functional, it was broken in 0.10.0
  • Fix searching in “_id” field
  • Fixed and documented Scripts and Foreign Scripts
  • BREAKING: .index shared by node
  • BREAKING: Keyword datatype term prefix is now ‘K’
  • BREAKING: Guessed IDs are now strings also for numeric IDs
  • BREAKING: URL drill selector now uses dot (’.’) directly, so document IDs cannot have dots in them now.
  • BREAKING: System-added fields in returned objects are no longer prefixed by ’#’
  • Never overwrite an existent database
  • Index only adds new indexes during write operations
  • Do not save empty data inside database
  • Numeric IDs are auto-incremented when creating new documents with POST
  • Added ’*” for using as multiple indexes

  • Docker build had missing symbols since 0.10.0
  • Fix exit code on errors

  • Default database path is /var/run/xapiand

  • Added _match_all and _match_none to :search command
  • Added the Elite Set Operator
  • BREAKING: Cluster using .index and .cluster databases
  • BREAKING: Indexes paths (endpoints) no longer have a trailing slash
  • Use OP_FILTER for terms in range queries
  • Flattened range queries
  • The default stopper during indexation is now stop_stemmed
  • Numeric ranges are now properly calculating the filtering terms
  • Geospatial queries are now calculating better the filtering trixels terms
  • Default accuracies changed
  • Removed _raw from QueryDSL
  • Fixed sort query param during search

  • Embedded Xapian core 1.5.0 in the source code
  • Accept application/x-ndjson as input for :restore command
  • Home and docs site with cosmetic improvements
  • Greatly improves performance
  • Documentation extended
  • Upgraded to fmtlib v5.3.1 library
  • Upgraded to ChaiScript v6.1.1 library
  • Sort accepting casts
  • QueryDSL with flattened queries
  • Fixes a few issues with the way geospatial locations were being filtered and indexed

  • Keep-Alive and Close connection support
  • Compress documents using LZ4 instead of gzip
  • Text fields no longer remove terms with Stopper
  • Fix bug with UUIDs and macOS

  • Added new Xapiand client (based on Elasticsearch client)
  • Added sorting for Aggregations
  • Added _keyed to aggregations optional. Keyed are off by default now
  • Added _shift to aggregations
  • Schema deprecated string type, it now only has text and keyword (string will be removed at some point)

  • Added bounds to standard deviation aggregation
  • Documentation about Aggregations
  • Remote and Replication protocol are now split in two
  • Range keys for aggregations fixed

  • Documentation updated
  • Fix Docker entrypoint

  • Uses /var/db/xapiand directory for database by default

  • Default bind address is now again 0.0.0.0

  • Fixes to multicast script for Kubernetes

  • Added daemon script for multicasting across k8s nodes
  • UDP usage tweaked (for Discovery)
  • Default TCP bind address is now the current IP

  • Explicit initialization of schema for cluster database
  • Cluster database endpoints normalized

  • Using fnv1ah64 for clustering hash (jump_consistent_hash)

  • Logo adjustments + banner updated

  • Fix Remote storage
  • Discovery protocol fixed issues with cluster

  • Discovery/Raft: Merged both into Discovery
  • Colorize all node names
  • Fix exit codes

  • Improve efficiency of nodes update after PING

  • Rewrite of Database Pool
  • Thread names changed

  • Cluster mode enabled
  • Adds replication and remote protocol

  • Prometheus metrics exposed at /:metrics

  • Added --flush-threshold command line option
  • MERGE and PATCH now require the document already exists
  • Renamed command line option --force-up to simply --force
  • Fix restore command memory usage
  • Fix HTTP protocol on malformed messages
  • Fix HTTP returning of large object bodies
  • Fix indexing documents with integer IDs
  • Fix io::read() to always return requested size when available
  • Fix date accuracies

  • Parallel restore commands
  • General optimizations
  • BREAKING: Geospatial fields changed generated terms. Reindexing needed!
  • Fixed geospatial terms and accuracy
  • Fixes storage (no longer returns 500)
  • Fixes multithreading bug with collected statistics

  • Added :dump and :restore endpoints
  • Added time it took (in milliseconds) to execute a :search query
  • Added sort, limit and offset to query DSL
  • Python: Xapiand client updated
  • BREAKING: Support for multi-content (by Content-Type) documents
  • Using C++17
  • Updated FreeBSD port
  • GCC 7 compatibility
  • Fixes problem with big body responses breaking the logs
  • Towards MultiarchSpec compatibility for Linux

  • Logo updated
  • Fixes issues while compiling with GCC + cleanups

  • A Homebrew tap was created
  • Versioning of the project is now officially in the “0.x” line, after going through many ghost versions.
  • Documentation was updated

  • Site is up and running
  • Documentation is finally cooking in the oven.
  • This release also features a new website (based on Jekyll’s theme)

  • Fixes issues with “already locked database”

  • First stable release
  • Lots of optimizations

  • Fixed a few issues

  • First beta release
  • Working under FreeBSD

  • Birthday!