[October 2026 edition]
Alopex DB v0.8.16: Performance Compared with Other Products
Published: Oct 27, 2026
Reading time: ~5 min
Introduction
We develop Alopex DB in Rust as a single engine for DataFrame, SQL, KV, and vector search workloads. It also provides a Python API. As the developers of a newer database, we use performance comparisons with established products and libraries as an important development benchmark.
Summary and Conclusion
We compared Alopex DB v0.8.16 with Polars, SQLite, PostgreSQL, DataFusion, LMDB, hnswlib, and Faiss HNSW in fixed test cases using Python APIs on a single host. Alopex DB was faster at DataFrame construction and persistent writes to an embedded KV store. It was slower at DataFrame computation, SQL, KV reads, and HNSW search.
Test Conditions
| Item | Condition |
|---|---|
| Versions | Python wheel alopex==0.8.16; alopex-server 0.8.16 |
| Environment | AMD Ryzen 5 3500U; Linux x86_64 / WSL2; Python 3.11.11 |
| Measurement | Fixed data; p50 for each trial; comparison based on the median of three trials |
| SQL comparison | Alopex DB and PostgreSQL ran in server/client configurations on the same host and reused existing connections |
| KV comparison | The embedded Python APIs of Alopex DB and LMDB ran on the same CPU |
| Cache | Warm cache |
These results apply to fixed data and Python APIs on a single shared host. We did not measure the Rust API, distributed processing, concurrent load, cold-disk I/O, or datasets larger than memory.
What We Measured
| Compared with | API type | Operations compared |
|---|---|---|
| Polars | DataFrame API | Construction from Python data, column computation after construction, conversion to a Python dict, and composite operations containing these steps |
| Polars | Lazy and streaming APIs | Reading CSV and Parquet, column computation, collect, and retrieval of all batches |
| SQLite | SQL API | SELECT, transactions, savepoints, prepared queries, and constrained INSERT operations |
| PostgreSQL | SQL API | Aggregation, sorting, retrieval of all results, persistent writes, point reads, updates, and deletes |
| DataFusion | SQL streaming API | Retrieval and complete consumption of rows matching a condition |
| LMDB | Embedded KV API | Persistent writes, point reads, overwrites, deletes, and range scans |
| hnswlib and Faiss HNSW | HNSW search API | Top-10 search at the same recall threshold |
We again did not measure random DataFrame access or file writes and exports.
Performance Results
The time ratio in each table is Alopex DB time divided by the comparison target’s time. A ratio below 1 means Alopex DB was faster; a ratio above 1 means the comparison target was faster. Each reported time is the median of three trials, with each trial measured at p50. We include only measurements whose processing results matched.
DataFrame and File Reads
| Compared with | Test case | Alopex DB time | Comparison time | Time ratio |
|---|---|---|---|---|
| Polars | Construct a DataFrame from Python data | 18.270 ms | 557.278 ms | 0.033 |
| Polars | Compute 12 columns on an existing DataFrame and collect | 5.405 ms | 3.286 ms | 1.645 |
| Polars | Convert 12 computed columns to a Python dict | 9.975 ms | 10.111 ms | 0.987 |
| Polars | Construction plus materialization of intermediate and final results | 214.644 ms | 775.785 ms | 0.277 |
| Polars | Previous composite operation, including construction, computation, and Python conversion | 171.293 ms | 1,264.226 ms | 0.135 |
| Polars | Lazy CSV, compute 12 columns, then collect | 13.843 ms | 7.226 ms | 1.916 |
| Polars | Lazy CSV, compute 12 columns, then retrieve all batches | 15.149 ms | 7.790 ms | 1.945 |
| Polars | Lazy Parquet, 11 numeric and other columns, then collect | 2.916 ms | 3.559 ms | 0.819 |
| Polars | Lazy Parquet, 11 numeric and other columns, then retrieve all batches | 3.648 ms | 4.257 ms | 0.857 |
| Polars | Fully consume CSV streaming output | 16.798 ms | 7.299 ms | 2.301 |
| Polars | Fully consume Parquet streaming output | 4.984 ms | 4.795 ms | 1.039 |
Alopex DB was faster at DataFrame construction, while Polars was faster at computation after construction. The composite operations include construction, computation, explode, concat, and Python conversion, so they are not comparisons of computation alone. The per-operation results appear in the individual rows above.
The trial ranges for conversion to a Python dict overlapped, so the result does not establish an advantage for either product. For file reads, Polars was faster with CSV, while Alopex DB was faster in the Parquet operations that we were able to measure.
In v0.8.16, Alopex DB does not support the Polars-compatible select, direct construction and addition of Series, direct construction of a LazyFrame, scan_csv, or scan_parquet operations. We could run the lazy comparison through Alopex DB’s own static scan, but the 12-column Parquet computation that included string concatenation failed because of a type mismatch between Utf8 and LargeUtf8. We do not claim Polars compatibility for these six APIs and operations or for Parquet string computation.
SQL
| Compared with | Test case | Alopex DB time | Comparison time | Time ratio |
|---|---|---|---|---|
| SQLite | SELECT, transaction, savepoint, prepared query, and constrained INSERT | 33.740 ms | 3.533 ms | 9.551 |
| PostgreSQL | COUNT | 72.274 ms | 5.353 ms | 13.501 |
| PostgreSQL | ORDER BY id LIMIT 16 | 36.333 ms | 1.339 ms | 27.132 |
| PostgreSQL | Retrieve all 17,125 rows | 242.455 ms | 42.426 ms | 5.715 |
| PostgreSQL | INSERT RETURNING plus rollback | 5.289 ms | 2.948 ms | 1.794 |
| PostgreSQL | Single-row INSERT plus commit | 9.302 ms | 4.336 ms | 2.145 |
| PostgreSQL | 100-row INSERT plus commit | 20.754 ms | 6.200 ms | 3.347 |
| PostgreSQL | 100 random-get hits | 294.737 ms | 119.212 ms | 2.472 |
| PostgreSQL | 100 random-get misses | 283.025 ms | 107.088 ms | 2.643 |
| PostgreSQL | Random UPDATE plus commit | 11.620 ms | 4.604 ms | 2.524 |
| PostgreSQL | Random DELETE plus commit | 220.362 ms | 4.403 ms | 50.053 |
| DataFusion | Fully consume SQL streaming output | 59.427 ms | 1.277 ms | 46.522 |
SQL took longer in Alopex DB than in the comparison target for every case measured. The gaps were particularly large for sequential reads compared with DataFusion and deletes compared with PostgreSQL. SQL needs improvement across both read and write operations.
In the DataFusion comparison, Alopex DB returns Python rows while DataFusion returns Arrow batches. The reported times therefore include not only SQL execution but also conversion to each result format and complete consumption of the results in Python.
Embedded KV and HNSW Search
| Compared with | Test case | Alopex DB time | Comparison time | Time ratio |
|---|---|---|---|---|
| LMDB | Single PUT plus commit | 0.996 ms | 2.137 ms | 0.466 |
| LMDB | 100 sequential PUT operations plus commit | 4.529 ms | 5.818 ms | 0.778 |
| LMDB | 100 shuffled-order PUT operations plus commit | 4.272 ms | 5.462 ms | 0.782 |
| LMDB | 100 random-get hits | 1.257 ms | 0.188 ms | 6.700 |
| LMDB | 100 random-get misses | 0.241 ms | 0.181 ms | 1.334 |
| LMDB | Random overwrite plus commit | 1.061 ms | 2.560 ms | 0.415 |
| LMDB | Random DELETE plus commit | 1.104 ms | 2.425 ms | 0.455 |
| LMDB | Range scan of 1,000 entries with all values retrieved | 8.797 ms | 0.502 ms | 17.538 |
| hnswlib | HNSW search at recall 0.99 or higher | 152.374 µs | 50.534 µs | 3.015 |
| Faiss HNSW | HNSW search at recall 0.99 or higher | 152.374 µs | 56.790 µs | 2.683 |
The embedded KV implementation was faster than LMDB for persistent writes, overwrites, and deletes. That is a strength of Alopex DB v0.8.16 in these fixed test cases. LMDB was faster for point reads and range scans, however, producing a clear contrast between write and read performance.
All products met the required search-recall threshold in the HNSW comparison. Under that condition, Alopex DB took longer to search and needs to reduce search time while preserving recall.
Planned Improvements
Our next development priorities are SQL streaming and random DELETE. We will also work on SQL reads and writes generally, KV point reads and range scans, DataFrame computation, lazy CSV, Parquet string computation, and the unsupported Polars-compatible APIs. For HNSW, we will reduce search time while maintaining recall of at least 0.99.
After making these improvements, we will measure again at the same input and API boundaries and publish the results.
Summary
Alopex DB v0.8.16 is strong at DataFrame construction and persistent writes to its embedded KV store. Its DataFrame computation, SQL, KV reads, and HNSW search need improvement. Some Polars-compatible operations also remain unsupported.
We will use these results as a baseline for further development and publish the performance after the changes.