Alopex DB v0.8.8 Tutorial, Part 5: Opening the Database with a Server

Up through Part 4, we opened the database inside a Python process. Part 5 opens the same data directory with alopex-server and runs the same SQL over HTTP.

The v0.8.8 release verification used only published packages and confirmed that the library, embedded, CLI, HTTP, gRPC, and cluster-aware entry points could read the same data. This part reproduces the server portion of that verification locally.

Separating a growing application’s database into another process usually requires dumping, converting, loading, and comparing the data. Alopex DB instead opens the data directory used since Part 1.

1. Environment

Install the server and CLI.

cargo install alopex-cli --version "=0.8.8" --locked
cargo install alopex-server --version "=0.8.8" --locked

2. Five entry points to the engine

Alopex DB provides five paths to the same data.

Database.new() from Part 1 is SF-MEM, and Database.open() is SF-FILE. Only SF-MEM is non-persistent; the other four can open the same data directory.

3. Creating a data directory

Create the articles table with the CLI. Save the following as setup.sql.

CREATE TABLE articles (
  id INTEGER PRIMARY KEY,
  title TEXT,
  author TEXT,
  views INTEGER,
  rating REAL
);
INSERT INTO articles VALUES (1, 'Rustで書くLSMツリー', 'mio', 1200, 3.5);
INSERT INTO articles VALUES (2, 'SQLパーサをNimで実装する', 'mio', 800, 4.5);
INSERT INTO articles VALUES (3, 'ベクトル検索の基礎', 'ren', 1500, 4.5);

Pass a directory with --data-dir to write the data there.

alopex --batch --output json --data-dir ./srv-data sql -f setup.sql

This creates ./srv-data, which Python can also read.

from alopex import Database

db = Database.open("./srv-data")
print(db.execute_sql("SELECT id, title, views FROM articles ORDER BY id"))

[{'id': 1, 'title': 'Rustで書くLSMツリー', 'views': 1200},
 {'id': 2, 'title': 'SQLパーサをNimで実装する', 'views': 800},
 {'id': 3, 'title': 'ベクトル検索の基礎', 'views': 1500}]

Python reads the data written by the CLI without conversion. It is the same kind of directory opened with Database.open() throughout the tutorial.

4. Opening the directory with the server

Open the same directory with alopex-server. Save the configuration as alopex.toml.

data_dir = "./srv-data"
http_bind = "127.0.0.1:18080"
grpc_bind = "127.0.0.1:19090"
admin_bind = "127.0.0.1:18081"

Stop the Python process before starting the server.

alopex-server --config alopex.toml

The log displays the applied configuration.

INFO alopex_server::server: Cluster startup configuration applied
  cluster_mode=SingleNode node_id=local role=Gateway
  http_bind=127.0.0.1:18080 grpc_bind=127.0.0.1:19090 admin_bind=127.0.0.1:18081

HTTP listens on port 18080, gRPC on 19090, and administration on 18081.

A data directory is opened by one process at a time. Do not start the server while Python still has it open.

5. Running the same SQL over HTTP

From another terminal, send a POST request to /api/sql/query.

curl -s -X POST http://127.0.0.1:18080/api/sql/query \
  -H 'Content-Type: application/json' \
  -d '{"sql":"SELECT id, title, views FROM articles ORDER BY id"}'

{"columns":[{"name":"id","data_type":"INTEGER"},
            {"name":"title","data_type":"TEXT"},
            {"name":"views","data_type":"INTEGER"}],
 "rows":[[{"Integer":1},{"Text":"Rustで書くLSMツリー"},{"Integer":1200}],
         [{"Integer":2},{"Text":"SQLパーサをNimで実装する"},{"Integer":800}],
         [{"Integer":3},{"Text":"ベクトル検索の基礎"},{"Integer":1500}]],
 "affected_rows":null}

The SQL is unchanged from the Python query in Section 3.

The JSON shown here extracts only columns, rows, and affected_rows. The actual response also includes results and routing_diagnostics.

The response shape differs. Python returns dictionaries, while HTTP separates column definitions from row arrays and attaches a type name to each value. Tests comparing these paths should compare values and column names, not the surrounding shape. Fixing metadata such as execution time in a test would make it fail after unrelated server changes.

The recording writes a row to the v0.8.8 server over HTTP. Its actual response shows affected_rows: 1 and the local_only routing decision.

6. Reading an HTTP-written row through the file API

Add one row over HTTP.

curl -s -X POST http://127.0.0.1:18080/api/sql/query \
  -H 'Content-Type: application/json' \
  -d "{\"sql\":\"INSERT INTO articles VALUES (4, 'HTTPから追加した記事', 'sora', 500, 4.0)\"}"

{"columns":[],"rows":[],"affected_rows":1}

Stop the server and reopen the directory from Python.

from alopex import Database

db = Database.open("./srv-data")
for row in db.execute_sql("SELECT id, title, author FROM articles ORDER BY id"):
    print(row)

{'id': 1, 'title': 'Rustで書くLSMツリー', 'author': 'mio'}
{'id': 2, 'title': 'SQLパーサをNimで実装する', 'author': 'mio'}
{'id': 3, 'title': 'ベクトル検索の基礎', 'author': 'ren'}
{'id': 4, 'title': 'HTTPから追加した記事', 'author': 'sora'}

The file path reads id = 4, which was inserted through HTTP. Every entry point writes to the same data directory.

7. healthz and admin/status

The administration port exposes a health check.

curl -s http://127.0.0.1:18081/healthz

Detailed status is available from /api/admin/status.

curl -s http://127.0.0.1:18080/api/admin/status

{"version": "0.8.8",
 "uptime_secs": 20,
 "cluster": {
   "mode": "single_node",
   "identity": {"node_id": "local", "role": "gateway",
                "lifecycle_state": "unconfigured"},
   "membership": {"source": "local_default", "members": []},
   "routing_capabilities": {"local_only": true,
                            "future_distributed_execution_required": true}}}

mode is single_node, and routing_capabilities.local_only is true. SQL execution remains within this node.

8. Starting in cluster-aware mode

Add a [cluster] section to alopex.toml to start in cluster-aware mode.

data_dir = "./srv-data"
http_bind = "127.0.0.1:18080"
grpc_bind = "127.0.0.1:19090"
admin_bind = "127.0.0.1:18081"

[cluster]
mode = "cluster_aware"
node_id = "node-1"
cluster_id = "blog-cluster"
advertised_endpoint = "127.0.0.1:19090"

node_id, cluster_id, and advertised_endpoint are all required. Omitting any of them stops startup.

failed to load config: invalid config: cluster.cluster_id is required when cluster.mode is cluster_aware

The contents of /api/admin/status change after startup.

{"cluster": {
   "mode": "cluster_aware",
   "identity": {"node_id": "node-1", "cluster_id": "blog-cluster",
                "advertised_endpoint": "127.0.0.1:19090",
                "role": "gateway", "lifecycle_state": "active"},
   "membership": {"source": "chirps", "members": []},
   "routing_capabilities": {"local_only": true},
   "degraded": false}}

lifecycle_state changes from unconfigured to active, and membership.source changes from local_default to chirps, while local_only remains true. This configuration does not distribute SQL across nodes. Operations that require distributed execution are rejected with future_distributed_execution_required. Cluster-aware mode exposes node identity and membership to external observers.

Monitor the degraded field. If the membership source is unavailable, the server starts with a single-node fallback and sets degraded to true. A check that only confirms process startup would miss this state.

9. Reading statistics with io_stats

Memory and I/O statistics are available through ordinary SQL.

alopex --batch --output json --data-dir ./srv-data sql "SELECT io_stats()"

{"io_stats": "{\"wal_write_bytes\":0,\"sstable_read_bytes\":0,
               \"buffer_pool_hit_rate\":1,\"buffer_pool_size_bytes\":0,
               \"memtable_size_bytes\":1656,\"compaction_bytes_written\":0}"}

The result includes WAL bytes written, SSTable bytes read, buffer pool hit rate and size, memtable size, and compaction bytes written. The value is a JSON string, so parse it again before use. Other available functions include memory_stats() and clear_cache().

Before sending these values to a dashboard, inspect the empty-database values and identify which counters increase after writes. That baseline makes abnormal values interpretable.

10. Commands and endpoints used in this part

11. Connecting from Rust through embedded and HTTP paths

Rust can open the data through two paths: directly opening the directory in-process or querying the server over HTTP.

Opening the embedded path

Read the id = 4 row added over HTTP in Section 6. Stop the server before running this program.

[dependencies]
alopex-embedded = "=0.8.8"

use alopex_embedded::{Database, SqlResult};
use std::path::Path;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let db = Database::open(Path::new("./srv-data"))?;
    if let SqlResult::Query(q) = db.execute_sql("SELECT id, title, author FROM articles ORDER BY id")? {
        for row in &q.rows {
            let cells: Vec<String> = row.iter().map(|v| format!("{v:?}")).collect();
            println!("{}", cells.join(" | "));
        }
    }
    Ok(())
}

Integer(1) | Text("Rustで書くLSMツリー") | Text("mio")
Integer(2) | Text("SQLパーサをNimで実装する") | Text("mio")
Integer(3) | Text("ベクトル検索の基礎") | Text("ren")
Integer(4) | Text("HTTPから追加した記事") | Text("sora")

Rust reads the directory created by the CLI and extended over HTTP. These are the same four rows returned by Python in Section 6. The write and read paths differ, but the data directory is shared.

Opening the HTTP path

Run the same query through the server. alopex-embedded is unnecessary; an HTTP client and JSON parser are sufficient.

[dependencies]
ureq = { version = "2", features = ["json"] }
serde_json = "1"

Restart the server and run the program.

use serde_json::json;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let body: serde_json::Value = ureq::post("http://127.0.0.1:18080/api/sql/query")
        .send_json(json!({"sql": "SELECT id, title, views FROM articles ORDER BY id"}))?
        .into_json()?;

    let names: Vec<&str> = body["columns"].as_array().unwrap().iter()
        .map(|c| c["name"].as_str().unwrap()).collect();
    println!("{}", names.join(" | "));

    for row in body["rows"].as_array().unwrap() {
        let cells: Vec<String> = row.as_array().unwrap().iter()
            .map(|v| v.as_object().unwrap().values().next().unwrap().to_string())
            .collect();
        println!("{}", cells.join(" | "));
    }
    Ok(())
}

id | title | views
1 | "Rustで書くLSMツリー" | 1200
2 | "SQLパーサをNimで実装する" | 800
3 | "ベクトル検索の基礎" | 1500
4 | "HTTPから追加した記事" | 500

Each cell is a one-entry object whose key is the type name, such as {"Integer": 1}. The code calls values().next() to retrieve that single value. It is the JSON representation of the embedded SqlValue::Integer(1).

Both programs use the same SQL. Only the connection changes: the embedded program points Database::open() at a directory, while the HTTP program points at a URL.

When an application moves from embedded operation to a server, this connection code is the part that changes. Its SQL and data directory remain available without conversion.

Summary

This part opened one ./srv-data directory through four paths. The CLI created articles with --data-dir; Python read it through Database.open(); server HTTP inserted id = 4; and Rust read all four rows through Database::open(). These are the SF-FILE and SF-HTTP surfaces from Section 2. The published v0.8.8 verification also confirmed mode parity for gRPC and cluster-aware operation.

The query SELECT id, title, views FROM articles ORDER BY id remained unchanged across the paths. Only the connection changed between Database.open("./srv-data") and POST /api/sql/query. No dump-and-load migration occurred.

Result representation varies by path. Python returns a list of dictionaries, embedded Rust returns SqlResult::Query, and HTTP returns JSON with separate columns and rows. Values and column names agree, so cross-path tests should compare those two elements rather than the wrappers.

Adding [cluster] mode=cluster_aware changes /api/admin/status: lifecycle_state becomes active, and membership.source becomes chirps. In v0.8.8, local_only remains true. A single cluster-aware member can start and read existing data, while operations requiring multi-node execution are rejected with future_distributed_execution_required. This configuration exposes node identity and membership.

References

• v0.8.8 release verification report