Rust Is Not a Silver Bullet — Language Parsers Belong in Nim or Roc

Introduction

Hello, Asopi Tech here.

I’ve now published two language-introduction articles — “Getting Started with Nim: A Multi-Paradigm Language That Transpiles to C” and “Getting Started with Roc: A Pure Functional Language Meant to Ride Alongside Rust” — and this is the main event. In this article we walk through the AlopexDB team’s actual comparison experiment: “What if we rewrote our Rust SQL parser in Nim and Roc?” — with real code and benchmark numbers.

After writing the same SQL parser in three languages, it became very clear both why Rust cannot practically implement large-scale SQL grammars, and which candidate language is the pragmatic choice going forward. The primary sources are the alopex-db/docs feature/document-reorganization branch and the reference implementations at alopex-db/alopex experiment/sql-parser-trial.

Prologue: why Rust wasn’t good enough

First, some context on why a migration was necessary at all. AlopexDB is a distributed database written in Rust, but the SQL parser layer inside it was getting harder and harder to make progress on.

Why Rust’s type system doesn’t suit SQL grammar

The current alopex-sql module has these enums:

• ExprKind — 10 variants
• StatementKind — 8 variants
• Token — 24 variants
• Keyword — 69 variants

Roughly 180 variants total, and the build is already painful. SQL parsers are fundamentally “many recursive enums,” and in Rust that leads to:

• Box<T> is mandatory for recursive enums (currently 12 uses, growing fast)
• #[allow(clippy::large_enum_variant)] becomes ambient
• RecursionCounter (default depth 50) is needed to avoid stack overflow
• Lifetime annotations propagate everywhere — Parser<'a> bleeds into every method
• match exhaustiveness checks force full sweeps on every new variant

This isn’t specific to Alopex; even the de facto sqlparser-rs has parser/mod.rs at ~18,000 lines, and it’s straining both compile time and maintainability.

As SQL grammar grows linearly, the type-system complexity grows exponentially in Rust. That’s the structural problem.

Is there a viable non-Rust alternative?

That’s where the AlopexDB team started asking: could we take Rust out of the parser layer, hand the parser to Nim or Roc, and keep Rust for the planner / executor? The experiment was to build the same SQL parser in both languages and compare LoC, build time, and how the type systems felt from the inside.

Trial implementation scope

The Rust alopex-sql test cases were ported to Nim and to Roc, and equivalent parsers were built. Coverage includes SELECT / INSERT / UPDATE / DELETE / CREATE TABLE / DROP TABLE, plus expressions (precedence, BETWEEN, LIKE, IN, IS NULL, function calls). That’s roughly 15–20% of full SQLite grammar.

1. AST definitions — tagged union vs Tag Union

The differences show up immediately in how each language represents AST types.

Nim: object variant

Nim expresses a tagged union with ref object + a case kind clause. No Box needed, feels similar to a C union:

type
  SqlNodeKind* = enum
    nkSelect, nkInsert, nkUpdate, nkDelete
    nkIdentifier, nkStringLit, nkIntLit, nkBoolLit, nkNull
    nkStar, nkColumnRef
    nkBinaryOp, nkUnaryOp, nkFunctionCall, nkAlias
    nkFromClause, nkWhereClause, nkJoin
    # ...

  BinaryOpKind* = enum
    opEq, opNeq, opLt, opLe, opGt, opGe
    opAdd, opSub, opMul, opDiv, opMod
    opAnd, opOr
    opLike, opNotLike, opIn, opNotIn, opBetween, opNotBetween, opIs

  SqlNode* = ref object
    case kind*: SqlNodeKind
    of nkIdentifier, nkStringLit:
      strVal*: string
    of nkIntLit:
      intVal*: int64
    of nkBinaryOp:
      binOp*: BinaryOpKind
      binLeft*, binRight*: SqlNode
    of nkJoin:
      joinKind*: JoinKind
      joinLeft*, joinRight*, joinCond*: SqlNode
    else:
      children*: seq[SqlNode]

Helper constructors stay small:

proc newBinaryOp*(op: BinaryOpKind, left, right: SqlNode): SqlNode =
  SqlNode(kind: nkBinaryOp, binOp: op, binLeft: left, binRight: right)

Roc: Tag Union

Roc has first-class Tag Unions. Each variant can carry a struct-like payload, so the AST reads almost like a grammar:

SqlNode : [
    Ident Str,
    StrLit Str,
    IntLit I64,
    BoolLit Bool,
    NullLit,
    StarLit,
    BinOp { op : BinaryOp, left : SqlNode, right : SqlNode },
    UnOp { op : UnaryOp, operand : SqlNode },
    FnCall { name : Str, args : List SqlNode },
    ColRef { table : Str, column : Str },
    SelectStmt {
        columns : List SqlNode,
        from : List SqlNode,
        where : [Some SqlNode, None],
        orderBy : List SqlNode,
        # ...
    },
    InsertStmt { table : Str, columns : List Str, values : List SqlNode },
    UpdateStmt { table : Str, sets : List { col : Str, val : SqlNode }, where : [Some SqlNode, None] },
    DeleteStmt { table : Str, where : [Some SqlNode, None] },
    CreateTableStmt { table : Str, columns : List SqlNode, ifNotExists : Bool },
    DropTableStmt { table : Str, ifExists : Bool },
]

Option types simply become [Some SqlNode, None], and Box-style indirection just isn’t visible in the user code. Coming from Rust’s enum + Box + Option, this feels dramatically lighter.

Rust: enum + Box, again

For contrast, the same shape in Rust:

#[allow(clippy::large_enum_variant)]
pub enum SqlNode {
    Ident(String),
    StrLit(String),
    IntLit(i64),
    BoolLit(bool),
    NullLit,
    StarLit,
    BinOp {
        op: BinaryOp,
        left: Box<SqlNode>,
        right: Box<SqlNode>,
    },
    SelectStmt {
        columns: Vec<SqlNode>,
        from: Vec<SqlNode>,
        where_: Option<Box<SqlNode>>,
        // ...
    },
    // ...
}

• Box<SqlNode> shows up everywhere
• Option<Box<...>> becomes double-wrapped — semantically just [Some SqlNode, None], but indirection and nullability live in different layers
• You need #[allow(clippy::large_enum_variant)] to keep lint quiet
• Every new variant triggers a full sweep of every match

Concretely, just the AST portion is Nim 140 lines / Roc 93 lines / Rust 478 lines across 5 files.

2. Parsing strategy — imperative Pratt vs pure combinator

Parsing style diverges just as sharply.

Nim: imperative Pratt parser

Nim lets you write the exact same “one function per precedence level” style Rust uses. It’s an imperative style with a mutable Parser passed around:

proc parseMulDiv(p: var Parser): SqlNode =
  result = p.parsePrimary()
  while p.current.kind in {tkStar, tkSlash, tkPercent}:
    let op = case p.current.kind
      of tkStar:    opMul
      of tkSlash:   opDiv
      of tkPercent: opMod
      else: opMul  # unreachable
    discard p.advance()
    result = newBinaryOp(op, result, p.parsePrimary())

proc parseAddSub(p: var Parser): SqlNode =
  result = p.parseMulDiv()
  while p.current.kind in {tkPlus, tkMinus}:
    let op = if p.current.kind == tkPlus: opAdd else: opSub
    discard p.advance()
    result = newBinaryOp(op, result, p.parseMulDiv())

proc parseAndExpr(p: var Parser): SqlNode =
  result = p.parseComparison()
  while p.check(tkAnd):
    discard p.advance()
    result = newBinaryOp(opAnd, result, p.parseComparison())

proc parseExpr(p: var Parser): SqlNode =
  result = p.parseAndExpr()
  while p.check(tkOr):
    discard p.advance()
    result = newBinaryOp(opOr, result, p.parseAndExpr())

var Parser (Nim’s mutable reference) walks the token stream while building the AST. Nothing surprising if you’re coming from C or Rust.

Alternatively, npeg lets you write the grammar directly as PEG:

import npeg

let parser = peg("input"):
  input       <- select_stmt * !1
  select_stmt <- i"SELECT" * ws * columns * ws * i"FROM" * ws * >table_name
  columns     <- column * *(',' * ws * column)
  column      <- >+Alpha
  table_name  <- +Alpha
  ws          <- +' '

let r = parser.match("SELECT name, age FROM users")
echo r.captures  # @["name", "age", "users"]

Macros expand the PEG into Nim functions at compile time, so there’s no runtime grammar interpretation cost. Code size grows linearly with grammar size, not with type-system complexity.

Roc: recursive descent with pure functions

Everything is immutable in Roc. The token list and current position are passed as arguments; parsers return Result ParseResult Str (parse result + next position):

ParseResult : { node : SqlNode, pos : U64 }

parseExpr : List Token, U64 -> Result ParseResult Str
parseExpr = |tokens, pos|
    parseOr(tokens, pos)

parseOr : List Token, U64 -> Result ParseResult Str
parseOr = |tokens, pos|
    when parseAnd(tokens, pos) is
        Ok({ node: left, pos: pos2 }) ->
            parseOrTail(tokens, pos2, left)
        Err(msg) -> Err(msg)

parseOrTail : List Token, U64, SqlNode -> Result ParseResult Str
parseOrTail = |tokens, pos, left|
    when getToken(tokens, pos) is
        Ok(tok) ->
            if tok.kind == TkOr then
                when parseAnd(tokens, pos + 1) is
                    Ok({ node: right, pos: pos2 }) ->
                        parseOrTail(tokens, pos2, BinOp({ op: Or, left, right }))
                    Err(msg) -> Err(msg)
            else
                Ok({ node: left, pos })
        Err(_) -> Ok({ node: left, pos })

while is replaced by tail recursion, which looks more verbose, but the payoff is zero side effects and pure input-to-expected-value tests. All 56 tests in the Alopex trial pass with a single expect each.

LoC comparison

Nim and Roc each land at 35–37% of the Rust LoC for the same functionality.

3. Integration with the Rust host — Nim’s C shared library vs Roc’s Platform/Host

Once you commit to “parser in another language,” integration with the Rust-side planner and executor becomes the make-or-break question. The two languages take very different approaches.

Nim: --app:lib emits a C shared library directly

Nim transpiles to C, and from Rust’s perspective the output is a plain .so + .h. That’s the biggest advantage. The entry point:

import std/[json]
import ast, parser

type
  ParseResultKind {.exportc.} = enum
    prkOk = 0
    prkError = 1

  CParseResult {.exportc.} = object
    kind: ParseResultKind
    json_ptr: cstring
    json_len: cint
    error_ptr: cstring
    error_len: cint

proc NimMain() {.importc.}

proc alopex_parser_init*() {.exportc, dynlib, cdecl.} =
  ## Initialize Nim runtime. Must be called once.
  NimMain()

proc alopex_parse_sql*(input: cstring, length: cint): CParseResult
    {.exportc, dynlib, cdecl.} =
  let sql = if length > 0: ($input)[0 ..< length] else: $input
  try:
    let astNode = parseSql(sql)
    let jsonStr = $toJson(astNode)
    let copied = cast[cstring](alloc(jsonStr.len + 1))
    copyMem(copied, cstring(jsonStr), jsonStr.len + 1)
    result = CParseResult(
      kind: prkOk,
      json_ptr: copied,
      json_len: cint(jsonStr.len),
      error_ptr: nil,
      error_len: 0,
    )
  except ParseError:
    let errMsg = getCurrentExceptionMsg()
    let copied = cast[cstring](alloc(errMsg.len + 1))
    copyMem(copied, cstring(errMsg), errMsg.len + 1)
    result = CParseResult(
      kind: prkError, json_ptr: nil, json_len: 0,
      error_ptr: copied, error_len: cint(errMsg.len),
    )

Three key ideas:

• Serialize the AST to JSON to return it — never expose complex structs across the C ABI
• Memory is allocated with alloc → Rust side calls alopex_free_string when done
• NimMain runs once first — required to initialize the ORC runtime

The Rust side is straightforward:

#[repr(C)]
#[derive(Debug)]
pub enum ParseResultKind { Ok = 0, Error = 1 }

#[repr(C)]
pub struct CParseResult {
    pub kind: ParseResultKind,
    pub json_ptr: *const c_char,
    pub json_len: c_int,
    pub error_ptr: *const c_char,
    pub error_len: c_int,
}

extern "C" {
    fn alopex_parser_init();
    fn alopex_parse_sql(input: *const c_char, length: c_int) -> CParseResult;
    fn alopex_free_string(p: *const c_char);
}

pub fn parse_sql(sql: &str) -> Result<serde_json::Value, String> {
    unsafe { alopex_parser_init(); }  // once
    let cs = CString::new(sql).unwrap();
    let r = unsafe { alopex_parse_sql(cs.as_ptr(), sql.len() as c_int) };
    match r.kind {
        ParseResultKind::Ok => { /* json_ptr → serde_json::Value */ }
        ParseResultKind::Error => { /* error_ptr → String */ }
    }
}

Nim parser → JSON → Rust serde_json completely avoids type-shape mismatches across FFI when passing complex ASTs.

Roc: Platform / Host model

Roc, on the other hand, exports C ABI functions from a Roc app to a Rust host:

Roc App (parser, pure functions)
   ↓ exported as C ABI (roc_app_main, etc.)
Rust Platform Host (alopex-sql)
   ↓ extern "C" linking
alopex-core / alopex-embedded

This is the same architecture basic-webserver uses (Rust + hyper + tokio calling into Roc functions), and it embodies Roc’s design philosophy well.

But — as Alopex’s docs point out — the following are heavier obstacles for Roc than for Nim today:

• Library-output workflow isn’t well established — Roc’s primary use case is still “executable binary”
• No header auto-generation
• Static linking unverified
• Surgical linker issue forces --linker=legacy

4. Runtime performance — will SQL parsing actually be slower than Rust?

The intuitive worry that “changing languages will make it slower” doesn’t actually hold up for SQL parsing. Aggregated general benchmarks:

The workloads where Rust wins — numeric, regex, parallelism — aren’t really used in SQL parsing. The workloads that dominate a parser are:

1. Massive AST node allocation (heap-heavy)
2. String comparison (keyword matching)
3. Recursive tree construction (pattern matching + node stitching)

All three closely resemble Binary Trees. Nim’s ORC beats Rust’s Box + Drop on “many small object lifecycles,” which is why Binary Trees shows Nim 46% faster than Rust. Therefore: for AST-heavy workloads, Nim is likely at least as fast as Rust, possibly faster.

Roc doesn’t have public benchmark data against Rust yet, but the Perceus design gives it a similar expected profile.

5. Build times — the difference that changes how you iterate

For SQL parser development, this is arguably the deciding factor.

Right now Rust already takes release 2m45s / tests 2m04s, and extrapolating to SQLite-scale means 4+ minute releases and 3+ minute test cycles. A 3–4 minute edit-test loop is simply not viable for iterative SQL parser work.

Nim at SQLite scale is release 16 s / tests 30 s. Roc is dev 5 s + tests 2 s. That’s a 10-60x iteration speed advantage over Rust.

6. Verdict — Nim wins the migration

Taken together, the AlopexDB project has picked Nim as the primary migration target for the SQL parser layer. The decision matrix:

“Usable today” × “Rust C ABI integration is settled” × “Wins the AST-heavy workload” — those three land squarely on Nim.

Roc is compelling as a design and worth re-evaluating once 1.0 arrives. The feature/document-reorganization docs in alopex-db/docs show that the Nim migration plan is already promoted to a formal spec (.spec-workflow/specs/nim-sql-parser-migration/) with these milestones:

• M0: preparation (1d)
• M1: Nim parser expansion — full Rust AST compatibility (3-5d)
• M2: JSON serialization layer (1-2d)
• M3: Rust FFI bridge (2-3d)
• M4: caller-side feature flag switching (1d)
• M5: verification & optimization (2-3d)
• M6: gradual Rust parser retirement (1d)

Total 11–16 days (2–3 weeks) to complete the migration — a very concrete plan.

7. The final architecture

The new Alopex SQL stack looks like:

┌─────────────────────────────────────────┐
│  SQL Parser (Nim)                       │
│  ・npeg PEG grammar or hand Pratt       │
│  ・AST via object variant               │
│  ・Serialized to JSON on the way out    │
│  Output: libalopex_sql_parser.so + .h   │
└──────────────┬──────────────────────────┘
               │ extern "C" FFI
               ▼
┌─────────────────────────────────────────┐
│  alopex-sql (Rust)                      │
│  ・AST(JSON) → LogicalPlan              │
│  ・Planner / Optimizer / Executor       │
│  ・Catalog & transactions               │
└──────────────┬──────────────────────────┘
               │
               ▼
┌─────────────────────────────────────────┐
│  alopex-core (Rust)                     │
│  ・Storage, replication, distribution   │
└─────────────────────────────────────────┘

“A Nim layer that stops scaling exponentially with grammar complexity” × “A Rust layer that keeps its edge in storage and distributed systems” — each language occupies the domain where it’s truly strong. With this decision, AlopexDB can put SQLite-level SQL support (5 JOIN kinds, subqueries, CTEs, etc.) on a realistic schedule for v0.6.

Wrapping up

Writing the same SQL parser in three languages surfaced these truths:

• AST expressiveness: Roc (ADT) > Nim (object variant) >> Rust (enum + Box)
• Parser ergonomics: Nim (Pratt) and Roc (pure) are a matter of taste; Rust is high-cost to change
• Rust interop: Nim’s C shared library + auto header is currently the strongest option
• Runtime performance: on AST-heavy workloads, Nim may actually beat Rust
• Build times: 10–60x faster iteration than Rust
• Production readiness: Nim today, Roc after 1.0

Language choice can devolve into holy war, but if you look carefully at workload characteristics × type-system fit, the discussion becomes surprisingly quantitative. AlopexDB’s decision — take Rust out of the parser layer and hand it to Nim — is not a knock on Rust. It’s an architectural move to let Rust do what Rust is uniquely good at.

If Nim or Roc caught your interest, please also check out “Getting Started with Nim: A Multi-Paradigm Language That Transpiles to C” and “Getting Started with Roc: A Pure Functional Language Meant to Ride Alongside Rust”.

See you next time!

References

• alopex-db/docs — AlopexDB research docs
• alopex-db/alopex experiment/sql-parser-trial
• alopex-db/alopex feature/nim-sql-parser — the actual migration branch
• sqlparser-rs (the de facto Rust implementation)
• npeg — PEG for Nim
• roc-parser — Parser Combinator for Roc
• std/parsesql — Nim’s stdlib SQL parser
• Zig, Parser Combinators, and Why They’re Awesome (Hexops) — bonus: comparison with Zig