March 2026 Update: No Corruption Bugs, 8x Faster Than Git, Semantic Writes Still Too Slow

TL;DR

• Workload testing worked: 500 real commits replayed with no state corruption bugs
• Semantic writes still hit a write-amplification bottleneck on large files (500ms+)
• Without the semantic layer, the file-write-plus-commit workflow is ~8x faster than Git
• April goal: sub 100ms for 10k row inserts

Workload testing

Last month we set out to do real workload testing in March to reveal performance bottlenecks and bugs that prevent production usage of lix.

The test replays 500 real commits from the paraglide-js repo. For each commit, it sets up the "before" state outside the timer, applies the same file changes, and measures how long Lix takes to commit. The simulated scenario: "I edited some files, now I'm committing."

Three findings came out of this.

Finding 1: It works

The best result from the workload replay is that it worked. Replaying 500 real commits did not reveal state corruption bugs. That matters more than the benchmark number because correctness is the prerequisite for everything else.

Finding 2: Semantic writes still bottleneck on write amplification

  contract.docx
       ↓
  paragraphs / tables / images
       ↓
  diff / merge / history on those units

The bottleneck is write amplification. A single file write fans out into many rows. Inserting a file with 10k rows means the engine has to process 10k rows. On the current path, semantic writes are multi-second operations. Any interaction above 100ms stops feeling instantaneous, so this needs to come down.

  contract.docx            Lix engine                    SQL database
  ┌──────────────────┐     ┌─────────────────────┐       ┌──────────────┐
  │ Paragraph 1      │     │ process 10,000      │       │              │
  │ Paragraph 2      │     │ rows         │       │ INSERT row 1 │
  │ Paragraph 3      │────►│                     │──────►│ INSERT row 2 │
  │ Table 1          │     │ validate, transform,│       │ ...          │
  │   Row 1          │     │ detect changes      │       │ INSERT row   │
  │   Row 2          │     │                     │       │   10,000     │
  │ Image 1          │     │ 💥 too slow         │       │              │
  │ ...              │     └─────────────────────┘       └──────────────┘
  │ Paragraph 4,291  │
  └──────────────────┘
  1 file write             N rows to process           N SQL row inserts

The engine is not fast enough to handle these large batches. The goal for April is to get 10k row inserts under 100ms.

Finding 3: Without the semantic layer, the file-write-plus-commit workflow is ~8x faster than Git

Unexpected good news. Without the semantic layer (treating files as blobs), Lix completes the same file-write-plus-commit workload in ~5 ms where Git takes ~39 ms.¹

The difference comes down to architecture. Lix applies mutations inside an open SQLite transaction. Committing is closing that transaction (~1 ms). The comparison runs git add -A followed by git commit, which scans the working tree, updates the index, and writes tree and commit objects.

This is encouraging, but it's the blob layer only. The semantic layer is what makes Lix useful for non-code files, and that's where the work is.

Why not skip the semantic layer entirely?

If Lix is already fast without the semantic layer, why not just store blobs and diff on the fly?

This is really a source-of-truth decision, not a storage decision. Lix can keep both a blob and semantic state, but only one can be authoritative:

  Option A: Blob is source of truth, diffs computed on the fly

  ┌──────────────┐       ┌──────────────┐
  │ contract.docx│──────►│  re-parse    │──────► diffs (computed every time)
  │   (blob)     │       │  on every op │
  └──────────────┘       └──────────────┘


  Option B: Diffs are source of truth, blob derived on demand

  ┌──────────────┐       ┌──────────────┐
  │    diffs     │──────►│  serialize   │──────► contract.docx (derived)
  │  (stored)    │       │  on demand   │
  └──────────────┘       └──────────────┘

If both are independently writable, they can drift.

Git gets away with blob-first storage because its default diff and merge model is line-oriented and works well for ordinary text. For smaller structured text files like JSON, re-parsing on demand can still be acceptable. But as files grow, the cost per operation grows with them:

OOXML files like .docx and .xlsx are ZIP packages made of many XML parts, so rebuilding semantic state from the blob on every merge, history read, or sync means repeatedly paying unzip, parse, and tree-diff costs. A cache avoids repeated rebuilds, but now there are two representations to keep consistent — every write path must update both, and bugs in that synchronization are silent data corruption.

So Lix makes semantic state canonical and materializes the blob on demand when someone actually needs the file bytes. The tradeoff is that blob writes pay an upfront parsing cost — which is the write-amplification bottleneck we're now fixing.

Long term, most app and agent writes should bypass blob parsing entirely. They will write rows directly, so the hot path avoids both blob parsing and blob serialization.

That means the semantic layer must be fast.

Prolly trees for cheap versioning

Solving write speed alone isn't enough — storage also needs to scale across versions. Without content deduplication, creating a new version means duplicating all row data. A 10k-row Word document across 5 versions = 50k rows stored.

  Without deduplication:

  version: main              version: draft
  ┌──────────────────┐       ┌──────────────────┐
  │ 10,000 rows  │       │ 10,000 rows  │  ← full copy
  └──────────────────┘       └──────────────────┘
  💥 10,000 rows              💥 10,000 rows (copied)

Prolly trees are the most promising fit for this. Rows are grouped into chunks with boundaries determined by content hashes. If one paragraph changes, only the chunk containing that paragraph is new. The rest is shared across versions.

  With Prolly trees:

  version: main                       version: draft
  (original)                          (paragraph 3 edited)
  ┌──────────────────┐                ┌──────────────────┐
  │ Paragraph 1      │                │ Paragraph 1      │
  │ Paragraph 2      │                │ Paragraph 2      │
  │ Paragraph 3      │                │ Paragraph 3 ✎    │
  │ Table 1          │                │ Table 1          │
  │ ...              │                │ ...              │
  │ Paragraph 4,291  │                │ Paragraph 4,291  │
  └──────────────────┘                └──────────────────┘
          │                                   │
          ▼                                   ▼
  ┌──────────────┐                    ┌──────────────┐
  │   chunk A  ──┼────────────────────┼── chunk A    │  ← shared
  │   chunk B    │                    │   chunk B'   │  ← different (contains edited paragraph 3)
  │   chunk C  ──┼────────────────────┼── chunk C    │  ← shared
  │   chunk D  ──┼────────────────────┼── chunk D    │  ← shared
  └──────────────┘                    └──────────────┘

  ✅ Creating a version = pointing to the same chunks
  ✅ Only changed chunks are stored separately

What's next in April

Goal: Make Lix ready for people to try out.

March proved the blob path works. April is about closing the gap so the semantic layer is fast enough and correct enough for real use.

1. 10k row inserts under 100 ms. SQLite can insert 10k rows in under 10 ms. That gives us ~90 ms of headroom to work with.
2. Prolly trees for cheap branching. Without content deduplication, every branch copies all row data. Prolly trees share unchanged chunks across versions, so branching a 10k-row document is nearly free.
3. Workload testing with the semantic layer on. March proved the blob path doesn't corrupt state across 500 real commits. April repeats that test with semantic writes enabled.