MemKV

Problem

Key-value stores like Redis get their simplicity from single-threaded command execution — no locks, no torn writes, deterministic ordering. I wanted to build that model from scratch in Go, a language whose whole runtime is built around concurrency, and see what breaks: how do you keep one logical owner of state while still handling networking, disk I/O, and replication concurrently without either serializing everything behind one thread or reintroducing the locks you were trying to avoid?

Architecture

Every client connection gets its own goroutine for RESP2 decoding and response encoding. Parsed commands don't touch state directly — they're submitted to a bounded queue owned by a single engine goroutine.

The engine is the only goroutine that touches the store, so the command path needs zero locks — reads, writes, transactions, TTL maintenance, and replication snapshot capture are all ordered by definition, not by coordination. This is a single-owner execution model, not a single-threaded process: networking, WAL fsync, and replication I/O still run concurrently around the engine. The trade-off is explicit — one slow command delays everything behind it in the queue, and a full queue applies backpressure to client goroutines rather than letting memory grow unbounded.

Storage model

Every key maps to one union Entry holding its type, payload, version, and absolute expiration deadline. Mutation invariants — version bumps, FIFO metadata, empty collection cleanup — live in the store itself, not scattered across command handlers, so WATCH can detect conflicts from one source of truth.

The sorted set deliberately trades write performance for clarity: mutation rebuilds the sorted slice in O(n log n) rather than maintaining a skip list. It was the right call for a project meant to be readable end to end, and the benchmark numbers below make the cost visible instead of hiding it.

Durability: making a slow disk safe, not fast

When the write-ahead log is enabled, its fsync is the durability commit point — memory is only mutated, and replication only notified, after the record is versioned, checksummed, and synced to disk. If the append or sync fails, the mutation is rejected outright rather than applied in memory and silently lost on crash.

Recovery is designed around the failure modes that actually happen: an incomplete final record from a crash mid-write is truncated, but corruption in the middle of the log fails recovery loudly instead of silently skipping a record — a silent skip would mean serving a value that's inconsistent with what was acknowledged. Snapshots pair with the WAL rather than replace it: a checkpoint stores its corresponding WAL byte offset, so recovery restores the snapshot and replays only the committed suffix instead of the whole log.

Replication without blocking the engine

Replication is asynchronous — a primary write is acknowledged without waiting on a replica, because a slow or dead replica should never become the primary's problem. A replica reconnecting within the backlog window gets a partial resync from the backlog; one that's been gone too long gets a full resync from a captured snapshot, with writes that land during the transfer handed off through the backlog so there's no snapshot-to-live gap. A replica whose queue overflows — because it can't keep up — is simply disconnected rather than allowed to apply backpressure to the primary.

Verifying it under adversarial conditions

Correctness claims for a storage engine are only as good as what you throw at them. Beyond unit tests, the suite includes:

• Fuzz targets for the RESP2 decoder, WAL record decoder, and snapshot decoder
• go test -race across engine ordering, transactions, and WATCH conflict detection
• Real TCP fragmentation and pipelining, not just in-process calls
• Deliberate WAL corruption and tail-repair scenarios
• Full and partial replication sync, including the sync-to-live handoff and slow-replica overflow

What the benchmarks actually show

Reproducible microbenchmarks (Go 1.26.4, darwin/arm64, Apple M1 Pro) surface the two costs the design consciously accepts:

The sorted-set cost grows with cardinality exactly as the design implies, and snapshot encoding's allocation count is the clearest next optimization target — both are documented as known trade-offs rather than surprises found in production.