Redis Clone (Python / asyncio)

A key-value store built from scratch, modeled on Redis: in-memory data structures, RDB-style disk persistence, and a real TCP server speaking a simple text protocol. Built as Project 4 of a systems engineering curriculum, following Projects 1–3 (multithreaded downloader, async HTTP reverse proxy, Redis-backed job queue).

What it supports

Hash — HSET, HGET, HGETALL, HDEL List — LPUSH, RPUSH, LPOP, RPOP, LRANGE String — SET, GET, INCR

Each Redis key has exactly one type, enforced centrally — trying to SET a key that already holds a hash (or vice versa) fails with a clean WrongTypeError, matching real Redis's WRONGTYPE behavior.

Project layout

redis_clone/
├── key_value_store.py   # Core engine: all commands, locking, persistence hooks
├── snapshot.py           # RDB-style save/load — tags each value with its type
├── server.py              # asyncio TCP server, command dispatch, graceful shutdown
└── shared/
    └── models.py          # KeyValueStoreTypes — single source of truth for
                            # which Python type backs each Redis type
                            # (dict = hash, deque = list, str = string)

shared/models.py exists specifically to avoid a circular import: snapshot.py needs to know the same type mapping key_value_store.py uses, but neither module should depend on the other.

Running it

python3 server.py

Starts listening on localhost:8003. First run prints No Snapshot found. Starting afresh. — normal, not an error.

Talking to it

No client library needed — it's just newline-terminated text over TCP, so nc (netcat) works directly:

nc localhost 8003

SET foo bar
GET foo
HSET job:1 status running
HGETALL job:1
RPUSH queue:1 a b c
LRANGE queue:1 0 -1
INCR counter

Or script it with raw sockets:

import socket
s = socket.create_connection(('localhost', 8003))
s.sendall(b'SET foo bar\n')
print(s.recv(4096).decode())

One connection can send multiple commands in sequence — it stays open until the client disconnects.

Design notes

Protocol. Deliberately simpler than HTTP: one line in, one line out, no headers, no Content-Length buffering. tokens[0] is the command name, everything after is passed via getattr(kv, tokens[0].lower())(*tokens[1:]) — so adding a new command to KeyValueStore doesn't require touching the dispatcher at all.

Persistence (RDB-style). KeyValueStore loads any existing snapshot.json on startup and saves periodically via a self-rescheduling threading.Timer (every 5s). Since JSON can't distinguish a deque from a list, each saved value is tagged with its type ({"type": "list", "data": [...]}) so it can be correctly reconstructed on load.

Concurrency. The TCP server is single-threaded asyncio — client commands never truly interleave, since none of KeyValueStore's methods contain an await, so each one runs atomically from the event loop's perspective. The only real race in the system is between the background save timer (a genuine OS thread) and everything else; that's what self.lock (a threading.RLock, reentrant to allow incr to call set internally without deadlocking) protects against — and nothing else needs it.

Graceful shutdown. On Ctrl+C, the server is meant to: stop accepting new connections, cancel and drain in-flight client tasks, cancel the save timer, then do one final synchronous save_to_disk() so a snapshot is never lost to an untimed process kill.

Not yet implemented

• EXPIRE / TTL on keys
• Multi-command pipelining beyond simple sequential reads
• AOF (append-only log) persistence — currently RDB-only
• Connection-level auth or any access control