Flash weight streaming for MLX: run massive models larger than your RAM on Apple Silicon.
Run the full unquantized LTX-2.5 (22B) video+audio model on a laptop's integrated GPU by streaming weights from NVMe
On-device inference engine running Qwen MoE models larger than the phone's RAM - offline, exact, at usable speed.
An LLM inference engine written in pure Rust, designed to run large models on hardware that would normally refuse them.
Run large MLX models on Apple Silicon with flash weight streaming, using native precision beyond RAM limits
A research toolbox for running and studying LLMs on constrained hardware, with memory, placement, inference, and benchmarking methods.
Run LLMs larger than your RAM — out-of-core local inference on consumer hardware (MoE, GGUF, llama.cpp). NVMe-as-memory with honest telemetry: real tok/s, page faults, disk traffic.
To associate your repository with the weight-streaming topic, visit your repo's landing page and select "manage topics."