FreeToken Architecture
How FreeToken runs giant MoE models on consumer hardware: CPU-GPU co-execution, q* expert routing, double-buffered prefill, and LRU expert cache.
What is the Architecture?
Architecture articles explain why FreeToken can run giant mixture-of-experts models on consumer hardware. They cover CPU-GPU co-execution, the q* expert routing policy, double-buffered prefill, the LRU expert cache, Fourier Token Weighting, elastic memory management, and the semantic cache that skips redundant computation.
Why understand the architecture?
Tune With Intent
Understand how offload, hybrid, and cache settings actually affect throughput
Read the Paper Confidently
Follow the q* policy and FTW explanations behind the benchmark tables
Diagnose Bottlenecks
Trace slow prefill or decode to bandwidth, cache, or expert placement causes
Featured & Essential
FreeToken quantization: Setup Guide for MoE Inference
Learn how FreeToken handles quantized MoE models, memory tiers, hardware requirements, caching, and local inference setup in 2026.
FreeToken paper: MoE Serving Comparison and Setup Guide
Explore the FreeToken paper, its bandwidth-adaptive MoE serving design, benchmark claims, hardware limits, and practical evaluation steps.
All Architecture Guides
FreeToken edge native moe serving: Architecture Guide
Learn how FreeToken uses bandwidth-adaptive execution, caching, and CPU-GPU coordination to serve large MoE models on edge hardware.
FreeToken github moe: Setup Guide & MoE Runtime Comparison
Learn how FreeToken serves oversized MoE models, what hardware it needs, and how its adaptive runtime compares with llama.cpp.
FreeToken: Edge-Native MoE Setup Guide & Comparison
Learn how FreeToken serves large mixture-of-experts models locally, compare its routing-aware cache with llama.cpp, and assess hardware fit.