- FreeToken install requires a supported CUDA-capable GPU, host memory, storage, and a compatible runtime environment.
- Release location: The system is announced for release through the official FlashML project channel.
- Core setup: Prepare model weights, host-resident expert storage, GPU cache space, and measured bandwidth values.
- Important limitation: The available reference material does not publish a copy-ready installation command.
- Best practice: Verify the current release instructions before converting model weights or launching the engine.
FreeToken install: What the System Requires
FreeToken is an edge-native serving system for large Mixture-of-Experts (MoE) models. It is not a game, downloadable character title, or redemption-code service. The installation process is therefore centered on GPU inference, host memory, CUDA support, model storage, and runtime configuration.
The published design targets personal computers, workstations, and laptops where the complete expert pool may exceed available VRAM. FreeToken keeps the full routed-expert pool in host memory while using GPU memory as an elastic expert cache. Non-expert weights remain on the GPU, and cache capacity can change as other applications consume or release VRAM.
Treat FreeToken as a systems deployment rather than a one-click application. Hardware profiling, model preparation, and memory planning are part of the setup because runtime scheduling depends on the actual machine.
GPU Tier
A discrete CUDA-capable GPU provides the main execution path. The reference evaluation spans 8 GB laptop GPUs through workstation-class hardware.
Host Memory
System memory stores the complete expert pool and supplies bandwidth for CPU-side expert execution when GPU cache misses occur.
Fast Storage
NVMe storage is important for startup. FreeToken can load prepared weights directly into their final host layout.
The most important distinction is between capacity and performance. The full model does not need to fit entirely in VRAM, but the host must hold the expert pool and provide enough transfer bandwidth for acceptable latency.
| Resource | Role in FreeToken | Setup priority |
|---|---|---|
| GPU VRAM | Stores non-expert weights, KV cache, and elastic expert slots | High |
| Host memory | Holds the complete routed-expert pool | High |
| PCIe link | Moves selected experts into GPU cache | High |
| CPU bandwidth | Executes residual expert misses from host memory | High |
| NVMe storage | Loads model data during startup | Medium |
| CUDA environment | Supports GPU kernels and captured execution paths | High |
The reference system reports that hardware balance matters more than specification-sheet assumptions. A laptop with limited PCIe bandwidth may favor more CPU execution, while a desktop with stronger GPU transfer bandwidth may benefit from filling more cache slots.
Before You Install FreeToken
Before obtaining the runtime, prepare the machine and model assets. The published system uses a normalized expert layout called the FreeToken Weight (FTW) format. This format organizes expert banks around logical layer–expert identifiers so that the CPU executor and GPU cache can use the same mapping.
The available reference does not provide a universal conversion command or package manager instruction. Do not invent a command from another inference engine. Instead, use the current release documentation associated with the official FreeToken project release channel when it becomes available.
A command copied from llama.cpp, Ollama, or another serving engine may not create the required FTW layout or configure FreeToken’s cache and bandwidth policies correctly.
Use this preparation table to organize the deployment:
| Preparation area | What to confirm | Why it matters |
|---|---|---|
| GPU | CUDA-compatible architecture and sufficient free VRAM | Determines cache size and kernel compatibility |
| Memory | Enough capacity for the complete host expert pool | Prevents paging and unstable execution |
| Storage | NVMe capacity for the selected checkpoint and FTW files | Reduces model loading time |
| Drivers | Current compatible NVIDIA driver and CUDA stack | Enables GPU execution and graph support |
| Model | Supported MoE checkpoint and precision variant | Determines expert size and memory demand |
| Workload | Prompt length, agent turns, and expected decode rate | Influences KV cache and prefill pressure |
FreeToken’s design supports several important runtime behaviors:
- Full-layer double buffering overlaps expert transfer with GPU computation during prefill.
- Semantic-aware state caching preserves useful prefixes around thinking blocks, tool calls, and conversation turns.
- Shared LRU expert caching follows changing token-level routing during decode.
- Bandwidth-adaptive execution divides cache misses between PCIe transfer and direct CPU execution.
- Elastic cache rebuilding changes the GPU expert-cache budget at scheduler-safe points without restarting the engine.
For a first deployment, use a single model and a controlled workload. Avoid testing several large checkpoints simultaneously because the host memory and storage requirements can become difficult to distinguish from runtime issues.
| Deployment choice | Lower-risk starting point | Higher-demand option |
|---|---|---|
| Model size | Smaller supported MoE checkpoint | Frontier-scale MoE checkpoint |
| Precision | Official precision with documented support | Specialized quantized layout |
| Workload | Short single-session test | Multi-turn agent workload |
| Cache policy | Default measured configuration | Manually tuned memory split |
| Host execution | Minimal CPU miss handling | Aggressive hybrid CPU–GPU execution |
Step-by-Step FreeToken Setup
Follow these steps in order. The sequence separates environment preparation from model conversion and runtime tuning, which makes failures easier to diagnose.
Prepare the host first, verify the GPU stack second, stage the model third, and tune the serving cache only after a basic request succeeds.
Prepare the host environment
Close memory-heavy applications and confirm that the GPU, CPU, host memory, and NVMe storage are available for serving. Record total VRAM, system memory, GPU model, PCIe link width, and the driver or CUDA versions. FreeToken is designed for changing edge resources, but a clean baseline makes the first test more reliable.
Obtain the official runtime
Use the current installation instructions published with the FreeToken release at FlashML. Confirm the supported operating system, CUDA version, Python or native dependencies, and model compatibility before installing. The reference material identifies the release location but does not specify a fixed command sequence.
Stage a supported MoE checkpoint
Download the model files through an authorized distribution channel and verify that the checkpoint matches the runtime’s supported expert representation. Keep enough storage for the original checkpoint and any converted FTW files. Do not assume that a generic model file is already in FreeToken’s optimized layout.
Create or obtain the FTW layout
Follow the release documentation for converting or downloading FreeToken Weight files. FTW stores expert banks in the layout expected by the runtime, allowing aligned direct reads into host memory and avoiding repeated tensor discovery or repacking during startup.
Launch a controlled test
Start with one request and a short prompt. Confirm that the host expert pool loads, the GPU initializes, the first request completes, and routed experts can move between host memory and the GPU cache. Only then test long contexts, tool calls, or concurrent agent sessions.
The initial validation should focus on function rather than peak throughput. Confirm that the runtime can serve a request with a cold cache. FreeToken does not require a separate warmup phase by design; early requests can populate the cache through ordinary execution.
| Validation point | Expected observation | If it fails |
|---|---|---|
| Model discovery | The selected checkpoint or FTW assets are recognized | Recheck model format and release compatibility |
| Host loading | Expert data reaches the host-resident pool | Check memory capacity, file paths, and permissions |
| GPU startup | Non-expert layers and runtime state initialize | Check CUDA, driver, and VRAM availability |
| First request | Cold-cache inference completes | Inspect transfer, kernel, and CPU backend logs |
| Repeated decode | Recently used experts begin producing cache hits | Confirm cache allocation and routing support |
FreeToken Performance and Memory Tuning
FreeToken’s key tuning concept is the relationship between host-side bandwidth and PCIe transfer bandwidth. The runtime measures or profiles two values:
- Bₚ: pinned expert-transfer bandwidth over PCIe.
- Bₕ: effective host-side bandwidth available to CPU expert execution.
For a decode step with m missing experts, the system estimates the cache-fill count with the relationship:
q* ≈ m × Bₚ / Bₕ
This is not a user-facing promise or a fixed hardware rule. It is a scheduling model that balances concurrent GPU cache fills against CPU execution. The appropriate split changes across laptops, desktops, PCIe configurations, memory channels, and background load.
Use measured bandwidth from the deployed machine instead of relying only on advertised PCIe or memory specifications. FreeToken’s scheduling policy is intended to reflect the hardware actually running the model.
The GPU memory budget also needs planning. FreeToken divides available VRAM between the KV cache and complete expert slots. Longer contexts increase KV-cache demand, while a larger expert cache can reduce decode misses. Because the system can rebuild the expert cache at safe points, this balance does not need to remain fixed for the entire session.
Prefill
Prioritize transfer overlap and prefix reuse. Long prompts expose expert movement and recomputation costs.
Decode
Prioritize routing locality and a shared LRU cache. Recent expert usage is more useful than static placement.
Agent Workloads
Preserve semantic checkpoints around tool calls and edited context blocks.
Shared Desktop
Leave VRAM headroom for browsers, displays, games, and other applications.
The reference evaluation gives useful scale indicators without turning them into guaranteed results. On an RTX 5090 test system, FreeToken reported 77–83 tokens per second on Qwen3.6-35B and 22–25 tokens per second on DeepSeek-V4-Flash across the tested workloads. Results depend on model precision, cache capacity, host bandwidth, prompt length, and background applications.
| Tuning target | Practical action | Trade-off |
|---|---|---|
| Faster first token | Enable prepared model layout and transfer overlap | Requires correct staging and sufficient storage |
| Higher decode rate | Increase useful expert-cache capacity | Leaves less VRAM for KV cache |
| Longer context | Reserve more VRAM for KV pages | May increase expert cache misses |
| Better miss handling | Allow measured CPU–GPU cooperation | Uses host bandwidth and CPU resources |
| More stable desktop use | Reduce serving memory budget | Can lower cache-hit rates |
Installation Checklist and Troubleshooting
Use this checklist before moving from a basic test to a production-style local endpoint or agent workload.
When a test is slow, identify whether the bottleneck is storage loading, prefill transfer, decode misses, CPU bandwidth, or an undersized KV cache before changing settings.
Deployment Readiness:
- Confirm the GPU, driver, CUDA environment, and available VRAM
- Reserve enough host memory for the complete expert pool
- Use a supported MoE checkpoint and the documented FTW layout
- Run one cold-cache request before testing long agent sessions
- Record measured PCIe and host-side bandwidth for tuning
| Symptom | Likely area | First response |
|---|---|---|
| Startup takes too long | Storage or unprepared weight layout | Use the documented FTW path and fast local storage |
| GPU sits idle during prefill | Transfer and computation are serialized | Confirm that full-layer double buffering is enabled |
| Decode is CPU-bound | Too many misses or limited host bandwidth | Review cache size and measured bandwidth balance |
| Long sessions slow down | KV cache consumes the original expert budget | Rebalance VRAM at a safe runtime point |
| Runtime cannot serve a model | Unsupported representation or insufficient host memory | Check the release’s model and platform matrix |
| Desktop becomes unstable | Competing applications consume VRAM | Close background workloads or reduce serving allocation |
FreeToken’s fallback behavior is also important. If the complete expert pool cannot be pinned or registered for DMA because of operating-system or driver restrictions, the runtime can use a pure-CPU MoE backend. This improves deployability but trades away the faster transfer path. Non-expert layers can remain on the GPU while activation-sized inputs, routing metadata, and outputs cross the device boundary.
Do not judge installation quality from a single short prompt. Prefill and decode stress different parts of the system, and multi-turn agent sessions add repeated context processing. A useful test plan includes one short request, one long prompt, repeated decode, and a context-editing workload.
FreeToken Install FAQ
Q: Is FreeToken a game or a redemption-code platform?
No. FreeToken is an edge-native MoE model-serving system. Its installation concerns GPU inference, host memory, model files, CUDA support, and runtime configuration.
Q: Where should I get the FreeToken installer or release package?
The published project description announces the system at FlashML. Use the current official release documentation at https://flashml.ai/ for package names, dependencies, and commands because the reference material does not provide a fixed launcher command.
Q: Do all model weights need to fit in VRAM?
No. FreeToken keeps the complete routed-expert pool in host memory and uses GPU memory as an elastic expert cache. VRAM still needs to hold non-expert weights, runtime state, KV cache pages, and cached experts.
Q: Why does FreeToken need bandwidth profiling?
Decode misses can either be transferred to the GPU or executed directly on the CPU. FreeToken uses measured PCIe and host-side bandwidth to choose a practical split between those paths.
Installation details can change as the 2026 release evolves. Confirm the official package, supported model formats, and platform requirements before running conversion or serving commands.