DeepSeek V4.1 Flash on an RTX 3090: What 64 GB of RAM Gets You

The author of Delta Inference Engine has released version 0.93 and announced it on r/LocalLLaMA: DeepSeek-V4.1-Flash-1017 on a single 24 GB RTX 3090 now runs on PCs with 64, 96 and 128 GB of RAM, not only with 192 GB. The engine also fully supports the new quantized versions of the model from Unsloth.

Generation speed in tokens per second, one chat at a time, greedy decoding. For the 2-bit UD-IQ2_XXS (301 GB): 41 at 192 GB, 34.0 at 128 GB, 29.3 at 96 GB and 21.0 at 64 GB. For the 1-bit UD-IQ1_S (233 GB): 41.0, 40.6, 36.5 and 30.6. For the 2-bit UD-Q2_K_XL (335 GB): 31.3, 26.6, 22.9 and 15.2. The rig: an i5-13600K, four 48 GB sticks of DDR5-5600, a PCIe 4.0 NVMe drive, Windows with no WSL. The 128, 96 and 64 GB rows come from the same PC with the rest of the RAM locked away.

Speed at 192 GB stayed at 41 tokens per second, but quality went up. On the author’s own 99-question test, UD-IQ2_XXS scored 57 correct answers, the engine’s previous default model scored 52, and the full uncompressed model scored 59. A new native reader for the Engram tables and CPU kernel work helped with speed. The first token of a reply now appears about 40% sooner. The release also adds EXL3 support, basic multi-GPU use and public CI for the project.

Running huge models on ordinary hardware has become a theme of the past few days. On November 3, KRAKOBIT wrote about colibri, which reads experts from disk and about the FreeToken engine running 290B+ models on a gaming PC. And on November 5, DeepSeek took 53% of OpenRouter tokens in one day.