Can Qwen3.5 be used as a productivity tool? Local deployment of 27B and 35B-A3B ultra-low precision Q2_K quantization, reverse prompt testing.
10:33
24.8 tokens per second, running Qwen3.5-35B-A3B on a 3060-level 12GB GPU with Q4_K_M quantized GGUF version, disabling think mode, 4x speedup after compiling llama.cpp.
11:30
12GB VRAM at 30 tokens/sec, detailed method to compile llama.cpp running Qwen3.5-35B-A3B-UD-IQ2_M quantization.
14:30
Only 483MB. Is Qwen3.5's 0.8B small model actually useful?
05:57
llama.cpp local deployment Qwen3.6-35B-A3B, 12GB VRAM 20 tokens per second.
03:32
Qwen3.8-27B disrecommend, meaningless for average users.
07:05
Qwen3.8 27B new quantization, available with 12G low VRAM, Bonsai 2 27B, end-to-end ternary language weights.
03:51
New technique for saving VRAM in large language models, locally compiled Strata adapted for modified cards to run Qwen3.8-Flash-Next-GSQ-RCO-Coder, maxing out memory and CPU.
