#MoE
6 articles with this tag
Most read in #MoE, last 30 days
DeepSeek V4.1 Flash is cheap, fast and visual
DeepSeek V4.1 Flash 1017 brings native vision and CED architecture with 8B/16B active parameters and 1M context at $0.10/$0.50.
Cursor Open-Sources MoE Megakernel
Cursor open-sources Mixture-of-Kittens, a specialized MoE training kernel for NVL72 systems that fuses communication and computation for significant speedups.
SkewAdam: Rethinking MoE Optimizer Memory
SkewAdam drastically cuts MoE training memory by tailoring optimizer state to parameter populations, achieving superior perplexity and enabling training on accessible hardware.
UniPool: Rethinking MoE Efficiency
The UniPool MoE architecture redefines expert capacity, pooling resources globally and enabling sub-linear parameter growth for enhanced efficiency and performance.
DeepSeek V4 Pro Hits Together AI
Together AI launches DeepSeek V4 Pro, a 1.6T MoE model with a 512K context window and new cached input pricing for cost-effective long-context reasoning.
Cursor's Warp Decode Boosts MoE Inference
Cursor's new 'warp decode' technique dramatically boosts MoE model inference speed and accuracy on Blackwell GPUs by rethinking parallelism.
