#LLM Architecture

3 articles with this tag

UniPool: Rethinking MoE Efficiency
AI Research

UniPool: Rethinking MoE Efficiency

The UniPool MoE architecture redefines expert capacity, pooling resources globally and enabling sub-linear parameter growth for enhanced efficiency and performance.

about 1 month ago
LLMs Gain Persistent, Verifiable Memory
AI Research

LLMs Gain Persistent, Verifiable Memory

New hybrid LLM architecture augments parametric knowledge with structured ontological memory for persistent, verifiable, and enhanced reasoning.

about 2 months ago
MoDA: Unlocking LLM Depth Scaling
AI Research

MoDA: Unlocking LLM Depth Scaling

Mixture-of-Depths Attention (MoDA) tackles LLM signal degradation by enabling cross-layer attention, boosting performance with minimal overhead.

3 months ago