Recently, Tsinghua-affiliated AI startup Sand.ai announced the open-source release of its latest video generation model, MAGI-2-preview. Boasting a total of 114B parameters, the model leverages an advanced MoE (Mixture of Experts) architecture to activate only about 6B parameters per forward pass. This milestone not only delivers stellar video quality but also slashes inference costs dramatically. Based on current 8x H100 market rates, generating a 10-second 1080P HD video costs a mere 0.5 RMB (approx. $0.07), roughly one-tenth the cost of mainstream dense models.
Ranked sixth on the AA video generation leaderboard, MAGI-2-preview achieves results close to first-tier closed-source models with minimal active compute. Scaling video models poses a massive financial challenge compared to discrete text tokens in LLMs. Video models must split every frame into high-dimensional spatial patches while tracking temporal movements, object relationships, and camera transitions across consecutive frames. Under dense architectures, the computational and communication overhead of processing such ultra-long sequences makes scaling practically prohibitive.
To overcome these bottlenecks, Sand.ai re-engineered the underlying system. The model retains a single-stream architecture (pioneered in daVinci-MagiHuman), feeding text, video, and audio into a unified Transformer network. This unified formulation models cross-modal associations from the very first layer, dropping end-to-end latency and easing #MoE scaling. Furthermore, the model implements a Multi-Head LatentMoE design, slicing the 3072-dimensional hidden state into 12 distinct 256-dimensional heads. Across its 36 layers, the architecture hosts 3,072 independent expert units per layer. Tokens dynamically select 6 experts per head, activating a total of 72 mini-experts, offering far more granular routing than models like DeepSeek-V4-Pro.
To support such granular experts, Sand.ai developed its proprietary MagiMoE kernel library to fuse routing, sorting, and expert computation, mitigating VRAM transfer overhead. They also introduced Head Parallel routing, keeping communication costs constant regardless of active expert numbers. For optimization, they coupled Muon for weight matrices with AdamW for expert parameters. In data engineering, Sand.ai focused on 'data organization' rather than heavy filtering, training the model to map diverse relationships during pre-training before tackling preference alignment and safety in post-training.
[AgentUpdate Depth Analysis] The open-source release of MAGI-2-preview marks a pivotal shift toward cost-effective, high-throughput #multimodal intelligence. Historically, the high computational barrier of video generation has crippled the scalable deployment of autonomous AI Agents, especially in embodied intelligence and spatial reasoning. By open-sourcing a 114B MoE model with a 90% cost reduction, Sand.ai democratizes the capability for AI Agents to construct real-time 'world models' of their environments. Compared to closed-source dense paradigms, this ultra-low-latency, granular MoE architecture allows Agents to iteratively predict, simulate, and generate multi-sensory feedback at a fraction of the cost. This leap accelerates the transition of AI Agents from static text-based assistants to dynamic, physically-grounded decision-making systems capable of seamless spatial and audio-visual reasoning.
