Open weight models are now powerful enough to execute complex multi-step agentic workflows, advanced reasoning, and long-horizon coding. However, hosting multi-trillion parameter architectures demands specialized infrastructure, high-end GPUs, and highly optimized serving frameworks. On July 27, 2026, Moonshot AI released Kimi K3, a 2.8 trillion parameter Mixture of Experts (MoE) model. It represents the first open-weight system to breach the 3-trillion parameter class. To help organizations run this frontier-level model on their own infrastructure, this post outlines deploying #Kimi K3 on AWS using Amazon #SageMaker HyperPod and Amazon EKS.
Kimi K3 features a highly optimized architecture powered by Kimi Delta Attention (KDA), Gated Multi-Head Latent Attention (MLA), and a Stable LatentMoE framework. It distributes its 2.8T parameters across 896 specialist experts, activating only 16 experts per token. This translates to roughly 104 billion active parameters during any single forward pass, delivering a 2.5x scaling efficiency improvement over Kimi K2. Additionally, Kimi K3 boasts a 1-million token context window, native multimodal support, structured outputs, and an always-on thinking mode.
The open weights for Kimi K3 are available on Hugging Face under moonshotai/Kimi-K3 in MXFP4 (Microscaling Floating Point 4-bit) format, providing an optimized balance between model quality and GPU memory efficiency. To serve a model of this scale, developers can leverage specialized vLLM containers, specifically using the dedicated branch #vllm/vllm-openai:kimi-k3. vLLM provides native support for MoE architectures, tensor parallelism, and MXFP4 quantization, simplifying large-scale production deployments.
[AgentUpdate Depth Analysis] The open-weight availability of Moonshot AI's Kimi K3, coupled with enterprise-grade deployment on AWS, signals a massive shift in the AI Agent landscape. Traditionally, orchestrating complex, multi-step agentic workflows required API-dependent frontier models, raising serious concerns over data privacy, latency, and cost. By open-sourcing a 2.8T parameter MoE model that runs efficiently via vLLM and MXFP4, Moonshot AI has democratized super-computing-grade intelligence. Kimi K3's ultra-long 1M context and MLA architecture are specifically tailor-made for hosting stateful, multi-agent coordination environments. This deployment paradigm on AWS EKS and SageMaker HyperPod empowers enterprises to self-host bespoke, highly secure, and continuously 'thinking' AI agents, marking a decisive step from simple prompt-response interactions to fully autonomous enterprise workflows.
