China’s Kimi K3 puts a 2.8T open‑weight model on the table — and the cost math shifts
Moonshot AI’s K3 pairs a 1M‑token context with quantization that cuts weights to about 1.4TB, and cached input pricing down to $0.30 per million tokens. Research also advances longer robot memory and agents that track evidence.
One-Line Summary
Frontier-scale, open-weight AI arrives with Kimi K3 as research pushes longer memories for robots and more reliable, stateful web-search agents.
LLM & SOTA Models
Kimi K3 puts a 2.8T open-weight model with 1M context into developers' hands
Moonshot AI released Kimi K3, a large language model (LLM) for text and vision that developers can access now via API, with full public weights scheduled for release on Jul 27; it packs 2.8 trillion parameters, the first open-weight model in the 3-trillion class, and supports a 1,000,000‑token context window. The company frames K3 as frontier‑range while noting it still trails the very top proprietary systems. 1
Under the hood, K3 uses a Mixture of Experts (MoE) design with 896 experts and activates 16 per token, plus two new components — Kimi Delta Attention and Attention Residuals — to move information efficiently over long sequences. It trains with quantization‑aware training (QAT), storing weights in Microscaling FP4 (MXFP4) and using FP8 activations; in practice, the full 2.8T model needs about 1.4TB for weights (versus roughly 5.6TB in FP16) and targets modern graphics processing units (GPUs) like Nvidia Blackwell and AMD MI400 for deployment. 1
On vendor‑reported benchmarks, K3 scores 67.5 on DeepSWE, 88.3 on Terminal‑Bench 2.1, and 42.0 on SWE Marathon (best among the compared models), plus 91.2 on BrowseComp and 30.8 on Automation Bench; vision tests include 81.6 on MMMU‑Pro and 94.3 on MathVision. Pricing lists cached input at $0.30 per million tokens, cache‑miss input at $3.00/MTok, and output at $15.00/MTok, with Moonshot reporting 90%+ cache hit rates on coding workloads; the lab explicitly says K3 still trails Claude Fable 5 and GPT‑5.6 Sol overall. 2
Beyond Moonshot’s own numbers, Reuters reports K3 as the world’s largest open‑weight AI system at 2.8T parameters and cites independent checks placing it near the frontier: Arena.ai ranks it first on a web interface‑building benchmark and Vals AI places it second overall behind Fable 5 and ahead of GPT‑5.6. What to watch next is whether independent teams can reproduce these results once the weights drop. 3
Open Source & Repos
agmsg connects command-line coding agents across vendors
agmsg is a tiny bus that lets command‑line interface (CLI) coding agents from different vendors pass messages to each other on one machine, using Bash scripts and a local SQLite store — no daemon, no framework. It ships a simple llms.txt registry so agents can discover peers. 4
It’s aimed at developers who want Claude Code, Codex, Gemini, and Copilot to coordinate lightweight tasks without standing up new services. A prerelease build dated 2026‑07‑18 (diag‑383‑01) targets a Windows cursor‑blink issue in Codex CLI output, signaling active iteration on real‑world terminal quirks. 4
Research Papers
Local glimpses help visual models generalize over longer tasks
This paper asks whether reading an image through small, sequential “glimpses” can help models handle longer, more complex visual reasoning. The authors find that global, one‑shot input encourages shortcut learning and hurts length generalization on tasks that require aggregating local information. 5
They show that strictly local, recurrent perception policies mitigate these failures, restoring length generalization on visual state‑tracking tasks. The result argues that local attention is an essential, overlooked ingredient for robust compositional generalization in vision — echoing length‑generalization findings in language models. 5
RoboTTT scales robot memory to 8K steps with test-time training
RoboTTT is a robot policy that updates “fast‑weight” parameters during inference — a technique called Test‑Time Training (TTT) — to condition on up to 8,000 timesteps of visuomotor history without increasing latency. This extends context by roughly three orders of magnitude beyond many prior policies. 6
On real‑robot manipulation, RoboTTT improves overall performance by 87% over a single‑step baseline and is the first to fully complete a five‑minute, ten‑stage assembly task; pretraining with 8K context outperforms 1K by 62%, highlighting context length as a new scaling axis for robot foundation models. 6
SearchOS‑V1 turns web‑search agents into a coordinated team
SearchOS‑V1 turns free‑form web browsing by tool‑using large language models (LLMs) into structured teamwork by keeping an explicit, shared state of what’s known, unknown, and already tried. It formulates open‑domain information seeking as filling a relational schema with grounded citations. 7
A system called Search‑Oriented Context Management (SOCM) maintains a Frontier Task, Evidence Graph, Coverage Map, and Failure Memory while pipeline‑parallel scheduling overlaps sub‑agents and avoids loops. On the WideSearch and GISA benchmarks, SearchOS leads all reported single‑ and multi‑agent baselines. 7
Why It Matters
K3 puts frontier‑range capability into an open‑weight format with quantization that fits on multi‑node clusters, changing who can self‑host, customize, and control high‑end AI — and at what cost. 1
At the same time, labs are pushing longer, explicit memory and coordination: robots that learn during execution and search agents that track evidence and failures. These shifts point to AI that can sustain longer workflows more reliably, not just answer single prompts. 6
This Week to Try
- Read Kimi K3’s model overview: See how MXFP4 and the 1M‑token context work in practice. https://huggingface.co/blog/ResterChed/kimi-k3-model-overview-mxfp4-quantization-open-wei
- Install agmsg: Wire up your CLI coding agents to message each other locally. https://github.com/fujibee/agmsg
Comments (0)