Meta releases 30B local agent model under Apache 2.0
Muse Glimmer’s open weights target on‑device agents that plan, call tools, and check their own work, fitting on a single 24–32GB GPU with 4‑bit quantization and speculative decoding. NVIDIA details how to run it locally, while new papers tackle long‑video evaluation and cheaper long‑context retrieval.
One-Line Summary
Meta brings local AI agents to high-end PCs with a 30B open-weight model, while NVIDIA shows how to run it locally and new papers improve long-video evaluation and long-context retrieval efficiency.
LLM & SOTA Models
Meta releases Muse Glimmer for on-device agents under Apache 2.0
Meta releases Muse Glimmer, a 30-billion-parameter model you can run on a Mac or PC with a single high-end graphics processing unit (GPU). It is built for agentic work—planning, calling tools, checking results, and recovering from failures—and ships with open weights under the permissive Apache 2.0 license. The model accepts interleaved text and images via a perception encoder and is designed to work with common agent scaffolds. 1
To fit on consumer hardware, Meta provides roughly 4-bit quantized builds that keep the language model under 20GB of video RAM (VRAM), leaving room for the key-value (KV) cache, perception encoder, and a DFlash speculative-decoding “drafter” inside a 24GB or 32GB envelope. Meta reports speedups from DFlash of 3.1× on an RTX 5090 (74.9 to 233.4 tokens/sec), 1.8× on an Apple M5 Max (26.6 to 50.2), and 1.5× on an M4 Max (23.7 to 37.8), with identical output quality to standard decoding. 2
Glimmer is trained around the agent loop and distilled from the larger Muse Spark through logit distillation, mid-training on longer-context, agent-heavy data, and post-training that mixes supervised fine-tuning and reinforcement signals. In parallel, Meta makes Muse Spark 1.1 available to external developers through a new application programming interface (API) and reports that, after mitigations, Spark 1.1 operates within “moderate or lower” risk across catastrophic-risk domains under the Advanced AI Scaling Framework. 3
What to watch: real-world throughput and toolchain maturity. NVIDIA’s technical note highlights local deployment paths and claims over 20K tokens per second per GPU on Blackwell Ultra, while community reports are probing performance trade-offs as software support rolls out. 4
Run local agentic AI workflows with Muse Glimmer on NVIDIA
NVIDIA outlines how Muse Glimmer runs fully on-device across its platforms—from GeForce RTX 5090 desktops to DGX Station and Jetson—touting reliability for long, multi-step agents and privacy by design since data never leaves the machine. The post points to a 120K+ token context window and dense-model predictability for sustained throughput. 4
For deployment, NVIDIA offers multiple paths: SGLang or vLLM for open-source serving, downloadable NIM containers for a one-command setup, and NeMo tooling for supervised fine-tuning (SFT) or low-rank adaptation (LoRA). The company reports over 20K tokens/sec/GPU on Blackwell Ultra and positions Glimmer for governed, on-prem agent pipelines. 4
Research Papers
CLIP-CC-Bench: evaluating paragraph-length video descriptions
This paper introduces CLIP-CC-Bench, a benchmark that tests whether video-language systems can write accurate paragraph-level descriptions for longer clips (about 90 seconds) rather than just single sentences. The dataset covers five hours of movie content and pairs each clip with an expert paragraph reference. 5
The evaluation uses an ensemble of five large language model (LLM)-based embedding models to reduce single-model bias, and scores outputs with both coarse- and fine-grained semantic matching. The authors evaluate 17 state-of-the-art models and publish standardized scripts and aggregation tools, reporting inter-judge agreement and bootstrap stability to quantify reliability. 5
CoinRAG: nugget-level KV reuse for faster long-context retrieval
CoinRAG improves Retrieval-Augmented Generation (RAG) for long contexts by reusing the key-value (KV) cache at a fine-grained “information nugget” level instead of entire chunks—cutting redundant prefill while keeping relevant detail. It selects query-relevant nuggets via two-stage retrieval, then assembles their sliced KV representations with minimal overhead. 6
On LongBench multi-hop QA tasks, CoinRAG sets a new Pareto frontier under a fast prefill latency budget and yields an average 5.3% relative F1 improvement in answer quality over baselines. The result: better accuracy at the same latency, or similar accuracy at lower cost—particularly useful when context windows grow. 6
Community Pulse
Hacker News (955↑) — Mixed views balance Glimmer’s concise, token-efficient behavior against variable throughput and software maturity. 7
"In my experience I am getting 23-24 t/s output with dflash off, and it craters to ~9 t/s with it on, miss rate exceeding 50%. And I'm using the same device as stated on their model page/card. We might need to wait for the software to catch up" — Hacker News 7
"Having spent a good part of the day with it, glimmer reminds me of Rorschach from The Watchmen. No unessential parts of speech, action oriented, brief and to the point. From a token perspective anyway it’s great, and it seems to hold its own well against more verbose models. I really do feel like it’s effective tok / s is way higher because it doesn’t waste them." — Hacker News 7
Why It Matters
Local-first agent workflows are moving from a demo to something teams can actually run, with open weights, permissive licensing, and concrete deployment recipes that reduce data exposure and per-token fees for sensitive, tool-using tasks. 1
Stronger evaluation (CLIP-CC-Bench) and efficiency techniques (CoinRAG) give builders clearer yardsticks and cheaper long-context options—useful complements to local agents that must read long documents, reason over time, and remain responsive. 6
Comments (0)