One-Line Summary

Systems papers show that how you schedule generations and allocate resources can outweigh raw sample count for speed, accuracy, and energy, while new RL training tricks shape agent behavior and a Mac-native server simplifies local LLM trials.

Research Papers

Test-time scaling's hidden variable: how you batch candidates

If you ask an AI to produce multiple answers and then pick or combine them, running all candidates at once is very different from running them one by one. A study of Large Language Models (LLMs) finds that increasing the number of samples from 1 to 8 improves accuracy by 8.4 points for Phi-3-mini and 18.4 points for Qwen2.5-1.5B on 500 GSM8K problems — but at a fixed eight-candidate budget, eight serial calls can consume 4.64–4.86× more gross graphics processing unit (GPU) device energy and show 5.77–6.12× higher 95th‑percentile (P95) latency than one batched call of eight candidates on A100 GPUs. ¹

The authors compare four schedules — 1×8, 2×4, 4×2, 8×1 — and measure latency, throughput, GPU‑hours, and gross GPU energy while keeping the total samples constant. The same pattern holds across three independently scheduled A100 nodes per model and on short‑output SciQ experiments with V100 GPUs. Their recommendation: report not just sample count and accuracy, but also the generation schedule and GPU‑level system metrics. ¹

A companion paper analyzes LLM‑based AI agents that mix remote model calls with local tools across tasks like Retrieval‑Augmented Generation (RAG) question answering, web search, and coding. It shows that task mix and concurrency expose bottlenecks in central processing unit (CPU), disk I/O, and memory; faster model responses or more CPU cores do not always speed up agents. Two simple schedulers — CPU‑aware tool admission and task‑aware CPU allocation — improve latency by about 5.4× on CPU‑sensitive tasks and reduce average latency by about 32% compared to native agents. ²

Taken together, these results show that “N samples” is not a complete budget: batching policy and resource scheduling materially change cost and responsiveness. Evaluations benefit from logging schedule plus system metrics such as P95 latency and GPU energy alongside accuracy. ²

How observation supervision and dual-axis RL change agent behavior

Training an agent only on its action tokens can teach it to act, but not to anticipate what the environment will show next. ActObs adds supervision on observation tokens already present in trajectories so the policy learns consequences without extra data or forward passes. After similar Supervised Fine‑Tuning (SFT), Group Relative Policy Optimization (GRPO) from ActObs yields higher pass@k on Qwen3‑4B at every sampling budget; on Qwen3‑8B it trades some pass@1 for higher pass@k (+3.4 points at pass@16) and solves more distinct tasks. It also lifts cross‑domain code editing (+4.2 points pass@1 at 4B on aider‑polyglot) and retains more entropy during Reinforcement Learning (RL), moving the policy less from its SFT start. ³

Another team frames RL for LLM agents along two axes: how to attribute feedback within a trajectory and how to aggregate gradients across trajectories. Their BATON method pairs Bayesian Feedback Attribution with Trajectory Mass Normalization, providing independent gains that combine to the strongest overall results across scales on ALFWorld, WebShop, and SearchQA with GRPO‑based training. ⁴

For tool‑use, MATCH closes the loop between curriculum and rewards. Model‑Aware Curriculum Learning keeps difficulty near the model’s evolving capability boundary, while Hierarchically Gated Rewards score tool name, argument keys, and values only when prerequisites hold. On API‑Bank and BFCL V3, MATCH reaches 72.19% and 62.87% overall accuracy, outperforming supervised and RL baselines, with consistent gains across four backbones from two model families. ⁵

Obstacle-aware harness improves safety for robot coding agents

“Coding agents” write the robot’s controller as a program, but when the task includes an obstacle not to touch, baseline agents mostly collide — they reason about the obstacle yet fail to prioritize it in planning. SafeHarness adds two obstacle‑aware harnesses: route planning over grounded bounding boxes with verification and replanning, and contact execution that selects a safe contact position. The result is 71.9% task success and 87.5% collision avoidance, surpassing the previous best by 6.5 and 27.0 points, and reaching 2.3× and 1.5× the scores of the same agent without harnesses. ⁶

The findings point to a simple rule: make safety constraints first‑class in both the path to the goal and at the moment of contact. Explicit obstacle‑aware route and contact planning substantially reduce collisions without robot‑specific pretraining. ⁶

Score centering stabilizes RL under training–inference mismatch

RL for language models often trains with one engine and deploys with another, leading to a training–inference mismatch that can destabilize learning. This work traces the instability to drift — a persistent bias between training and inference engines — and derives an additive “score centering” correction that cancels drift. Across models from 0.6B to 30B parameters, score centering alone matches or outperforms importance‑sampling methods under quantization, with the advantage growing as the mismatch worsens; composing it with importance sampling further outperforms pure importance‑sampling baselines under staleness. ⁷

Practically, you do not need perfect parity between the training and serving stacks to keep RL stable. A lightweight additive correction can deliver robustness with less rollout overhead. ⁷

Open Source & Repos

mlx-serve: a native LLM server for Apple Silicon

mlx‑serve runs Large Language Models (LLMs) locally on Apple Silicon with OpenAI‑ and Anthropic‑compatible Application Programming Interface (API) endpoints, no Python or cloud required. It ships a macOS app (MLX Core) with chat, agent mode, and tool calling. ⁸

The project claims faster throughput than LM Studio on identical MLX weights. A pre‑release (v26.9.5‑pre‑release.1, Sep 18, 2026) highlights support for a 2‑bit Qwen3.8‑27B variant (Ternary‑Bonsai‑2‑27B‑mlx‑2bit, text+vision) that loads and serves like a standard Qwen 27B. It targets developers who want drop‑in local endpoints compatible with existing client code. ⁸

Why It Matters

As teams scale reasoning by sampling more at inference and by building agents, today’s results show that system choices — batching schedule, CPU budget, and reward design — can decide both accuracy and energy. Expect scorecards and benchmarks to feature P95 latency and energy alongside accuracy, and tools like mlx‑serve to lower the friction of measuring these trade‑offs locally. ¹

This Week’s Hands-on

1. Install mlx‑serve on a Mac: download the latest pre‑release and start a local OpenAI‑compatible endpoint. https://github.com/ddalcu/mlx-serve ⁸
2. Compare “1×8 vs 8×1” in your pipeline: batch eight candidates in one call and then as eight serial calls, and log P95 latency and device energy as in the paper’s setup. https://arxiv.org/abs/2609.19499 ¹