One-Line Summary

Smarter data mixing and memory planning—not just bigger models—drive multilingual reasoning, reveal Mixture‑of‑Experts overfitting under repeated data, and lift long‑horizon robotics, alongside an agent repo focused on traceable actions.

Research Papers

Data mixing lifts in‑language reasoning to 93% across 60 languages

This paper shows a model can answer in the same language you ask it—so‑called in‑language reasoning—by carefully mixing training data, rather than translating everything into English internally. The team builds the 3.35B‑parameter Tiny Aya L2‑Thinker and reports an in‑language reasoning rate above 93% across 60 languages on six benchmarks spanning math, commonsense, instruction following, open‑ended generation, and cultural reasoning. ¹

The method is data‑centric: during Supervised Fine‑Tuning (SFT), they optimize composition and scheduling to pair broad multilingual, non‑reasoning text with a strong English reasoning backbone. The result generalizes to held‑out languages, suggesting reasoning is a language‑agnostic behavior that can transfer without per‑language supervision in a Large Language Model (LLM). ¹

They release model weights and multilingual reasoning data, positioning in‑language reasoning as an accessibility feature rather than a niche benchmark goal. For practitioners, the watch‑item is whether this data mixing holds up on domain‑specific tasks and low‑resource scripts. ¹

Negative self‑distillation steers away from bad reasoning

Instead of copying a perfect‑looking chain of thought, this method teaches a model to notice and avoid its own flawed reasoning. The authors argue On‑Policy Self‑Distillation (OPSD) can degrade complex reasoning by forcing students to imitate traces conditioned on privileged information; their Negative Self‑Distillation (NSD) generates a question‑specific “careless reasoner” as a negative condition and pushes the student distribution away from it. ²

A naive “unlearn the bad tokens” approach harms basic language ability, so NSD adds a dynamic gating mechanism that targets only reasoning‑critical tokens. It requires no external answers and consistently outperforms OPSD and other label‑free, self‑bootstrapping Reinforcement Learning (RL) baselines on reasoning tasks. ²

MoE models overfit sooner when data is repeated

When training data must be repeated, sparse Mixture‑of‑Experts (MoE) models degrade faster than dense Transformers. Across models with 80M–1B active (8.5B total) parameters, the authors find 80M dense models can repeat data over 8× with minimal degradation, while MoEs begin to suffer at 4× and can underperform dense models after 32×; the effect grows with sparsity and tracks total, not active, parameters. ³

Regularization helps but doesn’t erase the gap: dropout mitigates overfitting, and with strong masking‑based regularization MoEs can outperform dense models even when data is repeated beyond 64×—yet none of the methods match all‑unique data. The practical read: if you rely on repetition, plan stronger regularization and more diverse mixes for MoE. ⁴

Mechanistically, MoE routing stabilizes early and expert specialization correlates with memorization under high repetition, pointing to over‑concentration as a failure mode to watch. Teams should monitor gate usage and expert load balance during training. ³

Memory‑as‑plans boosts long‑horizon robot manipulation

Both papers separate “remembering” from “doing,” showing robots perform better when memory stays with the agent/planner and the motion policy just executes compact plans. The 2AM study makes a multimodal Agent the sole holder of task memory and a single red‑green‑blue (RGB) camera‑based, episodically stateless Action Model the executor; on LIBERO‑Mem, it reaches 76.3% average completion—61.5 points above a 14.8% baseline—plus 63.0% relaxed and 11.8% strict success, without depth, online geometry, or planner‑based object motion. It trains a Vision‑Language‑Action (VLA) policy to follow structured hints and to tolerate noise. ⁵

MaP‑WAM reframes “memory as plans”: it converts episodic context into segment‑level language plans plus visual guidance, then a World‑Action‑Progress (WAP) model executes each plan while predicting progress for adaptive transitions. Structured attention keeps the executor context fixed and enables key‑value (KV) caching; results include 83.3% on RMBench and 78.0% on real‑robot tasks, with approximately constant executor latency as history grows. ⁶

The common pattern is clear: compress long histories into steerable, segment‑level intentions, then keep the executor lean. Watch whether these interfaces generalize beyond the reported benchmarks and into messier, multi‑object scenes. ⁶

Open Source & Repos

OpenBot gives each AI coworker its own computer

OpenBot is an MIT‑licensed, alpha‑stage toolkit where each AI “coworker” runs on an isolated machine with its own real browser, logins, files, and only the tools you grant. Every action is decided before execution and recorded after, creating a review trail; you can bring your own agent. ⁷

Release v0.0.9 (Sep 11, 2026) includes a fix to keep a person’s last sign‑in visible on the People screen, alongside ongoing security checks. It’s a fit for teams that need agents to operate with auditable, pre‑approved steps. ⁷

Why It Matters

Today’s results reinforce a shift from “bigger model” thinking to “better scaffolding”: thoughtful data composition can produce strong in‑language reasoning even in small models. If your product must respond in users’ languages, investing in multilingual coverage plus an English reasoning backbone may pay off. ¹

For builders facing data scarcity or compliance limits, MoE training under repeated data needs heavier regularization and monitoring, while long‑running agents and robots benefit from keeping memory with the planner and logging actions end‑to‑end—patterns that tools like OpenBot make easier to operationalize. ⁷

Things to Try This Week

1. OpenBot quick start: spin up an isolated “AI coworker” and inspect its action log. https://github.com/CopilotKit/OpenBot
2. Skim the L2‑Thinker abstract: note how data mixing and SFT are arranged for in‑language reasoning. https://arxiv.org/abs/2609.10445