Million-token reinforcement learning fits on fixed GPUs — LongStraw shows a path
LongStraw demonstrates an execution stack that runs reinforcement learning post-training on 2.1M-token prompts within a fixed GPU budget, validating a 2.1M path across 78 layers on 32 H20 GPUs and stressing to 4.46M. Two papers also debut: a diffusion-based language model that lets tokens ‘appear’ at different speeds, and a multimodal scene representation that binds what and where.
One-Line Summary
LongStraw shows a way to train on million-token histories under fixed compute, while new work speeds up text generation with continuous diffusion and ties together what and where across vision–audio–language.
Research Papers
LongStraw demonstrates million-token reinforcement learning post-training on fixed GPUs
LongStraw is an execution stack that lets reinforcement learning (RL) post-training operate on million-token prompts while staying within a fixed graphics processing unit (GPU) budget. It evaluates the shared prompt without autograd, keeps only the model-specific state needed by later tokens, and replays short response branches one at a time; the authors instantiate it with Group Relative Policy Optimization (GRPO). 1
On eight H20 GPUs, LongStraw completes grouped Qwen scoring and response backward at 2.1 million positions for group sizes of 2 and 8; increasing the group size adds only 0.21 GB of peak allocated memory. A separate stress test reaches 4.46 million positions, indicating headroom in the execution approach. 1
On 32 H20 GPUs, the team validates an end-to-end execution path for a 2.1M‑token prompt across all 78 layers of GLM‑5.2, a compressed‑attention mixture‑of‑experts (MoE) model; they also implement LongStraw for the hybrid recurrent + full‑attention Qwen3.6‑27B. The paper emphasizes these experiments establish execution capacity rather than full training correctness, because the captured prompt state is detached and some distributed forward/gradient composition paths remain incomplete. 1
Why it matters: inference context windows are approaching a million tokens, but RL post‑training often stops at 256K and relies on length generalization. LongStraw narrows that gap for agents that accumulate long trajectories, while raising practical questions about throughput trade‑offs from replay and how learning quality holds up at these scales. 1
Diffusion language model that lets tokens appear at different speeds
Token Time Continuous Diffusion (TTCD) generates text by mapping Gaussian noise to a final token canvas in continuous space, with a per‑token “time” so some tokens resolve faster than others. This avoids the inaccuracies that come from parallel sampling in discrete space when pushing for high speedups. 2
A 160M‑parameter TTCD model trained on OpenWebText and then self‑distilled shows that at high speedups it is comparable in unconditional generation and outperforms in conditional generation versus similarly sized models trained on the same data and self‑distilled; it shows similar gains on Sudoku solving. The result points to a path for faster decoding without the quality drop‑offs common in aggressive parallel decoding. 2
SceneBind unifies what and where across vision, audio, and text
SceneBind is an omni‑modal scene representation that carries both semantics (what is present) and 3D spatial structure (where it is). Each scene combines a global semantic embedding with object‑centric semantic‑spatial slots, and SceneBind Matching fuses global scene similarity with object alignment for cross‑modal retrieval and object grounding. 3
The authors curate a real‑world binaural audio‑visual dataset with structured semantic and spatial annotations and propose a training protocol to align signals across modalities. SceneBind plugs into large pretrained semantic encoders, adds only a few extra tokens for spatial modeling, achieves state‑of‑the‑art scene and spatial retrieval, and transfers zero‑shot to tasks like audio‑visual localization. 3
Why It Matters
The three papers target core bottlenecks: compute‑efficient long‑context RL training (LongStraw), faster and more stable high‑speed decoding (TTCD), and multimodal perception that understands spatial layout (SceneBind). For product teams, that maps to agents learning from full histories, text generation that keeps quality at speed, and assistants that can ground answers in where events occur. 1
What to watch: independent replications and end‑to‑end training correctness for LongStraw’s path; latency‑versus‑quality curves for TTCD’s continuous decoding; and SceneBind‑style retrieval/localization accuracy in cluttered, real‑world scenes with noisy audio. 2
Comments (0)