One-Line Summary

Architectural tweaks, not just bigger models, set today’s pace: a three-phase residual stream boosts Transformer accuracy with near-2x faster convergence, while new RL, multimodal tuning, and system-level optimizers deliver practical gains.

Research Papers

Three-Phase Transformer (3PT)

This work restructures a Transformer’s internal highway (the residual stream) into three rotating channels so the model stays organized as it learns, leading to better predictions without adding size. The authors partition the hidden vector into N equal cyclic channels, apply per-channel normalization, and insert a small rotation each block; they also add a fixed “DC” side-channel for absolute position that composes cleanly with relative position (RoPE). At 123M parameters on WikiText-103, 3PT reduces perplexity by 7.20% (−2.62% bits-per-byte) over a matched RoPE-only baseline with just +1,536 parameters (0.00124%) and shows a 1.93x step-count convergence speedup (1.64x wall-clock). ¹

The design acts like a self-balancing system: per-phase normalization stabilizes each channel, a per-block 2D Givens rotation mixes attention and feed-forward effects in phase, and a Gabriel’s horn profile $r(p)=1/(p+1)$ is injected into a DC subspace for absolute positions. The approach composes orthogonally with RoPE, attention, and FFN, and the choice of N behaves like a parameter-sharing knob rather than a single best value—N=1 and N=3 are statistically similar at 123M across seeds. The authors also report geometric stability without hard constraints and a U-shaped drift profile of rotation angles across 12 layers. ¹

Why it matters: it’s a microscopic change with macroscopic impact—an accuracy and speed win almost “for free.” In parallel, alternative routing-first architectures also report big reasoning jumps: a Hierarchical Winner-Take-All (HWTA) circuit with 272K parameters beats a matched tree transformer by +44.0 points on CLUTRR k=10 under the same loop, and reaches 100% on several compositional tasks at tiny scales, suggesting the field is exploring beyond softmax attention. ²

Reinforcement Learning via Value Gradient Flow

This paper introduces a new way to fine-tune policies that starts from your reference behavior (a dataset in offline RL or a base LLM) and “flows” it toward a higher-value policy, like pushing many small particles along a hill using the value’s slope. The method, Value Gradient Flow (VGF), frames behavior-regularized RL as an optimal transport problem and solves it with discrete gradient flow guided by value gradients. This avoids explicit policy parameterization and enables adaptive test-time scaling by adjusting a “transport budget.” ³

Compared with reparameterized policy gradients or conservative reject sampling, VGF aims to move beyond the dataset safely while controlling over-optimization by how far mass is transported. The analysis shows regularization emerges implicitly from the transport constraint rather than add-on penalties. ³

In experiments, VGF reports state-of-the-art results on offline RL suites (D4RL, OGBench) and LLM RL tasks, indicating it scales to large generative models where classic policy gradients struggle. A project page provides code and runs for reproduction. ³

Boosting Visual Instruction Tuning with Self-Supervised Guidance

This study strengthens multimodal models’ visual reasoning by sprinkling in a small dose of vision-only tasks written as simple instructions, so the model must actually look rather than guess from language patterns. The authors reformulate classic self-supervised pretext tasks—like rotation prediction, color matching, and cross-view correspondence—into image–instruction–response triplets during instruction tuning. ⁴

Without changing architecture, adding labels, or extra training stages, injecting just 3–10% of these visually grounded instructions consistently improves vision-centric benchmarks across multiple models and regimes. The technique directly targets a common failure mode where many instruction-tuned MLLMs can answer “well enough” from text priors alone. Code is available for replication. ⁴

The broader context: recent industry analyses emphasize “mid-training” phases that reorganize model internals before RL. Together with this paper’s data-only tweak, it points to simple, well-placed training adjustments as powerful levers for reasoning—rather than solely scaling parameters. ⁵

Prism: Symbolic Superoptimization of Tensor Programs

Prism is a new compiler-time optimizer that searches whole families of tensor programs symbolically, then picks the fastest concrete plan—like drafting many route blueprints before committing to a drive. Its sGraph representation encodes program variants with symbolic parameters, enabling two-level search: build symbolic program families, then instantiate and auto-tune. Symbolic reasoning prunes provably suboptimal regions using operator semantics, algebraic identities, and hardware constraints. ⁶

On five common LLM workloads, Prism achieves up to 2.2x speedup over the best superoptimizers and up to 4.9x over leading compiler approaches, while cutting end-to-end optimization time by up to 3.4x—showing that smarter search can rival hand-tuned kernels. ⁶

In parallel, vendor stacks continue kernel-level gains: an open TensorRT-LLM PR optimizes causal_conv1d, reporting decode speedups from 1.90x to 3.32x (batch 1→256) and prefill 1.41x to 2.16x on an NVIDIA B300 for a 40-layer Mamba-style config; CI activity also documents skipped tests for known issues, illustrating the churn behind production inference speed. ^{7 8}

Open Source & Repos

Hermes Web UI: A dashboard for always-on personal agents

This web dashboard lets you manage chat sessions, channels (Telegram/Discord/Slack/WhatsApp), scheduled jobs, skills, and usage analytics for the open-source Hermes Agent from a single interface—aimed at teams running a persistent, self-hosted agent. Install is a single global npm command with a cross-platform UI. ⁹

Why people care: Hermes Agent positions itself as a self-hosted, persistent, self-improving agent with three-layer memory and auto-created skills; a recent overview claims it surpassed 60,000 GitHub stars within two months of launch in 2026, highlighting strong community pull. A full-featured dashboard helps operationalize that agent in real channels. ¹⁰

Who it’s for: operators who want data to stay local, coordinate multi-platform messaging, and schedule background tasks, while swapping among 200+ LLMs via the agent’s model routing. The UI centralizes configuration so non-developers can supervise an always-on agent. ⁹

HWTA Circuits: Attention-free compositional reasoning

This repo presents Hierarchical Winner-Take-All (HWTA) circuits—routing over fixed slots without softmax attention—that beat matched-parameter transformers by large margins on five compositional reasoning benchmarks. On CLUTRR k=10, HWTA (272K params) tops a tree transformer (268K) by +44.0 points under identical loops; several tiny variants reach 100% on SCAN/ListOps-style tasks. The key fix enabling depth generalization is letting messages carry source-slot state via a simple gather. ²

Monolith: Symbolic regression with one universal operator

Monolith explores differentiable trees built from a single operator eml(x,y)=exp(x)−ln(y), showing gradient descent can recover elementary functions from data with a minimal grammar. It’s a research proof-of-concept rather than a competitive tool, but clarifies the boundary of what gradient-only search can express. ¹¹

Why It Matters

Today’s updates underline a theme: small, principled changes at the right place—residual geometry, value transport, instruction mix, and compiler search—can unlock outsized gains without bigger models. That’s practical for teams constrained by cost or latency.

For readers, two mental models help: 1) structure the highway where information flows (3PT), and 2) move probability mass, not just parameters (VGF). Pair them with system optimizers (Prism, kernels) to turn “paper gains” into faster, cheaper deployments.