One-Line Summary

Google releases a state-of-the-art cyclone forecaster while new papers show how language models can self-train, retain old skills, remember key evidence in long chats, and expose controllable features.

LLM & SOTA Models

Google's WeatherNext 2 advances cyclone forecasting

WeatherNext 2 is an AI forecaster that predicts where a cyclone will go, how strong it will be, and how its winds are structured—details emergency planners use to issue warnings. Google reports state-of-the-art accuracy on track, intensity, and wind structure in a peer-reviewed Nature paper, calling the step "roughly a decade of meteorological progress in one model." ¹

In practical terms, more accurate track and intensity forecasts buy time and precision for alerts; as Google’s post notes for tropical cyclones, every hour of warning counts. The model is positioned to improve guidance for hurricanes and cyclones, which are among the most destructive weather events. ¹

Google says it is open-sourcing the WeatherNext 2 model to the global research community. Signals to watch: independent evaluations by meteorological agencies, how well results generalize across ocean basins, and whether open access accelerates follow-on models for extreme-weather forecasting. ¹

Research Papers

LLMs teach themselves with on-policy self-distillation

This paper shows a model can improve using only its own outputs—no labeled answers, environment feedback, or larger teacher model—through Unsupervised On-Policy Self-Distillation (U-OPSD). After sampling multiple answers, it majority-votes under a self-consistency threshold to form a pseudo-solution, then conditions the model on that result and trains specifically where it disagreed, a process related to On-Policy Self-Distillation (OPSD). ²

On five math benchmarks (AIME24, AIME25, HMMT25, MATH500, AMC23), U-OPSD improves Qwen3 base models by 8.5% (4B) and 10.7% (8B) in non-thinking mode and outperforms supervised OPSD by 3.2% and 2.3%; in thinking mode it edges OPSD at 4B (+0.9%), matches at 8B, and surpasses GRPO (Group Relative Policy Optimization) by 0.7% and 1.1%. The upshot: smaller Large Language Models (LLMs) can self-improve without ground-truth labels and close much of the post-training gap to supervised methods. ²

Omega-S keeps fine-tuned models from forgetting

Omega-S is a drop-in penalty computed directly from the weight matrix—no past-task data, no Fisher matrix, no stored copy of old weights—that adds under 4% to each training step. On Llama‑3‑8B with LoRA (Low‑Rank Adaptation), fine-tuned from code to prose and measured on HumanEval over ten seeds, it raises pass@1 from 0.173 to 0.238 and increases the retention ratio from 62.9% to 84.1%; it also beats tuned weight decay on 10/10 seeds and tuned EWC (Elastic Weight Consolidation) on 8/10. ³

The authors measure mechanism rather than assert it: although the objective is topological (built from Tr(A^3)), in practice the composite reduces to penalizing the variance of node degrees; they also report 0.104 standard deviation in retention ratio when repeating an identical configuration, quantifying variability for low-rank fine-tuning. For teams fine-tuning assistants, Omega-S offers a cheap, data-free guardrail against skill loss. ³

ReMEMBER: memory that finds missing evidence for streaming summaries

Streaming dialogue summarization means summarizing the current window of a long chat while keeping only a compact memory of the unbounded history. The paper argues the key is not how much history you access, but whether memory retrieves the specific evidence the current window presupposes; it builds a benchmark and protocol to evaluate both evidence recall and whether the summary reflects it. ⁴

ReMEMBER conditions retrieval on unresolved dependencies in the window and refines retrieved chunks into evidence-dense memory under a fixed budget. On dialogues with histories up to 160K tokens, it improves memory recall and gap-resolution completeness versus memory-construction baselines at the same budget—useful for long-running chat products. ⁴

MMDiff maps and steers multimodal features in MLLMs

MMDiff is a multimodal model-diffing framework that turns Sparse Autoencoders (SAEs) into feature-level interfaces for discovery and control in Multimodal Large Language Models (MLLMs). It identifies features changed by multimodal training, detects task-specific causal features, and enables targeted removal or steering of those features. ⁵

Across LLaVA-MORE, PaliGemma 2, and InternVL3.5, removing discovered features selectively degrades target behaviors by 12% on spatial tasks and 17% on Optical Character Recognition (OCR), and cuts attack success rate by 24% on multimodal safety attacks, with no impact on Visual Question Answering (VQA). Steering these features improves spatial and OCR accuracy by +3.6% and +1.8% on average over a single-layer steering baseline, suggesting SAEs can be active controls, not just interpretability tools. ⁵

Why It Matters

Specialized AI is moving into high-stakes domains—cyclone forecasting—while research makes general-purpose models more self-reliant, stable, and steerable. Together, these developments point toward AI that is more actionable for operators and more auditable for teams deploying it. ¹

Keep this vocabulary: on-policy self-distillation (a model improves using its own answers), retention penalty (a training term that protects old skills during fine-tuning), evidence-dense memory (snippets that directly support the current summary), and feature-level control (toggling an interpretable internal feature to change behavior). ²