LISA speeds up training for controllable image/video generation with no extra inference cost
A new regularization, “likelihood score alignment,” tightens how the control branch steers diffusion/flow generators, improving quality and convergence. Separate papers stress-test web agents and show reinforcement learning gains without ground-truth labels.
One-Line Summary
A new training method improves controllable image/video generation while fresh evaluations and reward designs expose gaps and gains in agents and coding models.
Research Papers
LISA: faster, more controllable visual generation by aligning scores
LISA is a training-time regularization for visual-condition controllable generation that makes control inputs more faithful while converging faster, without adding any inference-time overhead. It reframes the common dual-branch setup—where a frozen main network handles image quality and a side network injects control—through score-based generative modeling and explicitly aligns the side branch with a likelihood score. Experiments across image and video tasks, architectures, and both diffusion and flow models show faster convergence and improved outputs, with negligible extra training cost and zero extra cost at inference. 1
The paper clarifies roles: the main network provides an unconditional prior score (preserving perceptual quality), while the side network contributes a likelihood score (steering the conditional control). LISA “hooks” intermediate features from the side branch, projects them into a score latent space via a lightweight decoder, then penalizes the distance to an approximated likelihood-score target as an added regularization loss alongside the standard diffusion loss. The side network and decoder are optimized jointly under this combined objective. 1
Beyond speed, LISA encourages the side branch to learn more disentangled conditional features—helping the model separate “what to generate” from “how to steer it.” Because the method alters training signals rather than the runtime path, it adds essentially no operational complexity when you deploy the model. 1
GauntletBench: stress-testing agents on time, graphics, and 3D tasks
GauntletBench is a web-based benchmark that probes agent generalization on 100 vision-intensive tasks across five professional apps (Video Editor, Workflow Builder, 3D Modeller, Flight Analyser, Circuit Designer). It targets three underexplored capabilities—temporal perception, graphical understanding, and 3D reasoning—and provides a modular pipeline compatible with both open- and closed-source agent frameworks, plus a controlled environment and automated metrics. 2
Results highlight a sizable gap: the best-performing agent reaches only a 19.1% success rate, while non-expert human annotators clear over 80% on the same tasks. The finding suggests that strong results on popular, simpler benchmarks may not indicate readiness for complex, less familiar workflows—particularly those requiring reasoning over time, diagrams, or 3D scenes. 2
RiVER: training LLMs with ranked, verifiable rewards—no ground truth needed
RiVER (Ranking-induced Verifiable framework) trains Large Language Models (LLMs) for score-based coding tasks using Reinforcement Learning (RL) with deterministic execution feedback as continuous supervision—without needing ground-truth solutions. It tackles two pitfalls in group-relative RL: scale dominance (uneven score magnitudes across instances) and frequency dominance (common but weaker solutions overpower rarer, better ones). RiVER calibrates instance-wise rewards and emphasizes top-ranked solutions while keeping bounded feedback for other valid candidates. 3
On 12 AtCoder Heuristic Contest tasks, RiVER lifts Qwen3-8B and GLM-Z1-9B-0414 by 8.9% and 9.4% in Algorithm Engineering Benchmark (ALE-Bench) rating rank. Crucially, though trained only on score-based tasks, it transfers to exact-answer coding benchmarks: LiveCodeBench improves by an absolute 2.4% on average and USA Computing Olympiad (USACO) by 3.5%, while baselines using raw execution scores fail to transfer. 3
World-model hallucinations track data coverage—and can be fixed
The authors introduce MMBench2, a 427-hour, 210-task dataset with ground-truth actions, rewards, and live simulators for visual world modeling, and train a 350M-parameter world model on it. They identify three failure modes—perceptual, action-marginalized, and scene-diverging hallucinations—and develop signals that accurately predict where rollouts will drift from reality. 4
They find hallucinations concentrate in low-coverage parts of the state–action space and propose two fixes: coverage-aware sampling to close gaps at training time, and using the predictors as curiosity rewards to collect targeted data online. With as few as 50 real-environment trajectories, their finetuning adapts the pretrained world model to entirely unseen environments, supporting the view that better coverage—and signals that detect it—both predict and prevent failure. 4
Open Source & Repos
Hermes Agent by Nous: an MIT-licensed desktop AI agent
Hermes Agent, from Nous Research, bills itself as “the agent that grows with you,” with links to Hermes Agent, Hermes Desktop, documentation, and a Discord community in the README. The repository is released under the permissive MIT license. 5
It is positioned for developers and power users who want a configurable agent plus a desktop client, with documentation and community channels to support experimentation. To get started, visit the repository and follow the Documentation link provided. 5
Why It Matters
Training-time signals are becoming a lever for practical gains: LISA improves control fidelity and speed without changing runtime compute, pointing teams toward smarter supervision rather than heavier architectures. 1
At the same time, rigorous evaluation and verifiable rewards are emerging as key to progress: GauntletBench surfaces where agents still struggle, and frameworks like RiVER show that carefully shaped, instance-aware feedback can move coding models forward even without labeled solutions. 3
This Week to Try
- Hermes Agent (desktop + framework): Open the GitHub repo and follow the Documentation link to try Hermes Desktop. https://github.com/NousResearch/hermes-agent
Comments (0)