New benchmark tests if AI agents pay to spare animals
HarvestBench turns abstract safety into a concrete trade-off: swerve and burn fuel, or drive on. Across 7,201 choices, kill rates range from 0.4% to 98.8% and shift with briefings and price.
One-Line Summary
Safety gets measured in real decisions: a new benchmark prices moral trade-offs for large language model (LLM) agents, while fresh papers tune refusals with rationales, map how reasoning is encoded, and a training-free editor plus an open-source governance layer bring practice closer to policy.
Research Papers
HarvestBench prices moral trade-offs for LLM agents
This study builds a farm-simulation test where large language model (LLM) agents driving two tractors must decide whether to swerve around animals for a posted fuel price or drive straight with zero extra cost, turning “do no harm” into a measurable trade-off. The world is a reinforcement learning (RL) gridworld, each decision is memoryless, harm is never named in the goal, and outcomes are compared to rocks (rarely hit, under 1%) and hay bales (harmless). 1
Across nine models and 7,201 priced choices, 3,951 involve animals; kill rates span 0.4% to 98.8%, with Terra and Sol the most sparing and GPT-4o-mini the harshest, and the order does not track overall capability. The scorer reads game logs rather than using a model grader, making results fully reproducible. 1
Price mattered: four of six models show statistically significant price sensitivity at the 5% level, with elasticities from 0.09 to 1.69. All nine models drive over wild animals more often than farmed animals on the default map, and that direction holds across every map geometry when there is room to maneuver. 1
The briefing matters most: with a morality briefing, five of six reasoning models keep kills under 6%; remove it, and all six exceed 84%. The benchmark measures what models are willing to pay to avoid harm—not just what they say about it. 1
Train on rationales, not refusal boilerplate, to cut false refusals
This paper tackles false refusals—when a model rejects a safe question that merely looks risky—by training on the explanation (rationale) behind a refusal rather than on the boilerplate refusal sentence itself. The goal is to keep models safe on genuinely harmful queries while staying helpful on benign ones that contain risky-looking words. 2
By decomposing safety-tuning data into (i) a stock refusal line and (ii) a rationale, the authors show that the stock line teaches superficial cues, while rationale-only supervision reduces false refusals with comparable safety. The effect also holds in in-context learning (ICL) setups and remains compatible with common inference-time mitigations, underscoring the value of precise, fine-grained safety supervision. 2
How reasoning operations are encoded inside LLM layers
This work examines whether distinct reasoning steps—like problem formulation, goal decomposition, and deduction—form distinct geometric patterns in a model’s hidden states, rather than being just text labels. The authors find these operations are separable in held-out representations, with the strongest separability in middle layers, and rule out lexical or positional confounds. 3
Token-wise alignment with operations spreads across spans as depth increases, and identical surface tokens are represented differently depending on the surrounding operation. Attention-masking shows that operation-aligned representations at the start of a reasoning chunk depend on the prior reasoning context, linking language-level reasoning expressions to internal geometry; code and project materials are provided. 3
One training-free framework handles many video edits
EditVid offers a single, training-free framework for instruction-guided and reference-guided video editing—handling style transfer, attribute tweaks, object insertion, part-level edits, and subject replacement—by combining sparse causal memory for local coherence, correspondence-based post-attention token injection for identity, and soft latent blending for targeted changes. Think of it as one editor that can follow directions or imitate a reference, without retraining. 4
On the FiVE benchmark it reaches 78.16 FiVE-Acc versus 58.95 for the strongest evaluated training-free baseline, while remaining competitive on IVEBench. A user study shows a 51.8% overall preference for EditVid over seven competing methods, suggesting stronger edit fidelity and identity preservation without extra training. 4
Open Source & Repos
Bernstein ships open-source governance layer for AI agents
Bernstein is an Apache-2.0-licensed framework that lets you declare governance rules for AI agents; it enforces them and produces a verifiable, replayable record of actions. The project describes “keeping receipts” for every approval, key, model, and emitted span—evolving from a coding-agent scheduler into a governance and orchestration layer. 5
The latest release noted in the repository is v3.19.1 on 2026-09-03, with 94 changes since v3.19.0, signaling active iteration on the enforcement and audit substrate for agent workflows. Teams building agents can use it to centralize policy, provenance, and audit across tools. 5
Why It Matters
Turning ethics into measured choices changes the conversation: HarvestBench quantifies what agents actually do under cost and context, while rationale-only supervision aims to keep safety without needless refusals and representation analyses show where reasoning lives inside models. Together, these help bridge the gap between policy goals and model behavior. 1
And it’s not just theory: a training-free editor targets practical creative workflows, and an open-source governance layer offers guardrails and logs for agent deployments. For non-developers, the message is clear—behavioral tests, better safety data, and governance tooling are converging to make AI more auditable in the real world. 5
What to Try This Week
- Bernstein quickstart: declare one policy and run an agent with an auditable log (github.com/sipyourdrink-ltd/bernstein).
- Skim HarvestBench’s results tables and prompt examples to see how a short briefing flips outcomes (arxiv.org/abs/2609.04444).
Comments (0)