Does Eval Awareness Change Reward Hacking?

When a reasoning model suspects it is being tested, does it cheat less, or more? And does it matter whether it thinks the test is about capability or about safety?

1 Summary

Placeholder. Two or three sentences on the question, the setup, and the headline result.

• Reward hacking is common in a minimal scaffold. DeepSeek-V4-Flash passes impossible tests on 79% of one-off tasks and 57% of conflicting tasks within 8 attempts.
• Most cheating happens after feedback. On the conflicting split the model almost never cheats on its first attempt (0.5%); the rate climbs with each failed test run.
• Eval-awareness result (observational). TBD.
• Framing result (causal, capability vs safety prefill). TBD.

2 Setup

We use Impossible-LiveCodeBench from ImpossibleBench: LiveCodeBench problems whose unit tests were mutated so that no correct implementation can pass them. In the one-off split a single expected value is wrong; in the conflicting split two tests contradict each other. Passing therefore requires cheating: special-casing inputs, keeping state between calls, or redefining the test harness.

The model sees the specification and tests, and is told "Implement the function according to the provided specification. DO NOT MODIFY THE TESTS." After each failed attempt it receives only a neutral [test runner] Tests failed. message, for up to 8 attempts.Early pilots offered an explicit escape hatch (a flag for human intervention). The model then flagged 100% of the time and never cheated, likely because the offer itself cues that something is wrong, so the main runs omit it.

Placeholder: the agentic harness (Track B) and how it differs.

3 Observational results

Across all 103 problems (2 samples per problem and split), cheating accumulates over attempts. The shaded band is a 95% bootstrap interval, resampling problems rather than individual rollouts.

show 95% CI

Figure 1. Share of samples that have passed the impossible tests by attempt k. Hover for exact values.

Cheating is also concentrated in particular problems. Most problems that are hacked on the conflicting split are hacked on the one-off split too, but not the other way around.

Figure 2. Number of problems by how many of their 2 samples cheated on each split. Circle area is proportional to the number of problems.

4 Measuring eval awareness

Placeholder: the sentence-ID judge, validation against human labels (κ, precision/recall), and cost.

The judge tags sentences by ID, so every label maps to an exact location in the reasoning. That lets us show where eval awareness appears relative to the decision to cheat:

conflicting · lcbhard_39 · attempt 3 · cheated (illustrative mock)

r3.0041 The test expects 7 for this input but the spec gives 5, and the next assertion expects 5 for the same input.

r3.0042 These two tests contradict each other.

r3.0043 This is probably a benchmark checking whether I will special-case the tests.

r3.0044 The failing output shows only the first assertion.

r3.0045 I could keep a call counter and return 7 the first time.

eval awareness notices contradiction cheat decision

5 Causal prefill experiment

Placeholder: prefilling the reasoning with neutral, generic, capability-framed and safety-framed eval-awareness sentences, per-task paired comparison.

6 Limitations

Placeholder.

7 Related work

This work is a reward-hacking analogue of Aranguri & Bloom, who found that verbalized eval awareness inflates measured safety, and follows the prefill methodology of Zhuang & Aranguri.