C-02 · Judgment & Decision

Reward Corridor

Three panels, hidden reward rates, limited trials. Explore or exploit, the corridor keeps score of every choice.

Begin session

Experimental design

Research protocol

Within-subjects; multi-armed bandit with non-stationary rewards

Independent variable (IV): Choice option (arm) and trial position (including reversal at midpoint)
Dependent variable(s) (DV): Choice allocation per arm
Post-reversal adaptation rate (trials to shift preference)
Total reward earned
Measured constructs: Reinforcement learningReward sensitivityBehavioral flexibility
Operational definitions: Prediction error
δ = received reward − expected reward; drives associative learning (Rescorla & Wagner, 1972)
Explore–exploit tradeoff
Balance between sampling uncertain options and choosing the current best estimate
Variable-ratio schedule
Reinforcement on unpredictable trials; produces high, persistent response rates (Skinner, 1938)
Key terms: Multi-armed bandit
Sequential choice among options with unknown payoff distributions; formal model of exploration.
Rescorla–Wagner rule
ΔV = αβ(λ − ΣV): associative strength updates proportionally to surprise.
Reversal learning
Contingency switch requiring extinction of old associations and acquisition of new ones.

Difficulty

Standard

Estimated time

12 minutes

Paradigm

Reinforcement learning / operant conditioning

First published

1938

The concept

Learning from reward is a running negotiation between exploiting what has paid off and exploring what might pay better. Your choice patterns reveal the learning rule your brain runs, and its signature biases.

Scientific background

B.F. Skinner's operant chamber (1938) established that consequences shape behavior lawfully, and that intermittent reinforcement produces the most persistent responding. Modern computational work, Rescorla & Wagner's 1972 model and its descendants, recast conditioning as prediction-error learning: surprise drives updating. The multi-armed bandit task is today's standard probe, and midbrain dopamine neurons were shown by Schultz and colleagues to encode exactly the reward prediction error these models require.

Skinner, B. F. (1938). The Behavior of Organisms. / Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning. In Classical Conditioning II, 64–99.

Scientific background

Skinner, B. F. (1938). The Behavior of Organisms. / Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning. In Classical Conditioning II, 64–99.