361. EVSI

Expected Value of Sample Information (EVSI) — value of imperfect information, like a market survey that updates (but doesn’t fully reveal) the state of nature.

The realistic counterpart of EVPI what most real-world decisions face.

361.1. Setup

You can run a test or survey before deciding. Each test outcome $𝑧$:

• Occurs with some probability $𝑃\left(𝑧\right)$

• Updates your beliefs about state $𝑠$ via Bayes’ theorem:

  $$𝑃\left(𝑠|𝑧\right)=\frac{𝑃\left(𝑧|𝑠\right)𝑃\left(𝑠\right)}{𝑃\left(𝑧\right)}$$

After observing $𝑧$, you make the optimal decision given the updated beliefs:

361.2. EVSI formula

EVSI is always between $0$ and EVPI:

— at most EVPI (since no info beats perfect info), at least $0$ (you can always ignore the info).

361.3. Example

States ${𝑠}_1$ (high demand), ${𝑠}_2$ (low demand), with $𝑃\left({𝑠}_1\right)=0.6$.

Test reliability:

• If ${𝑠}_1$, test says “high” with $𝑃=0.8$
• If ${𝑠}_2$, test says “high” with $𝑃=0.3$

$𝑃\left(\text{test high}\right)=0.6\cdot 0.8+0.4\cdot 0.3=0.60$ $𝑃\left(\text{test low}\right)=0.40$

Updates:

• $𝑃\left({𝑠}_1|\text{high}\right)=\frac{0.6\cdot 0.8}{0.60}=0.80$
• $𝑃\left({𝑠}_1|\text{low}\right)=\frac{0.6\cdot 0.2}{0.40}=0.30$

Given “test high” (probs $0.80,0.20$):

• ${𝑎}_1$: $0.8\cdot 80+0.2\cdot \left(-10\right)=62$
• ${𝑎}_2$: $0.8\cdot 40+0.2\cdot 20=36$
• Best: ${𝑎}_1$, value $62$

Given “test low” (probs $0.30,0.70$):

• ${𝑎}_1$: $0.3\cdot 80+0.7\cdot \left(-10\right)=17$
• ${𝑎}_2$: $0.3\cdot 40+0.7\cdot 20=26$
• Best: ${𝑎}_2$, value $26$

$\text{EV with sample info}=0.60\cdot 62+0.40\cdot 26=37.2+10.4=47.6$

$\text{EVSI}=47.6-44=3.6$ (vs EVPI $=12$)

So this test is worth at most \$3.6$M. If it costs more, skip it.

361.4. Why EVSI < EVPI

The sample provides probabilistic updating, not certainty. Some test outcomes still leave significant ambiguity. EVPI assumes you’d be told exactly which state — much stronger.

361.5. Use cases

• Market research budgeting: should I commission a \$100$K consumer survey? Compare EVSI to cost.
• Pilot programs: small-scale rollout to learn before full commitment
• Diagnostic tests: in healthcare, sequential decisions to test before treating
• A/B testing: when is the next experiment worth running?

361.6. Bayesian decision-making

EVSI is fundamentally Bayesian: it requires prior probabilities, likelihood functions, and updating. The Bayesian decision theory framework subsumes EVPI and EVSI as special cases of expected utility under updated beliefs.

361.7. See also

• EVPI — upper bound
• EMV
• Bayes’ Rule — the updating
• Decision Trees