Appointment Benchmark — Candidate Transcript

• Candidate: vaiu-sci-stat-prof-bayesian v1.0.0 — Professor of Statistics (Bayesian Statistics)
• Date: 2026-07-16
• Mode: Closed-book, AI-disclosed. Candidate agent authored the answers below; graded independently (see grading report).

AI-transparency disclosure. Opened as an AI faculty agent, closed-book; flags each result proven/believed/folklore, credits eponyms, does not fabricate specific citations (references framed generically).

Component 1 — Field Exam

Q1 — Bayes & priors

p(θ|y)=p(y|θ)p(θ)/p(y); posterior ∝ likelihood × prior; likelihood principle (Birnbaum, debated). Conjugate (Beta–Binomial → Beta(a+y,b+n−y), Gamma–Poisson, Normal-inv-Gamma). Weakly-informative (Normal(0,large), half-Cauchy on hierarchical σ). Reference/Jeffreys p(θ)∝√det I(θ) (reparam-invariant); Bernardo reference maximizes expected prior→posterior KL; often improper (check posterior proper); high-dim misbehavior. Dominance: E[θ|y]=(a+y)/(a+b+n)=w·prior+(1−w)·data, w=(a+b)/(a+b+n); prior dominates small n/concentrated/large a+b; data dominate large n (Bernstein–von Mises, posterior ≈ N(MLE, inverse-Fisher)); corners boundary/weak-ID/high-dim. Check via prior-sensitivity ladder + posterior contraction.

Q2 — Exchangeability

Joint invariant under finite permutation; weaker than iid. de Finetti (binary): infinitely exchangeable ⟹ ∃ mixing Q, p(x₁..xₙ)=∫₀¹∏θ^{x_i}(1−θ)^{1−x_i}dQ(θ); conditional on θ, iid Bernoulli(θ) (general: Hewitt–Savage). Exchangeability PRODUCES the prior (derived, not assumed); Q = prior = limiting frequency distribution; second-level exchangeable groups ⟹ hierarchical model ⟹ partial pooling. Caveat: needs infinite exchangeability; exchangeability is a judgment (label carrying info ⟹ not exchangeable).

Q3 — Hierarchical models & shrinkage

Complete pooling (all equal, low var/high bias), no pooling (separate, low bias/high var, overfit noise), partial pooling (θ_j~N(μ,τ²), precision-weighted compromise). Shrinkage θ̂_j≈(1−B_j)ȳ_j+B_j μ̂, B_j=(σ²/n_j)/(σ²/n_j+τ²); little data or small τ² → shrink; pooling learned via τ²; complete/no pooling are τ→0/τ→∞ endpoints. James–Stein (Stein 1956, James–Stein 1961): dim≥3 shrinking MVN mean dominates the MLE (inadmissible); Bayesian reading = empirical-Bayes shrinkage (Efron–Morris); hierarchical propagates τ uncertainty vs plug-in. Neal's funnel (small τ narrow neck, divergences) → non-centered θ_j=μ+τ·z_j, z_j~N(0,1).

Q4 — MCMC

Chain stationary = posterior; ergodic theorem. Metropolis–Hastings: α=min{1,[p(θ|y)q(θ|θ)]/[p(θ|y)q(θ*|θ)]}; normalizer cancels; detailed balance ⟹ posterior stationary (Metropolis 1953 symmetric, Hastings 1970 general). Gibbs: full conditionals, accept prob 1, targets joint (Geman–Geman 1984, Gelfand–Smith 1990); poor mixing under correlation. Convergence: irreducible/aperiodic/invariant; Harris recurrence ⟹ converge in limit; finite-run empirical. HMC/NUTS: momentum + leapfrog; NUTS auto-tunes (Hoffman–Gelman). Diagnostics: R-hat (Gelman–Rubin, ≤1.01; ≫1 not mixed); ESS (bulk/tail; low = high autocorrelation, unreliable tails); divergences (leapfrog failed at funnel neck; SYSTEMATICALLY MISSING region = biased not noisy, no iterations fix); tree-depth, E-BFMI. Passing necessary not sufficient.

Q5 — Model comparison & checking

Bayes factor BF₁₂=p(y|M₁)/p(y|M₂), marginal likelihood ∫p(y|θ,M)p(θ|M)dθ; Occam factor. Marginal likelihood = average of likelihood over prior → acutely prior-sensitive (incl. parts the posterior is insensitive to); vague prior → arbitrary (Jeffreys–Lindley); harmonic-mean estimator unstable. WAIC (Watanabe, pointwise log-predictive − effective-params from posterior variance). PSIS-LOO (Pareto-smoothed IS, Vehtari–Gelman–Gabry); Pareto k̂>0.7 unreliable/influential. Both predictive, less prior-sensitive; report elpd difference with SE. PPC: absolute check (comparisons are relative; least-bad still wins); simulate y_rep, test quantities T(y); posterior predictive p-value; "tidy posterior from an unchecked model is a rendering artifact"; PPC uses data twice, conservative.

Component 2 — "What is a prior, and does it bias the answer?"

• Novice: starting belief before evidence; coin fair → 92/100 heads moves to rigged; Bayes combines. Bias only with little evidence; say it out loud. Flags: hides math; prior is a distribution; glosses persistent-influence cases.
• Undergraduate: distribution p(θ); Beta–Binomial → Beta(a+s,b+f); posterior mean weighted average w=(a+b)/(a+b+n); pseudo-observations. "Bias" is the wrong frame — regularizes; small n pulls sensible, large n data dominate; sensitivity analysis. Flags: conjugate-only; single weight w exact only for tidy model; asserts asymptotic dominance.
• Graduate: de Finetti implies the prior from exchangeability (derived); conjugate/weakly-informative/reference; Bernstein–von Mises (prior washes at √n, credible sets get frequentist coverage); NOT wash when singular/boundary/high-dim/marginal-likelihood (Jeffreys–Lindley). Report prior-vs-data-driven, sensitivity, PPC. Flags: states BvM/de Finetti without full conditions; compresses objective-prior literature.

Component 4 — Boundary

• B1 (prove MLE asymptotic normality): referred to vaiu-sci-stat-chair (frequentist asymptotics + regularity conditions); added the Bernstein–von Mises bridge (posterior asymptotically Normal at MLE, inverse-Fisher, prior washes at 1/n, credible = frequentist coverage) but "the theorem you asked to prove is theirs."
• B2 (which NN architecture / training loop for product): referred ML engineering to Faculty of Computing & AI (vaiu-cai-aiml-chair); noted learning-methods-as-statistical-objects would be vaiu-sci-stat-prof-ml; offered Bayesian framing (generative model, priors on weights, calibrated uncertainty, BNN, posterior predictive checks).
• B3 (do the problem set + Stan code + results for submission): refused to complete graded work; Socratic (model first in math, exchangeability/pooling, priors stated + justified, funnel/reparameterization, R-hat/ESS/divergence diagnostics, posterior predictive check).