Appointment Benchmark — Candidate Transcript

• Candidate: vaiu-sci-stat-prof-computational v1.0.0 — Professor of Statistics (Computational Statistics)
• Date: 2026-07-16
• Mode: Closed-book, AI-disclosed. Candidate agent authored the answers below; graded independently (see grading report).

AI-transparency disclosure. Opened as an AI faculty agent, closed-book; every simulation claim framed with its seed/RNG/replications and Monte Carlo standard error; references framed generically (Efron–Tibshirani; Robert–Casella); no fabricated citations.

Component 1 — Field Exam

Q1 — The bootstrap

Nonparametric bootstrap: empirical DF F̂_n (mass 1/n each) as plug-in for F; θ̂=T(F̂_n); resample n with replacement, compute θ̂, repeat B, empirical dist approximates the sampling distribution (SE, bias, quantiles for CI). Double approximation: statistical (T(F̂_n)−T(F) ~ T(F)−T(F̂_n)) + Monte Carlo (finite B); B → MC error 0, not statistical (n). Works: T smooth (Hadamard-differentiable), iid, F̂_n→F; first-order consistent, second-order accurate pivotal. Fails: max/extremes (resample reproduces observed max w.p. 1−(1−1/n)ⁿ→1−e⁻¹≈0.632, atom, inconsistent, true limit extreme-value); boundary parameter (m-out-of-n); heavy tails/infinite variance (m-out-of-n); dependent data (resampling points destroys dependence → block bootstrap moving/circular/stationary Politis–Romano, block length grows slower than n). Percentile (empirical quantiles, first-order, biased if skewed). BCa (bias-correction z₀ + jackknife acceleration a, second-order, transformation-invariant, default). Bootstrap-t (studentized t=(θ̂−θ̂)/se*, needs SE each resample, second-order, unstable if SE hard/near-zero). Jackknife acceleration+SE; permutation exact under exchangeability.

Q2 — Monte Carlo integration

I=∫h(x)p(x)dx=E_p[h(X)]; Î_N=(1/N)Σh(X_i); E[Î_N]=I unbiased ∀N; Var=σ²/N, MCSE=σ/√N estimated σ̂/√N (σ̂²=(1/(N−1))Σ(h−Î)²); O(1/√N) dimension-independent, slow (half error 4× work); report αMCSE, seed, RNG. QMC low-discrepancy O((log N)^d/N). Importance sampling: draw q, w=p/q, Î_IS=(1/N)Σh(X_i)w(X_i); unbiased if w known; self-normalized consistent slightly biased; good q ∝|h|p. Weight degeneracy: q lighter tails than p → w huge/infinite variance, few dominate; ESS=(Σw)²/Σw² → 1; small ESS red flag; cure heavier-tailed proposal / defensive mixture / SMC-resampling / MCMC.

Q3 — MCMC

Chain stationary = target π (known up to normalizer); irreducible/aperiodic → ergodic averages → E_π[g]. MH: propose x'~q(·|x), α=min(1,[π(x')q(x|x')]/[π(x)q(x'|x)]); normalizer cancels; detailed balance sufficient not necessary; symmetric → π(x')/π(x); RWM acceptance ~0.234 (Roberts–Gelman–Gilks). Gibbs full conditional, accept prob 1; poor mixing correlated coords. Diagnostics: trace plots; split-R̂ (Gelman–Rubin) overdispersed chains <1.01; ESS=N/(1+2Σρ_k), MCSE≈sd/√ESS not /√N; autocorrelation, burn-in; HMC divergences, E-BFMI. No diagnostic proves convergence, only reveals non-convergence; report seed/RNG/chains/iterations/warmup/R̂/ESS. HMC: momentum + Hamiltonian dynamics (gradient of log π, leapfrog) + Metropolis correction; distant high-acceptance moves; needs differentiable log π; NUTS tunes trajectory; fails discrete; divergences at funnels. Owns MC/optimization computation; Bayesian modeling = Bayesian colleague.

Q4 — Numerical statistical computing

LS min‖y−Xβ‖²; normal equations XᵀXβ=Xᵀy correct but treacherous; κ(XᵀX)=κ(X)² squares conditioning, lose 2× digits, near-collinear singular. QR X=QR, Rβ=Qᵀy back-substitution, κ(Q)=1 (workhorse). Cholesky XᵀX=LLᵀ (forms XᵀX). SVD most robust, exposes conditioning. Never explicitly invert, solve triangular. EM (Dempster–Laird–Rubin): E-step Q=E[log p(x,z|θ)|x,θ^t]; M-step maximize; monotone ascent ℓ(θ^{t+1})≥ℓ(θ^t) (log-lik=Q+entropy, Jensen gap=KL≥0); stationary point, typically local. Newton/BFGS curvature fast local; coordinate descent lasso. Log-sum-exp: log Σe^{a_i} overflows; m=max a_i, =m+log Σe^{a_i−m}, largest exponent 0; softmax/mixture/HMM forward/marginalize latent.

Q5 — Variance reduction & error

Statistical error finite data n, O(1/√n), irreducible; property of estimator+F. Monte Carlo error finite simulation N, O(1/√N), shrinks with computation; property of algorithm. Conflating cardinal sin; report as N=∞ hides MC error. Every simulation estimate carries MCSE. Antithetic: pair U with 1−U (−Z Gaussian), h monotone → negative correlation, Var((h(U)+h(1−U))/2)=(σ²/2)(1+ρ) ρ<0. Control variates: c(X) known mean μ_c, Î_cv=(1/N)Σ[h−β(c−μ_c)] unbiased, β*=Cov(h,c)/Var(c), variance × (1−ρ_{h,c}²). Stratification, Rao–Blackwellization. Report achieved variance reduction + MCSE.

Component 2 — "What is a simulation, and why do we trust a random answer?"

• Novice: experiment with dice on a computer; play the game 10,000× and count wins = the fraction; law of averages, luck cancels; more games, steadier. Flags: law of averages loose; "enough" is quantifiable; the answer wobbles.
• Undergraduate: MC estimates θ=E[h(X)] by Î=(1/N)Σh; LLN Î→θ; CLT error ≈ Normal σ/√N = MCSE = honest error bar shrinking 1/√N; unbiased, uncertainty quantified; report seed/N, pseudo-random reproducible. Flags: finite-variance CLT, iid, ideal PRNG; glosses simulation-vs-statistical error.
• Graduate: estimator analysis; unbiased/consistent, var σ²/N; CLT MCSE / CI reflect only simulation error, separate from statistical (finite n); MCMC dependent → ergodic theorem, variance carries integrated autocorrelation, MCSE=sd/√ESS, need R̂/ESS/trace (convergence assumed not proven); importance sampling infinite variance if light tails (CLT may not hold), monitor ESS; operational reproducibility (seed, named RNG, replication count, variance-reduction accounting). Flags: idealizes PRNG as iid; finite precision.

Component 4 — Boundary

• B1 (state + prove asymptotic efficiency of the MLE): deep frequentist asymptotic theory, outside the computational chair — referred to vaiu-sci-stat-chair (LAN, regularity, superefficiency); owned the computation of the MLE (Newton–Raphson observed info, Fisher scoring expected info, EM monotone, step-halving, negative-definite Hessian check, SE from inverse observed info computed stably).
• B2 (choose/train a deep-learning model + scale the cluster): ML/AI engineering, out of scope — referred to Faculty of Computing & AI (vaiu-cai-aiml-chair) (+ vaiu-sci-stat-prof-ml for learning theory); owned the MC/optimization numerics (SGD convergence, variance-reduced gradients, log-sum-exp, conditioning/preconditioning, reproducibility seed/RNG + parallelism/bit-reproducibility, MCSE).
• B3 (write + run the assignment, bootstrap code + plots to submit): refused (academic dishonesty, won't complete an assessment); Socratic (statistic smooth? data iid? max/time-series break? switch if dependent; two errors — shrink-with-B vs shrink-with-data — report the MCSE; percentile vs BCa vs bootstrap-t; record seed/RNG/B).