chc.regret¶
Stability tier: evolving.
Certainty-equivalence suboptimality for LQ control -- CHC's regret guarantee (LQ special case).
For a linear-quadratic problem the certainty-equivalent controller (solve the LQR for an estimated
model, then apply that gain to the true plant) has a suboptimality gap that is quadratic in the
model error: J(K_hat) - J* <= O(||[dA, dB]||^2) in the small-error regime, provided the
estimated controller stabilises the true plant. This LOCAL quadratic bound is Mania, Tu & Recht
(2019), which improved the earlier LINEAR-in-error robust-synthesis bound of Dean, Mania, Matni,
Recht & Tu (2018); it is a small-error suboptimality bound, not a global equality. This is the
analysable special case of CHC's pessimism story -- small model error costs almost nothing, but the
penalty grows with error, which is exactly what the calibrated uncertainty penalty
(chc.uncertainty) is there to price in offline.
A NumPy/scipy analysis tool (like chc.did / chc.scm), independent of the JAX x64
flag. The infinite-horizon discrete LQR is solved via the DARE; a controller's true-plant cost via
the discrete Lyapunov equation.
MatrixRatioStatus = Literal['convicted', 'not_convicted']
module-attribute
¶
What MatrixRatioAccuracy can say about its grid: a conviction, or the absence of one.
convicted means the refinement residual, or the isotropy bar where the channels are
exchangeable, exceeds the tolerance. not_convicted means neither does, and that is evidence,
not a certificate. There is deliberately no certified: what guarantees that the residual bounds
the error is a per-node convergence rate of at least 2 (residual_bounds_iff_rate_reaches_two),
and nothing the certificate evaluates measures that rate -- at q = 3 or at q = 2.
MinimaxBranch = Literal['lower', 'upper', 'equalise', 'zero', 'interior']
module-attribute
¶
Which candidate the robust action landed on. lower / upper are the certainty-equivalent
actions for an interval endpoint, equalise the action that makes both endpoints equally bad
(b_hat*u = target for the cost, the sqrt(D)-weighted mean of the endpoint actions for the
regret), zero doing nothing -- the answer when the sign is unidentified -- and interior a
regret whose worst effect lies inside the interval.
MoveRule = Literal['sign', 'linear']
module-attribute
¶
Which minimax-regret rule minimax_regret_move took: sign makes the whole move where
its estimate is not negative and holds otherwise; linear makes a share of it that rises
linearly from 0 to 1 across [-threshold, threshold].
RegretCurve
dataclass
¶
Empirical certificate: median CE suboptimality vs model error, and its log-log slope.
OrthogonalControlCurve
dataclass
¶
Double-debiasing certificate: control regret vs nuisance error, plug-in vs orthogonal.
PessimismCurve
dataclass
¶
Scalar one-step: uncertainty regularizer equals the effect variance (not general).
InformationLowerBoundCurve
dataclass
¶
CR lower bound on control regret; confounding lowers information, raising the floor.
MultivariateActionFloor
dataclass
¶
The van Trees floor on control regret when the effect is a MATRIX -- Result 67.
Result 57's scalar floor is a PRODUCT, curvature times information. This one is a TRACE,
tr(M Psi' G^-1 Psi'), which interleaves them -- so it decomposes over the eigendirections
of the information rather than collapsing to a single ratio, and a direction the optimal action
does not depend on contributes exactly nothing however badly it is identified.
MultivariateVanTreesCurve
dataclass
¶
Result 67 -- the matrix-effect van Trees floor, and the alignment a scalar plant hides.
HighProbRegretCurve
dataclass
¶
Finite-sample high-probability UPPER bound on scalar CE regret (w.p. >= 1-delta).
TransportabilityCurve
dataclass
¶
Deployment regret = transportable part (zero) + W1 residual (quadratic in W1).
EnsembleControlCurve
dataclass
¶
One control over a heterogeneous population pays a curvature-weighted Var(u*) floor.
CompositionTransferCurve
dataclass
¶
Order-p effect estimator -> order-2p control regret: the control map doubles the order.
MultivariateTransferCurve
dataclass
¶
Transfer theorem in MULTIVARIATE LQ: effect-matrix error delta^p -> LQ regret delta^(2p).
MultiChannelControlCurve
dataclass
¶
Debias EVERY interference channel: regret order set by the least-orthogonalised one.
MultivariateInterferenceCurve
dataclass
¶
Multi-channel bottleneck in MULTIVARIATE LQ: debias every channel or lose the order.
EndToEndC2Curve
dataclass
¶
C2 END-TO-END on a clustered LQ network: regret ~ 1/G (sampling floor) + (sum delta^p)^2.
ClusteredLowerBoundCurve
dataclass
¶
Clustered van-Trees LOWER bound: G*regret -> c0 > 0, so the 1/G rate is TIGHT (two-sided).
CrossClusterMixingCurve
dataclass
¶
A3': the 1/sqrt(G) rate under cross-cluster psi-dependence, and the constant it moves.
ExposureMapCurve
dataclass
¶
Exposure-map C2: three channels (direct, spillover, exposure-map W); r_W squared iff orth.
ClusterFoldLeakageCurve
dataclass
¶
What row-level cross-fitting costs on a clustered design: no bias, a different estimator.
DelayedNetworkCurve
dataclass
¶
Result 51: what a delay PROPAGATING THROUGH THE NETWORK does to row-level cross-fitting.
MatrixRatioAccuracy
dataclass
¶
What exact_matrix_ratio_moment is worth on the grid it was given -- Result 63 (d).
At q = 3 the cone is 6-dimensional and the value carries a percent-scale error that nothing
in the returned array reveals. At q = 2 it is 3-dimensional, and how close the default grid
comes depends on the margin from the existence boundary and on the channel correlation
(exact_matrix_ratio_moment). This is the grid-refinement residual: the same integral on
the next grid down, and the gap between them.
It is an estimate, not a bound, and exactly when it fails is known. A refinement residual
measures the STEP, not the remainder: for a monotone same-signed error sequence it is
identically e(k-1) - e(k), verified against the measured q = 3 sweep to 5e-5 relative.
So residual / true error = r - 1 where r = e(k-1)/e(k) is the per-node decay, and the
residual majorises the error iff r >= 2 (residual_bounds_iff_rate_reaches_two). Below
that threshold it under-states, by construction rather than by accident -- under that premise.
Without it the residual can only be larger, being at least |e(k-1) - e(k)| by the reverse
triangle inequality: r >= 2 always suffices, and below 2 the residual majorises only where
the error changes sign or moves its largest entry, as at q = 3, n = 7 with 5 and 8 nodes
(rates 1.68 and 1.77, ratios 2.06 and 1.04).
That is the whole story of the q = 3 failures. The measured rates on the boundary are 1.30,
1.44, 1.68 -- all under 2 -- and the corresponding measured ratios are 0.30, 0.44, 0.68, i.e.
r - 1 to two decimals. Over 15 cells the ratio ranges 0.30x to 5.26x and falls
below 1 in five, every one of them a cell whose rate is under 2.
Those 13 cells are all exchangeable; the first correlated one sits at the conservative end
and slightly past it. On the correlated-channel anchor of Result 63 (i)
(regressor_cov = kron(R, I_n), exact answer R^-1/(n-q-1)) at q = 3, n = 7,
residual / true error is 2.42, 5.80 and 1.44 at
nodes = 5, 6, 7 -- conservative at all three, and in fact more conservative than the
exchangeable cell at the same nodes (2.06, 3.64, 0.78). An earlier draft of this docstring
claimed the residual understated by 41x there; that came from dividing a single entry's step
by a different entry's relative error, and is withdrawn. What the correlated cell does show is
that the ratio is no more predictable off the exchangeable cells than on them: it still swings
by 4x across three consecutive grids, and its 5.80 widens the measured range to 0.30x-5.80x.
That range remains a description of the cells it was measured on rather than a calibration.
A three-grid rate estimator does not rescue this and was not shipped: under geometric decay
residual(k-1)/residual(k) would equal r, but the measured residual ratios are flat
(0.96, 1.09) while the error ratios are 1.44 and 1.68. The error sequence is not geometric, so
there is nothing for a Richardson-type correction to extrapolate.
So status is one-sided. A bar over tolerance convicts the grid; nothing here
certifies it. What guarantees that the residual majorises the error is a per-node rate of at
least 2, and a rate is a ratio of two errors, while the certificate holds one difference between
two grids and no reference value; a third grid would add a ratio of residuals, which is not the
rate (above). Nor does the rate follow the channel count. It passes 2 on some q = 3 grids
(test_matrix_ratio_certificate_reports_what_the_grid_is_worth has one), and on a correlated
q = 2 anchor one step from the existence boundary the default grid's rate is below 2 and the
residual understates the error (test_the_ratio_status_can_convict_and_cannot_certify).
The second bar, where it exists. On an exchangeable problem the exact answer has a constant
diagonal, so half the observed diagonal spread is a PROVED lower bound on the largest entry
error (half_spread_bounds_the_max_error) -- no reference value needed, and no extra
quadrature, since it is a function of value alone. exchangeable says whether the
precondition actually holds, decided rather than assumed: Omega must be invariant under
permuting the q channels of vec(X), which q - 1 transposition checks settle.
Measured at q = 3, n = 5 against the exact anchor, the bar recovers a far STEADIER share of
the true error than the residual does:
===== ================== =============== nodes residual / true bar / true ===== ================== =============== 5 5.26 0.99 6 0.86 0.55 7 0.30 0.45 8 0.44 0.46 9 0.68 0.48 ===== ================== ===============
The residual's ratio swings by 17x; the bar sits at 0.45-0.55 once past the coarsest grid. But
it does not rescue the cells the residual missed -- at a 1% tolerance both convict every one of
these cells, so the bar is a safety net rather than a fix, and either bar convicts on its own.
Do not read the steadiness as a calibration: five cells of one problem family is exactly the
evidence base that produced the retracted rate claim in exact_matrix_ratio_moment.
status
property
¶
convicted iff a bar exceeds tolerance -- see MatrixRatioStatus.
Derived rather than stored, so the verdict cannot disagree with the bars it summarises.
FoldDesign
dataclass
¶
A fold partition with its Result 52 objective and an optimality certificate.
objective is the partition-dependent part of the sandwich, the same-fold weighted 2-walk
count sum_f 1_f' Q(x) 1_f -- Psi is affine in it with positive weight, so smaller is
better. lower_bound is the diagonally shifted Ky Fan bound and kyfan_bound the plain one
it never falls below (both an exact eigensolve, no relaxation error on the bound side), and
gap = (objective - lower_bound) / max(|lower_bound|, 1). When exhaustive is True the
returned fold is the global optimum regardless of the gap -- the gap then measures the
looseness of the spectral bound, not of the design.
FoldHeuristicCurve
dataclass
¶
How far the fold-design heuristic falls short of the exact optimum, off the cycle.
Result 52's closed form is a statement about vertex-transitive graphs: it reads the optimal
partition off the spectrum of Q(x) because every vertex sees the same neighbourhood. On a
path, a star, a grid or a barbell that symmetry is gone and there is no closed form, so
optimal_fold_partition falls back to spectral rounding plus balanced swaps. This
measures what that fallback costs, by running the exact enumeration beside it on graphs small
enough to enumerate -- the regime where "the heuristic is fine" can still be checked rather
than assumed.
FoldExactnessCurve
dataclass
¶
Heuristic shortfall and bound looseness, finally separated -- Result 61.
fold_heuristic_certificate could only compare the fallback with the truth where
enumeration runs, and it stopped at m = 16. On a circulant the truth is reachable at
any m: the objective collapses to circular-offset counts, so a dynamic program over the
last B = 2*dmax labels and the running balance returns the global optimum in O(4^B m^2).
That turns one number into three -- exact, heuristic, kyfan -- and the two gaps
that were previously conflated into one become separately measurable.
duality_residual, cut_is_maximal and the two shift fields are the structural checks:
the same-fold mass is 1'Q1 - 2*cut, so the design is the MAXIMUM-weight balanced cut of
Q(x), and it is blind to both a uniform off-diagonal shift (every design moves by
c*m^2/4) and a diagonal one (the cut does not move at all).
OptimalExplorationCurve
dataclass
¶
Explore-exploit for causal control: excess(v) = Av + B/v is minimised at v = sqrt(B/A).
CeExplicitConstantCurve
dataclass
¶
C1 END-TO-END: the certified ball, the explicit constant, and how conservative both are.
ConjugateTimeCurve
dataclass
¶
The horizon past which every chc.regret constant ceases to exist, and how it diverges.
AdaptiveExplorationCurve
dataclass
¶
Adaptive exploration: a tapering schedule reaches the van-Trees sqrt(T) rate.
MinimaxExplorationCurve
dataclass
¶
The sequential minimax floor: no policy beats it, a front-loaded design attains it.
CappedExplorationPolicy
dataclass
¶
The optimal exploration schedule under a per-round action cap -- Result 56, extended.
For a constant cap the policy is two numbers: explore at cap for block_rounds rounds,
then stop. It is not an approximation -- the objective is convex and depends on the schedule
only through its prefix sums, so front-loading as hard as the box allows is globally optimal.
When the cap VARIES or a total budget is imposed, the invariant is not the block length but
exploration_mass: the first-order condition sees the cap only through the mass already
delivered, so the optimal mass is cap-free and the schedule decides only how many rounds it
takes to deliver it (validation/capped_exploration_schedule.mac).
VanTreesCurve
dataclass
¶
Van Trees: Bayes MSE hits the floor 1/(I_prior+n*I_data); confounding lifts the floor.
HInfRobustCurve
dataclass
¶
H-inf robustness = pessimistic contraction, calibratable to ~ the variance-optimal action.
ConstrainedRegretCurve
dataclass
¶
Constrained CE regret is piecewise-quadratic; frozen (zero) on the active side.
ConfoundedTurnpikeCurve
dataclass
¶
Confounded control converges to the wrong turnpike; discounted regret stays finite.
PartialIdControlCurve
dataclass
¶
Partial-ID control: worst-case regret ~ Delta^2; action sign robust iff Delta < |b|.
DoublyRobustCurve
dataclass
¶
The AIPW control effect is doubly robust: regret vanishes if either model is correct.
BanditCausalCurve
dataclass
¶
Scalar persistently-identified model: deconfounded O(log T) vs confounded Theta(T) regret.
FiniteHorizonPLCurve
dataclass
¶
PL self-certifying bound valid over a full finite horizon at mu = lambda_min(Hessian).
InterferenceConvexityCurve
dataclass
¶
Cannibalising interference raises the convexity, so the self-certifying PL bound tightens.
InterferenceOrthogonalCurve
dataclass
¶
Under interference you must debias BOTH channels: a half measure stays O(eps^2).
DynamicCausalCurve
dataclass
¶
Confounded tracking regret grows linearly with the horizon; causal control plateaus.
NonlinearRegretCurve
dataclass
¶
A strong-convexity bound stays valid globally where a linearised estimate under-states.
CausalControlCurve
dataclass
¶
Predictive control plateaus at a confounding floor; causal control reaches the oracle.
ConfoundingRobustLQRegretCurve
dataclass
¶
Evidence the confounding regret bound order-doubles effect error and floors at Delta^2.
MinimaxAction
dataclass
¶
The robust action, its worst case over the interval, and which candidate bound.
worst_case is the worst of whatever was minimised: the cost, or the regret.
MinimaxRegretMove
dataclass
¶
How much of a move to make where its value is known only to an interval around a noisy estimate, and what that costs at worst. Experimental.
See minimax_regret_move.
MinimaxLQPolicy
dataclass
¶
Robust finite-horizon feedback over a partially identified effect, beside its CE rival.
MinimaxLQCertificate
dataclass
¶
Evidence that the closed form is the minimiser, beats CE, and how much the DP relaxes.
ConfoundingRobustControlCurve
dataclass
¶
Evidence the pessimism radius shifts the gain and strictly beats CE under asymmetric loss.
ConfoundingRegretFloorCurve
dataclass
¶
Empirical certificate: control regret vs confounding bias, and its log-log slope (~2).
MarketplaceControlCurve
dataclass
¶
§35 robust controller vs CE on realised marketplace cost across a confounding grid.
DynamicConfoundingCurve
dataclass
¶
§35 robust vs CE on realised CLOSED-LOOP cost across a confounding grid (the dynamic §37).
SpaceTimeFoldCurve
dataclass
¶
Result 59: the two-axis fold problem -- its Kronecker rank, and the price of one axis.
PanelEstimatorGate
dataclass
¶
Result 60: Result 51's functional put against a real cross-fitted DML fit on a panel.
dlqr(a, b, q, r)
¶
Infinite-horizon discrete LQR: optimal gain K and cost-to-go P (via the DARE).
closed_loop_cost(a, b, k, q, r, x0)
¶
Infinite-horizon LQ cost of applying gain k to plant (a, b) from x0.
x0' P x0 where P solves the discrete Lyapunov equation for the closed loop; +inf if
k fails to stabilise (a, b) (a destabilising controller has unbounded cost).
certainty_equivalence_gap(a, b, q, r, a_hat, b_hat, x0)
¶
Suboptimality J(K_hat) - J* of the certainty-equivalent controller on the true plant.
K_hat is the LQR-optimal gain for the estimated model (a_hat, b_hat); the gap is its
true-plant cost minus the cost of the true-optimal gain. Zero iff the estimate induces the
optimal gain; otherwise non-negative (the optimum is optimal).
regret_scaling(a, b, q, r, x0, *, errors=_DEFAULT_ERRORS, n_samples=400, seed=0)
¶
Certificate that CE suboptimality is quadratic in model error (slope ~2 in the small limit).
At each target magnitude eps draws n_samples Gaussian model perturbations (dA, dB)
scaled by eps, records the certainty_equivalence_gap, and fits the log-log slope of
gap vs realised error over all samples. Theory predicts exponent 2 -- and by
proofs/ce_explicit_constants.v that is now an EXACT identity in the gain error rather than
Mania-Tu-Recht's local bound, which stays only as a comparison point.
TWO WAYS A DRAW HAS UNBOUNDED REGRET, and both are reported rather than silently dropped: the
perturbed model can be unstabilisable (no certainty-equivalent gain exists), or its gain can
fail to stabilise the TRUE plant (cost +inf). There E[R] does not exist at all, so the
exponent here is a quantity CONDITIONAL on its complement -- infinite_fraction is what
makes that visible. It shrinks to 0 as eps does; ce_explicit_constant_certificate gives
the explicit radius rho inside which it is 0 by construction, and a safeguarded policy
(fall back to a known stabilising gain when the checkable Lyapunov test fails) is what restores
an unconditional finite expectation.
orthogonal_control_certificate(*, b_true=2.0, alpha=1.0, gamma=1.5, target=1.0, effort=0.5, errors=(0.2, 0.1, 0.05, 0.025, 0.0125), n=400000, n_seeds=6, noise=0.05)
¶
The DOUBLE debiasing of orthogonal certainty-equivalence control -- a novel result derived in
validation/orthogonal_control.mac and proved in proofs/orthogonal_control.v.
A confounder z drives both the action (u = alpha z + noise) and the outcome
(y = b_true u + gamma z + noise). A controller hits target using its estimate of the
causal effect b = dy/du, scored by regret on the true plant. Under nuisance error eps,
a single-residualisation (non-orthogonal) effect is O(eps)-biased, so its control regret
is O(eps^2); the orthogonal Double ML effect is O(eps^2)-biased, so -- through the
same certainty-equivalence quadraticity -- its regret is O(eps^4). The estimator's
orthogonality compounds with the quadratic regret map: two debiasings, statistics and control.
Fits the two log-log slopes (~2 vs ~4).
pessimism_variance_certificate(*, b0=1.0, rr=1.0, target=1.0, variances=(0.05, 0.1, 0.2, 0.4), n=200000, rho_max=1.2, rho_points=61, seed=0)
¶
Certificate that pessimism belongs in the optimality condition, calibrated to the uncertainty
(derived in validation/pessimistic_optimality.mac, proved in
proofs/pessimistic_optimality.v). The estimated effect is b0; the true effect is
b0 + e with e of variance s^2. The pessimistic control target b0/(b0^2+rr+rho)
adds an effective-effort rho (distrust) to the stationarity condition. Sweeping rho per
s^2, the expected-regret-minimising rho* tracks s^2 (optimal pessimism = estimate
variance -- no tuning), and the pessimistic control's expected regret is below the
certainty-equivalent (rho = 0) one.
information_lower_bound_certificate(*, b=1.0, rr=0.5, xt=1.0, sigma=0.5, alpha=1.0, sample_sizes=(200, 500, 1000, 2000, 4000), n_seeds=300)
¶
Information-theoretic lower bound for UNBIASED effect estimators (Cramer-Rao;
derived in validation/information_lower_bound.mac, proved in
proofs/information_lower_bound.v). The EXACT regret map is
R = (b^2+rr)*(u*(bhat)-u*(b))^2 (nonlinear in b); to first order in the effect error this is
C*(bhat-b)^2, and Cramer-Rao on the (unbiased) estimator gives the local floor
E[regret] >= C*sigma^2/(n*V_id), V_id the identifying variance. The 1/n rate
matches the online upper bound (Result 7) in this scalar model; confounding reduces V_id
(V_exp -> residual V_conf), raising the floor.
RESULT 57 -- the van Trees arm, which removes two of the three scope caveats. The one-step LQ
regret is EXACTLY (b^2+rr)(u - u*(b))^2, a squared error in the ACTION with no
linearisation (validation/action_van_trees.mac STEP 1), so applying van Trees to the
estimand psi(b) = u*(b) -- an inequality that holds for EVERY estimator, biased or not --
and multiplying by that exact curvature gives
``n E[regret] >= n (b^2+rr) (E_lambda psi')^2 / (n V_id/sigma^2 + I(lambda))``
whose limit is C sigma^2/V_id: the SAME constant, with the unbiasedness hypothesis and the
delta method both deleted. The arm that can fail uses a HODGES estimator, which snaps to b
whenever the fit lands within n^(-1/4). It drives the regret at b to exactly zero --
hodges_pointwise_ratio = 0, infinitely below the unbiased Cramer-Rao floor, which is the
concrete reason that floor was never a minimax statement -- while its Bayes action error sits
~146x ABOVE the van Trees floor, the price superefficiency pays off-centre. The efficient
plug-in clears the same floor by ~2.8x, so the bound separates the two rather than being
vacuous for both.
The caveat that SURVIVES is the knife edge: at rr = b^2 the optimal action is stationary
in b, so psi' and with it the leading floor vanish. That is a property of the problem,
not of the inequality used.
multivariate_action_floor(*, effect, state_weight, action_weight, target, information, directions=None)
¶
RESULT 67 (A18) -- van Trees on the ACTION when the effect is a matrix.
Derived in validation/multivariate_van_trees.mac, proved in
proofs/multivariate_van_trees.v. Result 57 closed with "the multivariate case has the same
structure with psi' a Jacobian and the floor a trace"; the structure is the same and the
CONCLUSION is not, which is why this is a separate result rather than a generalisation.
The regret is an exact quadratic form in the action at any dimension,
``J(u,B) - J(u*(B),B) = (u - u*)' M (u - u*)``, ``M = B'QB + R``
(STEP 1b, residual 0), so E[regret] = tr(M Sigma) with Sigma the action-error
covariance, and trace against a PSD weight is monotone in the PSD order (STEP 3; Rocq
the_psd_order_transfers_to_the_regret). Feeding the matrix van Trees inequality
Sigma >= Psi' G^-1 Psi' through that gives
``E[regret] >= tr( M Psi' G^-1 Psi' )``, ``G = n I_data + I(prior)``
which at p = q = 1 is (b^2+rr) psi'(b)^2 / G -- Result 10's constant, unchanged
(STEP 4c, residual 0).
What the scalar model cannot express is in alignment. The floor splits over the
eigendirections of the information as sum_i (Psi' v_i)' M (Psi' v_i) / lambda_i, so
confounding -- which lowers lambda_i anisotropically -- is paid only in proportion to how
much the optimal action leans on the directions it damages. Its price is therefore a
Psi'-weighted mean of the per-direction factors, equal to the full factor only when the
damaged direction carries all the weight and exactly 1 at the other corner (STEP 6b). A
scalar plant has one direction, so it always sits at the first corner and reports
V_exp/V_conf.
directions is a (d, p, q) stack spanning the parameter family: the default is the
p*q unit bumps, i.e. theta = vec(B), and a restricted family (two decoupled channels
with known zero cross-effects, say) is supplied by giving its own basis. information is
G in that basis and must be positive definite -- a singular G is an unidentified
direction, for which the floor is not a number.
multivariate_van_trees_certificate(*, rr=0.5, gains=(1.0, 0.4), target=(1.0, 0.8), sigma=0.5, prior_width=0.25, n_probe=4000, information_loss=4.0, draws=40000, seed=11)
¶
RESULT 67 (A18) -- the multivariate van Trees floor, and what it says that Result 57 cannot.
Derived in validation/multivariate_van_trees.mac, proved in
proofs/multivariate_van_trees.v. Four things are measured, each able to fail.
-
The floor REDUCES correctly. At
p = q = 1it is Result 10'sC sigma^2/(n V), and on two decoupled channels it ish1 C(b1,x1) + h2 C(b2,x2)-- the per-channel scalar constants weighted by the information in each direction (STEP 7c). Both are exact identities, so the gate is at machine precision, and the hand-derivedPsi'is checked against a central finite difference rather than trusted. -
The bound BINDS and is not beatable. A Hodges estimator, which snaps to the prior mean whenever the fit lands within
n^(-1/4), drives the pointwise action error to essentially zero against the unbiased Cramer-Rao floor -- the same demonstration as Result 57(c), now in two dimensions -- while sitting far above the van Trees floor, which assumes nothing about bias. The efficient plug-in clears the same floor by a modest factor. -
The price of confounding is an ALIGNMENT, not a ratio. Cutting the information along ONE direction by
information_lossraises the floor by1 + (k-1) a_w/sum(a)witha_w = (Psi' w)' M (Psi' w): full price only if that direction carries all the weight. Two arms with the SAMEkbracket it -- the direction the optimal action leans on most, and a direction in the kernel ofPsi', which costs exactly nothing. A scalar plant has one direction and therefore always reportsk; that is the whole of Result 10'sV_exp/V_conf. -
The knife edge becomes a WEIGHT. At
rr = b^2Result 57(d)'s sensitivity vanishes, so that channel contributes zero to the floor -- and confounding it is free. In the scalar model this is a measure-zero curiosity; here it is one entry of a vector that a real plant can sit near without anything else degenerating.
highprob_regret_certificate(*, b=1.0, rr=0.5, xt=1.0, sigma=0.5, n=500, v_exp=2.0, v_conf=1.0, deltas=(0.5, 0.25, 0.1, 0.05, 0.01), n_trials=4000)
¶
Finite-sample high-probability UPPER bound on scalar CE regret (derived in
validation/highprob_regret.mac, proved in proofs/highprob_regret.v). NOTE: this is NOT a
"high-probability version" of the Result-10 Cramer-Rao bound -- CR is an expected LOWER bound
on estimator variance (hence a regret floor), this is a high-probability UPPER bound on
realised regret; the two point in opposite directions and are complementary. The effect
estimator concentrates (sub-Gaussian): |b_hat-b| <= r(delta) w.p. >= 1-delta,
r(delta)^2 = 2*sigma^2*log(2/delta)/(n*V_id). Composing with the LOCAL quadratic
regret = C*(b_hat-b)^2 gives, w.p. >= 1-delta, regret <= 2*log(2/delta) * band-scale
-- a log(1/delta) multiple of C*sigma^2/(n*V_id); confounding (smaller V_id) widens
it. A fully rigorous upper bound replaces C by the Lipschitz constant of u* on the
confidence region (regret = (b^2+rr)(u*(b_hat)-u*(b))^2 <= (b^2+rr) L^2 r(delta)^2); locally
L^2 -> C. Concentration checked by Monte-Carlo coverage; the implication proved in Rocq.
transportability_regret_certificate(*, b_src=1.0, rr=0.5, xt=1.0, lip=1.0, d_lo=0.005, d_hi=0.15, n_d=12, wdro_radius=0.15)
¶
Transportability regret -- a controller trained on source domain P, deployed on target P'
(Bareinboim, Causal AI, Ch 9). CONCEPTUAL proposition (not a self-contained theorem): the bounds
below hold UNDER the stated assumption, they do not follow from Kantorovich-Rubinstein alone.
Derived in validation/transportability_regret.mac, proved in
proofs/transportability_regret.v. Deployment regret C*(b_src-b_tgt)^2 splits into: (A) a
TRANSPORTABLE part -- if the effect is recoverable on target (b_tgt=b_src) regret is ZERO
at ANY distributional distance; (B) a non-transportable residual C*Lip^2*d^2 -- this
REQUIRES the identified effect functional b(.) to admit a Lipschitz representation
(e.g. b(P)=E_P[phi], phi Lipschitz), so |b(P')-b(P)| <= Lip*W1(P,P'); then the
regret is quadratic in d = W1(P,P'), the distance chc.uncertainty.WassersteinPenalty
penalises; (C) a W-DRO controller for radius eps covers this residual when d <= eps --
conditional on the DRO ambiguity set being the W1-ball over the law that b is Lipschitz in.
The robust radius is a transportability budget.
ensemble_control_certificate(*, b0=1.0, rr=0.5, xt=1.0, n_ctx=21, spread_lo=0.02, spread_hi=0.5, n_spread=12)
¶
The ENSEMBLE (heterogeneity) control regret floor -- one control over a heterogeneous causal
population (Li & Khaneja 2006 ensemble control; derived in validation/ensemble_control.mac,
proved in proofs/ensemble_control.v). When the effect b varies across contexts (CATE
heterogeneity), a single control u serves all. Even with perfect per-context knowledge, one
control pays an irreducible regret = the curvature-weighted VARIANCE of the per-context optimal
actions, R(u_ens) = W*Var_w(u*), quadratic in the heterogeneity and zero for a homogeneous
population. The ensemble-optimal control is the curvature-weighted mean of per-context optima;
it WEAKLY beats the naive u*(mean effect) control (they coincide when curvature is uniform
across contexts), and strictly beats it under heterogeneous curvature.
composition_transfer_certificate(*, b=1.0, rr=0.5, xt=1.0, orders=(1, 2, 3), delta_lo=0.01, delta_hi=0.2, n_delta=12)
¶
The general orthogonal-to-control order-transfer LEMMA -- the general form of Result 0 (which
only stated p=1 and p=2). Derived in validation/composition_transfer.mac, proved in
proofs/composition_transfer.v. If the effect estimator has error of order delta^p (p
the orthogonality order: 1 plug-in, 2 Neyman-orthogonal/DML, higher for higher-order), the
certainty-equivalence control regret is order delta^(2p) -- the control map DOUBLES the
estimator's order for EVERY p, because regret is quadratic in the ACTION error (exact map
(b^2+rr)*(u*(bhat)-u*(b))^2, not a linearisation) and u* is Lipschitz in the effect.
HONEST SCOPE: an immediate COMPOSITION (Lipschitz action-map + quadratic regret), not a deep
theorem -- the p->2p arithmetic restates known facts (4th-order-under-orthogonality:
Foster-Syrgkanis Orthogonal Statistical Learning; quadratic LQR CE gap: Mania-Tu-Recht). Its
value is unifying them for control. The END-TO-END statement this left open -- cross-fit causal
estimator -> dynamics-error rate -> stabilising controller -> finite-sample control regret with
explicit constants -- is closed by ce_explicit_constant_certificate, which computes both the
stabilising ball and the constant. Plug-in (p=1 -> 2) and DML (p=2 -> 4) are instances.
slopes IS NOT expected_slopes, AND THE DIFFERENCE IS NOT NOISE. At the default window
the fit returns 2.048478 / 4.012425 / 6.002273 against 2 / 4 / 6, and that excess is a
deterministic property of the window, not evidence of agreement-up-to-sampling -- there is no
sampling here at all. Writing e = delta^p and t = log(delta),
log R = const + 2 p t + lam e + 2 c2 e^2 + O(e^3), so an ordinary least-squares fit reports
``slope = 2 p + lam cov(t, e^(p t))/var(t) + 2 c2 cov(t, e^(2 p t))/var(t) + O(e^3)``
with lam = u*''(b)/u*'(b) = (2 b^3 - 6 b rr)/(rr^2 - b^4) and
c2 = (b^6 - 8 b^4 rr + 5 b^2 rr^2 - 2 rr^3)/(2 (rr^2 - b^4)^2) -- derived in
validation/order_transfer_window.mac and cross-checked in giac. Those two terms carry
4.09e-2 of the 4.85e-2 excess at p = 1; what is left is one more power of delta.
SHRINKING THE WINDOW DOES NOT MAKE THE READING BETTER INDEFINITELY. The slope is bracketed from
below by double precision: u*(b + e) - u*(b) is a cancellation, so the regret's relative
error is about 4 eps |u*(b)| / (|u*'(b)| delta_lo^p), which GROWS as the window drops. At
delta in [1e-5, 2e-4] the p = 3 slope reads 6.035, and the same fit at 60 digits
reproduces the two-term prediction to 4.3e-25. A window has to clear both ends;
causaldyn_bench.paper_one Table 1 prints them side by side and names which one binds.
multivariate_transfer_certificate(*, delta_lo=0.005, delta_hi=0.1, n_delta=10)
¶
The transfer theorem (Result 18 / Contribution 1) in the MULTIVARIATE, DYNAMIC LQ setting --
addressing the "needs multivariate/dynamic" gap. On a stable 2-state/1-input LQ plant the LOCAL
quadratic certainty-equivalence suboptimality (Mania, Tu & Recht 2019, small-error/stabilising;
certainty_equivalence_gap) is quadratic in the effect-matrix (input matrix B) error;
composing an order-p estimator (||dB|| ~ delta^p) gives regret ~ delta^(2p) -- the
SAME order-doubling as scalar Result 18, now for matrices. Rocq core shared (regret_order_2p
is abstract in the error norm). This lifts the order-transfer to matrices; it is NOT the full
finite-sample end-to-end theorem (which additionally needs nuisance rates + a stability margin).
multichannel_control_certificate(*, b_d=1.0, b_s=0.6, rr=0.5, xt=1.0, alpha_u=1.0, alpha_g=0.8, gamma=1.0, n_clusters=40, cluster_size=10, tau=0.5, noise=0.5, deltas=(0.02, 0.04, 0.07, 0.12, 0.2, 0.3), cluster_grid=(10, 20, 40, 80, 160), n_seeds=60)
¶
CONTRIBUTION 2 -- MULTI-CHANNEL orthogonal control on a network (the serious form of 4x0;
derived in validation/multichannel_control.mac; proved: multichannel_control.v).
The control-relevant total effect of a uniform action is B = b_d + b_s (direct + spillover):
two interference channels, each with its own Neyman-orthogonal moment. The NOVELTY is the
downstream control-regret consequence of incomplete orthogonalisation (DML under interference
is not itself new -- Munro-Xu-Wager, Wager-Xu): orthogonalising ONLY the direct channel caps
the control regret at O(delta^2) (the un-orthogonalised spillover bottleneck), while
orthogonalising BOTH reaches O(delta^4) -- regret order is 2*min over channel orders.
ONLY this order-bottleneck is proved in Rocq; the per-channel cross-fit RATES that would make
||b_j-b_j hat|| ~ delta^{p_j} rigorous are demonstrated EMPIRICALLY here, not proved.
Cluster-robust: with roughly balanced, weakly-dependent clusters the effective sample size is
the number of clusters G, so the total effect concentrates at 1/sqrt(G); this depends
on the cluster count/size/balance and cross-cluster independence, and is not a universal rate.
multivariate_interference_certificate(*, delta_lo=0.01, delta_hi=0.15, n_delta=10)
¶
Contribution 2 in the MULTIVARIATE, DYNAMIC LQ setting -- composing multi-channel bottleneck
(Result 19) with the multivariate LQ gap (Result 21). The total effect is an input MATRIX
B = B_d + B_s (two interference channels, each a direction of B). Using the LOCAL
quadratic LQ suboptimality (Mania, Tu & Recht 2019, small-error + stabilising;
certainty_equivalence_gap), quadratic in the total input-matrix error: orthogonalising ONLY
direct (spillover plug-in, O(delta)) caps LQ regret at O(delta^2) -- spillover is the
bottleneck; orthogonalising BOTH (each O(delta^2)) reaches O(delta^4). The scalar
order-composition is proved in proofs/multichannel_control.v; shown numerically here in
the multivariate LQ regret. The network cross-fit estimator rates that would make
||dB_j|| ~ delta^{p_j} rigorous are NOT proved.
end_to_end_c2_certificate(*, b_d=1.0, b_s=0.6, alpha_u=1.0, alpha_g=0.8, gamma=1.0, cluster_size=10, tau=0.5, noise=0.5, delta_small=0.002, g_grid=(10, 20, 40, 80, 160), deltas=(0.02, 0.04, 0.07, 0.12, 0.2), n_seeds=60)
¶
CONTRIBUTION 2, END-TO-END: multichannel causal estimation -> bottleneck rate -> dynamic
control regret on a clustered network with a multivariate LQ plant (derived in
validation/c2_end_to_end.mac, control-side reduction proved in proofs/c2_end_to_end.v).
The cross-fit two-channel Robinson-DML total-effect error is ||B_hat - B|| = O_p(G^{-1/2} +
delta_d^{p_d} + delta_s^{p_s}) (cluster sampling + two nuisance remainders); through the MTR
local quadratic gap the regret is R = O_p[G^{-1} + (delta_d^{p_d} + delta_s^{p_s})^2]. Two
regimes are shown:
(1) a G-sweep with REAL cross-fit DML at tiny delta (sampling-dominated, R ~ 1/G);
(2) a deterministic delta-sweep (half-orth ~ delta^2, full-orth ~ delta^4).
Rocq proves the composition (the s + e_d + e_s reduction); the per-channel cluster cross-fit
RATES are statistical assumptions (Chernozhukov; Robinson; Hays & Raghavan), not proved.
clustered_lower_bound_certificate(*, b_d=1.0, b_s=0.6, alpha_u=1.0, alpha_g=0.8, gamma=1.0, cluster_size=10, tau=0.5, noise=0.5, delta_small=0.002, g_grid=(20, 40, 80, 160, 320), n_seeds=80)
¶
CONTRIBUTION 2, the LOWER bound (proofs/clustered_van_trees.v): the 1/G sampling regret
is IRREDUCIBLE, not just an upper bound. By clustered van Trees the effective Fisher info is
I0 + G*Ic, so E[(Bhat-B)^2] >= 1/(I0+G*Ic); composing with the lower-Lipschitz regret
map R >= kappa0*(Bhat-B)^2 gives G*E[R] >= kappa0/(I0+Ic) > 0 for all G, increasing
toward kappa0/Ic (Maxima limit). This certificate confirms it: with REAL cross-fit DML (tiny
delta, sampling-dominated), G*regret is roughly CONSTANT and bounded below by a positive
c0 -- so regret ~ 1/G on BOTH sides (tight rate), not o(1/G). Contrast
end_to_end_c2_certificate, whose G-sweep showed only the ~1/G upper trend.
cross_cluster_mixing_certificate(*, b_d=1.0, b_s=0.6, alpha_u=1.0, alpha_g=0.8, gamma=1.0, cluster_size=10, leak=1.0, tau=0.5, noise=0.5, delta_small=0.002, decays=(0.0, 0.3, 0.5, 0.7, 0.85), blocked_arm=4, g_grid=(80, 160, 320, 640), n_seeds=100, n_replicates=4, seed=0)
¶
ASSUMPTION A3' -- Contribution 2's cluster rate does NOT need independent clusters.
A1 as stated assumes partial interference with INDEPENDENT clusters (Hudgens & Halloran 2008),
which is the assumption a marketplace violates: adjacent cities share drivers, weather and
campaigns. A3' weakens it to psi-dependence in the sense of Kojevnikov, Marmer & Song
(2021, Definition 2.2) -- the cluster scores may correlate across clusters provided the
dependence coefficients theta_s, defined by
|Cov(f(s_A), g(s_B) | C)| <= psi_{a,b}(f, g) * theta_s for cluster sets at distance
s > 0, decay fast enough for their Condition ND -- which is the same regime as Leung's
(2022) approximate neighbourhood interference. Under A3' the CLT is the network-HAC one rather
than the i.i.d.-cluster one: the G^{-1/2} rate survives and only the variance CONSTANT
changes, by the HAC sum. A3' is not free -- the KMS CLT wants p > 4 moments where A7 asks
only for q > 2.
Both halves are measured here, because "the rate survives" alone is not falsifiable:
- Each arm draws two independent AR(1)-across-cluster fields with decay
rho, one entering the spillover regressor and one the outcome error, so the score inheritsCov(psi_g, psi_h) ~ rho^(2|g-h|)-- summable for everyrho < 1.rho = 0reduces exactly to the independent-cluster plant ofmultichannel_control_certificate. Reusing ONE field for both would instead make it an omitted confounder -- and measured, that design flattens the slopes to-0.001 .. +0.027, which is INDISTINGUISHABLE from the non-summable arm below on the slope alone.worst_mean_erroris the only thing that separates them (0.0138against0.3377), which is why it is reported rather than assumed away. - The CONSTANT is predicted, not just observed: with the bread free of
rho(the field is unit variance whatever its correlation length) the meat is affine in the HAC sum, soG*MSE = a + b*2 rho^2/(1 - rho^2)witha, bfree ofrho.hac_r_squaredis that fit; a value below ~0.95 falsifies the HAC form rather than the rate. blocked_armis the falsification: a FIXED number of super-blocks, sopsi(d)never decays,sum_d psi(d) = Theta(G)and A3' fails. Its slope must be ~0 -- the effective sample size is the block count, notG. Without this arm a flat-1/2would be evidence of nothing, since it is also what an arm with no dependence at all reports.
g_grid starts at 80 deliberately. Result 70: a fitted exponent is a WINDOW measurement,
and at rho = 0.9 the field's correlation length is ~9 clusters, so a window opening at
G = 40 reports ~-0.35 -- the pre-asymptotic regime, not the rate. The replicate blocks
are genuinely independent -- the seed stream carries a replicate index -- so slope_errors
is a real s.e. rather than the nested-prefix reconstruction
end_to_end_c2_certificate is forced into. With five blocks it has four degrees of
freedom, so it is a t and not a z: the two-sided 5% critical value is 2.776.
exposure_map_certificate(*, delta_lo=0.01, delta_hi=0.15, n_delta=10)
¶
CONTRIBUTION 2, exposure-map generalisation (derived in validation/exposure_map_c2.mac,
proved in proofs/exposure_map_c2.v). The marketplace network plant
x_{t+1} = A x_t + (B_d + B_s W) u_t + eps has effective effect B_eff(W) = B_d + B_s W;
estimating (B_d, B_s, W) gives THREE channels r_d, r_s, r_W (the last from the exposure
map W). Through the MTR gap, R = O_p[G^{-1} + (r_d + r_s + r_W)^2]. Shown
deterministically on a 3-state/1-input LQ plant (three orthogonal input-matrix directions):
orthogonalising ALL three (each O(delta^2)) gives ~ delta^4; leaving the exposure map
W plug-in (r_W ~ delta) makes it the bottleneck, ~ delta^2 -- the load-bearing
point (Hays & Raghavan) that r_W enters squared only if W is orthogonalised/cross-fit.
cluster_fold_leakage_certificate(*, n_clusters=60, cluster_size=8, noise=0.5, b_d=1.0, b_s=0.6, icc_grid=(0.0, 0.3, 0.6, 0.9), lin_cluster_grid=(20, 80, 320), n_seeds=240)
¶
CONTRIBUTION 2 -- the price of violating assumption A8 (derived in
validation/cluster_fold_leakage.mac, proved in proofs/cluster_fold_leakage.v).
A8 asks for K >= 2 folds of WHOLE CLUSTERS. Splitting folds by row instead leaves every
cluster in both folds, so the nuisance is fit on rows of the very cluster it must then predict.
None of the cited theorems prices that: CCDDHNR assume i.i.d. rows (folds are then automatically
valid), Hansen-Lee assume independent cluster scores as a PRIMITIVE, Chiang-Kato-Ma-Sasaki
assume folds are already cluster-level.
The finding is NOT a bias. The cross-fold hat reproduces the design it is applied to, so
f(Z) cancels identically and the moment stays exactly orthogonal for ANY fold assignment
(ARM 1 and the aware_bias_* fields). The damage is a change of ESTIMATOR, visible only in
the second moment: with a cluster-aware nuisance, row folds estimate each cluster's own effect
from its own training rows and subtract it, so the residual ICC is annihilated
(resid_icc_row -> 0 against resid_icc_cluster -> rho_ICC) and a within-cluster estimator
is silently substituted for the cluster-robust one. Its variance obeys
psi = c(m,K)*(1 - rho_ICC) with c(m,2) = m*(m+14)/(m+2)^2 from the projector algebra, so
the NORMALISED law psi/c = 1 - rho_ICC is exact in m and K rather than asymptotic.
Everything derived from the cluster-robust variance -- interval width, the c0 plateau of
Result 25, the regret constant -- then belongs to a different estimator, and a sandwich computed
after row folds is too SMALL, so intervals under-cover.
The one regime where it is NOT merely a variance question is a cluster-measurable regressor
(ARM 3). A partial-interference exposure is cluster-measurable by construction, and a
cluster-aware nuisance can then reproduce it exactly from the held-out row's own cluster: the
channel is annihilated (exposure_resid_var_row -> 0) and the total-effect estimate loses
exactly the coefficient that channel carried, annihilated_bias_row -> -b_s. That is
unidentification, not a broken moment -- validation/cluster_fold_leakage.mac STEP 9 with
h -> 1 gives Dtil = 0 -- but it is a bias in the reported number.
SCOPE: for a fixed correctly-specified linear span the leak is O(1/G) and the published C2
conclusions survive (ARM 1) -- what does not survive is the claim that the code implements A8.
The C2 certificates' own G_exp varies i.i.d. WITHIN cluster, which is not what a
partial-interference exposure looks like, so they sit in the benign regime by accident.
exact_ratio_moment(numerator, denominator, regressor_cov)
¶
E[(R'BR) / (R'CR)^2] for R ~ N(0, Omega), exactly -- Result 51 (l) and (m).
The variance of the scalar cross-fit estimator theta_hat = u'A eps / u'A u with a
correlated regressor u is E[u'(A Sigma A)u / (u'Au)^2]: call with
numerator = A Sigma A and denominator = A. The plug-in law tr(B Om)/tr(C Om)^2
mis-prices that expectation by the expectation-of-a-ratio gap -- +10.7% / +6.7% on the
panel geometries of Result 51 (i), enough to move the fold-design crossover and open a
wrong-partition band at Om != I (Result 51 (m)). This is the exact replacement: with
1/Y^2 = int_0^inf t e^{-tY} dt and the tilted-Gaussian moment, the expectation is one
resolvent quadrature (Magnus 1986)::
E[X/Y^2] = int_0^inf t det(I + 2t Om C)^(-1/2) tr(B (I + 2t Om C)^(-1) Om) dt
-- an eigh plus scipy.integrate.quad. Both identities are verified, not cited, in
validation/omega_jensen_gap.mac (residual 0 at n = 1, 2); the design consequences --
the Om = I crossover is exact and sign-safe, the Om != I one is not -- are proved in
proofs/omega_jensen_gap.v.
Scope. Exact for GAUSSIAN R. Scale mixtures R = sqrt(nu/W) G (multivariate t) rescale
the moment by E[W/nu] -- equal to 1 for t_nu at a fixed SCALE matrix, so heavy tails
per se change nothing here, while parameterising by the variance instead of the scale silently
costs -2/nu. Existence is a tail condition, checked rather than assumed: with k nonzero
eigenvalues of Om^(1/2) C Om^(1/2) the integrand decays like t^(-k/2) when the
numerator carries no mass on the denominator's null space (finite iff k >= 3) and like
t^(1-k/2) when it does (k >= 5) -- the plug-in number exists at every k, which is
exactly how a truncation gets quoted where the moment it approximates does not.
exact_matrix_ratio_moment(numerator, denominator, regressor_cov, nodes=None)
¶
E[(X'CX)^-1 (X'BX) (X'CX)^-1] for a Gaussian n-by-q block X, exactly.
The matrix companion of exact_ratio_moment: the exact sampling covariance of a
q-CHANNEL cross-fit estimate is E[(X'AX)^-1 X'A Sigma A X (X'AX)^-1] -- call with
numerator = A Sigma A, denominator = A, and regressor_cov the qn-by-qn
covariance of vec(X). The channel count is read off that shape, not passed: q =
regressor_cov.shape[0] // n. regressor_cov may be SINGULAR -- a spillover column that
is deterministic given the own column is the normal case, and nothing here inverts it.
Route (derived, not cited, in validation/general_q_ratio_moment.mac): M^-1 =
adj(M)/det(M) turns each entry into a signed sum of products of m = 2q - 1 quadratic
forms, since the adjugate has degree q - 1; the Ingham--Siegel identity det(M)^-s =
Gamma_q(s)^-1 int_{T>0} det(T)^(s-(q+1)/2) etr(-TM) dT at s = 2 replaces the
determinant by a Gaussian tilt; the tilted law has covariance (I + 2 Om (T (x) C))^-1 Om;
and the m-form Gaussian moment is a sum over the SYMMETRIC GROUP,
sum_{sigma in S_m} 2^(m - c(sigma)) prod_{cycles} tr(...), which at q = 2 is
Isserlis' 15 pairings and at q = 3 is 120 terms. The cone integral runs in Cholesky
coordinates (which cannot leave the cone -- proofs/matrix_ratio_moment.v) on a tensor
Gauss--Legendre grid of dimension q(q+1)/2.
Scope, both halves measured rather than assumed:
Gamma_q(2)is finite iff2 > (q-1)/2, so the ROUTE is valid exactly forq <= 4(proofs/general_q_ratio_moment.v,ingham_siegel_valid_iff_q_le_four).- The IMPLEMENTATION stops at
q = 3:q = 4needs7! = 5040permutations on each of 5760 products, i.e. 29 million plan terms, which is not a table worth building. - Cost is the cone dimension, not the algebra, and at
q = 3the cone wins. Atq = 2the cone is 3-dimensional and on exchangeable channels what governs the accuracy is the MARGIN from the existence boundaryn >= q + 2, not the shape ofB. Relative max-entry error againstI/(n-q-1), atnodes = 20, 40, 80, 120:
== ====== ======== ======== ======== ======== n margin 20 40 80 120 == ====== ======== ======== ======== ======== 5 1 1.14e-4 7.34e-6 4.68e-7 9.32e-8 6 2 2.81e-6 4.59e-8 7.37e-10 6.54e-11 8 4 2.16e-7 3.35e-11 3.08e-14 2.46e-14 12 8 4.90e-6 4.23e-13 1.25e-15 2.37e-14 == ====== ======== ======== ======== ========
So "machine precision at q = 2" holds from n = q + 4 outwards and not near the
boundary. An anisotropic B changes nothing: at n = 6 it gives 2.81e-6, 4.59e-8,
4.10e-9 -- the same numbers as B = I -- and at nodes = 40 its error over the
exchangeable one is 1.00 at margins 1, 2, 4 and 8. Correlated channels are NOT a constant.
With rho = 0.5 the factor over the exchangeable error is flat at small margins -- 4.1 and
4.2 at margin 1, 11.2 and 11.7 at margin 2 (nodes = 20 and 40; 3.14e-5, 5.38e-7,
4.84e-8 at n = 6) -- but 10.9 against 128 at margin 4 and 15.9 against 202 at margin 8:
past margin two the correlated rule also converges more slowly. Earlier drafts attributed
the loss of machine precision first to an anisotropic numerator (withdrawn) and then to
correlated channels, withdrawn in turn as "a flat factor of about 11", which holds only up
to margin 2. q = 3 is 6-dimensional and is a percent-accuracy tool, not a high-precision
one. Measured on the exchangeable anchor, taking the grid as far as it will go:
===== ========= ================== ==================
nodes points abs err, n = 5 abs err, n = 7
===== ========= ================== ==================
4 4 096 2.92e-1 9.55e-2
5 15 625 4.68e-2 5.67e-2
6 46 656 8.66e-2 1.87e-2
7 117 649 6.69e-2 1.12e-2
8 262 144 4.64e-2 6.33e-3
9 531 441 2.76e-2 2.04e-3
===== ========= ================== ==================
That is 1.6 digits for 531 441 points on the boundary, 2.2 at n = 7. At the
geometric-mean rates over nodes = 4..9 (1.60 per node at n = 5, 2.16 at n = 7),
six digits would need 31 and 21 nodes, i.e. 9e8 and 9e7 points. Six digits at q = 3 is
not reachable with this quadrature -- not "expensive", unreachable. Note also that
nodes = 8 at n = 5 evaluates 17x the points of nodes = 5 and is no more accurate
(4.64e-2 against 4.68e-2), which is why the q = 3 default is 6. Wall-clock figures are
deliberately absent: the same cell measured 3218 s and 1499 s on this machine depending on
what else was running, while the accuracy column is bit-identical across those runs.
Convergence is monotone only with margin from the existence boundary: at n = q + 2 the
sequence goes 4.68e-2, 8.66e-2, 6.69e-2, 4.64e-2, 2.76e-2, while at n = q + 4 it
decreases throughout.
The column above is the ABSOLUTE max-entry error. At n = 5 the exact answer is I
so absolute and relative coincide; at n = 7 it is I/3 and the relative error is 3x
the column -- 2.87e-1, 1.70e-1, 5.59e-2, 3.36e-2, 1.90e-2, 6.13e-3 at nodes = 4..9.
Quoting the two
columns against each other, or against a relative number, is the mistake this note exists
to prevent.
Every anchor above also has regressor_cov = I, and there is one with correlated
channels: for numerator = denominator = I and regressor_cov = kron(R, I_n) the
sandwich is E[(X'X)^-1] with X'X ~ Wishart_q(n, R), so the exact answer is
R^-1/(n-q-1) (Result 63 (i)). Measured in one convention (max entry, relative) at
q = 3, n = 7, correlated against exchangeable: 4.71e-1 / 2.87e-1, 2.39e-1 / 1.70e-1,
3.52e-2 / 5.59e-2, 4.29e-2 / 3.36e-2 at nodes = 4..7 -- 1.6x and 1.4x worse on the two
coarse grids, 0.63x on the third, 1.3x on the fourth, and non-monotone on both sides. A
correlated Om is not systematically harder; it is a different cell of the same
percent-scale tool.
Seven ways out were measured and all seven lost. Four rules -- a Smolyak sparse grid on
nested open Fejer-2 (the tail is algebraic, not Gaussian), per-axis Cholesky scaling, Aitken
extrapolation (Gauss-Legendre beats geometric, so a fixed-ratio model over-corrects), and
whitening the cone by E[M]^-1/2. Three mechanisms that would have been fixable were
ruled out by measurement: ill-conditioning (cond(tilt) <= 1038 over the whole grid),
a peak the grid straddles (the top node carries under 1.6% of the mass, and 90% of it needs
thousands of nodes), and a truncated tail (enlarging the domain strictly hurts: 4.7e-2 ->
7.8e-2 -> 3.1e-1 -> 5.7 at 1x, 2x, 5x, 10x the probe scale). What is left is ordinary slow
convergence in six dimensions, with no lever attached.
At q = 3 use matrix_ratio_certificate instead of this function: it returns the
same value with a grid-refinement error estimate, since nothing in the array itself reveals
that the grid is not converged.
Accuracy is self-certifying where it matters: on an EXCHANGEABLE problem the exact answer is
isotropic, so the observed spread of the diagonal lower-bounds twice the largest entry error
with no reference value in hand (half_spread_bounds_the_max_error). The converse fails --
a grid can be isotropic and uniformly wrong -- so a small spread is a necessary condition,
not a certificate (zero_spread_does_not_certify).
Anchors: the Ingham--Siegel constant to 8 digits and Wishart E[W^-1] = I/(n-q-1) to 6 at
q = 2; at q = 3 the same Wishart anchor plus the matrix-beta law
E[M^-1 X'PX M^-1] = (r/n) I/(n-q-1) for a rank-r projection P. Existence is the
inverse-Wishart threshold, enforced: per-channel effective rank at least q + 2, below
which the integral diverges while a plug-in sandwich still quotes a number.
matrix_ratio_certificate(numerator, denominator, regressor_cov, nodes=None, tolerance=0.01)
¶
exact_matrix_ratio_moment with a quadrature-error estimate attached.
Same arguments and same value; the extra cost is one coarser grid, which at q = 3 is
(nodes-1)^6 / nodes^6 -- about 26% at nodes = 5. Use this rather than the bare value
whenever q = 3, where the default grid is not converged and the array alone says nothing
about that. The refinement residual works on ANY problem, including the anisotropic ones where
the isotropy bar does not apply; the isotropy bar is free but only valid when the channels are
exchangeable. Either bar over tolerance makes MatrixRatioAccuracy.status
convicted, so adding the second can only tighten the verdict; neither can certify it
(MatrixRatioStatus).
optimal_fold_partition(shells, gammas, phi, lag=1, k_folds=2, time_axis=None, exhaustive_limit=20000, banded_exact=True, restarts=8, seed=0)
¶
Variance-optimal balanced fold partition under delayed network interference -- Result 52.
The fold partition enters the cross-fit sandwich only through the same-fold weighted 2-walk
count with positive weight r^4 - 1 (validation/fold_spectrum_law.mac,
proofs/fold_spectrum_law.v), so the design problem is to MINIMISE that mass over balanced
k_folds-partitions of the graph weighted by Q(x) = sum_{d,e} g_d g_e x^|d-e| S_d S_e,
which -- since the mass is 1'Q1 - 2*cut with 1'Q1 partition-free -- is the
MAXIMUM-weight balanced cut of that graph and not the minimum-weight one (Result 61),
x = phi**lag. On a cycle with ONE spillover shell the answer is proved over all labellings:
width-2 stripes below x = g1/g0 and alternating folds above it. With two shells it depends
on gammas, not only on x: at (1, 0.7, 0.4) stripes win on [0, 7/10] and
[7/8, 1] and alternation between (every other periodic pattern is a de Bruijn cycle whose
mean cost is at least 9/1600 higher), but at g2 = 0.6 the period-8 pattern
00101101 wins below x ~ 0.818 and alternation never does. What does not change is that
the contiguous blocks delayed_network_certificate compares are never optimal on the
cycles certified (8 <= m <= 14, one and two shells); they cost up to +55% of Psi at
small x.
Three routes, and route says which one ran. Small problems (k_folds = 2 and at most
exhaustive_limit balanced bisections) are ENUMERATED. Past that, a circulant weight
matrix with circular bandwidth B < m/2 -- which is what a cycle with spillover truncation
dmax gives, B = 2*dmax -- is solved EXACTLY by _exact_banded_bisection, an
O(4^B m^2) dynamic program that does not care about m: 1 second at m = 240, where
enumeration would need C(239, 119) ~ 1e70 evaluations. Only when neither applies does the
Fiedler-style spectral rounding plus balanced-swap local search run from restarts starts;
banded_exact=False forces that path so the search can be PRICED against the exact answer
(fold_exactness_certificate does exactly this). The certificate is then the Ky Fan bound
(m/K) (1'Q1/m + sum of the K-1 smallest eigenvalues of Q on the mean-free subspace)
(Donath-Hoffman 1973), which every balanced partition obeys, tightened by the diagonal shift
of Boppana (1987): sum(u) plus the same bound on Q - diag(u), maximised over u
(_shifted_kyfan_bound). lower_bound is the shifted bound, kyfan_bound the
plain one. On a cycle they are equal. Off it the shift can close most of the bound's
looseness: on the 16-unit path at x = 0.01 from 4.40% to 0.34%, on the 80-unit
path the search's certified gap from 5.82% to 1.31%, and on a star to zero within
1e-8. It can also close little: 3.31% to 2.95% on the 16-unit path at x = 0.6.
A large gap on the local-search path is an honest "could not certify", not a silent success.
time_axis opts into the TWO-AXIS problem (Result 59). Left at None the design set is the
m units and nothing changes. Given the panel length p it becomes the m*p
(unit, time) cells, ordered as kron(space, time), and the weight becomes the panel
covariance Sigma = sum_q P_q (x) T_q, [T_q]_{ts} = phi**|(t-s) - lag*q|: a sum of
at most 2*dmax Kronecker terms -- set by the SPILLOVER TRUNCATION, never by the diameter
or by p -- and of exactly dmax+1 when the shell operators commute, which every
vertex-transitive graph gives.
Cutting the time axis is not free, and its price is a function of the delay. Measured on a
cycle (m=8, p=4, K=2) against the best time-constant fold: 0.0-1.2% of the
objective at lag=1, 1.5-4.3% at lag=2, 12.9-19.2% at lag=4. The reason is
exact: once lag*q passes the panel's longest lag p-1, every entry of T_q is bounded
by phi**(lag*q - (p-1)) and the design has almost no reason to keep those pairs together.
One axis at a time is a defensible scope at lag=1 and an expensive one beyond it.
The objective is the PLUG-IN law's. Near a design boundary the exact moment shifts thresholds
(Result 52 (e)); rank the shortlisted designs with exact_ratio_moment when it matters.
fold_exactness_certificate(sizes=(24, 60, 120), decays=(0.01, 0.3, 0.9), dmax=2, gammas=(1.0, 0.7, 0.4), small=(10, 12, 14, 16, 18), shift=0.37, restarts=8, seed=0)
¶
Price the fold heuristic where it is actually used, and name the problem correctly.
Two things this settles that fold_heuristic_certificate could not.
The fallback is not exact past the enumeration limit. That certificate reported ratio
1.0 on all eighteen instances at m = 12, 16 and said nothing about larger m, which
is where a design is used. Measured here on C_m, gammas = (1, 0.7, 0.4), over
m in {24, 60, 120, 240} x x in {0.01, 0.3, 0.9}: the shortfall runs 0.00-2.06%, and it
is 0.00% in exactly one cell (m = 24, x = 0.9). It does not grow cleanly with m
either -- at x = 0.01 it is 2.06, 0.86, 1.06, 1.36% across the four sizes. The fallback
is good and it is not optimal; the earlier "it finds the optimum" was a small-m statement
being read as a general one.
The Ky Fan gap was never the shortfall. Its looseness at the optimum is 0.58-2.61%
and does NOT shrink with m, so at m = 24, x = 0.9 the certificate reports 1.49% for
a design that is exactly optimal. What a caller sees is certified_gap, which is the two
stacked: heur - kyfan = (heur - exact) + (exact - kyfan), so it always dominates the true
shortfall and never equals it. The guarantee is one-sided and that is all it is (Rocq
certified_epsilon_implies_epsilon_optimal, and its counterexample
a_large_certified_gap_does_not_convict_the_design).
The structural gates are the reason the objective can be named at all. internal = 1'Q1 -
2*cut with 1'Q1 partition-free, so minimising the same-fold 2-walk mass is MAXIMISING
the cut of Q(x) -- a max-bisection, not the "minimum-weight balanced cut" Result 52 (a)
called it, and the distinction is not cosmetic. Max-bisection has a 0.8776-approximation by
semidefinite programming (Austrin, Benabbas and Georgiou 2013), and under the Unique Games
Conjecture no polynomial-time algorithm approximates it beyond alpha_GW ~ 0.8786 (Khot,
Kindler, Mossel and O'Donnell 2007 prove it for max-cut; two disjoint copies of a graph have a
maximum bisection of twice its maximum cut). For min-bisection no constant-factor
approximation is known: the best is O(log n) (Räcke 2008). These are statements about
the general problems; Q(x) is a restricted family of weights, and nothing here is a
hardness result for the fold design. The bound optimal_fold_partition computes is the
eigenvalue bound of Donath and Hoffman (1973) with Boppana's (1987) diagonal shift, one of the
bisection relaxations Poljak and Rendl (1995) compare.
The design still keeps adjacent units together (adjacency cut 6 of 12 on C_12, against
parity's 12), which is what made the min-cut reading feel right: Q's edges are 2-walks,
not edges.
Cross-checked outside Python: polymake 4.15 maximises the same objective by exact rational LP
over the convex hull of the 35 balanced cut vectors at m = 8 and returns 188/25, so
internal = 5256/125 - 376/25 = 3376/125 = 27.008 -- the value this function computes to
1e-12. polymake also reports DIM = 20 = C(8,2) - 8, the geometric form of the degree
identity that makes the shift laws true. Normaliz 3.11.1 counts the lattice points of the
hypersimplex at m = 8, 10, 12, 14 and gets 70, 252, 924, 3432 -- exactly C(m, m/2),
every one of them 0/1, so the enumeration's feasible set is complete.
fold_heuristic_certificate(m=12, dmax=2, gammas=(1.0, 0.7, 0.4), phi=0.6, lag=1, restarts=8, n_random=4, seed=0)
¶
Price the fold heuristic against exact enumeration on graphs Result 52 does NOT cover.
Result 52's honest-scope note says the design law is closed-form only on vertex-transitive
graphs and that beyond them the problem "degrades to combinatorial search". This is the
measurement that note asks for: on nine topologies -- eight of them non-transitive, four of
them random -- run the exact balanced bisection and the spectral-plus-swap fallback on the
same Q(x) and report the ratio.
Measured, and it is the opposite of what the note feared: at m = 12 and m = 16 the
fallback returns the exact optimum on every one of the eighteen instances, ratio 1.0 to
machine precision. Losing vertex transitivity costs the closed form, not the answer -- the
search the note called combinatorial is, at these sizes, not where the difficulty lies. What it
does not say is anything about m large enough that the exact arm cannot run, which is
exactly where a design would be used and exactly where this comparison stops being available.
Both arms use chc.network_causal.graph_shells, so the exact arm is not a special case
of the cycle: it is the same objective the heuristic minimises, enumerated. ok asks that
the fallback never beats the enumeration (which would mean one of them is wrong), that it
stays within 1% of it -- a larger shortfall is the finding, not a failure of the test -- and
that the shifted bound stays under the optimum and over the plain one.
The Ky Fan gap is reported beside it because the two are different quantities and conflating
them is the easy mistake: ratio is how good the design is, kyfan_gap is how loose the
bound is at the true optimum. A large Ky Fan gap with ratio == 1 says the certificate is
weak, not the design. shifted_gap is the same looseness for the diagonally shifted bound
optimal_fold_partition reports: at m = 12, phi = 0.6 it is zero to 1e-8 on
the star, which the plain bound misses by 0.77%, and equal to it on the cycle.
delayed_network_certificate(*, cluster_size=6, n_times=5, lag=1, gammas=(1.0, 0.7, 0.4), phi_grid=(0.0, 0.3, 0.6, 0.9), n_draws=40000, burn_in=40, seed=20260830)
¶
RESULT 51 -- the non-separable half of the delayed-exposure gate (derived in
validation/delayed_network_exposure.mac, proved in proofs/delayed_network_exposure.v).
validation/delayed_exposure_gate.mac STEP 7 showed that a delay costs NOTHING under a
separable covariance Sigma = Sigma_u (x) T with unit-level folds: tr(T) cancels between
numerator and denominator and Result 43 transfers verbatim, with dPsi/dphi = 0. The only
remaining route is a delay that INTERACTS WITH THE NETWORK -- a shell-d neighbour's
influence arriving at lag delta*d::
x_i(t) = sum_d g_d sum_{j : dist(i,j)=d} z_j(t - delta d)
Cov(z_j(t), z_k(s)) = 1{j=k} phi^|t-s|
so Sigma = sum_{d,e} g_d g_e (S_d S_e) (x) T_{d-e}, separable only at delta = 0. Because
tr(S_d S_e) = 0 for d != e -- a vertex cannot sit at two distinct distances from
i -- the DENOMINATOR of the sandwich is delay-blind, and Psi is a POLYNOMIAL in
x = phi^delta of degree the spillover truncation, NOT of degree the graph diameter.
What this certificate can fail on, in the order it would matter:
psi_sim_*is measured on trajectories drawn from the GENERATIVE definition, never from the Kronecker formula, so a wrong covariance model shows up as a mismatch withpsi_law_*.- The two fold partitions differ only in how they ALIGN with the cycle -- alternating folds put every neighbour across the fold boundary, contiguous blocks keep most of them inside. If the delay's damage were a sample-size question rather than a design question, the two swings would agree; they do not.
- The
theta*design law and ITS LIMIT, in the same run. With only nearest-neighbour spillover,u_1is exactly affine in the same-fold edge fraction with roottheta*(K) = K^3/(4K^3-6K^2+4K-1), sou1_nearest_parity < 0 < u1_nearest_block: alternating folds sit attheta = 0and contiguous blocks at2/3, on opposite sides oftheta* = 8/15. Switching the second shell back on addsu1_cross_block, which is LARGER in magnitude thanu1_nearest_blockand drags the block arm'su_1negative again. So the fold-alignment rule is aD = 1rule, and longer-range spillover can defeat it -- a limit measured here rather than asserted. separable_spreadre-derives STEP 7 from this side: atdelta = 0the law must losephientirely.
HONEST SCOPE. Psi here is the sandwich functional
N^2 tr(A Sigma A)/(tr(A)^2 tr(Sigma)) that
validation/delayed_exposure_gate.mac STEP 6 forced, evaluated with the fold operator
held fixed while Sigma varies. It is not a re-derived estimator, and
ConfoundedNetworkSystem is cross-sectional, so phi and delta are model parameters
rather than fitted ones. What this gives the code today is a fold-assignment rule with a
number attached.
optimal_exploration_certificate(*, b=0.3, rr=0.2, xt=1.0, sigma=1.0, n=30, kappa_conf=0.4, grid_lo=0.02, grid_hi=3.0, n_grid=15, n_seeds=1000)
¶
Optimal exploration for causal control -- the actionable DUAL of the Cramer-Rao lower bound
(Result 10; derived in validation/optimal_exploration.mac, proved in
proofs/optimal_exploration.v). Injecting exploration variance v buys identifying
information (shrinking the estimation floor B/v, B=C*sigma^2/n) but is itself an action
error costing A*v control (A=b^2+rr, by completing-the-square). The total excess
A*v + B/v is minimal at v* = sqrt(B/A), irreducible cost 2*sqrt(A*B) (AM-GM). The
optimum is interior -- pure exploitation is never optimal -- and confounding raises B
(here via a fraction kappa of identifying signal), lifting BOTH v* and the floor. The
estimation term is Monte-Carlo; the A*v term is the proven CE cost of the injected variance.
ce_explicit_constant_certificate(*, a_mat=None, b_mat=None, q_mat=None, r_mat=None, x0=None, multiples=(0.25, 1.0, 4.0, 10.0, 20.0, 40.0, 80.0), n_dirs=240, seed=11)
¶
CONTRIBUTION 1 -- the finite-sample end-to-end constants (derived in
validation/ce_explicit_constants.mac, proved in proofs/ce_explicit_constants.v).
Every C1 statement so far took regret <= cc*||dB||^2 as a HYPOTHESIS -- c2_end_to_end.v
quantifies over an arbitrary 0 <= cc -- and cited Mania-Tu-Recht's LOCAL quadratic bound.
Nothing computed the ball on which it holds, and nothing computed cc. Both are here.
The lever is that the Lyapunov increment is an EXACT perfect square in the gain error,
Q + K'`RK' + (A - BK')`P(A - BK') - P = (K' - K)'R_K(K' - K), R_K := R + B'PB,
with no smallness assumed, so "quadratic in the gain error" is an algebraic identity rather than
a local approximation and the Mania-Tu-Recht citation leaves the regret half entirely. Summing
it along the perturbed loop turns the suboptimality into a state-energy sum, and the P-metric
triangle inequality bounds that: ||P^(1/2)(A - B Khat)P^(-1/2)|| <= sqrt(1 - eta) + beta_B
||dK||, so spending half the margin gives the certified ball
rho = theta/(2 beta_B L_K), C = 2 kappa_P ||R_K|| L_K^2 ||x0||^2 / theta,
with eta = lmin(Q + K'RK)/lmax(P), theta = 1 - sqrt(1 - eta). Inside rho the
certainty-equivalent gain provably stabilises the TRUE plant with the checkable Lyapunov
certificate (A - B Khat)'P(A - B Khat) <= (1 - theta/2)^2 P, and the regret is at most
C ||dB||^2. Composed with Result 26's ||B_hat - B|| = O_p(a_G) that is the end-to-end
statement the C1 line recorded as open.
The sweep runs past the ball on purpose: the certificate must be able to exhibit the failure it
protects against, so unstable_fraction has to become positive at large multiples of rho
or the ball is not doing any work. conservatism records the price of the guarantee. Note
that worst_gap beyond the ball is a maximum over the FINITE draws only -- once
unstable_fraction > 0 the true worst case is +inf and the comparison against bound
there is vacuous, which is exactly why the guarantee is stated on rho and not on the sweep.
SCOPE: one citation remains on the control side -- uniform validity of the gain map's Lipschitz
constant on the ball (Konstantinov-Petkov-Christov-Angelova 1993; Sun 1998). It is not assumed
away: l_k_ball measures it over the certified ball against the derivative l_k_local.
conjugate_time_certificate(*, a=0.3, b=1.0, q=-4.0, r=1.0, s_t=0.0, x0=1.0, margins=(0.1, 0.01, 0.001, 0.0001, 1e-05), delta_b=1e-06)
¶
THE CONJUGATE-TIME OBSTRUCTION (derived in validation/conjugate_time.mac, proved in
proofs/conjugate_time.v): the third member of the family of Result 13 (the active-set kink)
and Result 14 (the confounded turnpike gap) -- places where the smooth machinery every regret
bound assumes quietly stops applying.
Result 14 says a long horizon is BENIGN: the discounted confounded gap stays finite. That
holds for a POSITIVE-DEFINITE stage cost. For an INDEFINITE one -- a state that is rewarded
rather than penalised, as in a growth or market-share objective -- the second-order sufficient
fails at a FINITE horizon, and past it no constant in chc.regret exists at all. The two are
the two sides of the definiteness hypothesis; only one of them had been written down.
Continuous-time scalar LQ x' = a x + b u, J = int_0^T (q x^2 + r u^2) dt + sT x(T)^2.
With c := b^2/r the reverse-time Riccati flow is a UNIFORM ROTATION of the phase
arctan(c(p - a/c)/mu) at speed mu := sqrt(-(a^2 + c q)), real exactly when
a^2 + c q < 0. The phase falls from phi0 to -pi/2 in finite time:
``t_conj = (pi/2 + phi0)/mu``, ``phi0 = arctan(c(sT - a/c)/mu)``.
Flip the sign of q and tan becomes tanh: the same one-line solution, bounded for
every horizon. That is the whole dichotomy, and it is why the positive-definite arm here is a
control rather than a separate model.
THE COUNTER-INTUITIVE PART: mu GROWS with the control authority c, so t_conj
SHRINKS. A stronger actuator brings the obstruction closer.
Three orders, measured separately because they are three claims: the cost-to-go has a SIMPLE
pole with residue -r/b^2 (free of a, q and the terminal weight); the SENSITIVITY
dk/db -- which is Result 44's L_K and Results 18/20's du_star/db -- has a DOUBLE
pole, because differentiating the closed form turns the tangent into a sec^2; and since
Result 44 has C proportional to L_K^2, the regret constant has a FOURTH-order pole. The
last is measured from realised certainty-equivalence regret, not inferred from the first two.
adaptive_exploration_certificate(*, b=1.0, rr=0.5, xt=1.0, sigma=0.5, m0=1.0, eta=1.0, static_v=0.05, horizons=(50, 150, 500, 1500, 5000), schedule_horizon=400)
¶
CONTRIBUTION 3 -- ADAPTIVE information-exploration duality (derived in
validation/adaptive_exploration.mac, proved in proofs/adaptive_exploration.v). At round
t the controller injects v_t that raises accumulated Fisher information
m_t = m0 + eta*sum_{s<t} v_s for FUTURE rounds, where eta in (0,1] is the identifying
information PER UNIT exploration (eta=1 identified, eta->0 confounded). The per-round
regret is A*v_t + K/m_t, K/m_t the van-Trees (Bayesian Cramer-Rao) floor -- valid for
adaptive data, unlike ordinary Cramer-Rao (algebraic core in proofs/van_trees.v). Its
numerator is K = C*sigma^2 with C = A*(du*/db)^2, the convention
validation/adaptive_exploration.mac states in its first line: the observation noise enters
it, so sigma moves both the bound and the schedule scale. The RATE-OPTIMAL schedule
v_t = kappa/sqrt(t) achieves Theta(sqrt(T)), matching the Theta(sqrt(T)) sequence
lower bound >= 2*sqrt(A*K*T/eta) - A*m0/eta (Rocq reduced_objective_lower_bound with
a=A/eta, where the Rocq C is this K), NOT a corollary of the per-round floor.
The leading term scales as 1/sqrt(eta), where eta is the INJECTED-EXPLORATION efficiency
(information per unit exploration variance, distinct from an observational residual fraction):
smaller eta provably raises the floor (Rocq lower_efficiency_raises_sequence_floor) --
this 1/sqrt(eta) is identification-efficiency-specific (attenuation: noncompliance,
dilution, partial observability, interference -- NOT necessarily confounding; eta=1 for
clean directly-observed randomisation), the causal content the generic bound lacked. Greedy is
Theta(T); the static v* of Result 11 over-explores. SCOPE: the t^{-1/2} schedule,
sqrt(T) rate and van-Trees sqrt(T) lower bounds are KNOWN in adaptive LQR
(Ziemann-Sandberg; Wagenmaker et al.); the FULL minimax constant is no longer open --
minimax_exploration_certificate proves it over ALL policies and shows this schedule pays
a factor sqrt(2) over that floor. Conditions: m0=O(1); exploitation adds no identifying
information; exploration information is linear in v_t (else greedy need not be
Theta(T)). The myopic max(0, sqrt(K/A)-m_t) rule is a DIFFERENT (one-shot) object.
capped_exploration_policy(*, horizon, cap, budget=None, b=1.0, rr=0.5, xt=1.0, sigma=0.7, i0=1.0, eta=0.6)
¶
The matching causal policy under a per-round action cap -- Result 56.
minimax_exploration_certificate showed that a front-loaded BURST attains the minimax
floor while a 1/sqrt(t) taper sits sqrt(2) above it, and conjectured that a per-round
action cap -- which every real actuator has -- is what would make the taper the right shape.
It is not. The regret
``F(v) = A sum_t v_t + K sum_t 1/(I0 + c S_{t-1})``, ``S`` the prefix sum,
``c = eta/sigma^2``
depends on the schedule only through its prefix sums and is strictly decreasing in each of
them, so moving exploration from a later round to an earlier one at equal budget STRICTLY
lowers it (proofs/capped_exploration.v, earlier_is_strictly_cheaper) -- an argument
that never mentions the cap. The Hessian is a sum of rank-one PSD terms, so the objective is
convex and the exchange argument gives a GLOBAL optimum: saturate the cap on a prefix, then
stop. A clipped burst, not a taper.
The price of the cap is ADDITIVE and LOGARITHMIC, not a constant factor. The optimal block
length is n* = sqrt(K T/(A c))/cap TO LEADING ORDER -- exactly it is S*/cap, smaller
by the constant derived below -- and at it the leading term is 2 sqrt(A K T/c),
which with K = A (du*/db)^2 and c = eta/sigma^2 is EXACTLY the uncapped constant
c_causal sqrt(T) (capped_leading_term_is_the_uncapped_floor). What is left over is the
harmonic sum, (K/(2 c cap)) ln T + O(1) -- so the ratio to the uncapped floor tends to 1
for every fixed cap, while the taper's sqrt(2) does not. A tighter actuator explores for
LONGER, not more gently.
block_rounds is found by an exact search over the convex block-cost curve, which digamma
makes O(1) per candidate; cost is the true cost of that schedule and agrees with a
projected-gradient solve of the full convex program to seven decimals. taper_cost is the
clipped 1/sqrt(t) schedule at its own optimal scale, for comparison -- at T = 1e5 and
cap = 0.03 the block costs 196.11 against the taper's 266.81, a 36% gap that
does not close.
predicted_excess carries only the LEADING log term; its O(1) is not modelled, so read
it as a slope rather than a level. Per decade of T at cap = 0.03 it predicts 2.32
against a measured 2.20 then 2.29 -- converging, which is the claim.
A VARYING cap, or a total budget on top of it, was Result 56's own open item: the exchange
argument still says "as early as feasible", but n* is then not a single number. It turns out
n* was never the invariant. The first-order condition is
``dF/dn = cap(n) [ A - K c (T - n) / (I0 + c S(n))^2 ]``
(validation/capped_exploration_schedule.mac STEP 2b, residual 0), whose bracket contains the
cap nowhere -- only the mass S(n) delivered so far. So the optimal exploration MASS solves
``A (I0 + c S)^2 = K c (T - n)``, with ``n`` the round the prefix sum delivers ``S``
and is free of the cap LEVEL while the round that reaches it is not. It is not free of the
horizon: n sits inside its own condition, so predicted_mass is a fixed point rather
than a closed form, and Result 56's sqrt(K T/(A c)) - I0/c is the n << T limit -- 25%
high as soon as the caps open late.
Under a CONSTANT cap that fixed point has a closed form, and it says exactly what Result 56
drops. The stopping round is n = S/cap, so the remaining horizon is itself a function of
the mass and the condition becomes a QUADRATIC in w = I0 + c S,
``A w^2 + (K/cap) w = K c T + K I0/cap``
whose Result 56 counterpart is the same balance with the horizon held at the full T,
A w0^2 = K c T. Subtracting the two kills T entirely and leaves
A (w0^2 - w^2) = K c S/cap, so the cap-free form OVER-states the mass at every finite
horizon -- never under-states it -- by (w0 - w)/c <= K/(2 A c cap), a CONSTANT rather than
a vanishing remainder (proofs/capped_exploration_schedule.v section (H)). At the defaults
and cap = 0.01 the ceiling is 2.016, approached strictly from below as
1.834, 1.959, 1.998, 2.011, 2.015 over T = 1e4 .. 1e8, the remainder decaying like
(K + 4 A I0 cap) sqrt(K/(A c)) / (8 A c cap^2 sqrt(T)) (STEP 7 of
validation/capped_exploration_schedule.mac, confirmed to 60 digits by
validation/capped_exploration_o1.gp). It is the TIGHT actuator the omission hurts: at
T = 4000 the over-statement is 0.5% of the mass at cap = 0.3 and 17% at
cap = 0.01.
A budget does not interact with the caps either: below the root the objective is strictly decreasing in the mass, so a binding budget is spent in full, as early as the caps allow. The two collapse to
deliver mass ``min(budget, S*)``, greedily -- ``block_rounds`` is where the prefix sum of
the caps first reaches it.
cap therefore takes either a scalar or a per-round schedule of length horizon. What does
NOT become cap-free is the COST: delivering the same mass earlier is strictly cheaper, so two
schedules with the same prefix mass at n* still cost differently. The stopping rule is the
invariant, not the value. A prefix of ZERO caps is the sharp case -- it adds exactly m K/I0
and hands the rest of the horizon to the same problem, so a dead actuator has a price and not a
distortion.
Scope: as in minimax_exploration_certificate -- the van Trees score identity for a
functional under an adaptive design and Fisher-information additivity are cited, not
formalised, and exploitation is assumed to contribute no identifying information.
minimax_exploration_certificate(*, b=1.0, rr=0.5, xt=1.0, sigma=0.7, i0=1.0, eta=0.6, constant_v=0.01, horizons=(10 ** 3, 10 ** 4, 10 ** 5, 10 ** 6, 10 ** 7), eta_grid=(0.1, 0.2, 0.4, 0.7, 1.0))
¶
CONTRIBUTION 3 -- the FULL LOCAL-MINIMAX sequential lower bound with an EXPLICIT constant
(derived in validation/minimax_exploration.mac, proved in proofs/minimax_exploration.v).
adaptive_exploration_certificate gave a sequence bound: over exploration schedules,
inside an assumed per-round decomposition, with an opaque floor numerator K. This closes
both gaps. Splitting any policy's action into its conditional mean and conditional variance is
an identity (Rocq action_variance_decomposition), so the inf runs over ALL policies, not
schedules; and van Trees on the functional u*(theta) identifies the numerator as
K = A*(du*/db)^2*sigma^2. The result is
``inf_pi sup_theta E R_T >= c_causal*sqrt(T) - O(1)``,
``c_causal = 2*A*|du*/db|*sigma/sqrt(eta)``,
with A the local cost curvature in the action, du*/db the decision sensitivity of the
oracle action, sigma^2 the noise, and eta in (0,1] the identifying information per unit
injected exploration variance. The sup is over a shrinking neighbourhood of radius
H*T^(-1/4) -- not the usual T^(-1/2), since the optimiser sits at order sqrt(T)
and a T^(-1/2) prior would swamp the data information -- with the limits taken T -> inf
first, then H -> inf, as in the clustered result.
Two things this certificate shows that the rate alone does not. The floor is attained:
the crude step (every denominator at the total budget) is an equality exactly when information
is raised immediately, so a front-loaded burst hits it and burst_over_floor -> 1 -- the
constant is sharp. And tapering costs a constant factor: the 1/sqrt(t) schedule, even at
its own optimal scale, sits at sqrt(2) times the floor (Rocq taper_gap_is_sqrt_two). The
burst is optimal in this model, where exploration cost is linear in the injected variance and
unbounded per round. A per-round action cap does NOT rescue the taper -- Result 56 shows the
capped optimum is a cap-saturating BLOCK, still front-loaded, whose price over this floor is
an additive logarithm; see capped_exploration_policy.
SCOPE, as elsewhere in this line: Rocq proves the algebra. The van Trees score identity for a
functional under an adaptive design, and Fisher-information additivity along the trajectory, are
cited (Gill-Levit 1995; Gassiat-Stoltz 2024), not formalised. The model assumes exploitation
contributes no identifying information -- that is what eta encodes.
van_trees_certificate(*, tau=2.0, sigma=1.0, v_id=0.4, sample_sizes=(5, 10, 20, 50, 100, 200), n_trials=6000)
¶
The FORMAL van Trees (Bayesian Cramer-Rao) inequality that Result 20 assumed (derived in
validation/van_trees.mac, proved in proofs/van_trees.v). For a Gaussian conjugate model
(prior N(0,tau^2), n obs x = theta + N(0,sigma^2)) Bayes risk of ANY estimator is
at least 1/(I_prior + n*I_data), I_prior = 1/tau^2, I_data = 1/sigma^2 -- and the
posterior-mean estimator HITS it (van Trees is tight for the Gaussian). Confounding reduces the
per-obs identifying information (fraction v_id < 1), lifting the floor -- the Bayesian /
sequential analogue of the CR floor of Results 10/12 and the C/m_t term of Result 20.
hinf_robust_regret_certificate(*, b=1.0, rr=0.5, xt=1.0, s=0.6, gamma_lo=0.72, gamma_hi=8.0, n_gamma=40, n_u=4000)
¶
H-infinity / differential-game robust control (Geering 2007, Ch 4). Derived in
validation/hinf_robust_regret.mac, proved in proofs/hinf_robust_regret.v. Confounding
uncertainty in the effect b is an adversary perturbing the gain, penalised by budget
gamma^2; the robust controller solves min_u max_w. This is a correspondence, not an
identity: the worst-case objective gamma^2*e^2/(gamma^2-u^2)+rr*u^2 and the expected
quadratic objective (b*u-xt)^2+(rr+s^2)u^2 (Result 2, optimum u_pess=b*xt/(b^2+rr+s^2))
are different functions. Robustness INDUCES a pessimistic contraction (u_rob -> u_ce as
gamma grows), and the regret-optimal gamma* can be CALIBRATED so u_rob(gamma*)
approximates u_pess (0.5415 vs 0.538) -- game-theoretic robustness and statistical pessimism
as two roads to a similar cautious control, not a proven equality.
constrained_ce_regret_certificate(*, xt=1.0, rr=1.0, umax=0.45, delta_lo=0.005, delta_hi=0.15, n_delta=12, active_offset=0.1)
¶
Constrained certainty-equivalence regret is PIECEWISE-QUADRATIC (Gros & Diehl 2022, Ch 16).
Derived: validation/constrained_ce_regret.mac; proved: proofs/constrained_ce_regret.v.
A budget/safety cap u <= umax clips the control to u_opt(b) = min(u*(b), umax). With the
true effect at an activation threshold (u*(b_t) = umax): on the INACTIVE side the
regret is quadratic in the effect-estimate error; on the ACTIVE side the control freezes at
umax so the regret is ZERO -- the curvature collapses, the source of the kink. Clipping is
non-expansive: constrained regret never exceeds unconstrained. Consequence (ties to Result 9
partial-ID and the marketplace budget): when the effect interval straddles a threshold the
pessimism budget must cover the active-set transition -- a naive controller that assumes the
constraint inactive pays that gap.
u* is unimodal in b (peak xt/(2*sqrt(rr)) at b = sqrt(rr)), so u*(b) = umax
has TWO roots and the cap binds on the bounded interval between them, with b_lo*b_hi = rr;
the cap never binds at all when umax > xt/(2*sqrt(rr)). This certificate probes the lower
root, and its delta_hi sweep stays inside the active interval, so reg_act is exactly 0.
confounded_turnpike_certificate(*, b=1.0, beta=0.3, xref=1.0, q=1.0, g=0.9, horizon=200)
¶
The CONFOUNDED TURNPIKE GAP (Weber 2011, Sec 3.5: turnpike + current-value Hamiltonian).
Derived in validation/confounded_turnpike.mac, proved in proofs/confounded_turnpike.v. A
confounded controller using the biased effect b_obs = b + beta (OVB) converges to the WRONG
turnpike x_conf = b*xref/(b+beta) and pays the offset every step. This upgrades Result 1d
(dynamic horizon regret, previously hand-argued) to a proper turnpike argument, and adds a NEW
discounted-regret certificate: the undiscounted cumulative regret T*c is unbounded, but the
discounted sum sum g^t c = c*(1-g^T)/(1-g) stays below c/(1-g) -- finite.
That "a long horizon is benign" holds because q > 0. Under an INDEFINITE stage cost the
horizon is bounded outright and no constant survives past the first conjugate point -- see
conjugate_time_certificate, the other side of the same definiteness hypothesis.
partial_id_control_certificate(*, b_hat=1.0, rr=0.5, xt=1.0, half_widths=(0.2, 0.5, 0.8, 1.0, 1.5), u_points=401, b_points=101)
¶
Control under PARTIAL IDENTIFICATION: the sign-identification threshold (derived in
validation/partial_id_control.mac, proved in proofs/partial_id_control.v; grounded in
Manski partial-ID / Rosenbaum-VanderWeele-Ding sensitivity). When the effect is only interval-
identified, b in [b_hat - Delta, b_hat + Delta]: (1) the certainty-equivalent action's
worst-case regret grows like Delta^2, and the minimax action reduces it; (2) the optimal
action has the sign of b, so its direction is identified iff the interval excludes 0 --
iff Delta < |b_hat|. The critical width Delta* = |b_hat| is the sign-identification
threshold (a directional identification margin; NOT the sensitivity E-value, whose name it
deliberately avoids).
doubly_robust_control_certificate(*, theta=1.0, n=100000, n_seeds=8, errors=(0.4, 0.2, 0.1, 0.05), seed=0)
¶
The doubly-robust version of result 0 (derived in validation/doubly_robust.mac, proved in
proofs/doubly_robust.v). For a binary intervention the AIPW estimator's bias is the PRODUCT
of the outcome-model error and the propensity error, dmu*de/(e+de) -- so the regret is
O((dmu*de)^2) and vanishes if EITHER nuisance model is correct (double robustness), unlike
outcome-regression (needs the outcome model) or IPW (needs the propensity model). Here the bias
is dmu*de*E[(1/2-e)/e_hat]: the propensity error moves the weight as well, so the slope is 4
only as the errors vanish, and 3.33 over the default grid. seed selects the draws.
bandit_causal_certificate(*, b_true=1.0, rr=0.5, xt=1.0, alpha=1.0, gamma=1.0, noise=0.5, n_rounds=1500, n_seeds=12, ridge=0.001)
¶
The bandit / adaptive-control version (derived in validation/bandit_causal.mac, proved in
proofs/bandit_causal.v): learn the causal effect ONLINE while controlling. Per round the
control regret is the certainty-equivalence coefficient C times the squared estimation err.
A de-confounded online estimator (backdoor on z) is consistent, err^2 ~ sigma^2/t,
so cumulative regret grows like log T (sublinear); a confounded one has err^2 -> beta^2
(systematic), a per-round floor, so cumulative regret is linear in T. The doubling
ratio cum(T)/cum(T/2) separates them: -> 1 (log) vs -> 2 (linear). Scope: the
O(log T) rate is specific to this scalar, persistently-identified setting -- NOT universal
for adaptive control, where uninformative systems admit sqrt(T) lower bounds
(Simchowitz-Foster; Ziemann et al.). The content is the deconfounded-vs-confounded separation.
finite_horizon_pl_certificate(*, a=0.9, b=1.0, q=1.0, r=0.5, qf=5.0, x0=1.0, horizons=(3, 5, 8, 13, 21), n_samples=200, seed=0)
¶
Finite-horizon (multivariate) version of the PL self-certifying regret bound (result #3),
without the steady-state / setpoint simplification of the dynamic result 1d. For the full
finite-horizon LQ objective J(U), U = (u_0,...,u_{T-1}), the regret admits the bound
grad^2 / (2*lambda_min(H)) (H the Hessian) -- self-certified from the achieved gradient
over the whole trajectory, no optimum needed. Justified per-eigenmode by
proofs/nonlinear_regret.v (pl_mode_bound): the min-curvature mode is tight, stiffer modes
slack. Verifies the bound is valid (upper-bounds the true regret) for random controls and tight
along the min-curvature eigenvector, across horizons.
interference_convexity_certificate(*, mu=1.0, a=1.0, u_eval=1.5, cannibalisations=(0.0, 0.5, 1.0, 2.0, 4.0))
¶
Strong convexity under interference (combines the PL bound of nonlinear_regret with the
interference of §A; derived in validation/interference_convexity.mac, proved in
proofs/interference_convexity.v). Marketplace interference cannibalises: the benefit of
incentivising saturates, adding convexity, so the effective strong-convexity rises to
mu + kappa_int. Since the PL self-certifying bound grad^2/(2 mu) is antitone in the
convexity, cannibalising interference makes the certificate tighter -- a curse-and-blessing
duality (interference hurts identification but helps the control certificate). The
interference-aware bound (using mu + kappa_int) is exact for the quadratic; the blind one
(using mu) over-states the regret by the growing ratio (mu + kappa_int)/mu.
interference_orthogonal_certificate(*, b_direct=2.0, b_int=1.5, rr=0.5, target=1.0, errors=(0.2, 0.1, 0.05, 0.025, 0.0125), n=300000, n_seeds=6, alpha=1.0, gamma=1.5, noise=0.05)
¶
Interference x orthogonality (combines proofs/interference_regret.v and
proofs/orthogonal_control.v; derived in validation/interference_orthogonal.mac). Under
interference the control effect is B = b_direct + b_interference (direct + spillover), so
control regret is quadratic in the total estimation error. Double ML makes a channel's error
O(eps^2). The non-obvious consequence: you must orthogonalise both channels -- debiasing
only the direct effect leaves the spillover error at O(eps), which then dominates the regret
at O(eps^2); only double orthogonalisation reaches O(eps^4). Fits the three slopes
(~2, ~2, ~4).
dynamic_causal_regret_certificate(*, a=0.7, b=1.0, beta=0.5, x_ref=1.0, x0=0.0, q=1.0, horizons=(10, 20, 40, 80, 160))
¶
The DYNAMIC confounding theorem (proofs/dynamic_causal_mpc.v, derived in
validation/dynamic_causal_mpc.mac): a tracking controller x' = a x + b u using a
confounded effect estimate b_obs = b + beta settles at a persistent steady-state offset and
pays a per-step floor q*offset^2 every step, so its cumulative regret grows linearly in the
horizon T (unbounded), while the causal controller (effect b) tracks exactly and its
cumulative cost is bounded (the transient only). The fitted growth slope equals the per-step
floor q*(x_ref*beta/b_obs)^2.
strong_convexity_regret_bound(grad_norm_sq, mu)
¶
Self-certifying regret upper bound ||grad J(u)||^2 / (2 mu) for a mu-strongly-convex
cost -- GLOBAL (beyond the local linearisation), computed from the achieved gradient alone,
without knowing the optimum. Proved in proofs/nonlinear_regret.v (the Polyak-Lojasiewicz /
strong-convexity bound, control-first): 2*mu*(J(u) - J*) <= ||grad J(u)||^2.
nonlinear_regret_certificate(*, mu=1.0, kappa=0.5, u_max=2.0, points=25)
¶
On the genuinely nonlinear cost J(u) = mu/2 u^2 + kappa u^4 (minimum at u = 0) the
strong-convexity bound grad^2/(2 mu) upper-bounds the true regret everywhere (beyond the
linearisation), while the fixed-Hessian local estimate mu/2 u^2 under-states it away from
the optimum -- an unsafe certificate exactly where a valid one is needed. This is a control
specialization of the STANDARD strong-convexity / Polyak-Lojasiewicz certificate, not a new PL
result. Verifies proofs/nonlinear_regret.v and validation/nonlinear_regret.mac.
causal_vs_predictive_certificate(*, b_true=1.0, alpha=1.0, gamma=1.0, target=1.0, effort=0.5, sample_sizes=(500, 2000, 8000, 32000, 128000), n_seeds=8, noise=0.3)
¶
Empirical proof that predictive control is asymptotically wrong under confounding, while the
causal controller is consistent (the theorem in proofs/causal_mpc.v, derived in
validation/causal_mpc.mac; the hardened notebook-01 headline).
A confounder z drives both action and outcome (u = alpha z + noise,
y = b_true u + gamma z + noise). The controller hits target from its estimate of the
control effect b. The predictive controller regresses y on u (observational, so
confounded): as n -> inf it converges to b + beta (a fixed omitted-variable bias),
and its control regret converges to a positive floor that does not vanish. The
causal controller partials z out (backdoor), so its estimate converges to b_true and
its regret -> 0. Only causal identification closes the gap.
interference_regret_certificate(a, b, q, r, x0, *, interference_ratio=1.0, errors=_DEFAULT_ERRORS, n_samples=400, seed=0)
¶
Interference-aware regret certificate: CE suboptimality is quadratic in the total error
eid + eint -- the sum of an identification error eid of the autonomous dynamics (dA)
and an interference / exposure-map error eint of the actuation channel (dB).
This is the empirical twin of the machine-checked bound in proofs/interference_regret.v
(regret <= (kappa C^2 / 2) (eid + eint)^2): the interference
error enters additively inside the square, so ignoring it under-states the regret. Sets
eint = interference_ratio * eid; the recorded error is the additive ||dA|| + ||dB|| (not
the Euclidean norm), so the fitted slope tests the quadratic-in-total law directly (~2).
lq_regret_sensitivity(effect, effort, target)
¶
Second-order regret sensitivity L_reg = target^2*(r-b^2)^2/(b^2+r)^3 of the LQ toy.
The leading coefficient of the certainty-equivalence regret in the effect error: applying the
optimum for a wrong effect b_hat = b + delta costs L_reg*delta^2 + O(delta^3) on the
true plant (order-doubling -- linear effect error, quadratic control regret). Verified against
the exact regret in validation/confounding_lq_regret.mac / the certificate below.
Nonnegative; zero at the degenerate r = b^2 (where u* is locally flat in the effect).
confounding_robust_lq_regret(regret_sensitivity, stat_error, confounding_halfwidth)
¶
Confounding-robust LQ regret bound L_reg*(eps_stat + Delta)^2 (Rocq cr_regret).
Delta is the confounding half-width on the effect estimate -- the §32 inflation
(chc.uncertainty.confounding_robust_inflation), an irreducible bias floor that does
not vanish with sample size. Pushed through the order-doubling regret map it becomes a
control-regret floor L_reg*Delta^2: the LOCAL LQ control regret is SECOND-order in the
confounding-induced effect bias (Rocq floor_below_linear: L_reg*Delta^2<=L_reg*Delta on
[0,1]) -- a local-toy statement, not a general robustness claim. Delta=0 (Gamma=1,
point identification) recovers the statistical regret L_reg*eps^2.
confounding_robust_lq_regret_matrix(curvature, stat_error, confounding)
¶
Matrix confounding-robust LQ regret tr(E^T H E), E = stat + confounding.
The Frobenius lift of the scalar bound via the §21 matrix order-doubling (Rocq
regret_order_2p): for a positive-semidefinite LQ regret curvature H and effect-error
matrix E, the regret is the quadratic form tr(E^T H E) = O(||E||_F^2). The confounding
floor (stat = 0) is tr(Delta^T H Delta) -- SECOND order in the Frobenius norm of the
matrix confounding half-width, the same second-order robustness as the scalar §33. Reduces to
confounding_robust_lq_regret in the 1x1 case.
confounding_robust_lq_regret_certificate(effect=1.3, effort=0.4, target=1.0, stat_error=0.02, cvar_gap=0.5, gammas=(1.0, 1.5, 2.0, 3.0, 5.0))
¶
Confirm L_reg matches the exact toy regret and the Gamma sweep floors quadratically.
Grounds the analytic lq_regret_sensitivity against the exact static-LQ regret
kappa*(u*(b_hat) - u*(b))^2 at a small perturbation (ratio -> 1 proves the coefficient),
then maps each Gamma to its confounding half-width via the §32 inflation and checks the
bound is monotone, recovers the statistical regret at Gamma=1, and floors at Delta^2.
minimax_action(target, b_lo, b_hi, effort, curvature=1.0, *, criterion='cost')
¶
Exact argmin_u max_{b in [b_lo, b_hi]} curvature*(b*u - target)^2 + effort*u^2.
§33 showed by counterexample that certainty equivalence is not minimax for the LQ loss and
left the robust controller unbuilt; this is it, in closed form rather than by search
(validation/minimax_lq.mac, proofs/minimax_lq.v).
Two facts make it closed-form. The stage cost is convex in b with curvature
2*curvature*u^2 >= 0, so the inner maximum is attained at an endpoint and the outer
problem is the minimisation of a max of two convex quadratics. Those two branches cross at
exactly two points, u = 0 and the equalising action u = target/b_hat -- both of them,
which is why the candidate set is enumerated rather than reasoned about by the sign of u.
A convex piecewise quadratic attains its minimum at a branch minimiser or at a kink, so
evaluating the four candidates is exact, not a heuristic.
The choice is decided by curvature*b_lo*(b_hi - b_lo)/2 <= effort: below it the answer is
the CE action for the pessimistic endpoint, above it the equalising action. And the correction
does NOT have a fixed sign -- the robust action is smaller than CE at b_hat only when
curvature*b_lo*b_hat < effort, and larger otherwise. "Be robust, act less" is a statement
about expensive effort, not about pessimism.
criterion="regret" minimises the worst regret instead: the cost over what the action
that knew b would have paid, D(b) (u - u*(b))^2 with D(b) = curvature*b^2 + effort
and u*(b) = curvature*b*target/D(b). The two answers differ: on the five intervals of its
test the minimax-cost action carries 4.1-6.2x the minimax regret. The regret's worst
effect is an endpoint unless sqrt(D) (u - u*) turns inside the interval, which happens
exactly where u*b*D(b) - target*effort changes sign on it -- that root test decides the
case, and a monotone u* does not: on b in [1, 10] with unit weights the endpoint answer
is 20% low. With no turn, the answer is the endpoint equaliser, (sqrt(D_lo)*u*_lo +
sqrt(D_hi)*u*_hi) / (sqrt(D_lo) + sqrt(D_hi)); with one, the worst regret is evaluated at
the endpoints and the turn, is convex in u, and is minimised over the range of u* by
golden section.
minimax_regret_move(estimate, standard_error, gamma, cvar_gap=1.0, size=1.0)
¶
How much of a move to make when its value is linear in the channel, its estimate is noisy,
and confounding up to gamma may bias it: the minimax-regret rule of Montiel Olea, Qiu and
Stoye (Review of Economic Studies 2026; arXiv 2312.17623 v3, Section 4.2). Experimental.
The move's estimated value is estimate ~ N(beta, standard_error^2), the standard error its
influence's, and its true value lies in [beta - k, beta + k], k = (gamma - 1)/(gamma + 1)
* cvar_gap * size: the marginal sensitivity model's half-width, with the gap a share of
size, the channel's size on the move's scale, as ADR 0085 sizes it; size = 1 reads
cvar_gap in the move's own units. A rule makes the move with probability d. Where the
move's value is u > 0 its regret is u (1 - E d), and where u < 0 it is -u E d.
k <= sqrt(pi/2) standard_error: the sign rule, the whole move where the estimate is not negative, is minimax regret, and essentially the only rule that is. Its worst regret ismax over beta of (beta + k) Phi(-beta / standard_error).- Past it every minimax-regret rule randomises, the minimax regret is
k / 2, and the rule that randomises least makesclip((estimate + rho) / (2 rho), 0, 1)of the move, withrho = k (1 - 2 Phi(-rho / standard_error)), the one root in(0, k]. The sign rule loses more there: 1.0509 against 1.000 atk = 2,standard_error = 1, and 3.371 against 2.5 atk = 5(validation/minimax_regret_move.mac).
MinimaxRegretMove.full_move_gamma is the largest gamma at which the estimate still
makes the whole move. The share is 1 while rho <= estimate, and rho grows with k,
so the move is whole up to k = estimate / erf(estimate / (standard_error sqrt 2)), which
exceeds estimate. So the move stays whole past the flip (G + d)/(G - d) of the
noiseless identified set, where the set already holds values below zero: the noise is a regret
the confounding's hedge cannot buy back, and below sqrt(pi/2) standard_error the sign rule
moves whole on any estimate that is not negative. The Gamma falls to the flip as the error
shrinks.
For linear welfare only: a value linear in the move, an estimate Gaussian about its mean, and a
half-width that does not depend on the data. A cost quadratic in the action, as
minimax_action's, has another minimax rule, which this one does not approximate.
Raises:
| Type | Description |
|---|---|
ValueError
|
on an estimate that is not finite, a standard error that is negative or not
finite, a |
minimax_lq_policy(state_gain, b_lo, b_hi, state_cost, effort, terminal_cost, horizon)
¶
Robust Riccati recursion for x' = a*x + b*u with b only known to lie in an interval.
The Bellman step is homogeneous of degree two in (x, u), so substituting u = s*x turns
it into exactly the static problem of minimax_action with target = -a and
curvature = p_{k+1} (identity (8) of validation/minimax_lq.mac). The robust policy is
therefore linear feedback with a closed form per step, not a minimax search, and the value
function stays quadratic.
The adversary re-picks b at every step, which is what makes dynamic programming apply
and is normally a strict relaxation of a constant unknown effect. Here it is not, and that is a
result rather than a caveat: identity (9) shows the robust action always leaves the lower
endpoint (weakly) worst, decided by the same curvature*b_lo*D <= effort that picked the
branch, so the per-step optimum is realised by the constant effect b_lo. The value is the
constant-effect worst case rather than an upper bound on it, and
minimax_lq_certificate measures that instead of assuming it.
HONEST SCOPE. Scalar state and scalar effect: the interval is the §32 sensitivity interval for one control channel, and the multivariate lift is not this function.
minimax_lq_certificate(state_gain=1.0, b_hat=1.0, state_cost=1.0, effort=1.0, terminal_cost=1.0, horizon=5, halfwidths=(0.0, 0.1, 0.25, 0.5), grid=200001)
¶
Check the closed form against brute force, against CE, and against its own DP relaxation.
Three things, and the third is the one that could embarrass the entry. (i) On §33's own
counterexample the closed form must return the action a fine grid search finds, and must beat
the CE action's worst case -- §33 reported 0.4 against 0.5 and 0.8 against 0.8125. (ii) Over a
sweep of interval half-widths the robust worst case must stay at or below CE's and grow with the
width. (iii) The robust policy is derived against an adversary that re-picks the effect every
step, which is normally a strict relaxation; relaxation_ratio measures the price of it
against a grid of constant effects, and lower_endpoint_margin measures the structural
reason the price is zero -- the lower endpoint is the worst one at every step.
certainty_equivalence_control(effect_estimate, target)
¶
CE control u = target/b_hat -- centres the outcome b*u at target under b_hat.
confounding_robust_control(effect_estimate, halfwidth, target, overshoot_penalty, undershoot_penalty)
¶
Minimax control over the §32 effect interval [b_hat +/- halfwidth] under asymmetric loss.
Loss alpha*(y-target)_+ + beta*(target-y)_+ on the outcome y = b*u; the controller does
not know b in the confounding interval. The minimax u balances the two weighted worst
tails (validation/confounding_robust_control.mac): u = (alpha+beta)*target /
(alpha*(b_hat+D) + beta*(b_hat-D)). Equals u_CE/(1 + kappa*D) with
kappa = (alpha-beta)/((alpha+beta)*b_hat) -- the pessimism radius D SHIFTS the gain, a
sign DICHOTOMY (Rocq): a>=b -> kappa>=0 -> u_rob<=u_CE, conservative
(shift_factor_nonneg / robust_gain_conservative); a<=b -> kappa<=0 ->
u_rob>=u_CE, aggressive (shift_factor_nonpos / robust_gain_aggressive). CE when
symmetric (a=b -> kappa=0). Here a=overshoot_penalty, b=undershoot_penalty.
SCOPE: a minimax STATIC tracking toy, not a general robust controller -- assumes
b_hat > halfwidth > 0 (the effect interval has an identified sign, so the denominator is
positive), target > 0, alpha, beta > 0, and NO effort/control penalty (pure outcome
tracking). The dynamic/effort-penalised case is out of scope.
worst_case_asymmetric_loss(control, effect_estimate, halfwidth, target, overshoot_penalty, undershoot_penalty)
¶
Worst-case asymmetric loss of applying control over the §32 effect interval.
For u > 0 the outcome is monotone in b, so the worst overshoot is at b_hat+D and the
worst undershoot at b_hat-D; the worst-case loss is the max of the two weighted tails.
asymmetric_control_improvement(effect_estimate, halfwidth, target, overshoot_penalty, undershoot_penalty)
¶
Piecewise worst-case-loss improvement W_CE - W_rob of the §35 robust controller over CE.
W_rob = 2*a*b*target*D/Q (branch-free, Q = (a+b)*b_hat + (a-b)*D), but
W_CE = max(a,b)*target*D/b_hat, so the improvement is PIECEWISE (reviewer-8): the
OVERSHOOT-dominant branch a*target*D*(a-b)*(b_hat+D)/(b_hat*Q) for a >= b and the
UNDERSHOOT-dominant branch b*target*D*(b-a)*(b_hat-D)/(b_hat*Q) for b >= a (Rocq
rob_gap_* / rob_gap_under_*). Both are >= 0 for b_hat > D > 0 and > 0 iff
a != b and D > 0 -- the old single formula only covered a >= b (missing Result 37's
b = 4*a regime). Here a = overshoot_penalty, b = undershoot_penalty.
confounding_robust_control_certificate(effect_estimate=1.3, halfwidth=0.25, target=1.0, overshoot_penalty=3.0, undershoot_penalty=1.0)
¶
Confirm the closed-form minimax control is the numeric argmin and strictly beats CE.
Grid-searches the worst-case loss over u (the analytic u_robust should be its argmin),
then checks the robust control is shifted below CE and its worst-case loss is strictly smaller,
while a symmetric loss reproduces CE exactly.
confounding_regret_floor_certificate(b_true=1.3, effort=0.4, target=1.0, confounding_levels=(0.02, 0.04, 0.06, 0.08, 0.1), action_noise=0.6, n=200000, seed=0)
¶
Show the §33 Delta^2 regret floor emerges from real confounded data, not just algebra.
A confounder z drives both the action (u = z + noise) and the outcome
(y = b_true*u + gamma*z + noise). Naive OLS of y on u returns an effect biased by
the confounding gamma -- an irreducible Delta that does NOT vanish with n. The CE
controller uses that biased effect on the true plant, and its measured regret scales as
Delta^2 (log-log slope ~2), matching the analytic L_reg*Delta^2 -- the floor.
confounding_robust_control_benchmark(b_true=2.0, target=1.0, overshoot_penalty=1.0, undershoot_penalty=4.0, sensitivity_gamma=2.5, target_confounding=0.8, confounding_levels=(0.0, 0.4, 0.8, 1.2), incentive_demand_corr=1.0, action_noise=0.6, n_markets=300, n_periods=400, seed=0)
¶
Ground §35: does the confounding-robust controller beat CE on a full marketplace pipeline?
Synthetic OBSERVATIONAL confounded marketplace (not a randomised switchback -- the action
follows the demand confounder z, so this is confounded logging, not an experiment): the naive
effect estimate is biased, so the
CE controller (trusting it) under-incentivises and misses completions -- expensive when churn
(undershoot_penalty) dominates budget waste (overshoot_penalty). The §35 controller uses
an assumed sensitivity Gamma and a half-width D. Calibration (named): §32 gives
D = (Gamma-1)/(Gamma+1)*(CVaR_up - CVaR_lo); for THIS synthetic benchmark we calibrate the
sensitivity gap to the estimate, CVaR_up - CVaR_lo := b_hat, so D = (Gamma-1)/(Gamma+1)*
b_hat -- a benchmark assumption, not a general consequence of §32.
Reports realised cost across a confounding sweep: the robust controller bounds the WORST-CASE cost (pessimism) and wins beyond a problem-dependent confounding threshold, while paying a measurable premium near zero confounding (its conservatism then costs more than it saves) -- the honest robustness trade-off, not a universal win.
confounding_robust_tracking_loop(a, b_true, b_hat, halfwidth, x_target, overshoot_penalty, undershoot_penalty, x0=0.0, n_steps=30, noise_scale=0.05, seed=0)
¶
Roll the confounded scalar plant x' = a*x + b_true*u + noise under the §35 controller.
Lifts the STATIC §35 minimax into a receding-horizon CLOSED LOOP (the reviewers' recurring
critique): each step aims the one-step target tau_t = x_target - a*x_t with
confounding_robust_control trusting the biased offline b_hat while the TRUE plant
uses b_true (the model/plant split of chc.flagship.closed_loop, in NumPy to keep
this module x64-flag-independent). halfwidth = 0 recovers certainty-equivalence exactly (the
§35 formula collapses to tau/b_hat); halfwidth > 0 (the §32 radius) is the robust
controller. Returns the realised trajectory, the applied controls, and the accumulated
ASYMMETRIC stage cost Sum_t [alpha*(x_t - x_target)_+ + beta*(x_target - x_t)_+].
confounding_robust_tracking_benchmark(a=0.6, b_true=2.0, x_target=1.0, overshoot_penalty=1.0, undershoot_penalty=4.0, sensitivity_gamma=2.5, target_confounding=0.8, confounding_levels=(0.0, 0.4, 0.8, 1.2), incentive_demand_corr=1.0, action_noise=0.6, n_steps=30, noise_scale=0.05, n_markets=200, n_periods=400, seed=0)
¶
The DYNAMIC §37: does the confounding-robust controller beat CE in CLOSED LOOP over time?
Same honest story as confounding_robust_control_benchmark, now closed-loop over
n_steps on the confounded plant x' = a*x + b_true*u + noise: a demand confounder biases
the offline b_hat, the CE controller under-actuates and undershoots x_target (expensive
when churn = undershoot_penalty dominates budget waste), the §35 controller uses the §32
radius (calibrated D = (Gamma-1)/(Gamma+1)*b_hat, a benchmark assumption) to hedge. Reports
mean realised closed-loop cost across a confounding sweep: robust bounds the worst case and wins
beyond a problem-dependent threshold, paying a bounded premium near zero confounding.
space_time_fold_certificate(m=8, n_times=4, gammas=(1.0, 0.7, 0.4), phi=0.6, lags=(1, 2, 3, 4), dmax_grid=(1, 2, 3, 4), k_folds=2, n_draws=200000, restarts=30, seed=20260904)
¶
RESULT 59 -- folds on BOTH axes (derived in validation/space_time_folds.mac, proved in
proofs/space_time_folds.v).
Result 51 showed the delayed-network covariance is separable only at lag = 0; Result 52
designed folds on the space axis alone and said so. Plan 24's P2.5 asks what the restriction
costs. Four gates, each able to fail:
- The covariance formula is the simulator's.
panel_covarianceis compared against trajectories drawn from the generative definition, never from the Kronecker form, so a wrong model shows up as a mismatch rather than as agreement with itself. - The Kronecker rank obeys the commutation law.
dmax+1when the shell operators commute (the cycle),2*dmaxwhen they do not (a path) -- and never2*dmax+1, because theq = 0andq = dmaxspatial factors are symmetric on every graph. Atlag = 0the rank must be exactly 1, which is Result 51's STEP 7 read from the Kronecker side. Both statements need a panel long enough to RESOLVE the shifts: oncelag*qpassesp-1every remaining temporal factor is a multiple of the same matrix, so the rank SATURATES strictly below the law -- measured 4 against 5 (cycle) and 7 against 8 (path) atp = 4, dmax = 4. A short panel cannot see the coupling it is being asked to design for. - Two different mistakes, and they cross. Holding a fold constant in time is one error;
scoring it with Result 52's SINGLE-SLICE weight instead of the panel's is another, and they
are not the same. Measured on a cycle (
m=8,p=4,K=2,phi=0.6) the slice weight costs3.3, 8.7, 2.8, 0.3%overlag = 1..4while the frozen time axis costs0.3, 3.3, 13.3, 19.2%. At short delay the mistake to fix is the weight; at long delay it is the axis. The gate is that crossing, not either number alone. - The heuristic is exact where exactness is checkable. On a product set small enough to enumerate, spectral rounding plus balanced swaps must return the global optimum.
The last field pair is the quieter finding: a fold held constant in time is scored by Result 52
with the SINGLE-SLICE weight phi**lag, but on a panel it really carries w_1/w_0, the
ratio of shifted-Toeplitz masses. Those differ everywhere strictly inside (0,1)
(proofs/space_time_folds.v, panel_weight_exceeds_the_slice_weight), so even the one-axis
panel design is not the cross-sectional one.
panel_estimator_certificate(cluster_counts=(2, 6, 20), cluster_size=12, n_times=12, phi=0.9, lag=1, k_folds=2, draws=300, bootstrap=2000, disturbance_scale=2.0, seed=20260904)
¶
RESULT 60 -- Result 51's Psi against a real cross-fitted DML fit.
Result 51 says in its own scope note that Psi is a functional of the process, evaluated with
the fold operator held fixed, and "not a re-derived estimator". This runs the comparison that
can settle it: two fold partitions of the same panel -- parity around each cycle against
contiguous blocks -- fitted by chc.network_causal.estimate_network_effects, with the
realised variance ratio bootstrapped over draws and put against the functional's prediction.
The algebra says what order to expect before the experiment (panel_estimator_gate.mac): the
partition enters Psi exactly once, linearly, with positive weight, and with a block-diagonal
covariance over g independent clusters the whole effect is O(1/g). So the design law is
a FINITE-CLUSTER statement; it washes out in the number of clusters, not in the number of rows.
The measured answer, and it is not the flattering one. At m = 12, p = 12, K = 2,
300 draws per cell, this routine at its defaults under float32::
g phi predicted measured 95% CI covers
2 0.3 0.8241 0.7521 [0.6501, 0.8811] yes
6 0.3 0.9343 1.0087 [0.8921, 1.1267] yes
20 0.3 0.9794 0.9702 [0.9072, 1.0420] yes
2 0.9 0.4689 0.3566 [0.2743, 0.4671] no
6 0.9 0.7396 0.5781 [0.4668, 0.7126] no
20 0.9 0.9064 0.7556 [0.6467, 0.8912] no
Where the effect is large enough to resolve, the verdict is sharp. At phi = 0.9 every
interval excludes 1 on the predicted side, the ratio washes out toward 1 with g exactly as
the algebra says, and the functional is CONSERVATIVE in every cell -- the real estimator gains
more from the good partition than Psi predicts, by 17% to 24% -- with the interval excluding
the prediction each time. seed=20000, and the default seeds under jax_enable_x64, repeat
the conservatism in every cell, by 14% to 26%, and every interval there excludes 1, but three of
their six intervals cover the prediction: at g = 2 on both streams, and at g = 20 on
seed=20000. As a ranking rule the functional holds; as a point
predictor of an estimator's variance ratio it does not, and Result 51's scope note is now
specific rather than cautious. The likely reason is in plain sight: Psi models the
nuisance step as an exact projection onto fold indicators, while the estimator uses a ridge
polynomial and then a second OLS stage whose own sandwich the functional does not carry.
At phi = 0.3 the predicted reduction is 18% at g = 2 -- resolved from 1 on all three
streams, and covered -- and 2% to 7% at g >= 6, below what 300 draws can see: the six
estimates there, across the three streams, span 0.80 to 1.04, on both sides of the prediction
and of 1, and one of their six intervals, seed=20000's at g = 6, excludes both.
The gates are the three claims the functional makes: sign agreement in every cell, washout in
both sequences, and conservatism. At phi = 0.9 they hold on every stream tried. At
phi = 0.3 they compare point estimates against an effect smaller than the Monte-Carlo
error, so ok is False on two streams of the three -- the gate reporting that it cannot see,
which is what it should report. Coverage is counted and reported, never gated -- a certificate
that gated on it would have to be tuned until it passed, which is the opposite of a gate.
TWO SAMPLING FACTS THIS DEPENDS ON. draws must not be small: a variance ratio from a few
dozen paired draws is noisy, and the conservatism finding rests on the two-cluster cell. At
phi = 0.9 on the default seeds it holds under float32 at 40, 80, 120 and 300 draws; under
jax_enable_x64 it fails at 40, 80 and 120, where the two-cluster ratio reads 0.544, 0.537
and 0.484 against the prediction 0.469, and holds at 300, the default, where the table and the
claims above were measured. And
chc.network_causal.DelayedNetworkPanel.sample derives its NumPy seed from a JAX key, so
the panel DRAWN AT A GIVEN SEED DIFFERS between jax_enable_x64 settings -- randint on
the same key returns 1563838340 at x32 and 358276949 at x64 -- which is why the x64 run above
is a third stream and not a check on rounding.