chc.offpolicy

Stability tier: evolving.

Off-policy evaluation of a one-step policy value from logged data.

Given logs (x, u, r) collected under a behaviour policy, off_policy_value estimates by inverse-propensity weighting the value of a candidate policy on the logger's own states: E_{x ~ d_b} E_{u ~ pi(.|x)} [r], the contextual-bandit value. For a policy that moves the state -- a feedback plan on a plant with memory -- that is not the value of deploying it. On a loop whose candidate spreads the state past twice the logger's variance, SNIPS converges to -2.92 against a deployed value of -5.76, with the overlap flag set (test_the_one_step_estimand_is_not_the_deployed_value).

Overlap is summarised by the effective sample size of the one-step weights, and overlap_ok reports whether its fraction clears a threshold. Nothing here refuses: the flag is the caller's to read, and it sees one-step overlap only -- the stationary state-action ratio a deployed plan needs can have infinite variance while it reads True. chc.evaluation.evaluate_plan estimates the deployed value, and refuses by name when the logs cannot give it.

GaussianPolicy

Bases: Module

Diagonal-Gaussian policy u ~ N(W x + b, diag(exp(log_std))^2).

fit_behavior_policy(xs, us)

Least-squares Gaussian fit of u ~ N(W x + b, sigma^2) from logged (x, u).

One sigma per action for every state. On logs whose noise varies with the state the fitted density is not the logger's, and weights built on it are biased: a candidate identical to the logger has mean weight 1 + chi^2(pi_b || pi_hat), which is r^2 / sqrt(2 r^2 - 1) for a standard-deviation misfit r and infinite at r^2 <= 1/2. In a sequential deployment gate simulated on heteroscedastic logs, that bias took the type-I error from 0.013 to 1.000. Where the propensity was logged at decision time, use it instead.

off_policy_value(data, target, behavior, ess_fraction_threshold=0.1)

IPS / self-normalised estimates of the one-step value, plus overlap diagnostics.

data has keys x (N, n), u (N, m), r (N,). Returns IPS and self-normalised (SNIPS) estimates of E_{x ~ d_b} E_{u ~ pi}[r] (the module docstring says why that is not a deployed plan's value), the effective sample size and its fraction, max_weight -- the largest normalised weight, max w / sum w, a share of the total rather than a weight -- and overlap_ok, whether the ESS fraction clears ess_fraction_threshold.