报告摘要
Offline policy learning aims at utilizing observations collected a priori (from either fixed or adaptively evolving behavior policies) to learn an optimal individualized decision rule that achieves the best overall outcomes for a given population. Existing policy learning methods rely on a uniform overlap assumption, i.e., the propensities of exploring all actions for all individual characteristics must be lower bounded. As one has no control over the data collection process, this assumption can be unrealistic in many situations, especially when the behavior policies are allowed to evolve over time with diminishing propensities for certain actions.
In this work, we propose Pessimistic Policy Learning (PPL), a new algorithm that optimizes lower confidence bounds (LCBs) -- instead of point estimates -- of the policy values. In our theoretical analysis, we develop a new self-normalized type concentration inequality for inverse-propensity-weighting estimators, generalizing the well-known empirical Bernstein's inequality to unbounded and non-i.i.d.~data. We complement our theory with an efficient optimization algorithm via Majorization-Minimization and policy tree search, as well as extensive experiments that demonstrate the efficacy of PPL.
嘉宾简介
Ying Jin is a fifth-year PhD candidate in the Department of Statistics at Stanford University, advised by Professors Emmanuel Candès and Dominik Rothenhäusler. Her research interests include conformal prediction, selective inference, distribution robustness, and data-driven decision-making.
直播分享时间:2024年4月27日