Technical Comment on ‘‘Policy impacts of statistical uncertainty and privacy’’

Link (Science)

Steed et al. (1) illustrates the crucial impact that the quality of official statistical data products may exert on the accuracy, stability, and equity of policy decisions on which they are based. The authors remind us that data, however responsibly curated, can be fallible. With this comment, we underscore the importance of conducting principled quality assessment of official statistical data products. We observe that the quality assessment procedure employed by Steed et al. needs improvement, due to (i) the inadmissibility of the estimator used, and (ii) the inconsistent probability model it induces on the joint space of the estimator and the observed data. We discuss the design of alternative statistical methods to conduct principled quality assessments for official statistical data products, showcasing two simulation-based methods for admissible minimax shrinkage estimation via multilevel empirical Bayesian modeling. For policymakers and stakeholders to accurately gauge the context-specific usability of data, the assessment should take into account both uncertainty sources inherent to the data and the downstream use cases, such as policy decisions based on those data products.

(1) R. Steed, T. Liu, Z. S. Wu, A. Acquisti, Policy impacts of statistical uncertainty and privacy. Science 377, 928–931 (2022).