Judicious judgment meets unsettling updating: dilation, sure loss, and Simpson’s paradox (with discussion)

Link to paper (PDF)

Link to rejoinder (PDF)

Statistical learning using imprecise probabilities is gaining more attention because it presents an alternative strategy for reducing irreplicable findings by freeing the user from the task of making up unwarranted high-resolution assumptions. However, model updating as a mathematical operation is inherently exact, hence updating imprecise models requires the user’s judgment in choosing among competing updating rules. These rules often lead to incompatible inferences, and can exhibit unsettling phenomena like dilation, contraction and sure loss, which cannot occur with the Bayes rule and precise probabilities. We revisit a number of famous “paradoxes”, including the three prisoners/Monty Hall problem, revealing that a logical fallacy arises from a set of marginally plausible yet jointly incommensurable assumptions when updating the underlying imprecise model. We establish an equivalence between Simpson’s paradox and an implicit adoption of a pair of aggregation rules that induce sure loss. We also explore behavioral discrepancies between the generalized Bayes rule, Dempster’s rule and the Geometric rule as alternative posterior updating rules for Choquet capacities of order 2. We show that both the generalized Bayes rule and Geometric rule are incapable of updating without prior information regardless of how strong the information in our data is, and that Dempster’s rule and the Geometric rule can mathematically contradict each other with respect to dilation and contraction. Our findings show that unsettling updates reflect a collision between the rules’ assumptions and the inexactness allowed by the model itself, highlighting the invaluable role of judicious judgment in handling low-resolution information, and the care we must take when applying learning rules to update imprecise probabilities.