Differential privacy protects individuals’ confidential information by subjecting data summaries to probabilistic perturbation mechanisms, carefully designed to minimize undue sacrifice of statistical efficiency. When properly integrated, differentially private data are conducive to exact inference when approximate computation techniques are employed. This paper shows that approximate Bayesian computation (ABC), a practical suite of methods to simulate from approximate posterior distributions of complex Bayesian models, produces exact posterior samples when applied to differentially private perturbation data. A parallel importance sampling implementation within Monte Carlo expectation-maximization for likelihood inference is also discussed. The results illustrate a duality between approximate computation on exact data, and exact computation on approximate data. A cleverly designed inferential procedure exploits the alignment between the statistical tradeoff of privacy versus efficiency, and the computational tradeoff of approximation versus exactness, so that paying the cost of one gains the benefit of both.