Improving Statistical Inference Through Flexible Approximations