Characterizing and mitigating phase distortions in quantum and nonlinear optical systems