Imperfect Quantum Photonic Integrated Circuits With Quantum Dot Phase Shifters