Towards Efficient Privacy-Preserving Deep Learning: HE-Friendly Structures, Flexible Pruning, HE-Efficient Architectures, and Secure Transformer Token Drop