Think Twice Before Enforcing A Notion: A Reflection and a Case Study on Fairness in Machine Learning