Target encoding is a technique used in machine learning to convert categorical variables into numerical values by replacing each category with the mean of the target variable for that category. This method can help improve model performance by incorporating the predictive power of categorical features, while also mitigating the risk of overfitting through techniques like smoothing and cross-validation-based encoding.