Data leakage occurs when information from outside the training dataset is used to create a model, leading to overly optimistic performance metrics that won't generalize to new data. It compromises the validity of predictive models by inadvertently allowing them access to data that they wouldn't have in a real-world scenario.