Zansa › Answers › Forecasting and data science
What is data leakage and how can it ruin a forecast?
Data leakage is when information that would not be available at prediction time sneaks into model training or testing, making results look far better than they will be in real use. Examples are using future sales as an input, random train-test splits on time data, or features calculated using the whole dataset.
Leakage is a leading reason models perform well in a demo and badly in production. A typical case: a churn model uses a field such as 'cancellation reason', which only exists after the customer has left.
Protect yourself by:
- Splitting by time, not randomly.
- Asking of each feature: would I know this at the moment of prediction?
- Computing summaries only from data before the forecast date.
- Being suspicious of unusually high accuracy.
Want this applied to your business? Zansa builds dashboards and forecasts for small and mid-sized companies worldwide. Start with a free intro call.
Book a call