The data not used in parameterizing your model is very valuable. Unleash the value in the out-of-sample data in both its validation and test roles. Chris will discuss the following approaches:

• Bootstrap samples to provide statistical tests of the difference between alternative models, without assuming an underlying distribution
• Measurement not only of the fit at a granular level, but also measurement of bias, most easily seen at an aggregated level
• Provide empirical distributions of residuals, capturing process, parameter, and model risk, and reflecting differences in the distribution across different variables. Use bootstrapping techniques to flesh out the distribution.