The Musigma webinar series

2021-02: Get the Most Out of Your Test Data

February 4th, 2021

Speaker: Chris Gross, Gross Consulting

The data not used in parameterizing your model is very valuable. Unleash the value in the out-of-sample data in both its validation and test roles. Chris will discuss the following approaches:

  • Bootstrap samples to provide statistical tests of the difference between alternative models, without assuming an underlying distribution
  • Measurement not only of the fit at a granular level, but also measurement of bias, most easily seen at an aggregated level
  • Provide empirical distributions of residuals, capturing process, parameter, and model risk, and reflecting differences in the distribution across different variables. Use bootstrapping techniques to flesh out the distribution.