Feature quantization for parsimonious and interpretable predictive models

Adrien Ehrhardt (1) , Christophe Biernacki (1) , Vincent Vandewalle (1) , Philippe Heinrich (2)

1 MODAL - MOdel for Data Analysis and Learning
2 LPP - Laboratoire Paul Painlevé - UMR 8524

For regulatory and interpretability reasons, the logistic regression is still widely used by financial institutions to learn the refunding probability of a loan from applicant's historical data. To improve prediction accuracy and interpretability, a preprocessing step quantizing both continuous and categorical data is usually performed: continuous features are discretized by assigning factor levels to intervals and, if numerous, levels of categorical features are grouped. However, a better predictive accuracy can be reached by embedding this quantization estimation step directly into the predictive estimation step itself. By doing so, the predictive loss has to be optimized on a huge and untractable discontinuous quantization set. To overcome this difficulty, we introduce a specific two-step optimization strategy: first, the optimization problem is relaxed by approximating discontinuous quan-tization functions by smooth functions; second, the resulting relaxed optimization problem is solved via a particular neural network and stochas-tic gradient descent. The strategy gives then access to good candidates for the original optimization problem after a straightforward maximum a posteriori procedure to obtain cutpoints. The good performances of this approach, which we call glmdisc, are illustrated on simulated and real data from the UCI library and Crédit Agricole Consumer Finance (a major Eu-ropean historic player in the consumer credit market). The results show that practitioners finally have an automatic all-in-one tool that answers their recurring needs of quantization for predictive tasks.

Domaines: Méthodologie [stat.ME]

Adrien Ehrhardt, Christophe Biernacki, Vincent Vandewalle, Philippe Heinrich. Feature quantization for parsimonious and interpretable predictive models. 2019. ⟨hal-01949135v2⟩