DOCTOR: A Simple Method for Detecting Misclassification Errors

Federica Granese (1) , Marco Romanelli (2) , Daniele Gorla (3) , Catuscia Palamidessi (1) , Pablo Piantanida (2)

1 COMETE - Concurrency, Mobility and Transactions
2 L2S - Laboratoire des signaux et systèmes
3 UNIROMA - Università degli Studi di Roma "La Sapienza" = Sapienza University [Rome]

Deep neural networks (DNNs) have shown to perform very well on large scale object recognition problems and lead to widespread use for real-world applications, including situations where DNN are implemented as "black boxes". A promising approach to secure their use is to accept decisions that are likely to be correct while discarding the others. In this work, we propose DOCTOR, a simple method that aims to identify whether the prediction of a DNN classifier should (or should not) be trusted so that, consequently, it would be possible to accept it or to reject it. Two scenarios are investigated: Totally Black Box (TBB) where only the soft-predictions are available and Partially Black Box (PBB) where gradient-propagation to perform input pre-processing is allowed. Empirically, we show that DOCTOR outperforms all state-of-the-art methods on various well-known images and sentiment analysis datasets. In particular, we observe a reduction of up to 4% of the false rejection rate (FRR) in the PBB scenario. DOCTOR can be applied to any pre-trained model, it does not require prior information about the underlying dataset and is as simple as the simplest available methods in the literature.

Domaines: Apprentissage [cs.LG]; Cryptographie et sécurité [cs.CR]

Federica Granese, Marco Romanelli, Daniele Gorla, Catuscia Palamidessi, Pablo Piantanida. DOCTOR: A Simple Method for Detecting Misclassification Errors. Advances in Neural Information Processing Systems (NeurIPS), 2021, Virtual event, United States. pp.5669--5681. ⟨hal-03624023v1⟩