Impact of the subjective dataset on the performance of image quality metrics

Sylvain Tourancheau (1) , Florent Autrusseau (1) , Parvez Z. M. Sazzad (2) , Yuukou Horita (2)

1 IRCCyN - Institut de Recherche en Communications et en Cybernétique de Nantes
2 School of Science [Univ of Toyama]

The interest in objective quality assessment have significantly increased over the past decades. Several objective quality metrics have been proposed and made publicly available, moreover, several subjective quality assessment databases are distributed in order to evaluate and compare the metrics. However, several question arises: are the objective metrics behaviours constant across databases, contents and distortions? how significantly the subjective scores might fluctuate on different displays (i.e. CRT or LCD)? which objective quality metric might best evaluate a given distortion? In this article, we analyse the behaviour of four objective quality metrics (including PSNR) tested on three image databases. We demonstrate that the performances of the quality metrics can strongly fluctuate depending on the database used for testing. We also show the consistency of all metrics for two distinct displays.

Domaines: Traitement des images [eess.IV]; Traitement du signal et de l'image [eess.SP]; Traitement du signal et de l'image [eess.SP]

Sylvain Tourancheau, Florent Autrusseau, Parvez Z. M. Sazzad, Yuukou Horita. Impact of the subjective dataset on the performance of image quality metrics. IEEE International Conference on Image Processing 2008. ICIP 2008., Oct 2008, San Diego, CA, United States. ⟨hal-00321663⟩