Skip to Main content Skip to Navigation
Conference papers

Comparison of subjective methods, with and without explicit reference, for quality assessment of 3D graphics

Abstract : Numerous methodologies for subjective quality assessment exist in the field of image processing. In particular, the Absolute Category Rating with Hidden Reference (ACR-HR) and the Double Stimulus Impairment Scale (DSIS) are considered two of the most prominent methods for assessing the visual quality of 2D images and videos. Are these methods valid/accurate to evaluate the perceived quality of 3D graphics data? Is the presence of an explicit reference necessary , due to the lack of human prior knowledge on 3D graphics data compared to natural images/videos? To answer these questions, we compare these two subjective methods (ACR-HR and DSIS) on a dataset of high-quality colored 3D models, impaired with various distortions. These subjective experiments were conducted in a virtual reality (VR) environment. Our results show differences in the performance of the methods depending on the 3D contents and the types of distortions. We show that DSIS outperforms ACR-HR in term of accuracy and points out a stable performance. Results also yield interesting conclusions on the importance of a reference for judging the quality of 3D graphics. We finally provide recommendations regarding the influence of the number of observers on the accuracy.
Document type :
Conference papers
Complete list of metadatas

Cited literature [24 references]  Display  Hide  Download
Contributor : Yana Nehme <>
Submitted on : Tuesday, October 1, 2019 - 3:44:06 PM
Last modification on : Tuesday, January 5, 2021 - 4:26:08 PM


Files produced by the author(s)



Yana Nehmé, Jean-Philippe Farrugia, Florent Dupont, Patrick Le Callet, Guillaume Lavoué. Comparison of subjective methods, with and without explicit reference, for quality assessment of 3D graphics. ACM Symposium on Applied Perception 2019, Sep 2019, Barcelona, Spain. pp.1-9, ⟨10.1145/3343036.3352493⟩. ⟨hal-02297220v2⟩



Record views


Files downloads