Guided Fine-Tuning for Large-Scale Material Transfer

Valentin Deschaintre (1, 2, 3, 4) , George Drettakis (1, 4) , Adrien Bousseau (1, 4)

1 UniCA - Université Côte d'Azur
2 ANSYS
3 Imperial College London
4 GRAPHDECO - GRAPHics and DEsign with hEterogeneous COntent

SVBRDF exemplar Rendering HD Input SVBRDF exemplar Rendering Figure 1: Our method transfers the appearance of one or a few exemplar SVBRDFs to a target picture. This approach allows the capture of large planar surfaces taken with ambient lighting (far left), by extracting the SVBRDF exemplars from close-up flash pictures (lower left), as well as the creation of plausible SVBRDFs from internet pictures by using existing artist-designed materials as exemplars (right). Please see supplemental materials for high-resolution SVBRDF parameter maps and animated renderings of all our results, which give a much better impression of the material properties. Abstract We present a method to transfer the appearance of one or a few exemplar SVBRDFs to a target image representing similar materials. Our solution is extremely simple: we fine-tune a deep appearance-capture network on the provided exemplars, such that it learns to extract similar SVBRDF values from the target image. We introduce two novel material capture and design workflows that demonstrate the strength of this simple approach. Our first workflow allows to produce plausible SVBRDFs of large-scale objects from only a few pictures. Specifically, users only need take a single picture of a large surface and a few close-up flash pictures of some of its details. We use existing methods to extract SVBRDF parameters from the close-ups, and our method to transfer these parameters to the entire surface, enabling the lightweight capture of surfaces several meters wide such as murals, floors and furniture. In our second workflow, we provide a powerful way for users to create large SVBRDFs from internet pictures by transferring the appearance of existing, pre-designed SVBRDFs. By selecting different exemplars, users can control the materials assigned to the target image, greatly enhancing the creative possibilities offered by deep appearance capture.

Mots clés: CCS Concepts • Computing methodologies → Reflectance modeling; Image processing; Keywords: material transfer; material capture; appearance cap- ture; SVBRDF; deep learning; fine tuning

Domaines: Traitement des images [eess.IV]

Valentin Deschaintre, George Drettakis, Adrien Bousseau. Guided Fine-Tuning for Large-Scale Material Transfer. Computer Graphics Forum, 2020, 39. ⟨hal-02869651v2⟩