Match: differentiable material graphs for procedural material capture
© 2020 Owner/Author. We present MATch, a method to automatically convert photographs of material samples into production-grade procedural material models. At the core of MATch is a new library DiffMat that provides differentiable building blocks for constructing procedural materials, and automatic t...
| Main Authors: | , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Association for Computing Machinery (ACM)
2021
|
| Online Access: | https://hdl.handle.net/1721.1/134067 |
| _version_ | 1826210758254395392 |
|---|---|
| author | Shi, Liang Li, Beichen Hašan, Miloš Sunkavalli, Kalyan Boubekeur, Tamy Mech, Radomir Matusik, Wojciech |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Shi, Liang Li, Beichen Hašan, Miloš Sunkavalli, Kalyan Boubekeur, Tamy Mech, Radomir Matusik, Wojciech |
| author_sort | Shi, Liang |
| collection | MIT |
| description | © 2020 Owner/Author. We present MATch, a method to automatically convert photographs of material samples into production-grade procedural material models. At the core of MATch is a new library DiffMat that provides differentiable building blocks for constructing procedural materials, and automatic translation of large-scale procedural models, with hundreds to thousands of node parameters, into differentiable node graphs. Combining these translated node graphs with a rendering layer yields an end-to-end differentiable pipeline that maps node graph parameters to rendered images. This facilitates the use of gradient-based optimization to estimate the parameters such that the resulting material, when rendered, matches the target image appearance, as quantified by a style transfer loss. In addition, we propose a deep neural feature-based graph selection and parameter initialization method that efficiently scales to a large number of procedural graphs. We evaluate our method on both rendered synthetic materials and real materials captured as flash photographs. We demonstrate that MATch can reconstruct more accurate, general, and complex procedural materials compared to the state-of-the-art. Moreover, by producing a procedural output, we unlock capabilities such as constructing arbitrary-resolution material maps and parametrically editing the material appearance. |
| first_indexed | 2024-09-23T14:54:57Z |
| format | Article |
| id | mit-1721.1/134067 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T14:54:57Z |
| publishDate | 2021 |
| publisher | Association for Computing Machinery (ACM) |
| record_format | dspace |
| spelling | mit-1721.1/1340672023-10-05T20:17:23Z Match: differentiable material graphs for procedural material capture Shi, Liang Li, Beichen Hašan, Miloš Sunkavalli, Kalyan Boubekeur, Tamy Mech, Radomir Matusik, Wojciech Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory © 2020 Owner/Author. We present MATch, a method to automatically convert photographs of material samples into production-grade procedural material models. At the core of MATch is a new library DiffMat that provides differentiable building blocks for constructing procedural materials, and automatic translation of large-scale procedural models, with hundreds to thousands of node parameters, into differentiable node graphs. Combining these translated node graphs with a rendering layer yields an end-to-end differentiable pipeline that maps node graph parameters to rendered images. This facilitates the use of gradient-based optimization to estimate the parameters such that the resulting material, when rendered, matches the target image appearance, as quantified by a style transfer loss. In addition, we propose a deep neural feature-based graph selection and parameter initialization method that efficiently scales to a large number of procedural graphs. We evaluate our method on both rendered synthetic materials and real materials captured as flash photographs. We demonstrate that MATch can reconstruct more accurate, general, and complex procedural materials compared to the state-of-the-art. Moreover, by producing a procedural output, we unlock capabilities such as constructing arbitrary-resolution material maps and parametrically editing the material appearance. 2021-10-27T19:57:55Z 2021-10-27T19:57:55Z 2020 2021-01-29T19:47:41Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/134067 en 10.1145/3414685.3417781 ACM Transactions on Graphics Creative Commons Attribution 4.0 International license https://creativecommons.org/licenses/by/4.0/ application/pdf Association for Computing Machinery (ACM) ACM |
| spellingShingle | Shi, Liang Li, Beichen Hašan, Miloš Sunkavalli, Kalyan Boubekeur, Tamy Mech, Radomir Matusik, Wojciech Match: differentiable material graphs for procedural material capture |
| title | Match: differentiable material graphs for procedural material capture |
| title_full | Match: differentiable material graphs for procedural material capture |
| title_fullStr | Match: differentiable material graphs for procedural material capture |
| title_full_unstemmed | Match: differentiable material graphs for procedural material capture |
| title_short | Match: differentiable material graphs for procedural material capture |
| title_sort | match differentiable material graphs for procedural material capture |
| url | https://hdl.handle.net/1721.1/134067 |
| work_keys_str_mv | AT shiliang matchdifferentiablematerialgraphsforproceduralmaterialcapture AT libeichen matchdifferentiablematerialgraphsforproceduralmaterialcapture AT hasanmilos matchdifferentiablematerialgraphsforproceduralmaterialcapture AT sunkavallikalyan matchdifferentiablematerialgraphsforproceduralmaterialcapture AT boubekeurtamy matchdifferentiablematerialgraphsforproceduralmaterialcapture AT mechradomir matchdifferentiablematerialgraphsforproceduralmaterialcapture AT matusikwojciech matchdifferentiablematerialgraphsforproceduralmaterialcapture |