A spin glass model for the loss surfaces of generative adversarial networks
We present a novel mathematical model that seeks to capture the key design feature of generative adversarial networks (GANs). Our model consists of two interacting spin glasses, and we conduct an extensive theoretical analysis of the complexity of the model’s critical points using techniques from Ra...
| Main Authors: | , , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
Springer
2022
|
| _version_ | 1797054290840256512 |
|---|---|
| author | Baskerville, N Keating, J Mezzadri, F Najnudel, J |
| author_facet | Baskerville, N Keating, J Mezzadri, F Najnudel, J |
| author_sort | Baskerville, N |
| collection | OXFORD |
| description | We present a novel mathematical model that seeks to capture the key design feature of generative adversarial networks (GANs). Our model consists of two interacting spin glasses, and we conduct an extensive theoretical analysis of the complexity of the model’s critical points using techniques from Random Matrix Theory. The result is insights into the loss surfaces of large GANs that build upon prior insights for simpler networks, but also reveal new structure unique to this setting which explains the greater difficulty of training GANs.
|
| first_indexed | 2024-03-06T18:55:09Z |
| format | Journal article |
| id | oxford-uuid:119653a3-56b5-4177-9bc8-829f659f6b61 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-06T18:55:09Z |
| publishDate | 2022 |
| publisher | Springer |
| record_format | dspace |
| spelling | oxford-uuid:119653a3-56b5-4177-9bc8-829f659f6b612022-03-26T10:03:08ZA spin glass model for the loss surfaces of generative adversarial networksJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:119653a3-56b5-4177-9bc8-829f659f6b61EnglishSymplectic ElementsSpringer2022Baskerville, NKeating, JMezzadri, FNajnudel, JWe present a novel mathematical model that seeks to capture the key design feature of generative adversarial networks (GANs). Our model consists of two interacting spin glasses, and we conduct an extensive theoretical analysis of the complexity of the model’s critical points using techniques from Random Matrix Theory. The result is insights into the loss surfaces of large GANs that build upon prior insights for simpler networks, but also reveal new structure unique to this setting which explains the greater difficulty of training GANs. |
| spellingShingle | Baskerville, N Keating, J Mezzadri, F Najnudel, J A spin glass model for the loss surfaces of generative adversarial networks |
| title | A spin glass model for the loss surfaces of generative adversarial networks |
| title_full | A spin glass model for the loss surfaces of generative adversarial networks |
| title_fullStr | A spin glass model for the loss surfaces of generative adversarial networks |
| title_full_unstemmed | A spin glass model for the loss surfaces of generative adversarial networks |
| title_short | A spin glass model for the loss surfaces of generative adversarial networks |
| title_sort | spin glass model for the loss surfaces of generative adversarial networks |
| work_keys_str_mv | AT baskervillen aspinglassmodelforthelosssurfacesofgenerativeadversarialnetworks AT keatingj aspinglassmodelforthelosssurfacesofgenerativeadversarialnetworks AT mezzadrif aspinglassmodelforthelosssurfacesofgenerativeadversarialnetworks AT najnudelj aspinglassmodelforthelosssurfacesofgenerativeadversarialnetworks AT baskervillen spinglassmodelforthelosssurfacesofgenerativeadversarialnetworks AT keatingj spinglassmodelforthelosssurfacesofgenerativeadversarialnetworks AT mezzadrif spinglassmodelforthelosssurfacesofgenerativeadversarialnetworks AT najnudelj spinglassmodelforthelosssurfacesofgenerativeadversarialnetworks |