Neural Teleportation
In this paper, we explore a process called neural teleportation, a mathematical consequence of applying quiver representation theory to neural networks. Neural teleportation <i>teleports</i> a network to a new position in the weight space and preserves its function. This phenomenon comes...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-01-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/11/2/480 |
| _version_ | 1797438932825145344 |
|---|---|
| author | Marco Armenta Thierry Judge Nathan Painchaud Youssef Skandarani Carl Lemaire Gabriel Gibeau Sanchez Philippe Spino Pierre-Marc Jodoin |
| author_facet | Marco Armenta Thierry Judge Nathan Painchaud Youssef Skandarani Carl Lemaire Gabriel Gibeau Sanchez Philippe Spino Pierre-Marc Jodoin |
| author_sort | Marco Armenta |
| collection | DOAJ |
| description | In this paper, we explore a process called neural teleportation, a mathematical consequence of applying quiver representation theory to neural networks. Neural teleportation <i>teleports</i> a network to a new position in the weight space and preserves its function. This phenomenon comes directly from the definitions of representation theory applied to neural networks and it turns out to be a very simple operation that has remarkable properties. We shed light on the surprising and counter-intuitive consequences neural teleportation has on the loss landscape. In particular, we show that teleportation can be used to explore loss level curves, that it changes the local loss landscape, sharpens global minima and boosts back-propagated gradients at any moment during the learning process. |
| first_indexed | 2024-03-09T11:45:26Z |
| format | Article |
| id | doaj.art-59d74c6ce50c4c649a3610ef20b917f9 |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-09T11:45:26Z |
| publishDate | 2023-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-59d74c6ce50c4c649a3610ef20b917f92023-11-30T23:22:47ZengMDPI AGMathematics2227-73902023-01-0111248010.3390/math11020480Neural TeleportationMarco Armenta0Thierry Judge1Nathan Painchaud2Youssef Skandarani3Carl Lemaire4Gabriel Gibeau Sanchez5Philippe Spino6Pierre-Marc Jodoin7Department of Computer Science, Université de Sherbrooke, Sherbrooke, QC J1K 2R1, CanadaDepartment of Computer Science, Université de Sherbrooke, Sherbrooke, QC J1K 2R1, CanadaDepartment of Computer Science, Université de Sherbrooke, Sherbrooke, QC J1K 2R1, CanadaDepartment of Computer Science, Université de Bourgogne Franche-Comte, 21000 Dijon, FranceDepartment of Computer Science, Université de Sherbrooke, Sherbrooke, QC J1K 2R1, CanadaDepartment of Computer Science, Université de Sherbrooke, Sherbrooke, QC J1K 2R1, CanadaDepartment of Computer Science, Université de Sherbrooke, Sherbrooke, QC J1K 2R1, CanadaDepartment of Computer Science, Université de Sherbrooke, Sherbrooke, QC J1K 2R1, CanadaIn this paper, we explore a process called neural teleportation, a mathematical consequence of applying quiver representation theory to neural networks. Neural teleportation <i>teleports</i> a network to a new position in the weight space and preserves its function. This phenomenon comes directly from the definitions of representation theory applied to neural networks and it turns out to be a very simple operation that has remarkable properties. We shed light on the surprising and counter-intuitive consequences neural teleportation has on the loss landscape. In particular, we show that teleportation can be used to explore loss level curves, that it changes the local loss landscape, sharpens global minima and boosts back-propagated gradients at any moment during the learning process.https://www.mdpi.com/2227-7390/11/2/480quiver representationsneural networksteleportation |
| spellingShingle | Marco Armenta Thierry Judge Nathan Painchaud Youssef Skandarani Carl Lemaire Gabriel Gibeau Sanchez Philippe Spino Pierre-Marc Jodoin Neural Teleportation Mathematics quiver representations neural networks teleportation |
| title | Neural Teleportation |
| title_full | Neural Teleportation |
| title_fullStr | Neural Teleportation |
| title_full_unstemmed | Neural Teleportation |
| title_short | Neural Teleportation |
| title_sort | neural teleportation |
| topic | quiver representations neural networks teleportation |
| url | https://www.mdpi.com/2227-7390/11/2/480 |
| work_keys_str_mv | AT marcoarmenta neuralteleportation AT thierryjudge neuralteleportation AT nathanpainchaud neuralteleportation AT youssefskandarani neuralteleportation AT carllemaire neuralteleportation AT gabrielgibeausanchez neuralteleportation AT philippespino neuralteleportation AT pierremarcjodoin neuralteleportation |