On the decision boundaries of neural networks: a tropical geometry perspective
This work tackles the problem of characterizing and understanding the decision boundaries of neural networks with piecewise linear non-linearity activations. We use tropical geometry, a new development in the area of algebraic geometry, to characterize the decision boundaries of a simple network of...
| Main Authors: | , , , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
IEEE
2022
|
| _version_ | 1811139514324746240 |
|---|---|
| author | Alfarra, M Bibi, A Hammoud, H Gaafar, M Ghanem, B |
| author_facet | Alfarra, M Bibi, A Hammoud, H Gaafar, M Ghanem, B |
| author_sort | Alfarra, M |
| collection | OXFORD |
| description | This work tackles the problem of characterizing and understanding the decision boundaries of neural networks with piecewise linear non-linearity activations. We use tropical geometry, a new development in the area of algebraic geometry, to characterize the decision boundaries of a simple network of the form (Affine, ReLU, Affine). Our main finding is that the decision boundaries are a subset of a tropical hypersurface, which is intimately related to a polytope formed by the convex hull of two zonotopes. The generators of these zonotopes are functions of the network parameters. This geometric characterization provides new perspectives to three tasks. (i) We propose a new tropical perspective to the lottery ticket hypothesis, where we view the effect of different initializations on the tropical geometric representation of a network's decision boundaries. (ii) Moreover, we propose new tropical based optimization reformulations that directly influence the decision boundaries of the network for the task of network pruning. (iii) At last, we discuss the reformulation of the generation of adversarial attacks in a tropical sense. We demonstrate that one can construct adversaries in a new tropical setting by perturbing a specific set of decision boundaries by perturbing a set of parameters in the network. |
| first_indexed | 2024-09-25T04:07:18Z |
| format | Journal article |
| id | oxford-uuid:8e186540-0f1b-42de-b8d5-d551fbda6adf |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-09-25T04:07:18Z |
| publishDate | 2022 |
| publisher | IEEE |
| record_format | dspace |
| spelling | oxford-uuid:8e186540-0f1b-42de-b8d5-d551fbda6adf2024-05-30T11:01:03ZOn the decision boundaries of neural networks: a tropical geometry perspectiveJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:8e186540-0f1b-42de-b8d5-d551fbda6adfEnglishSymplectic ElementsIEEE2022Alfarra, MBibi, AHammoud, HGaafar, MGhanem, BThis work tackles the problem of characterizing and understanding the decision boundaries of neural networks with piecewise linear non-linearity activations. We use tropical geometry, a new development in the area of algebraic geometry, to characterize the decision boundaries of a simple network of the form (Affine, ReLU, Affine). Our main finding is that the decision boundaries are a subset of a tropical hypersurface, which is intimately related to a polytope formed by the convex hull of two zonotopes. The generators of these zonotopes are functions of the network parameters. This geometric characterization provides new perspectives to three tasks. (i) We propose a new tropical perspective to the lottery ticket hypothesis, where we view the effect of different initializations on the tropical geometric representation of a network's decision boundaries. (ii) Moreover, we propose new tropical based optimization reformulations that directly influence the decision boundaries of the network for the task of network pruning. (iii) At last, we discuss the reformulation of the generation of adversarial attacks in a tropical sense. We demonstrate that one can construct adversaries in a new tropical setting by perturbing a specific set of decision boundaries by perturbing a set of parameters in the network. |
| spellingShingle | Alfarra, M Bibi, A Hammoud, H Gaafar, M Ghanem, B On the decision boundaries of neural networks: a tropical geometry perspective |
| title | On the decision boundaries of neural networks: a tropical geometry perspective |
| title_full | On the decision boundaries of neural networks: a tropical geometry perspective |
| title_fullStr | On the decision boundaries of neural networks: a tropical geometry perspective |
| title_full_unstemmed | On the decision boundaries of neural networks: a tropical geometry perspective |
| title_short | On the decision boundaries of neural networks: a tropical geometry perspective |
| title_sort | on the decision boundaries of neural networks a tropical geometry perspective |
| work_keys_str_mv | AT alfarram onthedecisionboundariesofneuralnetworksatropicalgeometryperspective AT bibia onthedecisionboundariesofneuralnetworksatropicalgeometryperspective AT hammoudh onthedecisionboundariesofneuralnetworksatropicalgeometryperspective AT gaafarm onthedecisionboundariesofneuralnetworksatropicalgeometryperspective AT ghanemb onthedecisionboundariesofneuralnetworksatropicalgeometryperspective |