On the limitations of representing functions on sets
Recent work on the representation of functions on sets has considered the use of summation in a latent space to enforce permutation invariance. In particular, it has been conjectured that the dimension of this latent space may remain fixed as the cardinality of the sets under consideration increases...
| Main Authors: | , , , , |
|---|---|
| Format: | Conference item |
| Published: |
Journal of Machine Learning Research
2019
|
| _version_ | 1797064039403094016 |
|---|---|
| author | Wagstaff, E Fuchs, F Engelcke, M Posner, I Osborne, M |
| author_facet | Wagstaff, E Fuchs, F Engelcke, M Posner, I Osborne, M |
| author_sort | Wagstaff, E |
| collection | OXFORD |
| description | Recent work on the representation of functions on sets has considered the use of summation in a latent space to enforce permutation invariance. In particular, it has been conjectured that the dimension of this latent space may remain fixed as the cardinality of the sets under consideration increases. However, we demonstrate that the analysis leading to this conjecture requires mappings which are highly discontinuous and argue that this is only of limited practical use. Motivated by this observation, we prove that an implementation of this model via continuous mappings (as provided by e.g. neural networks or Gaussian processes) actually imposes a constraint on the dimensionality of the latent space. Practical universal function representation for set inputs can only be achieved with a latent dimension at least the size of the maximum number of input elements. |
| first_indexed | 2024-03-06T21:08:35Z |
| format | Conference item |
| id | oxford-uuid:3d5730ad-0070-46e9-b173-5b139d122dba |
| institution | University of Oxford |
| last_indexed | 2024-03-06T21:08:35Z |
| publishDate | 2019 |
| publisher | Journal of Machine Learning Research |
| record_format | dspace |
| spelling | oxford-uuid:3d5730ad-0070-46e9-b173-5b139d122dba2022-03-26T14:19:00ZOn the limitations of representing functions on setsConference itemhttp://purl.org/coar/resource_type/c_5794uuid:3d5730ad-0070-46e9-b173-5b139d122dbaSymplectic Elements at OxfordJournal of Machine Learning Research2019Wagstaff, EFuchs, FEngelcke, MPosner, IOsborne, MRecent work on the representation of functions on sets has considered the use of summation in a latent space to enforce permutation invariance. In particular, it has been conjectured that the dimension of this latent space may remain fixed as the cardinality of the sets under consideration increases. However, we demonstrate that the analysis leading to this conjecture requires mappings which are highly discontinuous and argue that this is only of limited practical use. Motivated by this observation, we prove that an implementation of this model via continuous mappings (as provided by e.g. neural networks or Gaussian processes) actually imposes a constraint on the dimensionality of the latent space. Practical universal function representation for set inputs can only be achieved with a latent dimension at least the size of the maximum number of input elements. |
| spellingShingle | Wagstaff, E Fuchs, F Engelcke, M Posner, I Osborne, M On the limitations of representing functions on sets |
| title | On the limitations of representing functions on sets |
| title_full | On the limitations of representing functions on sets |
| title_fullStr | On the limitations of representing functions on sets |
| title_full_unstemmed | On the limitations of representing functions on sets |
| title_short | On the limitations of representing functions on sets |
| title_sort | on the limitations of representing functions on sets |
| work_keys_str_mv | AT wagstaffe onthelimitationsofrepresentingfunctionsonsets AT fuchsf onthelimitationsofrepresentingfunctionsonsets AT engelckem onthelimitationsofrepresentingfunctionsonsets AT posneri onthelimitationsofrepresentingfunctionsonsets AT osbornem onthelimitationsofrepresentingfunctionsonsets |