Open Vocabulary Scene Parsing
Recognizing arbitrary objects in the wild has been a challenging problem due to the limitations of existing classification models and datasets. In this paper, we propose a new task that aims at parsing scenes with a large and open vocabulary, and several evaluation metrics are explored for this prob...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Institute of Electrical and Electronics Engineers (IEEE)
2020
|
| Online Access: | https://hdl.handle.net/1721.1/123479 |
| _version_ | 1811085375949504512 |
|---|---|
| author | Zhao, Hang Puig Fernandez, Xavier Zhou, Bolei Fidler, Sanja Torralba, Antonio |
| author2 | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science |
| author_facet | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Zhao, Hang Puig Fernandez, Xavier Zhou, Bolei Fidler, Sanja Torralba, Antonio |
| author_sort | Zhao, Hang |
| collection | MIT |
| description | Recognizing arbitrary objects in the wild has been a challenging problem due to the limitations of existing classification models and datasets. In this paper, we propose a new task that aims at parsing scenes with a large and open vocabulary, and several evaluation metrics are explored for this problem. Our approach is a joint image pixel and word concept embeddings framework, where word concepts are connected by semantic relations. We validate the open vocabulary prediction ability of our framework on ADE20K dataset which covers a wide variety of scenes and objects. We further explore the trained joint embedding space to show its interpretability. Keywords: streaming media; vocabulary; training; semantics; predictive models; visualization |
| first_indexed | 2024-09-23T13:08:36Z |
| format | Article |
| id | mit-1721.1/123479 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T13:08:36Z |
| publishDate | 2020 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| record_format | dspace |
| spelling | mit-1721.1/1234792022-10-01T13:17:05Z Open Vocabulary Scene Parsing Zhao, Hang Puig Fernandez, Xavier Zhou, Bolei Fidler, Sanja Torralba, Antonio Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Recognizing arbitrary objects in the wild has been a challenging problem due to the limitations of existing classification models and datasets. In this paper, we propose a new task that aims at parsing scenes with a large and open vocabulary, and several evaluation metrics are explored for this problem. Our approach is a joint image pixel and word concept embeddings framework, where word concepts are connected by semantic relations. We validate the open vocabulary prediction ability of our framework on ADE20K dataset which covers a wide variety of scenes and objects. We further explore the trained joint embedding space to show its interpretability. Keywords: streaming media; vocabulary; training; semantics; predictive models; visualization National Science Foundation (U.S.) (Grant 1524817) Samsung Electronics Co. (Grant 1524817) 2020-01-20T18:41:32Z 2020-01-20T18:41:32Z 2017-12-25 2019-07-11T16:43:19Z Article http://purl.org/eprint/type/ConferencePaper 9781538610329 9781538610336 2380-7504 https://hdl.handle.net/1721.1/123479 Zhao, Hang et al. "Open Vocabulary Scene Parsing." 2017 IEEE International Conference on Computer Vision (ICCV), October 2017, Venice, Italy, Institute of Electrical and Electronics Engineers (IEEE), December 2017 © 2017 IEEE en http://dx.doi.org/10.1109/iccv.2017.221 2017 IEEE International Conference on Computer Vision (ICCV) Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute of Electrical and Electronics Engineers (IEEE) MIT web domain |
| spellingShingle | Zhao, Hang Puig Fernandez, Xavier Zhou, Bolei Fidler, Sanja Torralba, Antonio Open Vocabulary Scene Parsing |
| title | Open Vocabulary Scene Parsing |
| title_full | Open Vocabulary Scene Parsing |
| title_fullStr | Open Vocabulary Scene Parsing |
| title_full_unstemmed | Open Vocabulary Scene Parsing |
| title_short | Open Vocabulary Scene Parsing |
| title_sort | open vocabulary scene parsing |
| url | https://hdl.handle.net/1721.1/123479 |
| work_keys_str_mv | AT zhaohang openvocabularysceneparsing AT puigfernandezxavier openvocabularysceneparsing AT zhoubolei openvocabularysceneparsing AT fidlersanja openvocabularysceneparsing AT torralbaantonio openvocabularysceneparsing |