Practical and optimal LSH for angular distance
We show the existence of a Locality-Sensitive Hashing (LSH) family for the angular distance that yields an approximate Near Neighbor Search algorithm with the asymptotically optimal running time exponent. Unlike earlier algorithms with this property (e.g., Spherical LSH (Andoni-Indyk-Nguyen-Razensht...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Neural Information Processing Systems Foundation
2018
|
| Online Access: | http://hdl.handle.net/1721.1/113844 https://orcid.org/0000-0002-7983-9524 https://orcid.org/0000-0002-3962-721X https://orcid.org/0000-0002-9603-7056 |
| _version_ | 1811083058881757184 |
|---|---|
| author | Andoni, Alexandr Laarhoven, Thijs Indyk, Piotr Razenshteyn, Ilya Schmidt, Ludwig |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Andoni, Alexandr Laarhoven, Thijs Indyk, Piotr Razenshteyn, Ilya Schmidt, Ludwig |
| author_sort | Andoni, Alexandr |
| collection | MIT |
| description | We show the existence of a Locality-Sensitive Hashing (LSH) family for the angular distance that yields an approximate Near Neighbor Search algorithm with the asymptotically optimal running time exponent. Unlike earlier algorithms with this property (e.g., Spherical LSH (Andoni-Indyk-Nguyen-Razenshteyn 2014) (Andoni-Razenshteyn 2015)), our algorithm is also practical, improving upon the well-studied hyperplane LSH (Charikar 2002) in practice. We also introduce a multiprobe version of this algorithm and conduct an experimental evaluation on real and synthetic data sets.We complement the above positive results with a fine-grained lower bound for the quality of any LSH family for angular distance. Our lower bound implies that the above LSH family exhibits a trade-off between evaluation time and quality that is close to optimal for a natural class of LSH functions. |
| first_indexed | 2024-09-23T12:19:31Z |
| format | Article |
| id | mit-1721.1/113844 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T12:19:31Z |
| publishDate | 2018 |
| publisher | Neural Information Processing Systems Foundation |
| record_format | dspace |
| spelling | mit-1721.1/1138442022-09-28T01:01:55Z Practical and optimal LSH for angular distance Andoni, Alexandr Laarhoven, Thijs Indyk, Piotr Razenshteyn, Ilya Schmidt, Ludwig Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Indyk, Piotr Razenshteyn, Ilya Schmidt, Ludwig We show the existence of a Locality-Sensitive Hashing (LSH) family for the angular distance that yields an approximate Near Neighbor Search algorithm with the asymptotically optimal running time exponent. Unlike earlier algorithms with this property (e.g., Spherical LSH (Andoni-Indyk-Nguyen-Razenshteyn 2014) (Andoni-Razenshteyn 2015)), our algorithm is also practical, improving upon the well-studied hyperplane LSH (Charikar 2002) in practice. We also introduce a multiprobe version of this algorithm and conduct an experimental evaluation on real and synthetic data sets.We complement the above positive results with a fine-grained lower bound for the quality of any LSH family for angular distance. Our lower bound implies that the above LSH family exhibits a trade-off between evaluation time and quality that is close to optimal for a natural class of LSH functions. National Science Foundation (U.S.) Simons Foundation 2018-02-20T21:04:45Z 2018-02-20T21:04:45Z 2015-12 Article http://purl.org/eprint/type/ConferencePaper http://hdl.handle.net/1721.1/113844 Andoni, Alexandr et al. "Practical and optimal LSH for angular distance." Advances in Neural Information Processing Systems 28 (NIPS 2015), 7-12 December, 2015, Montreal, Canada, Neural Information Processing Systems Foundation, 2015. https://orcid.org/0000-0002-7983-9524 https://orcid.org/0000-0002-3962-721X https://orcid.org/0000-0002-9603-7056 en_US https://papers.nips.cc/paper/5893-practical-and-optimal-lsh-for-angular-distance Advances in Neural Information Processing Systems 28 (NIPS 2015) Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. application/pdf Neural Information Processing Systems Foundation Neural Information Processing Systems (NIPS) |
| spellingShingle | Andoni, Alexandr Laarhoven, Thijs Indyk, Piotr Razenshteyn, Ilya Schmidt, Ludwig Practical and optimal LSH for angular distance |
| title | Practical and optimal LSH for angular distance |
| title_full | Practical and optimal LSH for angular distance |
| title_fullStr | Practical and optimal LSH for angular distance |
| title_full_unstemmed | Practical and optimal LSH for angular distance |
| title_short | Practical and optimal LSH for angular distance |
| title_sort | practical and optimal lsh for angular distance |
| url | http://hdl.handle.net/1721.1/113844 https://orcid.org/0000-0002-7983-9524 https://orcid.org/0000-0002-3962-721X https://orcid.org/0000-0002-9603-7056 |
| work_keys_str_mv | AT andonialexandr practicalandoptimallshforangulardistance AT laarhoventhijs practicalandoptimallshforangulardistance AT indykpiotr practicalandoptimallshforangulardistance AT razenshteynilya practicalandoptimallshforangulardistance AT schmidtludwig practicalandoptimallshforangulardistance |