Making Hands: Neural Implicit Manifold Learning of Hand Gestures
The human hand is a complex and sophisticated biological machine. Hand gesturing is key to our understanding of and interacting with the world around us. Hand gesturing is also instrumental in design and making. I argue that understanding the geometry of the space of hand gestures leads to an intuit...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2023
|
| Online Access: | https://hdl.handle.net/1721.1/151622 |
| _version_ | 1826206086881869824 |
|---|---|
| author | Chatzinikolis, Dimitrios |
| author2 | Knight, Terry W. |
| author_facet | Knight, Terry W. Chatzinikolis, Dimitrios |
| author_sort | Chatzinikolis, Dimitrios |
| collection | MIT |
| description | The human hand is a complex and sophisticated biological machine. Hand gesturing is key to our understanding of and interacting with the world around us. Hand gesturing is also instrumental in design and making. I argue that understanding the geometry of the space of hand gestures leads to an intuitive human-computer interaction. I decompose the gesture into its constituent parts, i.e., the hand motion – global coordinate system – and the hand pose – local coordinate system. I propose modeling the configuration space of hands as a high-dimensional manifold via neural unsigned distance fields, and I define plausible hand poses as points on the manifold. Next, I apply a distance metric to their configuration space. A trajectory in that space is a finite or infinite sequence of hand poses. These trajectories represent the different ways that the hand gestures. To demonstrate my approach, I restrict my study to a dataset of hands grasping everyday objects, and I evaluate my model on unknown grasps. Extending the model, the learned manifold acts as a prior for hand pose denoising, hand pose interpolation, and hand pose synthesis. Constraining that space can be interpreted as excluding impossible hand poses while constraining the manifold can be interpreted as defining a set of desirable hand poses. The former emphasizes the importance of bridging deep learning with existing mathematical structures, while the latter underlines future directions for the fields of design and computational making. |
| first_indexed | 2024-09-23T13:23:53Z |
| format | Thesis |
| id | mit-1721.1/151622 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T13:23:53Z |
| publishDate | 2023 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1516222023-08-01T03:37:33Z Making Hands: Neural Implicit Manifold Learning of Hand Gestures Chatzinikolis, Dimitrios Knight, Terry W. Sitzmann, Vincent Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology. Department of Architecture The human hand is a complex and sophisticated biological machine. Hand gesturing is key to our understanding of and interacting with the world around us. Hand gesturing is also instrumental in design and making. I argue that understanding the geometry of the space of hand gestures leads to an intuitive human-computer interaction. I decompose the gesture into its constituent parts, i.e., the hand motion – global coordinate system – and the hand pose – local coordinate system. I propose modeling the configuration space of hands as a high-dimensional manifold via neural unsigned distance fields, and I define plausible hand poses as points on the manifold. Next, I apply a distance metric to their configuration space. A trajectory in that space is a finite or infinite sequence of hand poses. These trajectories represent the different ways that the hand gestures. To demonstrate my approach, I restrict my study to a dataset of hands grasping everyday objects, and I evaluate my model on unknown grasps. Extending the model, the learned manifold acts as a prior for hand pose denoising, hand pose interpolation, and hand pose synthesis. Constraining that space can be interpreted as excluding impossible hand poses while constraining the manifold can be interpreted as defining a set of desirable hand poses. The former emphasizes the importance of bridging deep learning with existing mathematical structures, while the latter underlines future directions for the fields of design and computational making. S.M. S.M. 2023-07-31T19:53:40Z 2023-07-31T19:53:40Z 2023-06 2023-07-13T21:34:53.010Z Thesis https://hdl.handle.net/1721.1/151622 In Copyright - Educational Use Permitted Copyright retained by author(s) https://rightsstatements.org/page/InC-EDU/1.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Chatzinikolis, Dimitrios Making Hands: Neural Implicit Manifold Learning of Hand Gestures |
| title | Making Hands: Neural Implicit Manifold Learning of Hand Gestures |
| title_full | Making Hands: Neural Implicit Manifold Learning of Hand Gestures |
| title_fullStr | Making Hands: Neural Implicit Manifold Learning of Hand Gestures |
| title_full_unstemmed | Making Hands: Neural Implicit Manifold Learning of Hand Gestures |
| title_short | Making Hands: Neural Implicit Manifold Learning of Hand Gestures |
| title_sort | making hands neural implicit manifold learning of hand gestures |
| url | https://hdl.handle.net/1721.1/151622 |
| work_keys_str_mv | AT chatzinikolisdimitrios makinghandsneuralimplicitmanifoldlearningofhandgestures |