On Learning Mechanical Laws of Motion From Video Using Neural Networks
In computer vision, physics plays an important role in several applications. In this work, we teach a machine to detect the mechanical laws of motion of physical objects using video, and show how the results can be useful for computer vision tasks. We assume no prior knowledge of physics, beyond a t...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10078253/ |
| _version_ | 1797855359608553472 |
|---|---|
| author | Pradyumna Chari Yunhao Ba Shijie Zhou Chinmay Talegaonkar Shreeram Athreya Achuta Kadambi |
| author_facet | Pradyumna Chari Yunhao Ba Shijie Zhou Chinmay Talegaonkar Shreeram Athreya Achuta Kadambi |
| author_sort | Pradyumna Chari |
| collection | DOAJ |
| description | In computer vision, physics plays an important role in several applications. In this work, we teach a machine to detect the mechanical laws of motion of physical objects using video, and show how the results can be useful for computer vision tasks. We assume no prior knowledge of physics, beyond a temporal stream of bounding boxes. The problem is very difficult because a machine must learn not only a governing equation (e.g. projectile motion) but also the existence of governing parameters (e.g. velocities). We evaluate our ability to represent the physical laws of motion in video, such as the movement of a projectile or circular motion, in both real and constructed videos. These elementary tasks have textbook governing equations and enable ground truth verification of our approach. To establish the importance of the proposed method, we show a real-world use case in the domain of object tracking in confounding scenes, where existing state-of-the-art algorithms fail. Incorporating physics into computer vision not only serves the purpose of curiosity-driven research, but also provides an inductive bias for computer vision applications like object tracking. |
| first_indexed | 2024-04-09T20:22:28Z |
| format | Article |
| id | doaj.art-c29f3a243cce421ca9334ef52c897fd4 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-09T20:22:28Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-c29f3a243cce421ca9334ef52c897fd42023-03-30T23:01:15ZengIEEEIEEE Access2169-35362023-01-0111301293014510.1109/ACCESS.2023.326040510078253On Learning Mechanical Laws of Motion From Video Using Neural NetworksPradyumna Chari0https://orcid.org/0000-0002-9610-0350Yunhao Ba1https://orcid.org/0000-0001-8664-7195Shijie Zhou2Chinmay Talegaonkar3Shreeram Athreya4https://orcid.org/0000-0001-5051-2723Achuta Kadambi5Department of Electrical and Computer Engineering, University of California at Los Angeles, Los Angeles, CA, USADepartment of Electrical and Computer Engineering, University of California at Los Angeles, Los Angeles, CA, USADepartment of Electrical and Computer Engineering, University of California at Los Angeles, Los Angeles, CA, USADepartment of Electrical and Computer Engineering, University of California at Los Angeles, Los Angeles, CA, USADepartment of Electrical and Computer Engineering, University of California at Los Angeles, Los Angeles, CA, USADepartment of Electrical and Computer Engineering, University of California at Los Angeles, Los Angeles, CA, USAIn computer vision, physics plays an important role in several applications. In this work, we teach a machine to detect the mechanical laws of motion of physical objects using video, and show how the results can be useful for computer vision tasks. We assume no prior knowledge of physics, beyond a temporal stream of bounding boxes. The problem is very difficult because a machine must learn not only a governing equation (e.g. projectile motion) but also the existence of governing parameters (e.g. velocities). We evaluate our ability to represent the physical laws of motion in video, such as the movement of a projectile or circular motion, in both real and constructed videos. These elementary tasks have textbook governing equations and enable ground truth verification of our approach. To establish the importance of the proposed method, we show a real-world use case in the domain of object tracking in confounding scenes, where existing state-of-the-art algorithms fail. Incorporating physics into computer vision not only serves the purpose of curiosity-driven research, but also provides an inductive bias for computer vision applications like object tracking.https://ieeexplore.ieee.org/document/10078253/Mechanical laws of motionvideo analysisobject tracking |
| spellingShingle | Pradyumna Chari Yunhao Ba Shijie Zhou Chinmay Talegaonkar Shreeram Athreya Achuta Kadambi On Learning Mechanical Laws of Motion From Video Using Neural Networks IEEE Access Mechanical laws of motion video analysis object tracking |
| title | On Learning Mechanical Laws of Motion From Video Using Neural Networks |
| title_full | On Learning Mechanical Laws of Motion From Video Using Neural Networks |
| title_fullStr | On Learning Mechanical Laws of Motion From Video Using Neural Networks |
| title_full_unstemmed | On Learning Mechanical Laws of Motion From Video Using Neural Networks |
| title_short | On Learning Mechanical Laws of Motion From Video Using Neural Networks |
| title_sort | on learning mechanical laws of motion from video using neural networks |
| topic | Mechanical laws of motion video analysis object tracking |
| url | https://ieeexplore.ieee.org/document/10078253/ |
| work_keys_str_mv | AT pradyumnachari onlearningmechanicallawsofmotionfromvideousingneuralnetworks AT yunhaoba onlearningmechanicallawsofmotionfromvideousingneuralnetworks AT shijiezhou onlearningmechanicallawsofmotionfromvideousingneuralnetworks AT chinmaytalegaonkar onlearningmechanicallawsofmotionfromvideousingneuralnetworks AT shreeramathreya onlearningmechanicallawsofmotionfromvideousingneuralnetworks AT achutakadambi onlearningmechanicallawsofmotionfromvideousingneuralnetworks |