Closed-Loop Region of Interest Enabling High Spatial and Temporal Resolutions in Object Detection and Tracking via Wireless Camera
The trade-off between spatial and temporal resolution remains a fundamental challenge in machine vision. A captured image often contains a significant amount of redundant information, and only a small region of interest (ROI) is necessary for object detection and tracking. In this paper, we first sy...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9454528/ |
| _version_ | 1818438260936933376 |
|---|---|
| author | Jack Chen Hen-Wei Huang Philipp Rupp Anjali Sinha Claas Ehmke Giovanni Traverso |
| author_facet | Jack Chen Hen-Wei Huang Philipp Rupp Anjali Sinha Claas Ehmke Giovanni Traverso |
| author_sort | Jack Chen |
| collection | DOAJ |
| description | The trade-off between spatial and temporal resolution remains a fundamental challenge in machine vision. A captured image often contains a significant amount of redundant information, and only a small region of interest (ROI) is necessary for object detection and tracking. In this paper, we first systematically characterize the effects of ROI on camera capturing, data transmission, and image processing. We then present the closed-loop ROI algorithm capable of high spatial and temporal resolution as well as wide scanning field of view (FOV) in single and multi-object detection and tracking via real-time wireless video streaming. With the feedback from real-time object tracking, the wireless camera is able to capture and transmit only the ROI which in turn enhances both the spatial and temporal resolution in object tracking. In addition, the proposed approach can still maintain a large FOV by processing regions outside of the ROI at lower spatial and temporal resolutions. When applied to a high spatial resolution wireless stream (5 MegaPixels), the closed-loop ROI algorithm improves the temporal resolution by up to <inline-formula> <tex-math notation="LaTeX">$10\times $ </tex-math></inline-formula> (from 2.4FPS to 22.5FPS). Specifically, camera processing is improved by up to <inline-formula> <tex-math notation="LaTeX">$4.7\times $ </tex-math></inline-formula>, data transmission is improved by up to <inline-formula> <tex-math notation="LaTeX">$160\times $ </tex-math></inline-formula>, and PC processing is improved by up to <inline-formula> <tex-math notation="LaTeX">$2.5\times $ </tex-math></inline-formula>. In a person tracking experiment, the closed-loop ROI algorithm enables a wide-angle camera to outperform both a normal wide-angle camera–which suffers from poor temporal resolution and motion blur–and a pan & tilt camera–which cannot automatically refresh tracking after the tracking is lost. |
| first_indexed | 2024-12-14T17:37:45Z |
| format | Article |
| id | doaj.art-30ec71b761d84dd2b93dba7f09d1ba6b |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-14T17:37:45Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-30ec71b761d84dd2b93dba7f09d1ba6b2022-12-21T22:52:56ZengIEEEIEEE Access2169-35362021-01-019873408735010.1109/ACCESS.2021.30864999454528Closed-Loop Region of Interest Enabling High Spatial and Temporal Resolutions in Object Detection and Tracking via Wireless CameraJack Chen0https://orcid.org/0000-0002-3892-1580Hen-Wei Huang1https://orcid.org/0000-0003-1921-8897Philipp Rupp2https://orcid.org/0000-0002-4637-4169Anjali Sinha3https://orcid.org/0000-0003-1957-8606Claas Ehmke4https://orcid.org/0000-0002-0618-5945Giovanni Traverso5https://orcid.org/0000-0001-7851-4077Division of Gastroenterology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USADivision of Gastroenterology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USADivision of Gastroenterology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USADepartment of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USAKoch Institute for Integrative Cancer Research, Massachusetts Institute of Technology, Cambridge, MA, USADivision of Gastroenterology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USAThe trade-off between spatial and temporal resolution remains a fundamental challenge in machine vision. A captured image often contains a significant amount of redundant information, and only a small region of interest (ROI) is necessary for object detection and tracking. In this paper, we first systematically characterize the effects of ROI on camera capturing, data transmission, and image processing. We then present the closed-loop ROI algorithm capable of high spatial and temporal resolution as well as wide scanning field of view (FOV) in single and multi-object detection and tracking via real-time wireless video streaming. With the feedback from real-time object tracking, the wireless camera is able to capture and transmit only the ROI which in turn enhances both the spatial and temporal resolution in object tracking. In addition, the proposed approach can still maintain a large FOV by processing regions outside of the ROI at lower spatial and temporal resolutions. When applied to a high spatial resolution wireless stream (5 MegaPixels), the closed-loop ROI algorithm improves the temporal resolution by up to <inline-formula> <tex-math notation="LaTeX">$10\times $ </tex-math></inline-formula> (from 2.4FPS to 22.5FPS). Specifically, camera processing is improved by up to <inline-formula> <tex-math notation="LaTeX">$4.7\times $ </tex-math></inline-formula>, data transmission is improved by up to <inline-formula> <tex-math notation="LaTeX">$160\times $ </tex-math></inline-formula>, and PC processing is improved by up to <inline-formula> <tex-math notation="LaTeX">$2.5\times $ </tex-math></inline-formula>. In a person tracking experiment, the closed-loop ROI algorithm enables a wide-angle camera to outperform both a normal wide-angle camera–which suffers from poor temporal resolution and motion blur–and a pan & tilt camera–which cannot automatically refresh tracking after the tracking is lost.https://ieeexplore.ieee.org/document/9454528/Machine visionobject detectionobject trackingreal-time systemsregion of interestspatial resolution |
| spellingShingle | Jack Chen Hen-Wei Huang Philipp Rupp Anjali Sinha Claas Ehmke Giovanni Traverso Closed-Loop Region of Interest Enabling High Spatial and Temporal Resolutions in Object Detection and Tracking via Wireless Camera IEEE Access Machine vision object detection object tracking real-time systems region of interest spatial resolution |
| title | Closed-Loop Region of Interest Enabling High Spatial and Temporal Resolutions in Object Detection and Tracking via Wireless Camera |
| title_full | Closed-Loop Region of Interest Enabling High Spatial and Temporal Resolutions in Object Detection and Tracking via Wireless Camera |
| title_fullStr | Closed-Loop Region of Interest Enabling High Spatial and Temporal Resolutions in Object Detection and Tracking via Wireless Camera |
| title_full_unstemmed | Closed-Loop Region of Interest Enabling High Spatial and Temporal Resolutions in Object Detection and Tracking via Wireless Camera |
| title_short | Closed-Loop Region of Interest Enabling High Spatial and Temporal Resolutions in Object Detection and Tracking via Wireless Camera |
| title_sort | closed loop region of interest enabling high spatial and temporal resolutions in object detection and tracking via wireless camera |
| topic | Machine vision object detection object tracking real-time systems region of interest spatial resolution |
| url | https://ieeexplore.ieee.org/document/9454528/ |
| work_keys_str_mv | AT jackchen closedloopregionofinterestenablinghighspatialandtemporalresolutionsinobjectdetectionandtrackingviawirelesscamera AT henweihuang closedloopregionofinterestenablinghighspatialandtemporalresolutionsinobjectdetectionandtrackingviawirelesscamera AT philipprupp closedloopregionofinterestenablinghighspatialandtemporalresolutionsinobjectdetectionandtrackingviawirelesscamera AT anjalisinha closedloopregionofinterestenablinghighspatialandtemporalresolutionsinobjectdetectionandtrackingviawirelesscamera AT claasehmke closedloopregionofinterestenablinghighspatialandtemporalresolutionsinobjectdetectionandtrackingviawirelesscamera AT giovannitraverso closedloopregionofinterestenablinghighspatialandtemporalresolutionsinobjectdetectionandtrackingviawirelesscamera |