Non-feature-based structure from motion and visual servo control
The demand for fully autonomous robotic system, such as unmanned aerial vehicle (UAV), mobile robot etc., is increasing day by day. It has been seen that robotic systems are more and more intelligent, thanks to the on-board sensors used and enhancement of the real-time processing capabilities for in...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | English |
| Published: |
2016
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/10356/69017 |
| _version_ | 1811678982512312320 |
|---|---|
| author | Liu, Zhichao |
| author2 | Wang Jianliang |
| author_facet | Wang Jianliang Liu, Zhichao |
| author_sort | Liu, Zhichao |
| collection | NTU |
| description | The demand for fully autonomous robotic system, such as unmanned aerial vehicle (UAV), mobile robot etc., is increasing day by day. It has been seen that robotic systems are more and more intelligent, thanks to the on-board sensors used and enhancement of the real-time processing capabilities for interpreting the captured sensor data. Vision sensors are commonly used in intelligent robotic systems and the amount of information available in visual data is very high compared to other sources of data. Visual servo control plays an important role for robotic intelligence and a lot of work have been done in this area in the past decades. Most of these work are based on the extracted features from the visual data, which are usually images. However, the problem of ascertaining which features in one image correspond to which features in another image, known as the correspondence problem, needs to be solved firstly. Many image processing methods have been proposed to track and match the features in different images. If the correspondence problem cannot be solved correctly, errors or even failures will happen during the whole procedure. Therefore it is necessary to carry out research to investigate visual servo control algorithms which can avoid the correspondence problem. As the structure information between the target and the camera is needed for visual servo control, and sensors that can measure this information is not always available, therefore structure from motion algorithms which can avoid the correspondence problem are also necessary. We call algorithms that can avoid the correspondence problem “non-feature-based”algorithms.
The major contribution of this thesis is the novel solution proposed to solve this problem. In the proposed system, modelling for structure from motion based on sets and image moments, observers designed for the models, a Set-based Direct Visual Servo control (SDVS) algorithm for gray images, and Fast Set-based Direct Visual Servo control algorithms for Binary (FSDVSB) and Gray images (FSDVSG) are developed. All these contributions are explained briefly next.
First, in order that the depth information from the object to the camera to be obtained for visual servo control, structure from motion problem is usually needed to be solved first. In the literature, several estimation algorithms have been developed for structure from motion tasks. Most existing methods address feature tracking and matching between several frames of images to solve the correspondence problem. In contrast to these methods, two methods of avoiding correspondence problem have been proposed. The first method is based on extracted feature points or segmented images which can be seen as sets, and a novel model based on set theory is proposed. Based on the model, a novel fast observer with the property of fast convergence is designed. With this method, depth as well as the orientation from the object to the camera can be estimated, while avoiding the correspondence problem. The second method is based on extracted feature points. Point-based image moments based on these points, which can be calculated directly from the coordinates of the points, is used to construct an alternative model that avoids correspondence problem. To fit this model, the observer in the first method can be easily modified and used to solve the latter model.
Next, with the depth information of the target, visual servo controllers can be designed. The control objective for the algorithm proposed in this thesis is to control the camera such that the current image coincides with the desired image. In contrast to most other approaches in this area, SDVS algorithm proposed in this thesis uses the pixel intensities in the image directly, which can avoid feature extraction, tracking and matching procedure by using typical image processing methods. As the non-feature-based algorithms usually require heavy computational load, FSDVSB and FSDVSG are proposed in this thesis to increase the computational speed for practical implementation. FSDVSB is restricted to at most 4 Degree of freedom (DoF). Instead of using the set itself, FSDVSB uses the boundary points of the set to compute the control signals, thus the computational speed is increased dramatically. FSDVSG can treat all 6 DoF for gray images, and its computational speed is faster
than normal SDVS.
In this thesis the proposed non-feature-based structure from motion and visual servo control schemes have been proved theoretically and their performances have been demonstrated through simulations or experimental results. It is concluded that the proposed method is suitable and realizable for non-feature-based structure form motion and visual servo control problem. |
| first_indexed | 2024-10-01T03:01:55Z |
| format | Thesis |
| id | ntu-10356/69017 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T03:01:55Z |
| publishDate | 2016 |
| record_format | dspace |
| spelling | ntu-10356/690172023-07-04T16:07:50Z Non-feature-based structure from motion and visual servo control Liu, Zhichao Wang Jianliang School of Electrical and Electronic Engineering DRNTU::Engineering The demand for fully autonomous robotic system, such as unmanned aerial vehicle (UAV), mobile robot etc., is increasing day by day. It has been seen that robotic systems are more and more intelligent, thanks to the on-board sensors used and enhancement of the real-time processing capabilities for interpreting the captured sensor data. Vision sensors are commonly used in intelligent robotic systems and the amount of information available in visual data is very high compared to other sources of data. Visual servo control plays an important role for robotic intelligence and a lot of work have been done in this area in the past decades. Most of these work are based on the extracted features from the visual data, which are usually images. However, the problem of ascertaining which features in one image correspond to which features in another image, known as the correspondence problem, needs to be solved firstly. Many image processing methods have been proposed to track and match the features in different images. If the correspondence problem cannot be solved correctly, errors or even failures will happen during the whole procedure. Therefore it is necessary to carry out research to investigate visual servo control algorithms which can avoid the correspondence problem. As the structure information between the target and the camera is needed for visual servo control, and sensors that can measure this information is not always available, therefore structure from motion algorithms which can avoid the correspondence problem are also necessary. We call algorithms that can avoid the correspondence problem “non-feature-based”algorithms. The major contribution of this thesis is the novel solution proposed to solve this problem. In the proposed system, modelling for structure from motion based on sets and image moments, observers designed for the models, a Set-based Direct Visual Servo control (SDVS) algorithm for gray images, and Fast Set-based Direct Visual Servo control algorithms for Binary (FSDVSB) and Gray images (FSDVSG) are developed. All these contributions are explained briefly next. First, in order that the depth information from the object to the camera to be obtained for visual servo control, structure from motion problem is usually needed to be solved first. In the literature, several estimation algorithms have been developed for structure from motion tasks. Most existing methods address feature tracking and matching between several frames of images to solve the correspondence problem. In contrast to these methods, two methods of avoiding correspondence problem have been proposed. The first method is based on extracted feature points or segmented images which can be seen as sets, and a novel model based on set theory is proposed. Based on the model, a novel fast observer with the property of fast convergence is designed. With this method, depth as well as the orientation from the object to the camera can be estimated, while avoiding the correspondence problem. The second method is based on extracted feature points. Point-based image moments based on these points, which can be calculated directly from the coordinates of the points, is used to construct an alternative model that avoids correspondence problem. To fit this model, the observer in the first method can be easily modified and used to solve the latter model. Next, with the depth information of the target, visual servo controllers can be designed. The control objective for the algorithm proposed in this thesis is to control the camera such that the current image coincides with the desired image. In contrast to most other approaches in this area, SDVS algorithm proposed in this thesis uses the pixel intensities in the image directly, which can avoid feature extraction, tracking and matching procedure by using typical image processing methods. As the non-feature-based algorithms usually require heavy computational load, FSDVSB and FSDVSG are proposed in this thesis to increase the computational speed for practical implementation. FSDVSB is restricted to at most 4 Degree of freedom (DoF). Instead of using the set itself, FSDVSB uses the boundary points of the set to compute the control signals, thus the computational speed is increased dramatically. FSDVSG can treat all 6 DoF for gray images, and its computational speed is faster than normal SDVS. In this thesis the proposed non-feature-based structure from motion and visual servo control schemes have been proved theoretically and their performances have been demonstrated through simulations or experimental results. It is concluded that the proposed method is suitable and realizable for non-feature-based structure form motion and visual servo control problem. Doctor of Philosophy (EEE) 2016-09-05T01:57:34Z 2016-09-05T01:57:34Z 2016 Thesis http://hdl.handle.net/10356/69017 en 185 p. application/pdf |
| spellingShingle | DRNTU::Engineering Liu, Zhichao Non-feature-based structure from motion and visual servo control |
| title | Non-feature-based structure from motion and visual servo control |
| title_full | Non-feature-based structure from motion and visual servo control |
| title_fullStr | Non-feature-based structure from motion and visual servo control |
| title_full_unstemmed | Non-feature-based structure from motion and visual servo control |
| title_short | Non-feature-based structure from motion and visual servo control |
| title_sort | non feature based structure from motion and visual servo control |
| topic | DRNTU::Engineering |
| url | http://hdl.handle.net/10356/69017 |
| work_keys_str_mv | AT liuzhichao nonfeaturebasedstructurefrommotionandvisualservocontrol |