Nanoparticle Recognition on Scanning Probe Microscopy Images Using Computer Vision and Deep Learning
Identifying, counting and measuring particles is an important component of many research studies. Images with particles are usually processed by hand using a software ruler. Automated processing, based on conventional image processing methods (edge detection, segmentation, etc.) are not universal, c...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-06-01
|
| Series: | Nanomaterials |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-4991/10/7/1285 |
| _version_ | 1797563745807892480 |
|---|---|
| author | Alexey G. Okunev Mikhail Yu. Mashukov Anna V. Nartova Andrey V. Matveev |
| author_facet | Alexey G. Okunev Mikhail Yu. Mashukov Anna V. Nartova Andrey V. Matveev |
| author_sort | Alexey G. Okunev |
| collection | DOAJ |
| description | Identifying, counting and measuring particles is an important component of many research studies. Images with particles are usually processed by hand using a software ruler. Automated processing, based on conventional image processing methods (edge detection, segmentation, etc.) are not universal, can only be used on good-quality images and need to set a number of parameters empirically. In this paper, we present results from the application of deep learning to automated recognition of metal nanoparticles deposited on highly oriented pyrolytic graphite on images obtained by scanning tunneling microscopy (STM). We used the Cascade Mask-RCNN neural network. Training was performed on a dataset containing 23 STM images with 5157 nanoparticles. Three images containing 695 nanoparticles were used for verification. As a result, the trained neural network recognized nanoparticles in the verification set with 0.93 precision and 0.78 recall. Predicted contour refining with 2D Gaussian function was a proposed option. The accuracies for mean particle size calculated from predicted contours compared with ground truth were in the range of 0.87–0.99. The results were compared with outcomes from other generally available software, based on conventional image processing methods. The advantages of deep learning methods for automatic particle recognition were clearly demonstrated. We developed a free open-access web service “ParticlesNN” based on the trained neural network, which can be used by any researcher in the world. |
| first_indexed | 2024-03-10T18:47:45Z |
| format | Article |
| id | doaj.art-b4a9060e921e4e329226e319569aa0ef |
| institution | Directory Open Access Journal |
| issn | 2079-4991 |
| language | English |
| last_indexed | 2024-03-10T18:47:45Z |
| publishDate | 2020-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Nanomaterials |
| spelling | doaj.art-b4a9060e921e4e329226e319569aa0ef2023-11-20T05:25:41ZengMDPI AGNanomaterials2079-49912020-06-01107128510.3390/nano10071285Nanoparticle Recognition on Scanning Probe Microscopy Images Using Computer Vision and Deep LearningAlexey G. Okunev0Mikhail Yu. Mashukov1Anna V. Nartova2Andrey V. Matveev3Novosibirsk State University Higher College of Informatics, Russkaja Str. 35, 630058 Novosibirsk, RussiaScientific-Educational Center “Machine Learning and Big Data Analysis”, Novosibirsk State University, Pirogova Str. 1, 630090 Novosibirsk, RussiaBoreskov Institute of Catalysis SB RAS, pr. Acad. Lavrentieva, 5, 630090 Novosibirsk, RussiaBoreskov Institute of Catalysis SB RAS, pr. Acad. Lavrentieva, 5, 630090 Novosibirsk, RussiaIdentifying, counting and measuring particles is an important component of many research studies. Images with particles are usually processed by hand using a software ruler. Automated processing, based on conventional image processing methods (edge detection, segmentation, etc.) are not universal, can only be used on good-quality images and need to set a number of parameters empirically. In this paper, we present results from the application of deep learning to automated recognition of metal nanoparticles deposited on highly oriented pyrolytic graphite on images obtained by scanning tunneling microscopy (STM). We used the Cascade Mask-RCNN neural network. Training was performed on a dataset containing 23 STM images with 5157 nanoparticles. Three images containing 695 nanoparticles were used for verification. As a result, the trained neural network recognized nanoparticles in the verification set with 0.93 precision and 0.78 recall. Predicted contour refining with 2D Gaussian function was a proposed option. The accuracies for mean particle size calculated from predicted contours compared with ground truth were in the range of 0.87–0.99. The results were compared with outcomes from other generally available software, based on conventional image processing methods. The advantages of deep learning methods for automatic particle recognition were clearly demonstrated. We developed a free open-access web service “ParticlesNN” based on the trained neural network, which can be used by any researcher in the world.https://www.mdpi.com/2079-4991/10/7/1285particle recognitiondeep neural networksscanning tunneling microscopyparticles |
| spellingShingle | Alexey G. Okunev Mikhail Yu. Mashukov Anna V. Nartova Andrey V. Matveev Nanoparticle Recognition on Scanning Probe Microscopy Images Using Computer Vision and Deep Learning Nanomaterials particle recognition deep neural networks scanning tunneling microscopy particles |
| title | Nanoparticle Recognition on Scanning Probe Microscopy Images Using Computer Vision and Deep Learning |
| title_full | Nanoparticle Recognition on Scanning Probe Microscopy Images Using Computer Vision and Deep Learning |
| title_fullStr | Nanoparticle Recognition on Scanning Probe Microscopy Images Using Computer Vision and Deep Learning |
| title_full_unstemmed | Nanoparticle Recognition on Scanning Probe Microscopy Images Using Computer Vision and Deep Learning |
| title_short | Nanoparticle Recognition on Scanning Probe Microscopy Images Using Computer Vision and Deep Learning |
| title_sort | nanoparticle recognition on scanning probe microscopy images using computer vision and deep learning |
| topic | particle recognition deep neural networks scanning tunneling microscopy particles |
| url | https://www.mdpi.com/2079-4991/10/7/1285 |
| work_keys_str_mv | AT alexeygokunev nanoparticlerecognitiononscanningprobemicroscopyimagesusingcomputervisionanddeeplearning AT mikhailyumashukov nanoparticlerecognitiononscanningprobemicroscopyimagesusingcomputervisionanddeeplearning AT annavnartova nanoparticlerecognitiononscanningprobemicroscopyimagesusingcomputervisionanddeeplearning AT andreyvmatveev nanoparticlerecognitiononscanningprobemicroscopyimagesusingcomputervisionanddeeplearning |