CLASSIFICATION OF OIL PAINTING USING MACHINE LEARNING WITH VISUALIZED DEPTH INFORMATION
<p>In the past few decades, a number of scholars studied painting classification based on image processing or computer vision technologies. Further, as the machine learning technology rapidly developed, painting classification using machine learning has been carried out. However, due to the...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2019-08-01
|
| Series: | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| Online Access: | https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLII-2-W15/617/2019/isprs-archives-XLII-2-W15-617-2019.pdf |
| _version_ | 1828775170920153088 |
|---|---|
| author | J. Kim J. Y. Jun M. Hong H. Shim J. Ahn |
| author_facet | J. Kim J. Y. Jun M. Hong H. Shim J. Ahn |
| author_sort | J. Kim |
| collection | DOAJ |
| description | <p>In the past few decades, a number of scholars studied painting classification based on image processing or computer vision technologies.
Further, as the machine learning technology rapidly developed, painting classification using machine learning has been carried
out. However, due to the lack of information about brushstrokes in the photograph, typical models cannot use more precise information
of the painters painting style. We hypothesized that the visualized depth information of brushstroke is effective to improve
the accuracy of the machine learning model for painting classification. This study proposes a new data utilization approach in machine
learning with Reflectance Transformation Imaging (RTI) images, which maximizes the visualization of a three-dimensional
shape of brushstrokes. Certain artist’s unique brushstrokes can be revealed in RTI images, which are difficult to obtain with regular
photographs. If these new types of images are applied as data to train in with the machine learning model, classification would
be conducted including not only the shape of the color but also the depth information. We used the Convolution Neural Network
(CNN), a model optimized for image classification, using the VGG-16, ResNet-50, and DenseNet-121 architectures. We conducted
a two-stage experiment using the works of two Korean artists. In the first experiment, we obtained a key part of the painting from
RTI data and photographic data. In the second experiment on the second artists work, a larger quantity of data are acquired, and the
whole part of the artwork was captured. The result showed that RTI-trained model brought higher accuracy than Non-RTI trained
model. In this paper, we propose a method which uses machine learning and RTI technology to analyze and classify paintings more
precisely to verify our hypothesis.</p> |
| first_indexed | 2024-12-11T15:34:22Z |
| format | Article |
| id | doaj.art-31281c03feda49d2b9eb6aa4f53d2c35 |
| institution | Directory Open Access Journal |
| issn | 1682-1750 2194-9034 |
| language | English |
| last_indexed | 2024-12-11T15:34:22Z |
| publishDate | 2019-08-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| spelling | doaj.art-31281c03feda49d2b9eb6aa4f53d2c352022-12-22T00:59:58ZengCopernicus PublicationsThe International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences1682-17502194-90342019-08-01XLII-2-W1561762310.5194/isprs-archives-XLII-2-W15-617-2019CLASSIFICATION OF OIL PAINTING USING MACHINE LEARNING WITH VISUALIZED DEPTH INFORMATIONJ. Kim0J. Y. Jun1M. Hong2H. Shim3J. Ahn4Graduate School of Culture Technology Korea Advanced Institute of Science Technology (KAIST) Daejeon, Republic of KoreaGraduate School of Culture Technology Korea Advanced Institute of Science Technology (KAIST) Daejeon, Republic of KoreaCulture Technology Research Institute Korea Advanced Institute of Science Technology (KAIST) Daejeon, Republic of KoreaGraduate School of Culture Technology Korea Advanced Institute of Science Technology (KAIST) Daejeon, Republic of KoreaGraduate School of Culture Technology Korea Advanced Institute of Science Technology (KAIST) Daejeon, Republic of Korea<p>In the past few decades, a number of scholars studied painting classification based on image processing or computer vision technologies. Further, as the machine learning technology rapidly developed, painting classification using machine learning has been carried out. However, due to the lack of information about brushstrokes in the photograph, typical models cannot use more precise information of the painters painting style. We hypothesized that the visualized depth information of brushstroke is effective to improve the accuracy of the machine learning model for painting classification. This study proposes a new data utilization approach in machine learning with Reflectance Transformation Imaging (RTI) images, which maximizes the visualization of a three-dimensional shape of brushstrokes. Certain artist’s unique brushstrokes can be revealed in RTI images, which are difficult to obtain with regular photographs. If these new types of images are applied as data to train in with the machine learning model, classification would be conducted including not only the shape of the color but also the depth information. We used the Convolution Neural Network (CNN), a model optimized for image classification, using the VGG-16, ResNet-50, and DenseNet-121 architectures. We conducted a two-stage experiment using the works of two Korean artists. In the first experiment, we obtained a key part of the painting from RTI data and photographic data. In the second experiment on the second artists work, a larger quantity of data are acquired, and the whole part of the artwork was captured. The result showed that RTI-trained model brought higher accuracy than Non-RTI trained model. In this paper, we propose a method which uses machine learning and RTI technology to analyze and classify paintings more precisely to verify our hypothesis.</p>https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLII-2-W15/617/2019/isprs-archives-XLII-2-W15-617-2019.pdf |
| spellingShingle | J. Kim J. Y. Jun M. Hong H. Shim J. Ahn CLASSIFICATION OF OIL PAINTING USING MACHINE LEARNING WITH VISUALIZED DEPTH INFORMATION The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| title | CLASSIFICATION OF OIL PAINTING USING MACHINE LEARNING WITH
VISUALIZED DEPTH INFORMATION |
| title_full | CLASSIFICATION OF OIL PAINTING USING MACHINE LEARNING WITH
VISUALIZED DEPTH INFORMATION |
| title_fullStr | CLASSIFICATION OF OIL PAINTING USING MACHINE LEARNING WITH
VISUALIZED DEPTH INFORMATION |
| title_full_unstemmed | CLASSIFICATION OF OIL PAINTING USING MACHINE LEARNING WITH
VISUALIZED DEPTH INFORMATION |
| title_short | CLASSIFICATION OF OIL PAINTING USING MACHINE LEARNING WITH
VISUALIZED DEPTH INFORMATION |
| title_sort | classification of oil painting using machine learning with visualized depth information |
| url | https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLII-2-W15/617/2019/isprs-archives-XLII-2-W15-617-2019.pdf |
| work_keys_str_mv | AT jkim classificationofoilpaintingusingmachinelearningwithvisualizeddepthinformation AT jyjun classificationofoilpaintingusingmachinelearningwithvisualizeddepthinformation AT mhong classificationofoilpaintingusingmachinelearningwithvisualizeddepthinformation AT hshim classificationofoilpaintingusingmachinelearningwithvisualizeddepthinformation AT jahn classificationofoilpaintingusingmachinelearningwithvisualizeddepthinformation |