Determining Chess Game State from an Image
Identifying the configuration of chess pieces from an image of a chessboard is a problem in computer vision that has not yet been solved accurately. However, it is important for helping amateur chess players improve their games by facilitating automatic computer analysis without the overhead of manu...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-06-01
|
| Series: | Journal of Imaging |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2313-433X/7/6/94 |
| _version_ | 1797531614335467520 |
|---|---|
| author | Georg Wölflein Ognjen Arandjelović |
| author_facet | Georg Wölflein Ognjen Arandjelović |
| author_sort | Georg Wölflein |
| collection | DOAJ |
| description | Identifying the configuration of chess pieces from an image of a chessboard is a problem in computer vision that has not yet been solved accurately. However, it is important for helping amateur chess players improve their games by facilitating automatic computer analysis without the overhead of manually entering the pieces. Current approaches are limited by the lack of large datasets and are not designed to adapt to unseen chess sets. This paper puts forth a new dataset synthesised from a 3D model that is an order of magnitude larger than existing ones. Trained on this dataset, a novel end-to-end chess recognition system is presented that combines traditional computer vision techniques with deep learning. It localises the chessboard using a RANSAC-based algorithm that computes a projective transformation of the board onto a regular grid. Using two convolutional neural networks, it then predicts an occupancy mask for the squares in the warped image and finally classifies the pieces. The described system achieves an error rate of 0.23% per square on the test set, 28 times better than the current state of the art. Further, a few-shot transfer learning approach is developed that is able to adapt the inference system to a previously unseen chess set using just two photos of the starting position, obtaining a per-square accuracy of 99.83% on images of that new chess set. The code, dataset, and trained models are made available online. |
| first_indexed | 2024-03-10T10:46:13Z |
| format | Article |
| id | doaj.art-c1090210fbaa4dd79cb5d6eedb49adf5 |
| institution | Directory Open Access Journal |
| issn | 2313-433X |
| language | English |
| last_indexed | 2024-03-10T10:46:13Z |
| publishDate | 2021-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Imaging |
| spelling | doaj.art-c1090210fbaa4dd79cb5d6eedb49adf52023-11-21T22:34:31ZengMDPI AGJournal of Imaging2313-433X2021-06-01769410.3390/jimaging7060094Determining Chess Game State from an ImageGeorg Wölflein0Ognjen Arandjelović1School of Computer Science, University of St Andrews, North Haugh, St Andrews KY16 9SX, Scotland, UKSchool of Computer Science, University of St Andrews, North Haugh, St Andrews KY16 9SX, Scotland, UKIdentifying the configuration of chess pieces from an image of a chessboard is a problem in computer vision that has not yet been solved accurately. However, it is important for helping amateur chess players improve their games by facilitating automatic computer analysis without the overhead of manually entering the pieces. Current approaches are limited by the lack of large datasets and are not designed to adapt to unseen chess sets. This paper puts forth a new dataset synthesised from a 3D model that is an order of magnitude larger than existing ones. Trained on this dataset, a novel end-to-end chess recognition system is presented that combines traditional computer vision techniques with deep learning. It localises the chessboard using a RANSAC-based algorithm that computes a projective transformation of the board onto a regular grid. Using two convolutional neural networks, it then predicts an occupancy mask for the squares in the warped image and finally classifies the pieces. The described system achieves an error rate of 0.23% per square on the test set, 28 times better than the current state of the art. Further, a few-shot transfer learning approach is developed that is able to adapt the inference system to a previously unseen chess set using just two photos of the starting position, obtaining a per-square accuracy of 99.83% on images of that new chess set. The code, dataset, and trained models are made available online.https://www.mdpi.com/2313-433X/7/6/94computer visionchessconvolutional neural networks |
| spellingShingle | Georg Wölflein Ognjen Arandjelović Determining Chess Game State from an Image Journal of Imaging computer vision chess convolutional neural networks |
| title | Determining Chess Game State from an Image |
| title_full | Determining Chess Game State from an Image |
| title_fullStr | Determining Chess Game State from an Image |
| title_full_unstemmed | Determining Chess Game State from an Image |
| title_short | Determining Chess Game State from an Image |
| title_sort | determining chess game state from an image |
| topic | computer vision chess convolutional neural networks |
| url | https://www.mdpi.com/2313-433X/7/6/94 |
| work_keys_str_mv | AT georgwolflein determiningchessgamestatefromanimage AT ognjenarandjelovic determiningchessgamestatefromanimage |