Computer vision detects inflammatory arthritis in standardized smartphone photographs in an Indian patient cohort
IntroductionComputer vision extracts meaning from pixelated images and holds promise in automating various clinical tasks. Convolutional neural networks (CNNs), a deep learning network used therein, have shown promise in analyzing X-ray images and joint photographs. We studied the performance of a C...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-11-01
|
| Series: | Frontiers in Medicine |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fmed.2023.1280462/full |
| _version_ | 1797629777301995520 |
|---|---|
| author | Sanat Phatak Somashree Chakraborty Pranay Goel |
| author_facet | Sanat Phatak Somashree Chakraborty Pranay Goel |
| author_sort | Sanat Phatak |
| collection | DOAJ |
| description | IntroductionComputer vision extracts meaning from pixelated images and holds promise in automating various clinical tasks. Convolutional neural networks (CNNs), a deep learning network used therein, have shown promise in analyzing X-ray images and joint photographs. We studied the performance of a CNN on standardized smartphone photographs in detecting inflammation in three hand joints and compared it to a rheumatologist’s diagnosis.MethodsWe enrolled 100 consecutive patients with inflammatory arthritis with an onset period of less than 2 years, excluding those with deformities. Each patient was examined by a rheumatologist, and the presence of synovitis in each joint was recorded. Hand photographs were taken in a standardized manner, anonymized, and cropped to include joints of interest. A ResNet-101 backbone modified for two class outputs (inflamed or not) was used for training. We also tested a hue-augmented dataset. We reported accuracy, sensitivity, and specificity for three joints: wrist, index finger proximal interphalangeal (IFPIP), and middle finger proximal interphalangeal (MFPIP), taking the rheumatologist’s opinion as the gold standard.ResultsThe cohort consisted of 100 individuals, of which 22 of them were men, with a mean age of 49.7 (SD 12.9) years. The majority of the cohort (n = 68, 68%) had rheumatoid arthritis. The wrist (125/200, 62.5%), MFPIP (94/200, 47%), and IFPIP (83/200, 41.5%) were the three most commonly inflamed joints. The CNN achieved the highest accuracy, sensitivity, and specificity in detecting synovitis in the MFPIP (83, 77, and 88%, respectively), followed by the IFPIP (74, 74, and 75%, respectively) and the wrist (62, 90, and 21%, respectively).DiscussionWe have demonstrated that computer vision was able to detect inflammation in three joints of the hand with reasonable accuracy on standardized photographs despite a small dataset. Feature engineering was not required, and the CNN worked despite a diversity in clinical diagnosis. Larger datasets are likely to improve accuracy and help explain the basis of classification. These data suggest a potential use of computer vision in screening and follow-up of inflammatory arthritis. |
| first_indexed | 2024-03-11T10:57:45Z |
| format | Article |
| id | doaj.art-ee139d1cfb004bc7bfeb402eeefa9ce1 |
| institution | Directory Open Access Journal |
| issn | 2296-858X |
| language | English |
| last_indexed | 2024-03-11T10:57:45Z |
| publishDate | 2023-11-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Medicine |
| spelling | doaj.art-ee139d1cfb004bc7bfeb402eeefa9ce12023-11-13T04:41:05ZengFrontiers Media S.A.Frontiers in Medicine2296-858X2023-11-011010.3389/fmed.2023.12804621280462Computer vision detects inflammatory arthritis in standardized smartphone photographs in an Indian patient cohortSanat Phatak0Somashree Chakraborty1Pranay Goel2KEM Hospital Research Centre, Pune, IndiaIndian Institute of Science, Education and Research, Pune, IndiaIndian Institute of Science, Education and Research, Pune, IndiaIntroductionComputer vision extracts meaning from pixelated images and holds promise in automating various clinical tasks. Convolutional neural networks (CNNs), a deep learning network used therein, have shown promise in analyzing X-ray images and joint photographs. We studied the performance of a CNN on standardized smartphone photographs in detecting inflammation in three hand joints and compared it to a rheumatologist’s diagnosis.MethodsWe enrolled 100 consecutive patients with inflammatory arthritis with an onset period of less than 2 years, excluding those with deformities. Each patient was examined by a rheumatologist, and the presence of synovitis in each joint was recorded. Hand photographs were taken in a standardized manner, anonymized, and cropped to include joints of interest. A ResNet-101 backbone modified for two class outputs (inflamed or not) was used for training. We also tested a hue-augmented dataset. We reported accuracy, sensitivity, and specificity for three joints: wrist, index finger proximal interphalangeal (IFPIP), and middle finger proximal interphalangeal (MFPIP), taking the rheumatologist’s opinion as the gold standard.ResultsThe cohort consisted of 100 individuals, of which 22 of them were men, with a mean age of 49.7 (SD 12.9) years. The majority of the cohort (n = 68, 68%) had rheumatoid arthritis. The wrist (125/200, 62.5%), MFPIP (94/200, 47%), and IFPIP (83/200, 41.5%) were the three most commonly inflamed joints. The CNN achieved the highest accuracy, sensitivity, and specificity in detecting synovitis in the MFPIP (83, 77, and 88%, respectively), followed by the IFPIP (74, 74, and 75%, respectively) and the wrist (62, 90, and 21%, respectively).DiscussionWe have demonstrated that computer vision was able to detect inflammation in three joints of the hand with reasonable accuracy on standardized photographs despite a small dataset. Feature engineering was not required, and the CNN worked despite a diversity in clinical diagnosis. Larger datasets are likely to improve accuracy and help explain the basis of classification. These data suggest a potential use of computer vision in screening and follow-up of inflammatory arthritis.https://www.frontiersin.org/articles/10.3389/fmed.2023.1280462/fullartificial intelligenceinflammatory arthritisdigital healthcomputer visionscreening |
| spellingShingle | Sanat Phatak Somashree Chakraborty Pranay Goel Computer vision detects inflammatory arthritis in standardized smartphone photographs in an Indian patient cohort Frontiers in Medicine artificial intelligence inflammatory arthritis digital health computer vision screening |
| title | Computer vision detects inflammatory arthritis in standardized smartphone photographs in an Indian patient cohort |
| title_full | Computer vision detects inflammatory arthritis in standardized smartphone photographs in an Indian patient cohort |
| title_fullStr | Computer vision detects inflammatory arthritis in standardized smartphone photographs in an Indian patient cohort |
| title_full_unstemmed | Computer vision detects inflammatory arthritis in standardized smartphone photographs in an Indian patient cohort |
| title_short | Computer vision detects inflammatory arthritis in standardized smartphone photographs in an Indian patient cohort |
| title_sort | computer vision detects inflammatory arthritis in standardized smartphone photographs in an indian patient cohort |
| topic | artificial intelligence inflammatory arthritis digital health computer vision screening |
| url | https://www.frontiersin.org/articles/10.3389/fmed.2023.1280462/full |
| work_keys_str_mv | AT sanatphatak computervisiondetectsinflammatoryarthritisinstandardizedsmartphonephotographsinanindianpatientcohort AT somashreechakraborty computervisiondetectsinflammatoryarthritisinstandardizedsmartphonephotographsinanindianpatientcohort AT pranaygoel computervisiondetectsinflammatoryarthritisinstandardizedsmartphonephotographsinanindianpatientcohort |