Proposal for Post Hoc Quality Control in Instrumented Motion Analysis Using Markerless Motion Capture: Development and Usability Study
BackgroundInstrumented assessment of motor symptoms has emerged as a promising extension to the clinical assessment of several movement disorders. The use of mobile and inexpensive technologies such as some markerless motion capture technologies is especially promising for la...
| Main Authors: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2022-04-01
|
| Series: | JMIR Human Factors |
| Online Access: | https://humanfactors.jmir.org/2022/2/e26825 |
| _version_ | 1797735204382572544 |
|---|---|
| author | Hanna Marie Röhling Patrik Althoff Radina Arsenova Daniel Drebinger Norman Gigengack Anna Chorschew Daniel Kroneberg Maria Rönnefarth Tobias Ellermeyer Sina Cathérine Rosenkranz Christoph Heesen Behnoush Behnia Shigeki Hirano Satoshi Kuwabara Friedemann Paul Alexander Ulrich Brandt Tanja Schmitz-Hübsch |
| author_facet | Hanna Marie Röhling Patrik Althoff Radina Arsenova Daniel Drebinger Norman Gigengack Anna Chorschew Daniel Kroneberg Maria Rönnefarth Tobias Ellermeyer Sina Cathérine Rosenkranz Christoph Heesen Behnoush Behnia Shigeki Hirano Satoshi Kuwabara Friedemann Paul Alexander Ulrich Brandt Tanja Schmitz-Hübsch |
| author_sort | Hanna Marie Röhling |
| collection | DOAJ |
| description |
BackgroundInstrumented assessment of motor symptoms has emerged as a promising extension to the clinical assessment of several movement disorders. The use of mobile and inexpensive technologies such as some markerless motion capture technologies is especially promising for large-scale application but has not transitioned into clinical routine to date. A crucial step on this path is to implement standardized, clinically applicable tools that identify and control for quality concerns.
ObjectiveThe main goal of this study comprises the development of a systematic quality control (QC) procedure for data collected with markerless motion capture technology and its experimental implementation to identify specific quality concerns and thereby rate the usability of recordings.
MethodsWe developed a post hoc QC pipeline that was evaluated using a large set of short motor task recordings of healthy controls (2010 recordings from 162 subjects) and people with multiple sclerosis (2682 recordings from 187 subjects). For each of these recordings, 2 raters independently applied the pipeline. They provided overall usability decisions and identified technical and performance-related quality concerns, which yielded respective proportions of their occurrence as a main result.
ResultsThe approach developed here has proven user-friendly and applicable on a large scale. Raters’ decisions on recording usability were concordant in 71.5%-92.3% of cases, depending on the motor task. Furthermore, 39.6%-85.1% of recordings were concordantly rated as being of satisfactory quality whereas in 5.0%-26.3%, both raters agreed to discard the recording.
ConclusionsWe present a QC pipeline that seems feasible and useful for instant quality screening in the clinical setting. Results confirm the need of QC despite using standard test setups, testing protocols, and operator training for the employed system and by extension, for other task-based motor assessment technologies. Results of the QC process can be used to clean existing data sets, optimize quality assurance measures, as well as foster the development of automated QC approaches and therefore improve the overall reliability of kinematic data sets. |
| first_indexed | 2024-03-12T12:55:12Z |
| format | Article |
| id | doaj.art-7f7fe1a7a74a4ec9b856e76a7c49adf8 |
| institution | Directory Open Access Journal |
| issn | 2292-9495 |
| language | English |
| last_indexed | 2024-03-12T12:55:12Z |
| publishDate | 2022-04-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | JMIR Human Factors |
| spelling | doaj.art-7f7fe1a7a74a4ec9b856e76a7c49adf82023-08-28T21:18:38ZengJMIR PublicationsJMIR Human Factors2292-94952022-04-0192e2682510.2196/26825Proposal for Post Hoc Quality Control in Instrumented Motion Analysis Using Markerless Motion Capture: Development and Usability StudyHanna Marie Röhlinghttps://orcid.org/0000-0002-3106-5830Patrik Althoffhttps://orcid.org/0000-0002-8044-4538Radina Arsenovahttps://orcid.org/0000-0003-3116-0464Daniel Drebingerhttps://orcid.org/0000-0002-5958-2872Norman Gigengackhttps://orcid.org/0000-0003-4852-9547Anna Chorschewhttps://orcid.org/0000-0002-5246-3635Daniel Kroneberghttps://orcid.org/0000-0001-5560-538XMaria Rönnefarthhttps://orcid.org/0000-0002-0846-8287Tobias Ellermeyerhttps://orcid.org/0000-0002-5217-9896Sina Cathérine Rosenkranzhttps://orcid.org/0000-0002-5228-4266Christoph Heesenhttps://orcid.org/0000-0001-8131-9467Behnoush Behniahttps://orcid.org/0000-0003-1361-2076Shigeki Hiranohttps://orcid.org/0000-0003-3473-2317Satoshi Kuwabarahttps://orcid.org/0000-0002-4716-8578Friedemann Paulhttps://orcid.org/0000-0002-6378-0070Alexander Ulrich Brandthttps://orcid.org/0000-0002-9768-014XTanja Schmitz-Hübschhttps://orcid.org/0000-0003-4833-5937 BackgroundInstrumented assessment of motor symptoms has emerged as a promising extension to the clinical assessment of several movement disorders. The use of mobile and inexpensive technologies such as some markerless motion capture technologies is especially promising for large-scale application but has not transitioned into clinical routine to date. A crucial step on this path is to implement standardized, clinically applicable tools that identify and control for quality concerns. ObjectiveThe main goal of this study comprises the development of a systematic quality control (QC) procedure for data collected with markerless motion capture technology and its experimental implementation to identify specific quality concerns and thereby rate the usability of recordings. MethodsWe developed a post hoc QC pipeline that was evaluated using a large set of short motor task recordings of healthy controls (2010 recordings from 162 subjects) and people with multiple sclerosis (2682 recordings from 187 subjects). For each of these recordings, 2 raters independently applied the pipeline. They provided overall usability decisions and identified technical and performance-related quality concerns, which yielded respective proportions of their occurrence as a main result. ResultsThe approach developed here has proven user-friendly and applicable on a large scale. Raters’ decisions on recording usability were concordant in 71.5%-92.3% of cases, depending on the motor task. Furthermore, 39.6%-85.1% of recordings were concordantly rated as being of satisfactory quality whereas in 5.0%-26.3%, both raters agreed to discard the recording. ConclusionsWe present a QC pipeline that seems feasible and useful for instant quality screening in the clinical setting. Results confirm the need of QC despite using standard test setups, testing protocols, and operator training for the employed system and by extension, for other task-based motor assessment technologies. Results of the QC process can be used to clean existing data sets, optimize quality assurance measures, as well as foster the development of automated QC approaches and therefore improve the overall reliability of kinematic data sets.https://humanfactors.jmir.org/2022/2/e26825 |
| spellingShingle | Hanna Marie Röhling Patrik Althoff Radina Arsenova Daniel Drebinger Norman Gigengack Anna Chorschew Daniel Kroneberg Maria Rönnefarth Tobias Ellermeyer Sina Cathérine Rosenkranz Christoph Heesen Behnoush Behnia Shigeki Hirano Satoshi Kuwabara Friedemann Paul Alexander Ulrich Brandt Tanja Schmitz-Hübsch Proposal for Post Hoc Quality Control in Instrumented Motion Analysis Using Markerless Motion Capture: Development and Usability Study JMIR Human Factors |
| title | Proposal for Post Hoc Quality Control in Instrumented Motion Analysis Using Markerless Motion Capture: Development and Usability Study |
| title_full | Proposal for Post Hoc Quality Control in Instrumented Motion Analysis Using Markerless Motion Capture: Development and Usability Study |
| title_fullStr | Proposal for Post Hoc Quality Control in Instrumented Motion Analysis Using Markerless Motion Capture: Development and Usability Study |
| title_full_unstemmed | Proposal for Post Hoc Quality Control in Instrumented Motion Analysis Using Markerless Motion Capture: Development and Usability Study |
| title_short | Proposal for Post Hoc Quality Control in Instrumented Motion Analysis Using Markerless Motion Capture: Development and Usability Study |
| title_sort | proposal for post hoc quality control in instrumented motion analysis using markerless motion capture development and usability study |
| url | https://humanfactors.jmir.org/2022/2/e26825 |
| work_keys_str_mv | AT hannamarierohling proposalforposthocqualitycontrolininstrumentedmotionanalysisusingmarkerlessmotioncapturedevelopmentandusabilitystudy AT patrikalthoff proposalforposthocqualitycontrolininstrumentedmotionanalysisusingmarkerlessmotioncapturedevelopmentandusabilitystudy AT radinaarsenova proposalforposthocqualitycontrolininstrumentedmotionanalysisusingmarkerlessmotioncapturedevelopmentandusabilitystudy AT danieldrebinger proposalforposthocqualitycontrolininstrumentedmotionanalysisusingmarkerlessmotioncapturedevelopmentandusabilitystudy AT normangigengack proposalforposthocqualitycontrolininstrumentedmotionanalysisusingmarkerlessmotioncapturedevelopmentandusabilitystudy AT annachorschew proposalforposthocqualitycontrolininstrumentedmotionanalysisusingmarkerlessmotioncapturedevelopmentandusabilitystudy AT danielkroneberg proposalforposthocqualitycontrolininstrumentedmotionanalysisusingmarkerlessmotioncapturedevelopmentandusabilitystudy AT mariaronnefarth proposalforposthocqualitycontrolininstrumentedmotionanalysisusingmarkerlessmotioncapturedevelopmentandusabilitystudy AT tobiasellermeyer proposalforposthocqualitycontrolininstrumentedmotionanalysisusingmarkerlessmotioncapturedevelopmentandusabilitystudy AT sinacatherinerosenkranz proposalforposthocqualitycontrolininstrumentedmotionanalysisusingmarkerlessmotioncapturedevelopmentandusabilitystudy AT christophheesen proposalforposthocqualitycontrolininstrumentedmotionanalysisusingmarkerlessmotioncapturedevelopmentandusabilitystudy AT behnoushbehnia proposalforposthocqualitycontrolininstrumentedmotionanalysisusingmarkerlessmotioncapturedevelopmentandusabilitystudy AT shigekihirano proposalforposthocqualitycontrolininstrumentedmotionanalysisusingmarkerlessmotioncapturedevelopmentandusabilitystudy AT satoshikuwabara proposalforposthocqualitycontrolininstrumentedmotionanalysisusingmarkerlessmotioncapturedevelopmentandusabilitystudy AT friedemannpaul proposalforposthocqualitycontrolininstrumentedmotionanalysisusingmarkerlessmotioncapturedevelopmentandusabilitystudy AT alexanderulrichbrandt proposalforposthocqualitycontrolininstrumentedmotionanalysisusingmarkerlessmotioncapturedevelopmentandusabilitystudy AT tanjaschmitzhubsch proposalforposthocqualitycontrolininstrumentedmotionanalysisusingmarkerlessmotioncapturedevelopmentandusabilitystudy |