Assessment of the Performances of the Protein Modeling Techniques Participating in CASP15 Using a Structure-Based Functional Site Prediction Approach: ResiRole
Background: Model quality assessments via computational methods which entail comparisons of the modeled structures to the experimentally determined structures are essential in the field of protein structure prediction. The assessments provide means to benchmark the accuracies of the modeling techniq...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-11-01
|
| Series: | Bioengineering |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2306-5354/10/12/1377 |
| _version_ | 1797381968796581888 |
|---|---|
| author | Geoffrey J. Huang Thomas K. Parry William A. McLaughlin |
| author_facet | Geoffrey J. Huang Thomas K. Parry William A. McLaughlin |
| author_sort | Geoffrey J. Huang |
| collection | DOAJ |
| description | Background: Model quality assessments via computational methods which entail comparisons of the modeled structures to the experimentally determined structures are essential in the field of protein structure prediction. The assessments provide means to benchmark the accuracies of the modeling techniques and to aid with their development. We previously described the ResiRole method to gauge model quality principally based on the preservation of the structural characteristics described in SeqFEATURE functional site prediction models. Methods: We apply ResiRole to benchmark modeling group performances in the Critical Assessment of Structure Prediction experiment, round 15. To gauge model quality, a normalized Predicted Functional site Similarity Score (PFSS) was calculated as the average of one minus the absolute values of the differences of the functional site prediction probabilities, as found for the experimental structures versus those found at the corresponding sites in the structure models. Results: The average PFSS per modeling group (gPFSS) correlates with standard quality metrics, and can effectively be used to rank the accuracies of the groups. For the free modeling (FM) category, correlation coefficients of the Local Distance Difference Test (LDDT) and Global Distance Test-Total Score (GDT-TS) metrics with gPFSS were 0.98239 and 0.87691, respectively. An example finding for a specific group is that the gPFSS for EMBER3D was higher than expected based on the predictive relationship between gPFSS and LDDT. We infer the result is due to the use of constraints imprinted by function that are a part of the EMBER3D methodology. Also, we find functional site predictions that may guide further functional characterizations of the respective proteins. Conclusion: The gPFSS metric provides an effective means to assess and rank the performances of the structure prediction techniques according to their abilities to accurately recount the structural features at predicted functional sites. |
| first_indexed | 2024-03-08T20:59:47Z |
| format | Article |
| id | doaj.art-7b8c79fc756141c8aaaef907718b6ccc |
| institution | Directory Open Access Journal |
| issn | 2306-5354 |
| language | English |
| last_indexed | 2024-03-08T20:59:47Z |
| publishDate | 2023-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Bioengineering |
| spelling | doaj.art-7b8c79fc756141c8aaaef907718b6ccc2023-12-22T13:54:05ZengMDPI AGBioengineering2306-53542023-11-011012137710.3390/bioengineering10121377Assessment of the Performances of the Protein Modeling Techniques Participating in CASP15 Using a Structure-Based Functional Site Prediction Approach: ResiRoleGeoffrey J. Huang0Thomas K. Parry1William A. McLaughlin2Department of Medical Education, Geisinger Commonwealth School of Medicine, 525 Pine Street, Scranton, PA 18509, USADepartment of Medical Education, Geisinger Commonwealth School of Medicine, 525 Pine Street, Scranton, PA 18509, USADepartment of Medical Education, Geisinger Commonwealth School of Medicine, 525 Pine Street, Scranton, PA 18509, USABackground: Model quality assessments via computational methods which entail comparisons of the modeled structures to the experimentally determined structures are essential in the field of protein structure prediction. The assessments provide means to benchmark the accuracies of the modeling techniques and to aid with their development. We previously described the ResiRole method to gauge model quality principally based on the preservation of the structural characteristics described in SeqFEATURE functional site prediction models. Methods: We apply ResiRole to benchmark modeling group performances in the Critical Assessment of Structure Prediction experiment, round 15. To gauge model quality, a normalized Predicted Functional site Similarity Score (PFSS) was calculated as the average of one minus the absolute values of the differences of the functional site prediction probabilities, as found for the experimental structures versus those found at the corresponding sites in the structure models. Results: The average PFSS per modeling group (gPFSS) correlates with standard quality metrics, and can effectively be used to rank the accuracies of the groups. For the free modeling (FM) category, correlation coefficients of the Local Distance Difference Test (LDDT) and Global Distance Test-Total Score (GDT-TS) metrics with gPFSS were 0.98239 and 0.87691, respectively. An example finding for a specific group is that the gPFSS for EMBER3D was higher than expected based on the predictive relationship between gPFSS and LDDT. We infer the result is due to the use of constraints imprinted by function that are a part of the EMBER3D methodology. Also, we find functional site predictions that may guide further functional characterizations of the respective proteins. Conclusion: The gPFSS metric provides an effective means to assess and rank the performances of the structure prediction techniques according to their abilities to accurately recount the structural features at predicted functional sites.https://www.mdpi.com/2306-5354/10/12/1377protein structure predictionprotein model quality assessmentfunctional site predictionbenchmarkingCASP15 assessment |
| spellingShingle | Geoffrey J. Huang Thomas K. Parry William A. McLaughlin Assessment of the Performances of the Protein Modeling Techniques Participating in CASP15 Using a Structure-Based Functional Site Prediction Approach: ResiRole Bioengineering protein structure prediction protein model quality assessment functional site prediction benchmarking CASP15 assessment |
| title | Assessment of the Performances of the Protein Modeling Techniques Participating in CASP15 Using a Structure-Based Functional Site Prediction Approach: ResiRole |
| title_full | Assessment of the Performances of the Protein Modeling Techniques Participating in CASP15 Using a Structure-Based Functional Site Prediction Approach: ResiRole |
| title_fullStr | Assessment of the Performances of the Protein Modeling Techniques Participating in CASP15 Using a Structure-Based Functional Site Prediction Approach: ResiRole |
| title_full_unstemmed | Assessment of the Performances of the Protein Modeling Techniques Participating in CASP15 Using a Structure-Based Functional Site Prediction Approach: ResiRole |
| title_short | Assessment of the Performances of the Protein Modeling Techniques Participating in CASP15 Using a Structure-Based Functional Site Prediction Approach: ResiRole |
| title_sort | assessment of the performances of the protein modeling techniques participating in casp15 using a structure based functional site prediction approach resirole |
| topic | protein structure prediction protein model quality assessment functional site prediction benchmarking CASP15 assessment |
| url | https://www.mdpi.com/2306-5354/10/12/1377 |
| work_keys_str_mv | AT geoffreyjhuang assessmentoftheperformancesoftheproteinmodelingtechniquesparticipatingincasp15usingastructurebasedfunctionalsitepredictionapproachresirole AT thomaskparry assessmentoftheperformancesoftheproteinmodelingtechniquesparticipatingincasp15usingastructurebasedfunctionalsitepredictionapproachresirole AT williamamclaughlin assessmentoftheperformancesoftheproteinmodelingtechniquesparticipatingincasp15usingastructurebasedfunctionalsitepredictionapproachresirole |