RoDLA: benchmarking the robustness of Document Layout Analysis models
Before developing a Document Layout Analysis (DLA) model in real-world applications, conducting comprehensive robustness testing is essential. However, the robustness of DLA models remains underexplored in the literature. To address this, we are the first to introduce a robustness benchmark for DLA...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Conference item |
| Language: | English |
| Published: |
IEEE
2024
|
| _version_ | 1817932396332318720 |
|---|---|
| author | Chen, Y Zhang, J Peng, K Zheng, J Liu, R Torr, P Stiefelhagen, R |
| author_facet | Chen, Y Zhang, J Peng, K Zheng, J Liu, R Torr, P Stiefelhagen, R |
| author_sort | Chen, Y |
| collection | OXFORD |
| description | Before developing a Document Layout Analysis (DLA) model in real-world applications, conducting comprehensive robustness testing is essential. However, the robustness of DLA models remains underexplored in the literature. To address this, we are the first to introduce a robustness benchmark for DLA models, which includes 450K document images of three datasets. To cover realistic corruptions, we propose a perturbation taxonomy with 12 common document perturbations with 3 severity levels inspired by realworld document processing. Additionally, to better understand document perturbation impacts, we propose two metrics, Mean Perturbation Effect (mPE) for perturbation assessment and Mean Robustness Degradation (mRD) for robustness evaluation. Furthermore, we introduce a self-titled model, i.e., Robust Document Layout Analyzer (RoDLA), which improves attention mechanisms to boost extraction of robust features. Experiments on the proposed benchmarks (PubLayNet-P, DocLayNet-P, andM6Doc-P) demonstrate that RoDLA obtains state-of-the-art mRD scores of 115.7, 135.4, and 150.4, respectively. Compared to previous methods, RoDLA achieves notable improvements in mAP of +3.8%, +7.1% and +12.1%, respectively. |
| first_indexed | 2024-12-09T03:37:15Z |
| format | Conference item |
| id | oxford-uuid:4c6ed205-761e-4fcc-8573-f92e4d7111c4 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-12-09T03:37:15Z |
| publishDate | 2024 |
| publisher | IEEE |
| record_format | dspace |
| spelling | oxford-uuid:4c6ed205-761e-4fcc-8573-f92e4d7111c42024-12-03T11:20:18ZRoDLA: benchmarking the robustness of Document Layout Analysis modelsConference itemhttp://purl.org/coar/resource_type/c_5794uuid:4c6ed205-761e-4fcc-8573-f92e4d7111c4EnglishSymplectic ElementsIEEE2024Chen, YZhang, JPeng, KZheng, JLiu, RTorr, PStiefelhagen, RBefore developing a Document Layout Analysis (DLA) model in real-world applications, conducting comprehensive robustness testing is essential. However, the robustness of DLA models remains underexplored in the literature. To address this, we are the first to introduce a robustness benchmark for DLA models, which includes 450K document images of three datasets. To cover realistic corruptions, we propose a perturbation taxonomy with 12 common document perturbations with 3 severity levels inspired by realworld document processing. Additionally, to better understand document perturbation impacts, we propose two metrics, Mean Perturbation Effect (mPE) for perturbation assessment and Mean Robustness Degradation (mRD) for robustness evaluation. Furthermore, we introduce a self-titled model, i.e., Robust Document Layout Analyzer (RoDLA), which improves attention mechanisms to boost extraction of robust features. Experiments on the proposed benchmarks (PubLayNet-P, DocLayNet-P, andM6Doc-P) demonstrate that RoDLA obtains state-of-the-art mRD scores of 115.7, 135.4, and 150.4, respectively. Compared to previous methods, RoDLA achieves notable improvements in mAP of +3.8%, +7.1% and +12.1%, respectively. |
| spellingShingle | Chen, Y Zhang, J Peng, K Zheng, J Liu, R Torr, P Stiefelhagen, R RoDLA: benchmarking the robustness of Document Layout Analysis models |
| title | RoDLA: benchmarking the robustness of Document Layout Analysis models |
| title_full | RoDLA: benchmarking the robustness of Document Layout Analysis models |
| title_fullStr | RoDLA: benchmarking the robustness of Document Layout Analysis models |
| title_full_unstemmed | RoDLA: benchmarking the robustness of Document Layout Analysis models |
| title_short | RoDLA: benchmarking the robustness of Document Layout Analysis models |
| title_sort | rodla benchmarking the robustness of document layout analysis models |
| work_keys_str_mv | AT cheny rodlabenchmarkingtherobustnessofdocumentlayoutanalysismodels AT zhangj rodlabenchmarkingtherobustnessofdocumentlayoutanalysismodels AT pengk rodlabenchmarkingtherobustnessofdocumentlayoutanalysismodels AT zhengj rodlabenchmarkingtherobustnessofdocumentlayoutanalysismodels AT liur rodlabenchmarkingtherobustnessofdocumentlayoutanalysismodels AT torrp rodlabenchmarkingtherobustnessofdocumentlayoutanalysismodels AT stiefelhagenr rodlabenchmarkingtherobustnessofdocumentlayoutanalysismodels |