Evaluating Human versus Machine Learning Performance in a LegalTech Problem
Many machine learning-based document processing applications have been published in recent years. Applying these methodologies can reduce the cost of labor-intensive tasks and induce changes in the company’s structure. The artificial intelligence-based application can replace the application of trai...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-12-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/1/297 |
| _version_ | 1827668768683196416 |
|---|---|
| author | Tamás Orosz Renátó Vági Gergely Márk Csányi Dániel Nagy István Üveges János Pál Vadász Andrea Megyeri |
| author_facet | Tamás Orosz Renátó Vági Gergely Márk Csányi Dániel Nagy István Üveges János Pál Vadász Andrea Megyeri |
| author_sort | Tamás Orosz |
| collection | DOAJ |
| description | Many machine learning-based document processing applications have been published in recent years. Applying these methodologies can reduce the cost of labor-intensive tasks and induce changes in the company’s structure. The artificial intelligence-based application can replace the application of trainees and free up the time of experts, which can increase innovation inside the company by letting them be involved in tasks with greater added value. However, the development cost of these methodologies can be high, and usually, it is not a straightforward task. This paper presents a survey result, where a machine learning-based legal text labeler competed with multiple people with different legal domain knowledge. The machine learning-based application used binary SVM-based classifiers to resolve the multi-label classification problem. The used methods were encapsulated and deployed as a digital twin into a production environment. The results show that machine learning algorithms can be effectively utilized for monotonous but domain knowledge- and attention-demanding tasks. The results also suggest that embracing the machine learning-based solution can increase discoverability and enrich the value of data. The test confirmed that the accuracy of a machine learning-based system matches up with the long-term accuracy of legal experts, which makes it applicable to automatize the working process. |
| first_indexed | 2024-03-10T03:49:15Z |
| format | Article |
| id | doaj.art-618a2220a46e44628c413d722f218070 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T03:49:15Z |
| publishDate | 2021-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-618a2220a46e44628c413d722f2180702023-11-23T11:10:47ZengMDPI AGApplied Sciences2076-34172021-12-0112129710.3390/app12010297Evaluating Human versus Machine Learning Performance in a LegalTech ProblemTamás Orosz0Renátó Vági1Gergely Márk Csányi2Dániel Nagy3István Üveges4János Pál Vadász5Andrea Megyeri6MONTANA Knowledge Management Ltd., H-1097 Budapest, HungaryMONTANA Knowledge Management Ltd., H-1097 Budapest, HungaryMONTANA Knowledge Management Ltd., H-1097 Budapest, HungaryMONTANA Knowledge Management Ltd., H-1097 Budapest, HungaryMONTANA Knowledge Management Ltd., H-1097 Budapest, HungaryMONTANA Knowledge Management Ltd., H-1097 Budapest, HungaryWolters Kluwer Hungary Ltd., Budafoki Way 187-189, H-1117 Budapest, HungaryMany machine learning-based document processing applications have been published in recent years. Applying these methodologies can reduce the cost of labor-intensive tasks and induce changes in the company’s structure. The artificial intelligence-based application can replace the application of trainees and free up the time of experts, which can increase innovation inside the company by letting them be involved in tasks with greater added value. However, the development cost of these methodologies can be high, and usually, it is not a straightforward task. This paper presents a survey result, where a machine learning-based legal text labeler competed with multiple people with different legal domain knowledge. The machine learning-based application used binary SVM-based classifiers to resolve the multi-label classification problem. The used methods were encapsulated and deployed as a digital twin into a production environment. The results show that machine learning algorithms can be effectively utilized for monotonous but domain knowledge- and attention-demanding tasks. The results also suggest that embracing the machine learning-based solution can increase discoverability and enrich the value of data. The test confirmed that the accuracy of a machine learning-based system matches up with the long-term accuracy of legal experts, which makes it applicable to automatize the working process.https://www.mdpi.com/2076-3417/12/1/297legal techdata analyticsartificial intelligenceIndustry 4.0 |
| spellingShingle | Tamás Orosz Renátó Vági Gergely Márk Csányi Dániel Nagy István Üveges János Pál Vadász Andrea Megyeri Evaluating Human versus Machine Learning Performance in a LegalTech Problem Applied Sciences legal tech data analytics artificial intelligence Industry 4.0 |
| title | Evaluating Human versus Machine Learning Performance in a LegalTech Problem |
| title_full | Evaluating Human versus Machine Learning Performance in a LegalTech Problem |
| title_fullStr | Evaluating Human versus Machine Learning Performance in a LegalTech Problem |
| title_full_unstemmed | Evaluating Human versus Machine Learning Performance in a LegalTech Problem |
| title_short | Evaluating Human versus Machine Learning Performance in a LegalTech Problem |
| title_sort | evaluating human versus machine learning performance in a legaltech problem |
| topic | legal tech data analytics artificial intelligence Industry 4.0 |
| url | https://www.mdpi.com/2076-3417/12/1/297 |
| work_keys_str_mv | AT tamasorosz evaluatinghumanversusmachinelearningperformanceinalegaltechproblem AT renatovagi evaluatinghumanversusmachinelearningperformanceinalegaltechproblem AT gergelymarkcsanyi evaluatinghumanversusmachinelearningperformanceinalegaltechproblem AT danielnagy evaluatinghumanversusmachinelearningperformanceinalegaltechproblem AT istvanuveges evaluatinghumanversusmachinelearningperformanceinalegaltechproblem AT janospalvadasz evaluatinghumanversusmachinelearningperformanceinalegaltechproblem AT andreamegyeri evaluatinghumanversusmachinelearningperformanceinalegaltechproblem |