Evaluation of Automatic Legal Text Summarization Techniques for Greek Case Law
The increasing amount of legal information available online is overwhelming for both citizens and legal professionals, making it difficult and time-consuming to find relevant information and keep up with the latest legal developments. Automatic text summarization techniques can be highly beneficial...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-04-01
|
| Series: | Information |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2078-2489/14/4/250 |
| _version_ | 1797604986429898752 |
|---|---|
| author | Marios Koniaris Dimitris Galanis Eugenia Giannini Panayiotis Tsanakas |
| author_facet | Marios Koniaris Dimitris Galanis Eugenia Giannini Panayiotis Tsanakas |
| author_sort | Marios Koniaris |
| collection | DOAJ |
| description | The increasing amount of legal information available online is overwhelming for both citizens and legal professionals, making it difficult and time-consuming to find relevant information and keep up with the latest legal developments. Automatic text summarization techniques can be highly beneficial as they save time, reduce costs, and lessen the cognitive load of legal professionals. However, applying these techniques to legal documents poses several challenges due to the complexity of legal documents and the lack of needed resources, especially in linguistically under-resourced languages, such as the Greek language. In this paper, we address automatic summarization of Greek legal documents. A major challenge in this area is the lack of suitable datasets in the Greek language. In response, we developed a new metadata-rich dataset consisting of selected judgments from the Supreme Civil and Criminal Court of Greece, alongside their reference summaries and category tags, tailored for the purpose of automated legal document summarization. We also adopted several state-of-the-art methods for abstractive and extractive summarization and conducted a comprehensive evaluation of the methods using both human and automatic metrics. Our results: (i) revealed that, while extractive methods exhibit average performance, abstractive methods generate moderately fluent and coherent text, but they tend to receive low scores in relevance and consistency metrics; (ii) indicated the need for metrics that capture better a legal document summary’s coherence, relevance, and consistency; (iii) demonstrated that fine-tuning BERT models on a specific upstream task can significantly improve the model’s performance. |
| first_indexed | 2024-03-11T04:54:38Z |
| format | Article |
| id | doaj.art-03266af84b8c4287b8c0fe8546914395 |
| institution | Directory Open Access Journal |
| issn | 2078-2489 |
| language | English |
| last_indexed | 2024-03-11T04:54:38Z |
| publishDate | 2023-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Information |
| spelling | doaj.art-03266af84b8c4287b8c0fe85469143952023-11-17T19:44:49ZengMDPI AGInformation2078-24892023-04-0114425010.3390/info14040250Evaluation of Automatic Legal Text Summarization Techniques for Greek Case LawMarios Koniaris0Dimitris Galanis1Eugenia Giannini2Panayiotis Tsanakas3Division of Computer Science, School of Electrical and Computer Engineering, National Technical University of Athens, Iroon Polytechniou 9, Zographou Campus, 15780 Athens, GreeceDivision of Computer Science, School of Electrical and Computer Engineering, National Technical University of Athens, Iroon Polytechniou 9, Zographou Campus, 15780 Athens, GreeceDepartment of Humanities Social Sciences and Law, School of Applied Mathematical and Physical Sciences, National Technical University of Athens, Iroon Polytechniou 9, Zographou Campus, 15780 Athens, GreeceDivision of Computer Science, School of Electrical and Computer Engineering, National Technical University of Athens, Iroon Polytechniou 9, Zographou Campus, 15780 Athens, GreeceThe increasing amount of legal information available online is overwhelming for both citizens and legal professionals, making it difficult and time-consuming to find relevant information and keep up with the latest legal developments. Automatic text summarization techniques can be highly beneficial as they save time, reduce costs, and lessen the cognitive load of legal professionals. However, applying these techniques to legal documents poses several challenges due to the complexity of legal documents and the lack of needed resources, especially in linguistically under-resourced languages, such as the Greek language. In this paper, we address automatic summarization of Greek legal documents. A major challenge in this area is the lack of suitable datasets in the Greek language. In response, we developed a new metadata-rich dataset consisting of selected judgments from the Supreme Civil and Criminal Court of Greece, alongside their reference summaries and category tags, tailored for the purpose of automated legal document summarization. We also adopted several state-of-the-art methods for abstractive and extractive summarization and conducted a comprehensive evaluation of the methods using both human and automatic metrics. Our results: (i) revealed that, while extractive methods exhibit average performance, abstractive methods generate moderately fluent and coherent text, but they tend to receive low scores in relevance and consistency metrics; (ii) indicated the need for metrics that capture better a legal document summary’s coherence, relevance, and consistency; (iii) demonstrated that fine-tuning BERT models on a specific upstream task can significantly improve the model’s performance.https://www.mdpi.com/2078-2489/14/4/250automatic text summarizationcase law summarizationlegal informationsummarization evaluation |
| spellingShingle | Marios Koniaris Dimitris Galanis Eugenia Giannini Panayiotis Tsanakas Evaluation of Automatic Legal Text Summarization Techniques for Greek Case Law Information automatic text summarization case law summarization legal information summarization evaluation |
| title | Evaluation of Automatic Legal Text Summarization Techniques for Greek Case Law |
| title_full | Evaluation of Automatic Legal Text Summarization Techniques for Greek Case Law |
| title_fullStr | Evaluation of Automatic Legal Text Summarization Techniques for Greek Case Law |
| title_full_unstemmed | Evaluation of Automatic Legal Text Summarization Techniques for Greek Case Law |
| title_short | Evaluation of Automatic Legal Text Summarization Techniques for Greek Case Law |
| title_sort | evaluation of automatic legal text summarization techniques for greek case law |
| topic | automatic text summarization case law summarization legal information summarization evaluation |
| url | https://www.mdpi.com/2078-2489/14/4/250 |
| work_keys_str_mv | AT marioskoniaris evaluationofautomaticlegaltextsummarizationtechniquesforgreekcaselaw AT dimitrisgalanis evaluationofautomaticlegaltextsummarizationtechniquesforgreekcaselaw AT eugeniagiannini evaluationofautomaticlegaltextsummarizationtechniquesforgreekcaselaw AT panayiotistsanakas evaluationofautomaticlegaltextsummarizationtechniquesforgreekcaselaw |