Framework for evaluating code generation ability of large language models
Large language models (LLMs) have revolutionized various applications in natural language processing and exhibited proficiency in generating programming code. We propose a framework for evaluating the code generation ability of LLMs and introduce a new metric, pass-ratio@n, which captures the granul...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Electronics and Telecommunications Research Institute (ETRI)
2024-02-01
|
| Series: | ETRI Journal |
| Subjects: | |
| Online Access: | https://doi.org/10.4218/etrij.2023-0357 |