Is JavaScript Call Graph Extraction Solved Yet? A Comparative Study of Static and Dynamic Tools
Code analysis is more important than ever because JavaScript is increasingly popular and actively used, both on the client and server sides. Most algorithms for analyzing vulnerabilities, finding coding issues, or inferring type depend on the call graph representation of the underlying program. Luck...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10066273/ |
| _version_ | 1797866987283546112 |
|---|---|
| author | Gabor Antal Peter Hegedus Zoltan Herczeg Gabor Loki Rudolf Ferenc |
| author_facet | Gabor Antal Peter Hegedus Zoltan Herczeg Gabor Loki Rudolf Ferenc |
| author_sort | Gabor Antal |
| collection | DOAJ |
| description | Code analysis is more important than ever because JavaScript is increasingly popular and actively used, both on the client and server sides. Most algorithms for analyzing vulnerabilities, finding coding issues, or inferring type depend on the call graph representation of the underlying program. Luckily, there are quite a few tools to get this job done already. However, their performance in vitro and especially in vivo has not yet been extensively compared and evaluated. In this paper, we compare several approaches for building JavaScript call graphs, namely five static and two dynamic approaches on 26 WebKit SunSpider programs, and two static and two dynamic approaches on 12 real-world Node.js programs. The tools under examination using static techniques were npm call graph, IBM WALA, Google Closure Compiler, Approximate Call Graph, and Type Analyzer for JavaScript. We performed dynamic analyzes relying on the nodejs-cg tool (a customized Node.js runtime) and the NodeProf instrumentation and profiling framework. We provide a quantitative evaluation of the results, and a result quality analysis based on 941 manually validated call edges. On the SunSpider programs, which do not take any inputs, so dynamic extraction could be complete, all the static tools also performed well. For example, TAJS found 93% of all edges while having a 97% precision compared to the precise dynamic call graph. When it comes to real-world Node.js modules, our evaluation shows that static tools struggle with parsing the code and fail to detect a significant amount of call edges that dynamic approaches can capture. Nonetheless, a significant number of edges not detected by dynamic approaches are also reported. Among these, however, there are also edges that are real, but for some reason the unit tests did not execute the branches in which these calls were included. |
| first_indexed | 2024-04-09T23:33:05Z |
| format | Article |
| id | doaj.art-8aa556cb15d14a7986f0f875dbbac3b4 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-09T23:33:05Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-8aa556cb15d14a7986f0f875dbbac3b42023-03-20T23:00:33ZengIEEEIEEE Access2169-35362023-01-0111252662528410.1109/ACCESS.2023.325598410066273Is JavaScript Call Graph Extraction Solved Yet? A Comparative Study of Static and Dynamic ToolsGabor Antal0https://orcid.org/0000-0002-3002-8624Peter Hegedus1https://orcid.org/0000-0003-4592-6504Zoltan Herczeg2Gabor Loki3Rudolf Ferenc4https://orcid.org/0000-0001-8897-7403Department of Software Engineering, University of Szeged, Szeged, HungaryDepartment of Software Engineering, University of Szeged, Szeged, HungaryDepartment of Software Engineering, University of Szeged, Szeged, HungaryDepartment of Software Engineering, University of Szeged, Szeged, HungaryDepartment of Software Engineering, University of Szeged, Szeged, HungaryCode analysis is more important than ever because JavaScript is increasingly popular and actively used, both on the client and server sides. Most algorithms for analyzing vulnerabilities, finding coding issues, or inferring type depend on the call graph representation of the underlying program. Luckily, there are quite a few tools to get this job done already. However, their performance in vitro and especially in vivo has not yet been extensively compared and evaluated. In this paper, we compare several approaches for building JavaScript call graphs, namely five static and two dynamic approaches on 26 WebKit SunSpider programs, and two static and two dynamic approaches on 12 real-world Node.js programs. The tools under examination using static techniques were npm call graph, IBM WALA, Google Closure Compiler, Approximate Call Graph, and Type Analyzer for JavaScript. We performed dynamic analyzes relying on the nodejs-cg tool (a customized Node.js runtime) and the NodeProf instrumentation and profiling framework. We provide a quantitative evaluation of the results, and a result quality analysis based on 941 manually validated call edges. On the SunSpider programs, which do not take any inputs, so dynamic extraction could be complete, all the static tools also performed well. For example, TAJS found 93% of all edges while having a 97% precision compared to the precise dynamic call graph. When it comes to real-world Node.js modules, our evaluation shows that static tools struggle with parsing the code and fail to detect a significant amount of call edges that dynamic approaches can capture. Nonetheless, a significant number of edges not detected by dynamic approaches are also reported. Among these, however, there are also edges that are real, but for some reason the unit tests did not execute the branches in which these calls were included.https://ieeexplore.ieee.org/document/10066273/Call graphcomparative studydynamic code analysisJavaScriptstatic code analysis |
| spellingShingle | Gabor Antal Peter Hegedus Zoltan Herczeg Gabor Loki Rudolf Ferenc Is JavaScript Call Graph Extraction Solved Yet? A Comparative Study of Static and Dynamic Tools IEEE Access Call graph comparative study dynamic code analysis JavaScript static code analysis |
| title | Is JavaScript Call Graph Extraction Solved Yet? A Comparative Study of Static and Dynamic Tools |
| title_full | Is JavaScript Call Graph Extraction Solved Yet? A Comparative Study of Static and Dynamic Tools |
| title_fullStr | Is JavaScript Call Graph Extraction Solved Yet? A Comparative Study of Static and Dynamic Tools |
| title_full_unstemmed | Is JavaScript Call Graph Extraction Solved Yet? A Comparative Study of Static and Dynamic Tools |
| title_short | Is JavaScript Call Graph Extraction Solved Yet? A Comparative Study of Static and Dynamic Tools |
| title_sort | is javascript call graph extraction solved yet a comparative study of static and dynamic tools |
| topic | Call graph comparative study dynamic code analysis JavaScript static code analysis |
| url | https://ieeexplore.ieee.org/document/10066273/ |
| work_keys_str_mv | AT gaborantal isjavascriptcallgraphextractionsolvedyetacomparativestudyofstaticanddynamictools AT peterhegedus isjavascriptcallgraphextractionsolvedyetacomparativestudyofstaticanddynamictools AT zoltanherczeg isjavascriptcallgraphextractionsolvedyetacomparativestudyofstaticanddynamictools AT gaborloki isjavascriptcallgraphextractionsolvedyetacomparativestudyofstaticanddynamictools AT rudolfferenc isjavascriptcallgraphextractionsolvedyetacomparativestudyofstaticanddynamictools |