Resource usage monitoring for web systems using real-time statistical analysis of log data
For Web-based systems accessible from the Internet, it is difficult to estimate workloads precisely. Precise estimation of resources necessary for the system is important for effective utilization of resources in a datacenter. Therefore, capacity planning to forecast the amount of IT resources neces...
| Main Authors: | , , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
2011
|
| _version_ | 1797102201508724736 |
|---|---|
| author | Yoshino, M Handa, A Komoda, N Oba, M |
| author_facet | Yoshino, M Handa, A Komoda, N Oba, M |
| author_sort | Yoshino, M |
| collection | OXFORD |
| description | For Web-based systems accessible from the Internet, it is difficult to estimate workloads precisely. Precise estimation of resources necessary for the system is important for effective utilization of resources in a datacenter. Therefore, capacity planning to forecast the amount of IT resources necessary for a system is important. In capacity planning, the amount of resources necessary for a system is calculated based upon numbers determined by the architecture and business requirements of the system. An example of a number determined by the architecture is the amount of memory required by a single user. An example of a number determined by business requirements is the estimated maximum number of simultaneous users. By multiplying these two numbers, a maximum memory requirement can be calculated. Usually, system memory consumption and the number of simultaneous users are monitored during operation, and if either value exceeds a threshold, an alarm is sent to operators. The authors propose a method to monitor memory consumption per user from memory consumption data and the number of users, and perform statistical significance testing in real time by applying a stream database. The window size used in a CQL statement for the test affects the precision of the test and memory consumption of the stream database. Through experimentation, the authors propose an optimal window size. |
| first_indexed | 2024-03-07T06:02:36Z |
| format | Journal article |
| id | oxford-uuid:ecb92f22-71a1-4c4a-a14a-c930216fec20 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T06:02:36Z |
| publishDate | 2011 |
| record_format | dspace |
| spelling | oxford-uuid:ecb92f22-71a1-4c4a-a14a-c930216fec202022-03-27T11:19:35ZResource usage monitoring for web systems using real-time statistical analysis of log dataJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:ecb92f22-71a1-4c4a-a14a-c930216fec20EnglishSymplectic Elements at Oxford2011Yoshino, MHanda, AKomoda, NOba, MFor Web-based systems accessible from the Internet, it is difficult to estimate workloads precisely. Precise estimation of resources necessary for the system is important for effective utilization of resources in a datacenter. Therefore, capacity planning to forecast the amount of IT resources necessary for a system is important. In capacity planning, the amount of resources necessary for a system is calculated based upon numbers determined by the architecture and business requirements of the system. An example of a number determined by the architecture is the amount of memory required by a single user. An example of a number determined by business requirements is the estimated maximum number of simultaneous users. By multiplying these two numbers, a maximum memory requirement can be calculated. Usually, system memory consumption and the number of simultaneous users are monitored during operation, and if either value exceeds a threshold, an alarm is sent to operators. The authors propose a method to monitor memory consumption per user from memory consumption data and the number of users, and perform statistical significance testing in real time by applying a stream database. The window size used in a CQL statement for the test affects the precision of the test and memory consumption of the stream database. Through experimentation, the authors propose an optimal window size. |
| spellingShingle | Yoshino, M Handa, A Komoda, N Oba, M Resource usage monitoring for web systems using real-time statistical analysis of log data |
| title | Resource usage monitoring for web systems using real-time statistical analysis of log data |
| title_full | Resource usage monitoring for web systems using real-time statistical analysis of log data |
| title_fullStr | Resource usage monitoring for web systems using real-time statistical analysis of log data |
| title_full_unstemmed | Resource usage monitoring for web systems using real-time statistical analysis of log data |
| title_short | Resource usage monitoring for web systems using real-time statistical analysis of log data |
| title_sort | resource usage monitoring for web systems using real time statistical analysis of log data |
| work_keys_str_mv | AT yoshinom resourceusagemonitoringforwebsystemsusingrealtimestatisticalanalysisoflogdata AT handaa resourceusagemonitoringforwebsystemsusingrealtimestatisticalanalysisoflogdata AT komodan resourceusagemonitoringforwebsystemsusingrealtimestatisticalanalysisoflogdata AT obam resourceusagemonitoringforwebsystemsusingrealtimestatisticalanalysisoflogdata |