Validity Tracking Based Log Management for In-Memory Databases
With in-memory databases (IMDBs), where all data sets reside in main memory for fast processing speed, logging and checkpointing are essential for achieving persistence in data. Logging of IMDBs has evolved to reduce run-time overhead to suit the systems, but this causes an increase in recovery time...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9509544/ |
| _version_ | 1811274444306382848 |
|---|---|
| author | Kwangjin Lee Hwajung Kim Heon Y. Yeom |
| author_facet | Kwangjin Lee Hwajung Kim Heon Y. Yeom |
| author_sort | Kwangjin Lee |
| collection | DOAJ |
| description | With in-memory databases (IMDBs), where all data sets reside in main memory for fast processing speed, logging and checkpointing are essential for achieving persistence in data. Logging of IMDBs has evolved to reduce run-time overhead to suit the systems, but this causes an increase in recovery time. Checkpointing technique compensates for these problems with logging, but existing schemes often incur high costs due to reduced system throughput, increased latency, and increased memory usage. In this paper, we propose a checkpointing scheme using validity tracking-based compaction (VTC), the technique that tracks the validity of logs in a file and removes unnecessary logs. The proposed scheme shows extremely low memory usage compared to existing checkpointing schemes, which use consistent snapshots. Our experiments demonstrate that checkpoints using consistent snapshot increase memory footprint by up to two times in update-intensive workloads. In contrast, our proposed VTC only requires 2% additional memory for checkpointing. That means the system can use most of its memory to store data and process transactions. |
| first_indexed | 2024-04-12T23:19:14Z |
| format | Article |
| id | doaj.art-1f048eb01ca249a898486ee9cd175339 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-12T23:19:14Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-1f048eb01ca249a898486ee9cd1753392022-12-22T03:12:34ZengIEEEIEEE Access2169-35362021-01-01911149311150410.1109/ACCESS.2021.31038629509544Validity Tracking Based Log Management for In-Memory DatabasesKwangjin Lee0https://orcid.org/0000-0001-7666-280XHwajung Kim1https://orcid.org/0000-0001-7134-823XHeon Y. Yeom2https://orcid.org/0000-0001-6865-1756School of Computer Science and Engineering, Seoul National University, Seoul, Republic of KoreaSchool of Computer Science and Engineering, Seoul National University, Seoul, Republic of KoreaSchool of Computer Science and Engineering, Seoul National University, Seoul, Republic of KoreaWith in-memory databases (IMDBs), where all data sets reside in main memory for fast processing speed, logging and checkpointing are essential for achieving persistence in data. Logging of IMDBs has evolved to reduce run-time overhead to suit the systems, but this causes an increase in recovery time. Checkpointing technique compensates for these problems with logging, but existing schemes often incur high costs due to reduced system throughput, increased latency, and increased memory usage. In this paper, we propose a checkpointing scheme using validity tracking-based compaction (VTC), the technique that tracks the validity of logs in a file and removes unnecessary logs. The proposed scheme shows extremely low memory usage compared to existing checkpointing schemes, which use consistent snapshots. Our experiments demonstrate that checkpoints using consistent snapshot increase memory footprint by up to two times in update-intensive workloads. In contrast, our proposed VTC only requires 2% additional memory for checkpointing. That means the system can use most of its memory to store data and process transactions.https://ieeexplore.ieee.org/document/9509544/Checkpointingin-memory databaseloggingpersistencesnapshot |
| spellingShingle | Kwangjin Lee Hwajung Kim Heon Y. Yeom Validity Tracking Based Log Management for In-Memory Databases IEEE Access Checkpointing in-memory database logging persistence snapshot |
| title | Validity Tracking Based Log Management for In-Memory Databases |
| title_full | Validity Tracking Based Log Management for In-Memory Databases |
| title_fullStr | Validity Tracking Based Log Management for In-Memory Databases |
| title_full_unstemmed | Validity Tracking Based Log Management for In-Memory Databases |
| title_short | Validity Tracking Based Log Management for In-Memory Databases |
| title_sort | validity tracking based log management for in memory databases |
| topic | Checkpointing in-memory database logging persistence snapshot |
| url | https://ieeexplore.ieee.org/document/9509544/ |
| work_keys_str_mv | AT kwangjinlee validitytrackingbasedlogmanagementforinmemorydatabases AT hwajungkim validitytrackingbasedlogmanagementforinmemorydatabases AT heonyyeom validitytrackingbasedlogmanagementforinmemorydatabases |