Causal Analysis Experiments on Log Extraction and Processing for Causal Insights

Recent decades have seen tremendous advancements in the design and implementation of data processing systems for various applications and use cases. However, even systems that support the most complex queries are mostly used for business reporting, prediction, and classification tasks based on the d...

Full description

Bibliographic Details
Main Author: Khine, Min Thet
Other Authors: Cafarella, Michael
Format: Thesis
Published: Massachusetts Institute of Technology 2023
Online Access:https://hdl.handle.net/1721.1/151420
Description
Summary:Recent decades have seen tremendous advancements in the design and implementation of data processing systems for various applications and use cases. However, even systems that support the most complex queries are mostly used for business reporting, prediction, and classification tasks based on the data. These systems do not necessarily inform users of the causal relationships that are inherent in the data. To this end, we design a new log-based data processing system that provides answers to causal questions based on timestamped logs. This thesis work focuses on improving the current log extraction methods and performing causal analysis experiments on inferred causal models extracted from the logs.