A Formal Framework for Knowledge Acquisition: Going beyond Machine Learning
Philosophers frequently define knowledge as justified, true belief. We built a mathematical framework that makes it possible to define learning (increasing number of true beliefs) and knowledge of an agent in precise ways, by phrasing belief in terms of epistemic probabilities, defined from Bayes’ r...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-10-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/24/10/1469 |
| _version_ | 1797473415003635712 |
|---|---|
| author | Ola Hössjer Daniel Andrés Díaz-Pachón J. Sunil Rao |
| author_facet | Ola Hössjer Daniel Andrés Díaz-Pachón J. Sunil Rao |
| author_sort | Ola Hössjer |
| collection | DOAJ |
| description | Philosophers frequently define knowledge as justified, true belief. We built a mathematical framework that makes it possible to define learning (increasing number of true beliefs) and knowledge of an agent in precise ways, by phrasing belief in terms of epistemic probabilities, defined from Bayes’ rule. The degree of true belief is quantified by means of active information <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msup><mi>I</mi><mo>+</mo></msup></semantics></math></inline-formula>: a comparison between the degree of belief of the agent and a completely ignorant person. Learning has occurred when either the agent’s strength of belief in a true proposition has increased in comparison with the ignorant person (<inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><msup><mi>I</mi><mo>+</mo></msup><mo>></mo><mn>0</mn></mrow></semantics></math></inline-formula>), or the strength of belief in a false proposition has decreased (<inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><msup><mi>I</mi><mo>+</mo></msup><mo><</mo><mn>0</mn></mrow></semantics></math></inline-formula>). Knowledge additionally requires that learning occurs for the right reason, and in this context we introduce a framework of parallel worlds that correspond to parameters of a statistical model. This makes it possible to interpret learning as a hypothesis test for such a model, whereas knowledge acquisition additionally requires estimation of a true world parameter. Our framework of learning and knowledge acquisition is a hybrid between frequentism and Bayesianism. It can be generalized to a sequential setting, where information and data are updated over time. The theory is illustrated using examples of coin tossing, historical and future events, replication of studies, and causal inference. It can also be used to pinpoint shortcomings of machine learning, where typically learning rather than knowledge acquisition is in focus. |
| first_indexed | 2024-03-09T20:14:11Z |
| format | Article |
| id | doaj.art-07dc24829558486596adbd6e7d0493f5 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-09T20:14:11Z |
| publishDate | 2022-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-07dc24829558486596adbd6e7d0493f52023-11-24T00:04:21ZengMDPI AGEntropy1099-43002022-10-012410146910.3390/e24101469A Formal Framework for Knowledge Acquisition: Going beyond Machine LearningOla Hössjer0Daniel Andrés Díaz-Pachón1J. Sunil Rao2Department of Mathematics, Stockholm University, SE-106 91 Stockholm, SwedenDivision of Biostatistics, University of Miami, Miami, FL 33136, USADivision of Biostatistics, University of Miami, Miami, FL 33136, USAPhilosophers frequently define knowledge as justified, true belief. We built a mathematical framework that makes it possible to define learning (increasing number of true beliefs) and knowledge of an agent in precise ways, by phrasing belief in terms of epistemic probabilities, defined from Bayes’ rule. The degree of true belief is quantified by means of active information <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msup><mi>I</mi><mo>+</mo></msup></semantics></math></inline-formula>: a comparison between the degree of belief of the agent and a completely ignorant person. Learning has occurred when either the agent’s strength of belief in a true proposition has increased in comparison with the ignorant person (<inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><msup><mi>I</mi><mo>+</mo></msup><mo>></mo><mn>0</mn></mrow></semantics></math></inline-formula>), or the strength of belief in a false proposition has decreased (<inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><msup><mi>I</mi><mo>+</mo></msup><mo><</mo><mn>0</mn></mrow></semantics></math></inline-formula>). Knowledge additionally requires that learning occurs for the right reason, and in this context we introduce a framework of parallel worlds that correspond to parameters of a statistical model. This makes it possible to interpret learning as a hypothesis test for such a model, whereas knowledge acquisition additionally requires estimation of a true world parameter. Our framework of learning and knowledge acquisition is a hybrid between frequentism and Bayesianism. It can be generalized to a sequential setting, where information and data are updated over time. The theory is illustrated using examples of coin tossing, historical and future events, replication of studies, and causal inference. It can also be used to pinpoint shortcomings of machine learning, where typically learning rather than knowledge acquisition is in focus.https://www.mdpi.com/1099-4300/24/10/1469active informationBayes’ rulecounterfactualsepistemic probabilitylearning, justified true beliefknowledge acquisition |
| spellingShingle | Ola Hössjer Daniel Andrés Díaz-Pachón J. Sunil Rao A Formal Framework for Knowledge Acquisition: Going beyond Machine Learning Entropy active information Bayes’ rule counterfactuals epistemic probability learning, justified true belief knowledge acquisition |
| title | A Formal Framework for Knowledge Acquisition: Going beyond Machine Learning |
| title_full | A Formal Framework for Knowledge Acquisition: Going beyond Machine Learning |
| title_fullStr | A Formal Framework for Knowledge Acquisition: Going beyond Machine Learning |
| title_full_unstemmed | A Formal Framework for Knowledge Acquisition: Going beyond Machine Learning |
| title_short | A Formal Framework for Knowledge Acquisition: Going beyond Machine Learning |
| title_sort | formal framework for knowledge acquisition going beyond machine learning |
| topic | active information Bayes’ rule counterfactuals epistemic probability learning, justified true belief knowledge acquisition |
| url | https://www.mdpi.com/1099-4300/24/10/1469 |
| work_keys_str_mv | AT olahossjer aformalframeworkforknowledgeacquisitiongoingbeyondmachinelearning AT danielandresdiazpachon aformalframeworkforknowledgeacquisitiongoingbeyondmachinelearning AT jsunilrao aformalframeworkforknowledgeacquisitiongoingbeyondmachinelearning AT olahossjer formalframeworkforknowledgeacquisitiongoingbeyondmachinelearning AT danielandresdiazpachon formalframeworkforknowledgeacquisitiongoingbeyondmachinelearning AT jsunilrao formalframeworkforknowledgeacquisitiongoingbeyondmachinelearning |