Incorporating inductive biases into machine learning algorithms
<p>Recently, significant advances in artificial intelligence (AI) have surpassed what was imaginable even five years ago. Today, we can instruct diffusion-based models to generate high-quality videos from human descriptions or prompt large language models (LLMs) to assist with writing, transla...
| Huvudupphovsman: | |
|---|---|
| Övriga upphovsmän: | |
| Materialtyp: | Lärdomsprov |
| Språk: | English |
| Publicerad: |
2024
|
| Ämnen: |
| _version_ | 1826314468182720512 |
|---|---|
| author | Miao, N |
| author2 | Rainforth, T |
| author_facet | Rainforth, T Miao, N |
| author_sort | Miao, N |
| collection | OXFORD |
| description | <p>Recently, significant advances in artificial intelligence (AI) have surpassed what was imaginable even five years ago. Today, we can instruct diffusion-based models to generate high-quality videos from human descriptions or prompt large language models (LLMs) to assist with writing, translation, and even mathematical reasoning. These remarkable abilities arise from training massive deep-learning models on huge amounts of data. However, we do not always have enough data. In some tasks, such as mathematical reasoning or molecule generation, available data are very limited. Furthermore, despite current LLMs utilizing nearly all available data on the Internet, they remain imperfect. Thus, it is a critical question how to enhance the performance of AI systems when it is difficult to increase the amount of training data.</p>
<p>In this thesis, we address this challenge from the perspective of inductive biases. Specifically, we investigate how to effectively use human knowledge about data or tasks to optimize the behavior of a machine learning algorithm, without requiring extra data. We will first give a brief review of research on inductive biases, and then we will show how to incorporate inductive biases during structure designing, training, and inference of a machine learning model, respectively. We also performed extensive experiments demonstrating that incorporating appropriate inductive biases can greatly boost model performance on a variety of tasks without the need for additional data.</p> |
| first_indexed | 2024-09-25T04:32:56Z |
| format | Thesis |
| id | oxford-uuid:193658cd-12f3-4ab8-901b-c4cf8ccdd7a5 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-09-25T04:32:56Z |
| publishDate | 2024 |
| record_format | dspace |
| spelling | oxford-uuid:193658cd-12f3-4ab8-901b-c4cf8ccdd7a52024-09-10T09:25:02ZIncorporating inductive biases into machine learning algorithmsThesishttp://purl.org/coar/resource_type/c_db06uuid:193658cd-12f3-4ab8-901b-c4cf8ccdd7a5machine learningEnglishHyrax Deposit2024Miao, NRainforth, TTeh, YW<p>Recently, significant advances in artificial intelligence (AI) have surpassed what was imaginable even five years ago. Today, we can instruct diffusion-based models to generate high-quality videos from human descriptions or prompt large language models (LLMs) to assist with writing, translation, and even mathematical reasoning. These remarkable abilities arise from training massive deep-learning models on huge amounts of data. However, we do not always have enough data. In some tasks, such as mathematical reasoning or molecule generation, available data are very limited. Furthermore, despite current LLMs utilizing nearly all available data on the Internet, they remain imperfect. Thus, it is a critical question how to enhance the performance of AI systems when it is difficult to increase the amount of training data.</p> <p>In this thesis, we address this challenge from the perspective of inductive biases. Specifically, we investigate how to effectively use human knowledge about data or tasks to optimize the behavior of a machine learning algorithm, without requiring extra data. We will first give a brief review of research on inductive biases, and then we will show how to incorporate inductive biases during structure designing, training, and inference of a machine learning model, respectively. We also performed extensive experiments demonstrating that incorporating appropriate inductive biases can greatly boost model performance on a variety of tasks without the need for additional data.</p> |
| spellingShingle | machine learning Miao, N Incorporating inductive biases into machine learning algorithms |
| title | Incorporating inductive biases into machine learning algorithms |
| title_full | Incorporating inductive biases into machine learning algorithms |
| title_fullStr | Incorporating inductive biases into machine learning algorithms |
| title_full_unstemmed | Incorporating inductive biases into machine learning algorithms |
| title_short | Incorporating inductive biases into machine learning algorithms |
| title_sort | incorporating inductive biases into machine learning algorithms |
| topic | machine learning |
| work_keys_str_mv | AT miaon incorporatinginductivebiasesintomachinelearningalgorithms |