Integrating inverse reinforcement learning into data-driven mechanistic computational models: a novel paradigm to decode cancer cell heterogeneity
Cellular heterogeneity is a ubiquitous aspect of biology and a major obstacle to successful cancer treatment. Several techniques have emerged to quantify heterogeneity in live cells along axes including cellular migration, morphology, growth, and signaling. Crucially, these studies reveal that cellu...
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2024-03-01
|
| Series: | Frontiers in Systems Biology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fsysb.2024.1333760/full |
| _version_ | 1827158199473537024 |
|---|---|
| author | Patrick C. Kinnunen Kenneth K. Y. Ho Siddhartha Srivastava Siddhartha Srivastava Chengyang Huang Wanggang Shen Krishna Garikipati Krishna Garikipati Krishna Garikipati Gary D. Luker Gary D. Luker Nikola Banovic Xun Huan Xun Huan Jennifer J. Linderman Jennifer J. Linderman Kathryn E. Luker Kathryn E. Luker |
| author_facet | Patrick C. Kinnunen Kenneth K. Y. Ho Siddhartha Srivastava Siddhartha Srivastava Chengyang Huang Wanggang Shen Krishna Garikipati Krishna Garikipati Krishna Garikipati Gary D. Luker Gary D. Luker Nikola Banovic Xun Huan Xun Huan Jennifer J. Linderman Jennifer J. Linderman Kathryn E. Luker Kathryn E. Luker |
| author_sort | Patrick C. Kinnunen |
| collection | DOAJ |
| description | Cellular heterogeneity is a ubiquitous aspect of biology and a major obstacle to successful cancer treatment. Several techniques have emerged to quantify heterogeneity in live cells along axes including cellular migration, morphology, growth, and signaling. Crucially, these studies reveal that cellular heterogeneity is not a result of randomness or a failure in cellular control systems, but instead is a predictable aspect of multicellular systems. We hypothesize that individual cells in complex tissues can behave as reward-maximizing agents and that differences in reward perception can explain heterogeneity. In this perspective, we introduce inverse reinforcement learning as a novel approach for analyzing cellular heterogeneity. We briefly detail experimental approaches for measuring cellular heterogeneity over time and how these experiments can generate datasets consisting of cellular states and actions. Next, we show how inverse reinforcement learning can be applied to these datasets to infer how individual cells choose different actions based on heterogeneous states. Finally, we introduce potential applications of inverse reinforcement learning to three cell biology problems. Overall, we expect inverse reinforcement learning to reveal why cells behave heterogeneously and enable identification of novel treatments based on this new understanding. |
| first_indexed | 2024-04-25T01:40:22Z |
| format | Article |
| id | doaj.art-be160e1e9e7845c4a5123e42183ab88b |
| institution | Directory Open Access Journal |
| issn | 2674-0702 |
| language | English |
| last_indexed | 2025-03-20T23:37:26Z |
| publishDate | 2024-03-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Systems Biology |
| spelling | doaj.art-be160e1e9e7845c4a5123e42183ab88b2024-08-03T13:48:50ZengFrontiers Media S.A.Frontiers in Systems Biology2674-07022024-03-01410.3389/fsysb.2024.13337601333760Integrating inverse reinforcement learning into data-driven mechanistic computational models: a novel paradigm to decode cancer cell heterogeneityPatrick C. Kinnunen0Kenneth K. Y. Ho1Siddhartha Srivastava2Siddhartha Srivastava3Chengyang Huang4Wanggang Shen5Krishna Garikipati6Krishna Garikipati7Krishna Garikipati8Gary D. Luker9Gary D. Luker10Nikola Banovic11Xun Huan12Xun Huan13Jennifer J. Linderman14Jennifer J. Linderman15Kathryn E. Luker16Kathryn E. Luker17Departments of Chemical Engineering, University of Michigan, Ann Arbor, MI, United StatesRadiology, University of Michigan, Ann Arbor, MI, United StatesMechanical Engineering, University of Michigan, Ann Arbor, MI, United StatesMichigan Institute for Computational Discovery and Engineering, University of Michigan, Ann Arbor, MI, United StatesMechanical Engineering, University of Michigan, Ann Arbor, MI, United StatesMechanical Engineering, University of Michigan, Ann Arbor, MI, United StatesMechanical Engineering, University of Michigan, Ann Arbor, MI, United StatesMichigan Institute for Computational Discovery and Engineering, University of Michigan, Ann Arbor, MI, United StatesMathematics, University of Michigan, Ann Arbor, MI, United StatesRadiology, University of Michigan, Ann Arbor, MI, United StatesBiomedical Engineering, University of Michigan, Ann Arbor, MI, United StatesElectrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, United StatesMechanical Engineering, University of Michigan, Ann Arbor, MI, United StatesMichigan Institute for Computational Discovery and Engineering, University of Michigan, Ann Arbor, MI, United StatesDepartments of Chemical Engineering, University of Michigan, Ann Arbor, MI, United StatesBiomedical Engineering, University of Michigan, Ann Arbor, MI, United StatesRadiology, University of Michigan, Ann Arbor, MI, United StatesBiointerfaces Institute, University of Michigan, Ann Arbor, MI, United StatesCellular heterogeneity is a ubiquitous aspect of biology and a major obstacle to successful cancer treatment. Several techniques have emerged to quantify heterogeneity in live cells along axes including cellular migration, morphology, growth, and signaling. Crucially, these studies reveal that cellular heterogeneity is not a result of randomness or a failure in cellular control systems, but instead is a predictable aspect of multicellular systems. We hypothesize that individual cells in complex tissues can behave as reward-maximizing agents and that differences in reward perception can explain heterogeneity. In this perspective, we introduce inverse reinforcement learning as a novel approach for analyzing cellular heterogeneity. We briefly detail experimental approaches for measuring cellular heterogeneity over time and how these experiments can generate datasets consisting of cellular states and actions. Next, we show how inverse reinforcement learning can be applied to these datasets to infer how individual cells choose different actions based on heterogeneous states. Finally, we introduce potential applications of inverse reinforcement learning to three cell biology problems. Overall, we expect inverse reinforcement learning to reveal why cells behave heterogeneously and enable identification of novel treatments based on this new understanding.https://www.frontiersin.org/articles/10.3389/fsysb.2024.1333760/fullinverse reinforcment learningmechanistic modelingmachine learningcellular heterogeneitylive-cell microscopy |
| spellingShingle | Patrick C. Kinnunen Kenneth K. Y. Ho Siddhartha Srivastava Siddhartha Srivastava Chengyang Huang Wanggang Shen Krishna Garikipati Krishna Garikipati Krishna Garikipati Gary D. Luker Gary D. Luker Nikola Banovic Xun Huan Xun Huan Jennifer J. Linderman Jennifer J. Linderman Kathryn E. Luker Kathryn E. Luker Integrating inverse reinforcement learning into data-driven mechanistic computational models: a novel paradigm to decode cancer cell heterogeneity Frontiers in Systems Biology inverse reinforcment learning mechanistic modeling machine learning cellular heterogeneity live-cell microscopy |
| title | Integrating inverse reinforcement learning into data-driven mechanistic computational models: a novel paradigm to decode cancer cell heterogeneity |
| title_full | Integrating inverse reinforcement learning into data-driven mechanistic computational models: a novel paradigm to decode cancer cell heterogeneity |
| title_fullStr | Integrating inverse reinforcement learning into data-driven mechanistic computational models: a novel paradigm to decode cancer cell heterogeneity |
| title_full_unstemmed | Integrating inverse reinforcement learning into data-driven mechanistic computational models: a novel paradigm to decode cancer cell heterogeneity |
| title_short | Integrating inverse reinforcement learning into data-driven mechanistic computational models: a novel paradigm to decode cancer cell heterogeneity |
| title_sort | integrating inverse reinforcement learning into data driven mechanistic computational models a novel paradigm to decode cancer cell heterogeneity |
| topic | inverse reinforcment learning mechanistic modeling machine learning cellular heterogeneity live-cell microscopy |
| url | https://www.frontiersin.org/articles/10.3389/fsysb.2024.1333760/full |
| work_keys_str_mv | AT patrickckinnunen integratinginversereinforcementlearningintodatadrivenmechanisticcomputationalmodelsanovelparadigmtodecodecancercellheterogeneity AT kennethkyho integratinginversereinforcementlearningintodatadrivenmechanisticcomputationalmodelsanovelparadigmtodecodecancercellheterogeneity AT siddharthasrivastava integratinginversereinforcementlearningintodatadrivenmechanisticcomputationalmodelsanovelparadigmtodecodecancercellheterogeneity AT siddharthasrivastava integratinginversereinforcementlearningintodatadrivenmechanisticcomputationalmodelsanovelparadigmtodecodecancercellheterogeneity AT chengyanghuang integratinginversereinforcementlearningintodatadrivenmechanisticcomputationalmodelsanovelparadigmtodecodecancercellheterogeneity AT wanggangshen integratinginversereinforcementlearningintodatadrivenmechanisticcomputationalmodelsanovelparadigmtodecodecancercellheterogeneity AT krishnagarikipati integratinginversereinforcementlearningintodatadrivenmechanisticcomputationalmodelsanovelparadigmtodecodecancercellheterogeneity AT krishnagarikipati integratinginversereinforcementlearningintodatadrivenmechanisticcomputationalmodelsanovelparadigmtodecodecancercellheterogeneity AT krishnagarikipati integratinginversereinforcementlearningintodatadrivenmechanisticcomputationalmodelsanovelparadigmtodecodecancercellheterogeneity AT garydluker integratinginversereinforcementlearningintodatadrivenmechanisticcomputationalmodelsanovelparadigmtodecodecancercellheterogeneity AT garydluker integratinginversereinforcementlearningintodatadrivenmechanisticcomputationalmodelsanovelparadigmtodecodecancercellheterogeneity AT nikolabanovic integratinginversereinforcementlearningintodatadrivenmechanisticcomputationalmodelsanovelparadigmtodecodecancercellheterogeneity AT xunhuan integratinginversereinforcementlearningintodatadrivenmechanisticcomputationalmodelsanovelparadigmtodecodecancercellheterogeneity AT xunhuan integratinginversereinforcementlearningintodatadrivenmechanisticcomputationalmodelsanovelparadigmtodecodecancercellheterogeneity AT jenniferjlinderman integratinginversereinforcementlearningintodatadrivenmechanisticcomputationalmodelsanovelparadigmtodecodecancercellheterogeneity AT jenniferjlinderman integratinginversereinforcementlearningintodatadrivenmechanisticcomputationalmodelsanovelparadigmtodecodecancercellheterogeneity AT kathryneluker integratinginversereinforcementlearningintodatadrivenmechanisticcomputationalmodelsanovelparadigmtodecodecancercellheterogeneity AT kathryneluker integratinginversereinforcementlearningintodatadrivenmechanisticcomputationalmodelsanovelparadigmtodecodecancercellheterogeneity |