Contested logistics simulation output analysis with approximate dynamic programming: a proposed methodology
Purpose – Rapid sensitivity analysis and near-optimal decision-making in contested environments are valuable requirements when providing military logistics support. Port of debarkation denial motivates maneuver from strategic operational locations, further complicating logistics support. Simulations...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Emerald Publishing
2022-12-01
|
| Series: | Journal of Defense Analytics and Logistics |
| Subjects: | |
| Online Access: | https://www.emerald.com/insight/content/doi/10.1108/JDAL-07-2022-0004/full/pdf |
| _version_ | 1797788783056257024 |
|---|---|
| author | Matthew Powers Brian O'Flynn |
| author_facet | Matthew Powers Brian O'Flynn |
| author_sort | Matthew Powers |
| collection | DOAJ |
| description | Purpose – Rapid sensitivity analysis and near-optimal decision-making in contested environments are valuable requirements when providing military logistics support. Port of debarkation denial motivates maneuver from strategic operational locations, further complicating logistics support. Simulations enable rapid concept design, experiment and testing that meet these complicated logistic support demands. However, simulation model analyses are time consuming as output data complexity grows with simulation input. This paper proposes a methodology that leverages the benefits of simulation-based insight and the computational speed of approximate dynamic programming (ADP). Design/methodology/approach – This paper describes a simulated contested logistics environment and demonstrates how output data informs the parameters required for the ADP dialect of reinforcement learning (aka Q-learning). Q-learning output includes a near-optimal policy that prescribes decisions for each state modeled in the simulation. This paper's methods conform to DoD simulation modeling practices complemented with AI-enabled decision-making. Findings – This study demonstrates simulation output data as a means of state–space reduction to mitigate the curse of dimensionality. Furthermore, massive amounts of simulation output data become unwieldy. This work demonstrates how Q-learning parameters reflect simulation inputs so that simulation model behavior can compare to near-optimal policies. Originality/value – Fast computation is attractive for sensitivity analysis while divorcing evaluation from scenario-based limitations. The United States military is eager to embrace emerging AI analytic techniques to inform decision-making but is hesitant to abandon simulation modeling. This paper proposes Q-learning as an aid to overcome cognitive limitations in a way that satisfies the desire to wield AI-enabled decision-making combined with modeling and simulation. |
| first_indexed | 2024-03-13T01:41:27Z |
| format | Article |
| id | doaj.art-1d304efc697142acbee0b3a95a0ebde7 |
| institution | Directory Open Access Journal |
| issn | 2399-6439 |
| language | English |
| last_indexed | 2024-03-13T01:41:27Z |
| publishDate | 2022-12-01 |
| publisher | Emerald Publishing |
| record_format | Article |
| series | Journal of Defense Analytics and Logistics |
| spelling | doaj.art-1d304efc697142acbee0b3a95a0ebde72023-07-03T12:52:22ZengEmerald PublishingJournal of Defense Analytics and Logistics2399-64392022-12-016212013310.1108/JDAL-07-2022-0004Contested logistics simulation output analysis with approximate dynamic programming: a proposed methodologyMatthew Powers0Brian O'Flynn1Operations Research, The MITRE Corporation, Washington, District of Columbia, USADivision of Simulation, Experimentation and Gaming, The MITRE Corporation, Washington, District of Columbia, USAPurpose – Rapid sensitivity analysis and near-optimal decision-making in contested environments are valuable requirements when providing military logistics support. Port of debarkation denial motivates maneuver from strategic operational locations, further complicating logistics support. Simulations enable rapid concept design, experiment and testing that meet these complicated logistic support demands. However, simulation model analyses are time consuming as output data complexity grows with simulation input. This paper proposes a methodology that leverages the benefits of simulation-based insight and the computational speed of approximate dynamic programming (ADP). Design/methodology/approach – This paper describes a simulated contested logistics environment and demonstrates how output data informs the parameters required for the ADP dialect of reinforcement learning (aka Q-learning). Q-learning output includes a near-optimal policy that prescribes decisions for each state modeled in the simulation. This paper's methods conform to DoD simulation modeling practices complemented with AI-enabled decision-making. Findings – This study demonstrates simulation output data as a means of state–space reduction to mitigate the curse of dimensionality. Furthermore, massive amounts of simulation output data become unwieldy. This work demonstrates how Q-learning parameters reflect simulation inputs so that simulation model behavior can compare to near-optimal policies. Originality/value – Fast computation is attractive for sensitivity analysis while divorcing evaluation from scenario-based limitations. The United States military is eager to embrace emerging AI analytic techniques to inform decision-making but is hesitant to abandon simulation modeling. This paper proposes Q-learning as an aid to overcome cognitive limitations in a way that satisfies the desire to wield AI-enabled decision-making combined with modeling and simulation.https://www.emerald.com/insight/content/doi/10.1108/JDAL-07-2022-0004/full/pdfSimulation modelingApproximate dynamic programmingContested logistics |
| spellingShingle | Matthew Powers Brian O'Flynn Contested logistics simulation output analysis with approximate dynamic programming: a proposed methodology Journal of Defense Analytics and Logistics Simulation modeling Approximate dynamic programming Contested logistics |
| title | Contested logistics simulation output analysis with approximate dynamic programming: a proposed methodology |
| title_full | Contested logistics simulation output analysis with approximate dynamic programming: a proposed methodology |
| title_fullStr | Contested logistics simulation output analysis with approximate dynamic programming: a proposed methodology |
| title_full_unstemmed | Contested logistics simulation output analysis with approximate dynamic programming: a proposed methodology |
| title_short | Contested logistics simulation output analysis with approximate dynamic programming: a proposed methodology |
| title_sort | contested logistics simulation output analysis with approximate dynamic programming a proposed methodology |
| topic | Simulation modeling Approximate dynamic programming Contested logistics |
| url | https://www.emerald.com/insight/content/doi/10.1108/JDAL-07-2022-0004/full/pdf |
| work_keys_str_mv | AT matthewpowers contestedlogisticssimulationoutputanalysiswithapproximatedynamicprogrammingaproposedmethodology AT brianoflynn contestedlogisticssimulationoutputanalysiswithapproximatedynamicprogrammingaproposedmethodology |