Well Construction Action Planning and Automation through Finite-Horizon Sequential Decision-Making

Well construction operations require continuous complex decision-making and multi-step action planning. Action selection at every step demands a careful evaluation of the vast action space, while guided by long-term objectives and desired outcomes. Current human-centric decision-making introduces a...

Full description

Bibliographic Details
Main Authors: Gurtej Singh Saini, Oney Erge, Pradeepkumar Ashok, Eric van Oort
Format: Article
Language:English
Published: MDPI AG 2022-08-01
Series:Energies
Subjects:
Online Access:https://www.mdpi.com/1996-1073/15/16/5776
_version_ 1797432235032313856
author Gurtej Singh Saini
Oney Erge
Pradeepkumar Ashok
Eric van Oort
author_facet Gurtej Singh Saini
Oney Erge
Pradeepkumar Ashok
Eric van Oort
author_sort Gurtej Singh Saini
collection DOAJ
description Well construction operations require continuous complex decision-making and multi-step action planning. Action selection at every step demands a careful evaluation of the vast action space, while guided by long-term objectives and desired outcomes. Current human-centric decision-making introduces a degree of bias, which can result in reactive rather than proactive decisions. This can lead from minor operational inefficiencies all the way to catastrophic health and safety issues. This paper details the steps in structuring unbiased purpose-built sequential decision-making systems. Setting up such systems entails representing the operation as a Markov decision process (MDP). This requires explicitly defining states and action values, defining goal states, building a digital twin to model the process, and appropriately shaping reward functions to measure feedback. The digital twin, in conjunction with the reward function, is utilized for simulating and quantifying the different action sequences. A finite-horizon sequential decision-making system, with discrete state and action space, was set up to advise on hole cleaning during well construction. The state was quantified by the cuttings bed height and the equivalent circulation density values, and the action set was defined using a combination of controllable drilling parameters (including mud density and rheology, drillstring rotation speed, etc.). A non-sparse normalized reward structure was formulated as a function of the state and action values. Hydraulics, cuttings transport, and rig state detection models were integrated to build the hole cleaning digital twin. This system was then used for performance tracking and scenario simulations (with each scenario defined as a finite-horizon action sequence) on real-world oil wells. The different scenarios were compared by monitoring state–action transitions and the evolution of the reward with actions. This paper presents a novel method for setting up well construction operations as long-term finite-horizon sequential decision-making systems, and defines a way to quantify and compare different scenarios. The proper construction of such systems is a crucial step towards automating intelligent decision-making.
first_indexed 2024-03-09T09:57:26Z
format Article
id doaj.art-7f98418c26754a6bbe63ec8eeda0d67b
institution Directory Open Access Journal
issn 1996-1073
language English
last_indexed 2024-03-09T09:57:26Z
publishDate 2022-08-01
publisher MDPI AG
record_format Article
series Energies
spelling doaj.art-7f98418c26754a6bbe63ec8eeda0d67b2023-12-01T23:39:05ZengMDPI AGEnergies1996-10732022-08-011516577610.3390/en15165776Well Construction Action Planning and Automation through Finite-Horizon Sequential Decision-MakingGurtej Singh Saini0Oney Erge1Pradeepkumar Ashok2Eric van Oort3Cockrell School of Engineering, The University of Texas at Austin, Austin, TX 78712, USACockrell School of Engineering, The University of Texas at Austin, Austin, TX 78712, USACockrell School of Engineering, The University of Texas at Austin, Austin, TX 78712, USACockrell School of Engineering, The University of Texas at Austin, Austin, TX 78712, USAWell construction operations require continuous complex decision-making and multi-step action planning. Action selection at every step demands a careful evaluation of the vast action space, while guided by long-term objectives and desired outcomes. Current human-centric decision-making introduces a degree of bias, which can result in reactive rather than proactive decisions. This can lead from minor operational inefficiencies all the way to catastrophic health and safety issues. This paper details the steps in structuring unbiased purpose-built sequential decision-making systems. Setting up such systems entails representing the operation as a Markov decision process (MDP). This requires explicitly defining states and action values, defining goal states, building a digital twin to model the process, and appropriately shaping reward functions to measure feedback. The digital twin, in conjunction with the reward function, is utilized for simulating and quantifying the different action sequences. A finite-horizon sequential decision-making system, with discrete state and action space, was set up to advise on hole cleaning during well construction. The state was quantified by the cuttings bed height and the equivalent circulation density values, and the action set was defined using a combination of controllable drilling parameters (including mud density and rheology, drillstring rotation speed, etc.). A non-sparse normalized reward structure was formulated as a function of the state and action values. Hydraulics, cuttings transport, and rig state detection models were integrated to build the hole cleaning digital twin. This system was then used for performance tracking and scenario simulations (with each scenario defined as a finite-horizon action sequence) on real-world oil wells. The different scenarios were compared by monitoring state–action transitions and the evolution of the reward with actions. This paper presents a novel method for setting up well construction operations as long-term finite-horizon sequential decision-making systems, and defines a way to quantify and compare different scenarios. The proper construction of such systems is a crucial step towards automating intelligent decision-making.https://www.mdpi.com/1996-1073/15/16/5776sequential decision-makingMarkov decision processreward shapingwell constructionhole cleaningdigital twinning
spellingShingle Gurtej Singh Saini
Oney Erge
Pradeepkumar Ashok
Eric van Oort
Well Construction Action Planning and Automation through Finite-Horizon Sequential Decision-Making
Energies
sequential decision-making
Markov decision process
reward shaping
well construction
hole cleaning
digital twinning
title Well Construction Action Planning and Automation through Finite-Horizon Sequential Decision-Making
title_full Well Construction Action Planning and Automation through Finite-Horizon Sequential Decision-Making
title_fullStr Well Construction Action Planning and Automation through Finite-Horizon Sequential Decision-Making
title_full_unstemmed Well Construction Action Planning and Automation through Finite-Horizon Sequential Decision-Making
title_short Well Construction Action Planning and Automation through Finite-Horizon Sequential Decision-Making
title_sort well construction action planning and automation through finite horizon sequential decision making
topic sequential decision-making
Markov decision process
reward shaping
well construction
hole cleaning
digital twinning
url https://www.mdpi.com/1996-1073/15/16/5776
work_keys_str_mv AT gurtejsinghsaini wellconstructionactionplanningandautomationthroughfinitehorizonsequentialdecisionmaking
AT oneyerge wellconstructionactionplanningandautomationthroughfinitehorizonsequentialdecisionmaking
AT pradeepkumarashok wellconstructionactionplanningandautomationthroughfinitehorizonsequentialdecisionmaking
AT ericvanoort wellconstructionactionplanningandautomationthroughfinitehorizonsequentialdecisionmaking