Automated Steering in Drilling: Model-Based Control

IPTC-19817-MS
Automated Steering with Real-Time Model-Based Control
Nazli Demirer, Umut Zalluhoglu, Julien Marck, and Robert Darbe, Halliburton
Copyright 2020, International Petroleum Technology Conference
This paper was prepared for presentation at the International Petroleum Technology Conference held in Dhahran, Saudi Arabia, 13 – 15 January 2020.
This paper was selected for presentation by an IPTC Programme Committee following review of information contained in an abstract submitted by the author(s).
Contents of the paper, as presented, have not been reviewed by the International Petroleum Technology Conference and are subject to correction by the author(s). The
material, as presented, does not necessarily reflect any position of the International Petroleum Technology Conference, its officers, or members. Papers presented at
IPTC are subject to publication review by Sponsor Society Committees of IPTC. Electronic reproduction, distribution, or storage of any part of this paper for commercial
purposes without the written consent of the International Petroleum Technology Conference is prohibited. Permission to reproduce in print is restricted to an abstract of
not more than 300 words; illustrations may not be copied. The abstract must contain conspicuous acknowledgment of where and by whom the paper was presented.
Write Librarian, IPTC, P.O. Box 833836, Richardson, TX 75083-3836, U.S.A., fax +1-972-952-9435.
Abstract
Automation of directional drilling processes has gained interest as drilling operators seek more costeffective solutions and consistent performance. Automation and control–although a mature topic in many
industries–has recently started to take place in the drilling world as well. Hence, there is a great potential
for rapid and significant improvements.
Currently, directional drilling operations are primarily managed by directional drillers who make steering
decisions based on available pre-job reports, real-time data and geometric calculations. The resulting
performance is contingent on directional drillers’ skill set, knowledge of the drilling tool, and familiarity
with the local geology. Automating this process brings the promise of standardized drilling decisions with
consistent accurate well placement with improved borehole quality, and the flexibility to adapt to new
technologies in drilling tools and sensors. Automation also increases safety level in operations by reducing
the on-site crew size since directional drillers can remotely manage multiple wells.
This work proposes a model-based control approach for automated steering in drilling operations.
Two different models are presented. First, a comprehensive wellbore propagation model which considers
multiple factors, such as the geometric feedback embedded in the wellbore trajectory, the drilling tool
mechanical properties and actuation principle, the bit/rock interaction laws and kinematic relations between
the bit motion and the newly created borehole, then a reduced-order model tailored for control applications.
The control approach presented is applicable for both rotary steerable systems (RSS) and mud motors.
Testing results are presented and discussed at the end of the paper.
Introduction
Automation in the drilling industry has started to supersede traditional labor-intensive methods in virtue of
recent developments in technology. More reliable and consistent operations are achieved with automation
using state-of-the-art technology. The drilling processes are monitored in remote real-time operation centers
(Annaiyappa 2013, Bello et al. 2014), where all drilling information including surface and downhole data
is accessible in real-time. The introduction of RSS which is equipped with sensors and fast processors is
another source of motivation for automation, since it requires real-time decision making in an automated
manner.
2
IPTC-19817-MS
Gaines et al. (2013), Matheus et al. (2014), and Langaker and Edvardsen (2015) considered the downhole
cruise control problem to drill straight sections with RSS based on real-time continuous inclination and
azimuth measurements. Other examples of drilling automation include advanced drilling advisory systems
which help directional drillers make important decisions on the fly (Dow et al. 2012, Chang et al.
2014, Bailey et al. 2017). Using these automated systems improves awareness of downhole conditions
significantly, facilitating faster, more informed, and consistent decision making.
This paper differs from the previously mentioned references because it provides a model-based control
approach for automated steering for both complex 3D wells and long laterals. Two different wellbore
propagation models are presented. First, a 2D wellbore propagation model is considered for the directional
response of the drilling tool in the vertical plane for the inclination dynamics of the borehole. Second,
a generalized 3D reduced-order model is considered where the inclination and azimuth dynamics are
expressed in the spatial domain. The proposed control method targets real-time smooth accurate trajectory
tracking along the desired well plan and can be applied to both RSS and mud motors. In terms of steering
control, similar control applications in literature are limited. Bayliss et al. (2015) consider a model with
kinematic equations with inclination and azimuth angles as states and uses model predictive control to hold
a constant attitude while handling temporal delays using Padé approximations. Further, Panchal et al. (2010)
consider an inclination- and azimuth-hold controller where a pole-placement method is used to design the
controller.
Automated Steering
In directional drilling, the objective is to drill the well with a specified Bottom Hole Assembly (BHA) and
well plan based on real-time sensor information. Currently, this is an open loop system managed by the
directional driller (DD), a "human in the loop". Hence, the resulting performance is contingent on directional
drillers’ skill set, knowledge of the drilling tool, and familiarity with the local geology. Additionally,
there are many disturbances that affect the drilling process such as formation changes that can alter the
tool performance, lateral and torsional vibrations that affect the controllability of the tool, bit wear that
reduces the rate of penetration and side cutting efficiency, and it can cause an increase in WOB leading
to vibrations or direction change. An experienced directional driller reacts to disturbances, making the
appropriate changes to hit the targeted reservoirs. But each directional change alters the resulting borehole.
The model-based control method presented here aims to automate the steering process by providing
drilling recommendations, which accounts for the aforementioned uncertainties in a consistent manner.
As part of the decision making process, the system considers the BHA configuration, bit selection, realtime data, operational constraints, and drilling objectives. Fig. 1 illustrates the general framework for the
automation platform and shows the primary inputs and outputs to each component. Further discussion is
provided for individual components, inputs, and outputs to provide an understanding of the entire system.
IPTC-19817-MS
3
Figure 1—Overview of steering automation and control process.
Continuous inclination and azimuth measurements are provided in real time as well as effective tool face
and steering magnitude, weight-on-bit, pressure-while-drilling (PWD), and tool health monitoring data. The
wellbore propagation model describes the borehole propagation dynamics for the given tool and drilling
conditions. In order to account for inherent biases related to unknown quantities (e.g., less than expected
pump efficiency), the physics-based model is calibrated continuously. The calibration process considers the
drilling conditions, which have continuous impact on tool performance, and local effects (e.g., formationspecific tendencies) that might alter the tool yield during shorter periods of time.
The wellbore propagation model is first used to estimate the state of the borehole at the bit including its
local attitude (inclination and azimuth), curvature (build and walk rate), and position in a Cartesian threedimensional space. The model also outputs the recent steering-performance parameters including estimates
of tool yield to the controller. Next, the model-based controller generates optimal steering commands based
on operational constraints and steering objectives. The related future projection for the borehole trajectory is
also provided, using the bit projection as initial condition. Steering commands include tool face and steering
ratio (percentage of the steering capability used or slide/rotate for mud motors). The recommendations for
steering commands are computed with the controller, which uses the calibrated model to adapt dynamically
to local and global tool performances while minimizing borehole tortuosity.
For RSS, the optimal steering commands can be downlinked to the tool automatically where the frequency
of steering-command generation and communication can be adjusted based on telemetry system capabilities,
rate of penetration, and/or borehole depth. In addition to its real-time application, the steering automation
system can be used during pre-operation designs of service to assess borehole trajectory and operational
parameters. Moreover, a sensitivity analysis can be performed on multiple challenging scenarios to build
confidence that the operation can be completed successfully.
Wellbore Propagation Model
In this section, two different wellbore propagation models are presented. First, a 2D wellbore propagation
model is considered for the directional response of the drilling tool in the vertical plane. This model thus
explains the inclination dynamics of the borehole. It can be easily expanded to the azimuthal dynamics as
well. A similar model has been used for designing model-based directional drilling controllers for rotary
steerable systems (Kremers et al. 2016) and mud motors (Zhao et al. 2019).
4
IPTC-19817-MS
The model considers three main components: a bit/rock interaction law that describes the relation between
the force and moment applied on the bit and its penetration, kinematic relationships that relate the bit motion
to the local borehole geometry, and a model for the BHA that computes the force and the moment acting on
the bit from the deflection of the BHA, subject not only to its own external loading but also to constraints
arising from its interaction with the wellbore. For a BHA with n stabilizers, the model can be represented
with the following distributed-delay algebraic equation in spatial domain. Curvilinear coordinate ξ along
the borehole is used to describe the hole depth during the drilling process (Eq. 1):
(1)
where〈Θ〉i(ξ) is the average inclination of the wellbore segment corresponding to the ith segment of the
BHA; coefficients ai, i = 1,2, …, n and au are constant coefficients calculated based on BHA configuration,
bit selection, rock formation, and applied weight on bit. Parameter u(ξ)is the effective steering control input
of the drilling tool projected on the vertical plane. When the drilling tool operates in a neutral mode, e.g.,
rotate mode for a mud motor, u=0. aG sin(〈Θ〉1(ξ))captures the gravity effect on wellbore propagation.
The distributed-delay algebraic equation may be discretized along the wellbore trajectory and be used
in an optimization algorithm to directly generate steering commands to navigate the tool downhole. Due to
the delays in the system, a simple treatment of directly converting delayed system dynamics to difference
equation in the discrete time domain may introduce a large number of state variables, which could potentially
prohibit real-time applications.
Second, a generalized 3D reduced-order model is considered where the inclination and azimuth dynamics
are expressed in the spatial domain (Eq. 2):
(2)
where x(ξ) is the state of the system at hole depth ξ and it can be a function of attitude (inclination and
azimuth), curvature (build rate and walk rate) and/or position. τ is the response depth (similar to time
constant) and Kθ and Kϕ) are the curvature-generation capabilities of the drilling tool in the planes where
inclination and azimuth are defined. Parameters τ,Kθ), and Kϕ) depend on the tool, BHA geometry, bit
selection, rock formation, and the operational parameters such as flow rate and Weight on Bit (WOB).
Parameter Kg) is the gravity-induced bias term that influences borehole propagation. u(ξ) is the input to the
dynamics. Using simple trigonometric relations, toolface and steering ratio can be converted to inputs in
inclination and azimuth planes.
In real-time control applications, a reduced-order model may be preferred for two reasons: (i) simplified
real-time tuning due to reduced (lumped) number of model parameters, and (ii) better computational
efficiency for model-based control methods. Selection of a reduced-order model can be considered as
finding a balance between physics-driven and data-driven control approaches. Nevertheless, in the case
where real-time data quality is poor (e.g., if detection issues are encountered) or unreliable, use of a more
comprehensive model for control may be preferred.
Model-based Control
The steering automation problem is formulated as a model-based control problem where a finite-horizon
optimization problem is solved in real time based on a given wellbore propagation model (Perneder et al.
2017, Marck and Zalluhoglu 2019, Zalluhoglu et al. 2019a). The wellbore propagation model describes the
inclination and azimuth dynamics given tool type, BHA geometry, bit selection, rock formation, operational
parameters, such as flow rate and WOB, and control input parameters, such as tool face and steering ratio.
Hence, the variables of the optimization problem are inclination, azimuth, rate of inclination change (build
IPTC-19817-MS
5
rate), and rate of azimuth change (walk rate). The cost function and problem constraints are expressed in
terms of these variables.
In the optimization problem, a predetermined cost function, subject to constraints, is minimized. The
cost function represents the drilling objectives, which can be defined in terms of a position along the well
plan, attitude (specified inclination and azimuth), or curvature. For example, the cost function can be the
deviation from the well plan when the objective is to control the position of the tool and drill close to the
reference well plan. When the objective is given in terms of attitude, the minimization of the cost function
ensures that the tool is steered based on the inclination and/or azimuth set point. These set points or reference
values can be well plan inclination and/or azimuth or a certain value that is an input to the system. This
type of objective can be extremely useful for geosteering applications during long lateral sections where
the well plan can become irrelevant and where target inclination and/or azimuth change frequently. Finally,
when the objective is given in terms of curvature, cost function can be formulated to hold a certain build
rate and/or walk rate.
The solution of the optimization problem provides the optimal series of steering commands as well as the
resulting future borehole projections along the horizon where the initial condition is obtained by estimating
the states (attitude, position, and curvature) at the bit based on the propagation model and real-time sensor
data. The horizon length of the problem can be selected so it extends to the next target point in the well
plan; it can be thousands of feet long. Using future projections, the method can generate early warnings on
whether the target is reachable.
Note that the optimization problem can be formulated to allow different types of objectives along the
inclination and azimuth planes. For example, when geosteering, if the objective is to decrease the True
Vertical Depth (TVD) by 10 ft while maintaining the azimuth, the operator could implement position control
in TVD and attitude control in azimuth. Additionally, by tuning the weights for cost function, control in one
plane can be prioritized based on drilling objectives demonstrated in the Field Test Results section.
In addition to steering objectives, the problem helps ensure that operational constraints are satisfied. For
example, with collision avoidance issues, a position constraint can be imposed to remain in a safe region
at all times. A constraint on curvature can help limit bending moments along the BHA and help improve
BHA life. Further, it can help construct smooth wellbores without high or fluctuating dog leg severities.
Given the general discussion on the formulation of the preceding problem, it can be expressed as follows
(Eq. 3):
(3)
where ξ represents Measured Depth (MD), and it takes values between MDstart (current bit depth) and MDfinal
(final depth when the target is reached). MDfinal-MDstart is the borehole length necessary to reach the target,
which is the horizon length. x is the state of the system, which is a function of attitude [i.e., inclination (θ),
azimuth (ϕ), build rate (κθ) and walk rate (κϕ)], such as x=[θ ϕ κθ κϕ)]; and u(ξ) is the control input, which is
a function of tool face and steering ratio. J is the objective function of the problem, which is formulized so
that the minimization of this value results in the optimal performance of the system based on the steering
6
IPTC-19817-MS
objective. fs represents the dynamic model of the system, which is used to help predict trajectory based on
the current state and future steering decisions.
Constraint gi is a function of state—the inequality represents the state inequality constraints that can be
used to put upper and lower bounds on the attitude, curvature, tortuosity, and/or position. For example, to
limit tortuosity, an upper bound on the change in curvature between two consecutive states can be imposed.
Constraint hj is a function of control input—the inequality represents input constraints that can be used
to bound the control inputs. Similar to tortuosity constraint, change in control between two consecutive
control commands can be bound to achieve smoother control. Finally, pk is a function of state—the equality
represents the state equality constraints that can be used to specify state values at a certain depth.
The given problem can be adapted for mud motors where the steering input is discrete (slide or rotate)
instead of continuous (steering ratio) by constraining the input u(ξ) to be either 0 or 1 where u(ξ) = 0
corresponds to rotate mode and u(ξ) = 1 corresponds to slide mode. The resulting optimization problem is
a mixed integer optimization problem and can be solved using a general-purpose mixed integer nonlinear
programming solver. For further details on the model predictive control application to directional drilling,
refer to Demirer et al. (2019) and Zhao et al. (2019).
Field Test Results
The proposed control method was recently tested and validated during multiple field trials (Zalluhoglu et
al. 2019b, 2019c). Using position or attitude control, multiple curve sections were successfully drilled with
both RSS and mud motors by maintaining proximity with the well plan. Results from one of the field tests
are discussed in this section.
Fig. 2 summarizes the results for the field test including the borehole trajectory as a side view [TVD vs.
vertical section (VS)] and as a top view (north/south vs. east/west) where black dots show the stationary
surveys taken while the steering commands are generated with the controller. The stationary surveys are
taken every 45ft. In this test, the prediction horizon is set up from the drill bit to the landing point. Given
the well plan and the drilling tool, Problem 1 is solved and the optimal solution is computed every 15ft.
The optimal values for toolface and steering ratio are implemented until the next solution is obtained.
During landing (2400-2500ft), the optimal solution is computed and implemented more frequently to land
accurately. Steering recommendations generated were automatically downlinked to the tool with minimal
human interaction; hence, the overall curve was drilled autonomously.
As it can be seen from the figure, the borehole starts ahead of the well plan and the controller attempts
to maintain the positive distance of 10ft until 70 degrees of inclination for safety. As the landing point
approaches, the controller is set to minimize the deviation from well plan and landed the well accurately
at target TVD.
The right-most plots in Fig. 2 show the toolface and steering ratio down-linked to the tool throughout
the run. In the toolface plot, the jump around 2300ft occurs because the tool changes direction and passes
through 0 to make a small correction in azimuth. Another important point to note is that the smoothing
constraint on control prevents abrupt changes in commands; hence, the steering ratio gradually goes from
100 to 0% during landing.
IPTC-19817-MS
7
Figure 2—Results from the well drilled entirely by model-based automated steering.
For the presented field test, because accurate landing at the target TVD was more important, inclination
control had more weight than azimuth control in terms of operational objectives. Hence, the position tracking
for inclination was intentionally controlled better than it was for azimuth throughout the well. Overall, the
well plan was followed with minimal deviation, and the system landed the well within the target window.
Conclusion
A model-based control approach is presented for steering automation; the solution ensures improved drilling
operations with higher accuracy and smoothness while operational objectives and constraints are satisfied.
Two models are considered: a high-fidelity borehole propagation model and a reduced order model which
can be tuned in real time. Optimal steering commands are generated for accurate tracking of the well plan
while producing a smooth wellbore with minimal tortuosity, as well as the resulting well plan projection
along the prediction horizon. The control method has been tried in various field tests and has successfully
generated accurate and smooth boreholes.
Acknowledgement
Authors gratefully acknowledge the management and technical publication review board of Halliburton for
their support and constructive comments on our paper.
References
Annaiyappa, P. 2013. IT Infrastructure Architectures to Support Drilling Automation. Presented at the SPE Digital Energy
Conference, The Woodlands, Texas, USA, 5–7 March. SPE-163719-MS. https://doi.org/10.2118/163719-MS.
Bayliss, M., Bogath, C., and Whidborne, J. 2015. MPC-Based Feedback Delay Compensation Scheme for Directional
Drilling Attitude Control. Presented at the SPE/IADC Drilling Conference and Exhibition, London, England, 17–19
March. SPE-173009-MS. https://doi.org/10.2118/173009-MS.
Bello, O., Srivastava, D., and Smith, D. 2014. Cloud-Based Data Management in Oil and Gas Fields: Advances,
Challenges, and Opportunities. Presented at the SPE Intelligent Energy Conference & Exhibition, Utrecht, The
Netherlands, 1–3 April. SPE-167882-MS. https://doi.org/10.2118/167882-MS.
8
IPTC-19817-MS
Chang, D-L., Payette, G. S., Pais, D. et al. 2014. Field Trial Results of a Drilling Advisory System. Presented
at the International Petroleum Technology Conference, Doha, Qatar, 19–22 January. IPTC-17216-MS. https://
doi.org/10.2523/IPTC-17216-MS.
Demirer, N., Zalluhoglu, U., Marck, J. et al. 2019. Autonomous Directional Drilling with Rotary Steerable Systems.
Presented at the American Control Conference (ACC), Philadelphia, Pennsylvania, 10–12 July.
Gaines, M., Neil, P., and Herrington, D. 2013. Novel Drilling System Offers Rotary Steerable System Performance with
Real Time Survey in Top Hole Sections While Reducing Overall Costs. Presented at the SPE Annual Technical
Conference and Exhibition, New Orleans, Louisiana, USA, 30 September–2 October. SPE-166467-MS. https://
doi.org/10.2118/166467-MS.
Kremers, N. A. H., Detournay, E., and van de Wouw, N., "Model-Based Robust Control of Directional Drilling Systems,"
in IEEE Transactions on Control Systems Technology, vol. 24, no. 1, pp. 226–239, Jan. 2016.
Langaker, I. and Edvardsen, I. 2015. Continuous Inclination Enhances TVD Wellbore Positioning at the Troll
Fields. Presented at the SPE Bergen One Day Seminar, Bergen, Norway, 22 April. SPE-173878-MS. https://
doi.org/10.2118/173878-MS.
Matheus, J., Ignova, M., and Hornblower, P. 2014. A Hybrid Approach to Closed-Loop Directional Drilling Control Using
Rotary Steerable Systems. Presented at the SPE Latin America and Caribbean Petroleum Engineering Conference,
Maracaibo, Venezuela, 21–23 May. SPE-169433-MS. https://doi.org/10.2118/169433-MS.
Marck, J. and Zalluhoglu U. 2019. On the Bifurcation Behavior of RSS Steering Capabilities. Presented at the 53rd US
Rock Mechanics / Geomechanics Symposium, American Rock Mechanics Association, New York City, New York,
23–26 June.
InverseProblem.co.nz. 2019. Optimization interface (OPTI) toolbox. 2018 https://www.inverseproblem.co.nz/OPTI/.
Panchal, N., Bayliss, M., and Whidborne, J. 2010. Robust Linear Feedback Control of Attitude for Directional Drilling
Tools. IFAC Proceedings Volumes, 43 (9): 92–97. https://doi.org/10.3182/20100802-3-ZA-2014.00022.
Perneder, L., Marck, J., and Detournay, E. 2017. A Model of Planar Borehole Propagation. SIAM Journal on Applied
Mathematics, 77 (4): 1089–1114. https://doi.org/10.1137/16M1094518.
Zalluhoglu, U., Marck, J., Gharib H. et al. 2019a. Borehole Propagation with Undergauged Stabilizers: Theory and
Validation. ASME Journal of Dynamic Systems, Measurement and Control, 141 (5): 051 013–051 013–11.
Zalluhoglu, U., Demirer, N., Marck, J. et al. 2019b. Steering Advisory System for Rotary Steerable Systems. Presented
at the SPE/IADC Drilling Conference and Exhibition, The Hague, The Netherlands, 5–7 March. SPE-194090-MS.
https://doi.org/10.2118/194090-MS.
Zalluhoglu, U., Demirer, N., Marck, J. et al. 2019c. Steering Advisory System for Mud Motors. Presented at the SPE/
IADC International Drilling Conference and Exhibition, The Hague, The Netherlands, 5–7 March. SPE-194077-MS.
https://doi.org/10.2118/194077-MS.
Zhao, Y., Zalluhoglu, U., Marck, J. et al. 2019. Model Predictive Control for Mud Motor Operation in Directional Drilling.
Presented at the American Control Conference (ACC), Philadelphia, Pennsylvania, 10–12 July.