## Abstract

To achieve walking and push recovery successfully, a biped robot must be able to determine if it can maintain its current contact configuration or transition into another one without falling. In this study, the ability of a humanoid robot to maintain single support (SS) or double support (DS) contact and to achieve a step are represented by balanced and steppable regions, respectively, as proposed partitions of an augmented center-of-mass-state space. These regions are constructed with an optimization method that incorporates full-order system dynamics, system properties such as kinematic and actuation limits, and contact interactions with the environment in the two-dimensional sagittal plane. The SS balanced, DS balanced, and steppable regions are obtained for both experimental and simulated walking trajectories of the robot with and without the swing foot velocity constraint to evaluate the contribution of the swing leg momentum. A comparative analysis against one-step capturability, the ability of a biped to come to a stop after one step, demonstrates that the computed steppable region significantly exceeds the one-step capturability of an equivalent reduced-order model. The use of balanced regions to characterize the full balance capability criteria of the system and benchmark controllers is demonstrated with three push recovery controllers. The implemented hip–knee–ankle controller resulted in improved stabilization with respect to decreased foot tipping and time required to balance, relative to an existing hip–ankle controller and a gyro balance feedback controller.

## 1 Introduction

In comparison to non-legged robots, biped robots are able to perform a greater selection of tasks with increased versatility, such as navigating uneven terrain. However, the improved versatility provided by legs also introduces inherent instability into these systems. In addition, the combination of underactuated dynamics, high dimensionality, and mechanical redundancies in humanoid robots present additional challenges when designing balance controllers to stabilize their motion [1].

The notion of stability appears in various forms across a broad range of disciplines, ranging from structural [2,3] to dynamic [4] and control [5] stability. In this article, stability refers to the maintenance and recovery (or loss) of balance in a legged, specifically biped, system. It should be noted that, for example, an unstable controller does not always lead to the loss of balance of a biped robot, while a loss of balance may still result from a stable controller.

Simple proxies are often used as fall detection criteria in humanoid robots. These include upper body angular kinematics [6], center of mass (COM) acceleration [7], angular momentum [8], orbital energy [9], and other such state variables that can be directly measured or estimated. The goal of maintaining balance is then recast as the goal of keeping the proxy below a specified threshold. While there are analytic approaches to control based on inverted pendulums [10,11], these do not take into account the extremes of the balance capability of a given robot or provide accurate thresholds. At the other end of complexity are data-driven criteria [12–14], typically based on machine learning techniques, which encompass empirical data from the real world or a simulation environment, without or accompanying the use of an explicit model.

More rigorous approaches to balance stability that use explicit models of the system to derive the relevant dynamics fall into the broad categories of limit cycle, point-centered, and partition-centered criteria. Limit cycle-based approaches to balance stability derived from nonlinear dynamical systems theory include, e.g., Poincare maps [15], Lyapunov exponents [16], and sensitivity norms [17]. Typically, local stability, i.e., the sensitivity of the biped system in response to infinitesimal perturbations [16], is evaluated for a periodic gait or other specified motions. While these approaches relate to the ability of a controller to track a desired limit cycle or designed trajectory, they do not inform a system's ability to balance itself. In general, the balance stability of a robot subject to non-infinitesimal perturbations must be considered not only for periodic tasks such as walking and running but also for general tasks (e.g., standing push recovery) and aperiodic or nonsteady gait.

### 1.1 Zero-Moment Point and Capture Point.

Point-centered approaches recast the problem of balance stability as keeping a reference point within a defined range corresponding to an existing region. In the case of ground reference points, such as the zero-moment point (ZMP) [18] or its extensions like the foot rotation indicator [18,19], typically the base of support (BOS) of the robot serves as the criterion for dynamic balance. Most such criteria lead to incomplete or inaccurate assessments and are neither necessary nor sufficient conditions for balance [20], because the system's kinematic and actuation limits, response to large perturbations, complete contact constraints, and other relevant physical components are typically not considered in their formulations. For example, when the ZMP criterion is satisfied (i.e., the ZMP is inside the BOS), the motion can still result in other elements of the system coming into contact with the ground (i.e., fallen). Even when the ZMP criterion is not satisfied, e.g., when the ZMP is at the edge of the BOS during foot tipping, the system may or may not be balanced depending on whether the tipped foot returns to full contact with the ground (rebalanced) or rolls over (fallen). Conventional ZMP-based controllers [21] allow a robot to walk without falling, but are overly restrictive in that they rely on feedback control that forces the ZMP to follow a specific planned trajectory mostly within a subset of the BOS selected by the designer, and thus, they do not explore the full range of available state space or even allow the ZMP to explore the complete BOS. These controllers are unable to adapt to an unexpected perturbation by pursuing another trajectory derived from some strategic control policy for active stabilization (i.e., maintaining or recovering balance). When the ZMP (or other reference points) is bounded to lie within the BOS or its subset selected by the designer, that limit does not reflect a fundamental demarcation between the system being balanced or falling. The satisfaction of ZMP at one moment in time merely implies that falling, if it is occurring, has not (yet) led to tipping of the foot. Recent efforts to make advancements in this regard include the mapping of the ZMP criterion to a partition of COM position space [22] and extension of the ZMP to 3D noncoplanar contacts [23].

The capture point is the point on the ground plane where the system can place its foot to bring itself to a complete stop (i.e., full static equilibrium where all joint velocities are zero), and the set of all capture points is the capture region [24]. The instantaneous capture point is such a point if the foot placement is assumed to occur instantaneously (without the effects of swing foot dynamics). In either notion, the stability of the system is determined by whether the point can eventually be brought within the BOS. For the case of the linear inverted pendulum model (LIPM), the instantaneous capture point is equivalent to the ground projection of the divergent component of motion (DCM) [25] and the extrapolated COM [26]. The need for more realistic bounds on the instantaneous capture point has led to the introduction of allowable offsets of the DCM for robust step timing and adjustment at a predefined walking speed [27]. Other variations of the capture point include the 3D extension of the DCM [25] and the corrected capture point, which incorporates a COM offset term to account for modeling error (i.e., non-LIPM effects) and external forces [28].

Reference points are often based on reduced-order models (e.g., LIPM or its variations) due to their computational simplicity that is beneficial within the scope of typical online control (e.g., COM trajectory planning with LIPM). While the use of reduced-order models can suffice for control within a certain limited scope, the few parameters that define these models, such as the total mass and total COM state of the actual system, cannot accurately capture the capabilities and limitations of a given robot. They do not reflect the full attributes of an actual robot's reachability, i.e., the satisfaction of kinematic, dynamic, and actuation limits. For instance, the ranges of motion and actuation limits of various joints in a full-order system cannot be projected into a LIPM with a flywheel.

As a result, the corresponding criteria may be overly restrictive or insufficient and thus require the imposition of additional conditions (e.g., torso tilt angles) to enable a robot to perform a task without falling. For example, robot controllers typically use constant or sinusoidally varying COM height and predetermined foot trajectories to convert walking ZMP trajectories into joint-level motion [6]. The use of task-oriented desired trajectories (e.g., joint, COM, or foot placement) and conditions, along with the associated feedback control efforts, are usually sufficient for fall prevention under manageably small disturbances, but do not encompass the full balance capability of the robot in general tasks and cannot be easily applied to other systems.

### 1.2 Partitioned Space for Stability.

Partition-centered approaches consider stability through the perspective of keeping a point within a derived region of some space, which is often a projection of the system's state space, commonly COM states (positions and velocities) or their extensions [29,30]. One such partition is the viability kernel of a system, the set of all its initial states from which there exists at least one evolution that never results in a failed or nonviable (unbalanced) state [31]. By definition, the state of a humanoid robot is viable (balanced) if and only if it lies within its viability kernel. However, the evaluation of the viability kernel is a computationally intractable problem for typical robots of interest due to the common issues of high dimensionality, nonlinearity, and the redundancy in system dynamics and constraints [24,31]. As an alternative partition-based method, a viable-capture basin can be computed, based not on viability but a similar concept of one-step capturability, defined as the ability of the humanoid robot to arrive at full static equilibrium by taking at most one step [1,24].

The evaluation of capture basins is computationally tractable when approximate centroidal dynamics or equivalent reduced-order models are used as an alternative to the full nonlinear dynamics of the robot. Analytic capture basins have been derived for the LIPM [32], along with necessary conditions for balance of a planar variable-height inverted pendulum with a point foot [33]. Recently, robust walking was demonstrated with a framework based on the capture point for a LIPM [34].

### 1.3 Rationale and Contributions.

Studies of push recovery tend to apply stability criteria more explicitly and extensively, while studies of walking tend to apply stability criteria implicitly and restrictively. Push recovery studies seek to fully explore and exploit a system's balancing capability, testing the extremes of its behavior with few or no priors on what a nominal push recovery motion should look like. Conversely, although walking does require some degree of push recovery, robotic walking control usually begins from a set of nominal motions (e.g., representative step cycles that are tuned or scaled to generate varying gaits) or parameters (e.g., a ZMP trajectory based on a series of pre-planned footsteps), where stability criteria are applied as a guardrail to ensure that a specified gait is stable. The history of humanoid robot locomotion began with conservative gait where the ZMP or the ground projection of the COM is kept close to the centroid of the BOS [35]. With the adoption of new criteria, advanced platforms and controllers for dynamic gait have been developed where the ground projection of the COM is no longer restricted to the BOS [34,36–42].

In this article, a framework is established for the bipedal balance stability of the single support (SS) and double support (DS) contact configurations of a humanoid robot and the feasibility of stepping as the contact transitions. The approach is demonstrated for the analysis of walking and push recovery control for the robot in the 2D sagittal plane. Unlike previous methods that rely on reduced-order models [30] or only consider the lower body [43,44], the proposed approach explicitly characterizes and quantifies the full balancing capabilities of a humanoid robot by including the effect of full-order, nonlinear system dynamics, and the system-specific kinematic and actuation limits. The computed stability regions with and without the swing foot velocity constraint are compared to other existing criteria and with results from experiments and simulations, in addition to being used to evaluate existing controllers.

## 2 Definitions and Concepts

The proposed partition is based on mutually exclusive and collectively exhaustive definitions that are consistent with an intuitive sense of balance stability and can be formulated and incorporated into computational algorithms.

For the purpose of this study, a humanoid robot's foot–ground interaction can only be through SS or DS, which are defined by the set of all possible contact points such that exactly one foot and two feet are in contact with the ground, respectively. Building on prior notions [20,24,44], the complete and general definitions of stability states (i.e., points within the partitioned space) used in this study for a given robot and specified current and desired contact configurations (excluding flight phase) are as follows:

*Definition:*Balanced state. There exists a (controlled) trajectory starting from the state such that the system does not ever need to alter its contacts.*Definition:*Unbalanced state. All (controlled) trajectories starting from the state will lead to an inevitable change in the system's contacts.

The set of all unbalanced states can be further partitioned into the following states:

*Definition:*Steppable state (in SS). There exists a (controlled) trajectory starting from the state, such that the non-stance foot can reach the desired step before an undesired contact is made.*Definition:*Falling state. All (controlled) trajectories starting from the state will lead to an undesired contact before the nonstance foot can reach the desired step in SS or without detaching the existing contacts in DS.

In this study, the term stability region is used to refer to both balanced and steppable (unbalanced) regions. While the whole-body dynamics in this framework includes both joint and Cartesian spaces, the partitioned space is obtained by augmenting the COM states, the dimensionality of which has been shown to be computationally and analytically tractable for both reduced-order [30,45] and lower-body models with joint kinematic and actuation limits [43,44], with the contact status (SS/DS) and relative feet position and velocity (current/desired). Each boundary in this augmented COM-state space is obtained as the set of maximum COM velocities in a desired direction across a range of COM positions, subject to relevant constraints (Fig. 1).

The boundary of the balanced region is the set of all states with maximum allowable velocity perturbations a humanoid robot can sustain while remaining in a specified contact configuration. The boundary of the steppable region is the set of all maximum allowable velocity perturbations, such that the robot can make a contact transition from the current SS to a DS with a desired step configuration. The magnitude of the allowable velocity perturbation increases as the initial swing foot position approaches the desired step and, in theory, goes to infinity right before the SS to DS transition when the desired step can be made almost instantaneously. The steppable region represents the reachability of the SS to DS transition, a complementary concept to one-step capturability criteria that incorporates effects such as swing leg dynamics, and any kinematic or actuation constraints related to foot placement. By definition, all one-step capturable states must be steppable for a given step length.

A currently balanced SS and DS state can transition to a DS (by stepping) and SS (by intended lifting of a foot) state, respectively, that are either balanced or unbalanced. If the desired transitions occur between two balanced states (e.g., static gait [46] by some humanoid robots that can be halted at any instant without becoming unbalanced), the robot, with possible additional balancing actions, can remain in its original contact configuration at any time during the transition process. Since balanced regions only guarantee the existence of a balanced trajectory, it is possible for a robot to evolve from a balanced to an unbalanced state as a result of its own internal control or external perturbations. It should be noted that the proposed definition also allows for balanced states that avoid a fall only by remaining in a certain motion (e.g., as in a spinning top) and so cannot attain a final static equilibrium.

An unbalanced DS state may result in a fall in DS or transition to a SS state (by unavoidable lifting of a foot) that may be balanced, falling, or steppable. Note that stepping does not always indicate immediate balance recovery. From the steppable state, balance may be recovered (with respect to the new contacts) if the DS state immediately after stepping is within the new contacts’ balanced region. Otherwise, if the DS state immediately after stepping is unbalanced, the aforementioned evaluations can be applied recursively. A robot that is in a falling SS state with respect to some desired step may still be able to make a different step (e.g., with a shorter step length).

## 3 Kinematic and Dynamic Models

The stability boundaries of a DARwIn-OP humanoid robot (ROBOTIS Co., Ltd, Seoul, South Korea) are computed from a series of constrained optimization problems given its state and its current and future (if any) desired contacts. Each optimization problem includes the full-order dynamics of the robot and its interaction with the environment through contact(s). The full-order dynamics is modeled under rigid body assumptions in both the joint and Cartesian spaces. The joint-space dynamics [44] is used for whole-body joint trajectories, the Cartesian-space dynamics is used to formulate the COM states and linear and angular momenta, and both are used to model the contact wrench applied to each foot–ground interaction.

### 3.1 Tree-Structure Robot Model With Floating Base.

The DARwIn-OP humanoid robot is modeled using the Denavit–Hartenberg (DH) convention in the sagittal plane as a planar kinematic tree structure attached to the global reference frame {*X*, *Y*} with a floating base at the pelvis. The total degrees-of-freedom (DOFs) of the tree structure include the joint variables {*q*_{1}, *…*, *q*_{11}} of the system, which are all in-plane rotational DOFs (flexion/extension), and the additional fictitious joint variables {*q _{f}*

_{1},

*q*

_{f}_{2},

*q*

_{f}_{3}}, which are the two translational and one rotational DOFs, respectively, that represent translation and rotation of the floating base in the sagittal plane (Fig. 2). The tree structure model comprises 5 branches and 12 links. The mass, COM, and inertia tensor of each link are modeled based on data from the manufacturer (ROBOTIS Co., Ltd.) [47]. Under the assumption of full contact, the BOS of each contact spans the full foot length

*fl*. For the case of DS contact, both feet remain in contact with the ground, separated by the step length

*sl*, and the DS BOS is the convex hull of both the front and rear feet. The index

_{desired}*p*= 1, 2 indicates the (stance or swing) foot located at and away from the origin, respectively (Fig. 2).

Each joint angle limit was directly measured with the motor encoder from its most extreme positions. The joint actuator torque and velocity limits of the robot are modeled based on the performance parameters of the MX-28 servomotor [48], with a refined linear approximation of the torque–speed curve [44]. In each direction, the maximum joint velocity and actuation torque limits are the no-load speed and the stall torque, respectively, where the latter corresponds to continuous operation at zero angular velocity [49].

### 3.2 Resultant Moment for 2D Contact Wrench Distribution in Sagittal Plane.

**M**

*about the total COM is equal to the total rate of change of the centroidal angular momentum*

_{c}**H**

*of the system about the total COM:*

_{c}*k*is the link index,

*m*is the link mass, $\rho k$ is the link COM position vector with respect to the total system COM, $\omega k$ is the link absolute angular velocity, and

_{k}*I*is the link inertia tensor with respect to the link COM frame. Since the whole-body kinematic structure is represented using the DH convention, the components of $\omega k$ and $\omega \u02d9k$ can be extracted from the angular velocity matrix $\omega ~k$ and angular acceleration matrix $\omega \u02d9~k$ [50], which are their respective duals, using the following relationships:

_{k}*R*and its time-derivatives can be obtained from the first 3 × 3 submatrices of the 4 × 4 homogenous transformation matrix and its time-derivatives, which transform the global reference frame to the

_{k}*k*th link local frame and are used for joint-space formulations. The link inertia tensor is obtained from $Ik=RkIk\u2032RkT$, where $Ik\u2032$ is the time-invariant inertia tensor with respect to a local frame, which is attached at the link COM and has the same orientation as the DH local frame of the link, from the manufacturer data.

*p*th contact wrench is applied at

**r**

*= (*

_{p}*r*

_{p}_{,x},

*r*

_{p}_{,y}), the

*X*- and

*Y*-position of the center of the

*p*th foot sole, which is coincident with the origin of the local frame of each foot link (Fig. 2). For SS, the contact wrench at the stance foot is computed directly from the COM and centroidal angular momentum dynamics [44]:

*Z*) component of $H\u02d9c$;

*F*

_{p}_{,x},

*F*

_{p}_{,y}, and

*M*

_{p}_{,z}are the friction force, normal force, and ground reaction moment, respectively, applied at

**r**

*;*

_{p}*m*is the total system mass; (

*x*,

*y*) are the COM

*X*- and

*Y*-positions, respectively; and

*g*is the gravitational acceleration as in

**g**= [0, −

*g*].

*α*(

*t*),

*β*(

*t*),

*γ*(

*t*)} that determine the relative scaling of the contact wrenches across both feet, as follows [44]:

*μ*is the friction coefficient and {

*α*(

*t*),

*β*(

*t*),

*γ*(

*t*)} are normalized variables within the interval [0,1] that are time-discretized to be determined from optimization.

These Cartesian-space contact wrenches are exerted on the humanoid robot and incorporated into the full-order joint-space dynamics. This approach allows for the mutual dynamic (time-varying) interactions between the whole-body motion and the contact wrenches while addressing the problem of mechanical indeterminacy. Furthermore, the combination of the two distinct formulations (recursive Lagrangian for joint space and Newton–Euler for Cartesian space) of dynamics allows for rigorous evaluation of the accuracy of the dynamic model and the associated computations. Accurate computations of the contact wrenches will result in floating-base forces and torques that are zero or within a numerical tolerance *ɛ*, which is used to monitor the errors in the computations.

## 4 Optimization Problems Formulation

The stability boundaries are evaluated over a partitioned space of augmented system states. For fore-aft perturbation analysis, the partitioned space is the COM-state space (*x*, *y*, $x\u02d9$) augmented by (*r*_{2,x}, *r*_{2,y}, $r\u02d92,x$). The stability boundaries are evaluated over a discretized domain of the system state: the set of points (* ^{i}x*,

*,*

^{i}y^{i}

*r*

_{2,x},

^{i}

*r*

_{2,y}, $ir\u02d92,x$), where the index

*i*identifies each point. Any method of discretization can be used, such as general schemes (e.g., sampling points on a defined grid) or task-specific schemes (e.g., sampling points along a specific trajectory or within its vicinity) to sample the state space. For the analysis of walking, the discretized points are sampled along the COM and swing foot trajectories during walking. For push recovery, they are sampled from the COM workspace at a constant COM

*Y*-position.

*X*-component of initial COM velocity $x\u02d9(0)$ in both forward and backward directions, subject to general and partition-specific (balanced or steppable) constraints. In general, COM velocities in any direction may be considered. The general constraints are those related to the system properties, contact interactions with the environment, initial conditions, and boundary conditions that are common for balanced and steppable states. Each discretized point (

*,*

^{i}x*,*

^{i}y^{i}

*r*

_{2,x},

^{i}

*r*

_{2,y}, $ir\u02d92,x$) in the partitioned space is imposed as initial conditions.

### 4.1 General Constraints.

*T*], where

*T*is the terminal time, as follows:

**q**, $q\u02d9$, and $\tau $ are the joint variable, velocity, and actuation torque vectors, respectively;

*τ*

_{f3}is the torque of the out-of-plane fictitious rotational DOF of the floating base;

*ɛ*is a numerical tolerance that is either explicitly imposed (for walking) or monitored (for push recovery); and

*Φ*

_{p,z}is the relative angle between the foot and the ground in the sagittal plane. The upper and lower bounds are indicated by the superscripts

*UB*and

*LB*, respectively. For the SS and DS contact configurations, the constraints are applied for

*p*= 1 and

*p*= 1, 2, respectively. The explicit floating-base actuation limits bound the errors of the inverse dynamics and contact wrench computations, which are sensitive to the numerical tolerances of stance feet kinematic constraints. In addition, according to the proposed definitions of balanced and steppable states, any non-foot contact forces are set to zero at all times.

### 4.2 Partition-Specific Constraints.

*Y*-position is constrained to remain above the ground when generating the SS balanced and steppable boundaries.

*t*=

*T*as final conditions that represent the corresponding SS or DS balanced states’ reachability for a typical fully actuated robot:

*x*and

_{f}*y*are the COM

_{f}*X*- and

*Y*-positions, respectively, of a specified balanced final pose in SS or DS contact.

*r*and

_{p,x}^{ankle}*r*are the

_{p,y}^{ankle}*X*- and

*Y*-positions of the swing foot ankle, respectively, which are used instead of

**r**

*to avoid extreme ankle angles during optimization. These constraints drive the relative feet position to have at least the desired step length in the desired direction, and their satisfaction within a sufficient*

_{p}*T*indicates that the desired step can be made before any other undesired contact occurs from a SS unbalanced state. The desired step conditions are evaluated at each time step, and, through the if-else structure within each optimization iteration, all constraints are imposed only until the first time instant, identified as

*t*

_{step}∈ [0,

*T*], when they are satisfied, i.e., $r2,xankle(tstep)\u2212r1,xankle(0)\u2265sldesired$ and $r2,yankle(tstep)\u2212r1,yankle(0)\u22640.1m$. This approach allows for a flexible solution interval [0,

*t*] within the fixed terminal time

_{step}*T*that is assigned for the collocation scheme.

### 4.3 Numerical Optimization and Initial Solution Scheme.

The solution to the optimization problem at each discretized point of the partitioned space includes the maximized $x\u02d9(0)$, the corresponding joint kinematic and actuation trajectories, and the time-varying contact wrenches applied at the stance foot/feet over the time interval. The optimization variables in the direct collocation scheme [44] include the control vertices of the third order B-splines that are used to parameterize the joint kinematics and the normalized variables for the contact wrenches. The terminal time is chosen based on numerical experiments. The constrained nonlinear trajectory optimization problems corresponding to the discretized points are solved as a sequence of subproblems by using the sequential quadratic programming algorithm [51]. All the analytical gradients for the cost and constraint functions are implemented into the algorithm.

Each optimization problem for balanced states is solved repeatedly within an initial solution generation algorithm to achieve improved optimality (Algorithm 1). Before optimization, the feasibility of each discretized point is evaluated according to the constraints imposed. The stability region is then generated through multiple rounds of optimization, where the previous optimal or feasible solutions are used as initial solutions to obtain the new ones in the next round. This algorithm was applied to the discretized points of walking trajectories.

## 5 Push Recovery Control

To demonstrate the utility of the computed stability regions with simple push recovery strategies, two existing controllers from prior work and an improved implementation of an existing controller are tested in a simulation environment. The strategies considered include the ankle strategy, which maintains balance only by modulating the center of pressure without directly regulating angular momentum, and the hip strategy, which does both [29,52].

The default balance controller of the DARwIn-OP humanoid robot regulates its torso angular velocity with proportional control by adjusting the knee angle biases $\Delta q7,10=KP,kneegyro\omega gyro$ and ankle angle biases $\Delta q8,11=KP,anklegyro\omega gyro$, where $KP,kneegyro$ and $KP,anklegyro$ are the proportional gains of the knee and ankle, respectively, and ω_{gyro} is the torso angular velocity measured by an attached gyro sensor [6].

*PD*control by adjusting $\Delta q8,11=KP,ankleCOMx+KD,ankleCOMx\u02d9$, where $KP,ankleCOM$ and $KD,ankleCOM$ are the proportional and derivative gains, respectively. The hip subcontroller employs a bang-bang-like control strategy that sets the hip angle biases as follows:

*q*

_{6,9},

*t*is the time when the COM

^{bb}*X*-velocity reaches a user-defined threshold to trigger the bang-bang control, and 2

*T*

_{H}_{1}and

*T*

_{H}_{2}are the durations of pelvis motion away from and then back to its initial position, respectively [53]. At all other times, the biases are zero.

The hip–knee–ankle controller (Fig. 3) is an improved version of the existing hip–ankle controller that adds proportional control of the knee to assist hip motion for additional stabilization capability. The knee subcontroller adjusts $\Delta q7,10=KP,kneehip(q6,9desired\u2212q6,9measured)$, where $KP,kneehip$ is the proportional gain, which is approximated from the average hip and knee motion of the optimal solution trajectories of the computed stability regions. The superscripts *desired* and *measured* denote the joint angles of the desired final DS pose and measured by the motor encoders, respectively.

## 6 Experiments and Simulations

Walking experiments were conducted with the DARwIn-OP humanoid robot at an intermediate walking speed of 0.096 m/s [44]. The robot's default gait generation algorithm, *walk_tuner* (ROBOTIS Co., Ltd.), consists of a walking pattern generator based on coupled sinusoidal trajectories and a gyro balance feedback controller based on a proportional control, where the angle biases of the knee pitch, ankle pitch, hip roll, and ankle roll DOFs are proportional to the torso angular velocity (measured by a gyro sensor) by their respective control gains [6]. Each joint is controlled by *PID* feedback. All control gains were set to their default values, including the arm swing gains for walking pattern generation. The SS and DS phases were segmented based on sensor data from force sensing resistors embedded in the foot soles of the robot. One representative step cycle, from a complete SS to the following complete DS, was selected from the walking trials. The COM and swing foot trajectories and contact phases were estimated using forward kinematics from the motor encoder data obtained at each DOF. The step length, step duration, step height, and the ratio of DS to SS durations of the selected step cycle were 0.057 m, 0.533 s, 0.03 m, and 0.1, respectively.

The Webots (Cyberbotics Ltd., Switzerland) simulation environment was used to obtain a walking trajectory with the same gait parameters as those of the walking experiment. Due to differences between the experiment and simulation setups, the step length of the simulated trajectory was slightly adjusted to 0.054 m. Perturbation experiments were also conducted in simulations to test the push recovery capability of the evaluated controllers in response to forward velocity perturbations from an upright DS pose without stepping. To simulate only sagittal planar dynamics, self-collision was disabled, the lateral widths of the bounding objects of the feet were increased, and large torque limits and zero velocity limits were imposed on out-of-plane DOFs (e.g., hip roll and ankle roll) such that lateral stability was not a concern. The step length, controller step time, $KP,kneegyro$, $KP,anklegyro$, $KP,ankleCOM$, $KD,ankleCOM$, $\Delta qhipbb$, 2*T _{H}*

_{1},

*T*

_{H}_{2}, and $KP,kneehip$ were 0.057 m, 8 ms, 0.25, 0.75, 0, 0.2, 40 deg, 0.3 s, 0.3 s, and 0.5, respectively. All gains except for $KP,kneehip$ were tuned manually.

## 7 Results and Discussion

Stability regions are constructed for the analysis of both experimental and simulated walking trajectories and the comparison of the three push recovery controllers tested. In this study, only the *X*-components of COM-state space are considered. The SS balanced and steppable (only for forward perturbations) regions and DS balanced regions for walking were obtained for the SS and DS phases of gait, respectively. The experimental walking COM trajectory was sampled at equally spaced COM *X*-positions to obtain the discretized domain for the space partitions. The DS balanced region for push recovery was obtained at all COM *X*-positions corresponding to the *sl _{desired}* and constant COM

*Y*-position of 0.057 m and 0.2081 m, respectively. When computing the regions, ɛ ranged from 0.01 to 0.2 Nm. Comparative analyses with other balance criteria such as the instantaneous capture velocity, ZMP, and upper body tilt angle are also presented.

### 7.1 Walking Trajectory Analysis.

The contribution of the linear and angular momenta of the swing leg, which is critical in balancing during SS, is confirmed by the reduction in the area of the SS balanced region when the constraint on initial swing foot velocity $r\u02d92,x(0)$ is imposed (Fig. 4). Constraining $r\u02d92,x(0)$ to match that of the walking trajectory leads to a more reasonable estimate of the balanced states during gait. The reduction in the backward and forward COM *X*-velocities is greatest during the beginning and end of SS, respectively, when the COM *X*-velocity direction and the swing leg position relative to the stance leg (backward or forward) are the same.

The computed stability regions are compared with one-step capturability [24] (Fig. 4) using the instantaneous capture velocity $x\u02d9cap=(sldesired+fl/2\u2212x)g/y$, which is based on the instantaneous capture point of the LIPM representing the humanoid robot. The velocity corresponds to when the *X*-position of the instantaneous capture point is coincident with the leading edge of the desired DS BOS (*sl _{desired}* +

*fl*/2; Fig. 2). The boundary is an approximation of the maximum capability of the system to capture itself within one step. The instantaneous capture velocity decreases as the COM

*X*-position increases and goes to zero as the COM

*X*-position approaches

*sl*+

_{desired}*fl*/2. The situation of this boundary between those of the balanced and steppable regions is physically plausible from the respective definitions of the proposed and existing concepts and indicates their validity. When the COM

*X*-position is at the instantaneous capture point, an infinitesimal forward velocity perturbation will cause the equivalent LIPM to become unbalanced [30]. It should be noted that the LIPM does not take into account the actual system's kinematic and actuation limits or swing leg dynamics.

The steppable boundary velocity increases with increasing COM *X*-position because the reachability of the step length is greater when the swing foot position nears the final footstep location (Fig. 4). The comparative analysis with capturability also demonstrates that the computed steppable region significantly exceeds the instantaneous capture velocity boundary. This combined analysis integrates the reachability information provided by the steppable region with the capturability of an approximate equivalent reduced-order model to provide a fuller picture of the entire system that cannot be obtained with either component alone.

The DS balanced region lies almost entirely within the instantaneous capture velocity boundary. At the end of the walking COM trajectory, the two boundaries are similar, suggesting that the reduced-order model approximates the full-order system well with respect to comparing balance stability and one-step capturability at that point.

The location of the different balanced regions relative to each other, such as the discontinuity between the SS and DS balanced regions at the SS to DS transition, offer potential insights into the range of possible gaits in response to forward perturbations that increase the COM *X*-velocity.

Although both the simulation and walking experiments used the same gait parameters, the different dynamics of the simulation environment and the real world led to different COM trajectories (Fig. 5). In the simulation, the COM *X*-position was shifted forward and the COM *Y*-position variation was reduced relative to that of the real world. Despite these differences, both trajectories have similar SS balanced regions over their overlapping range of COM *X*-positions.

### 7.2 Comparison of Push Recovery Controllers.

Both the hip–ankle and hip–knee–ankle controllers recovered from larger force perturbations than did the gyro balance feedback controller (Fig. 6). While the additional knee motion of the hip–knee–ankle controller did not increase its balance capability in terms of sustaining larger perturbations, it reduced both the balancing time, defined as the time elapsed for $x\u02d9$ to reach 5% of its post-perturbation magnitude, and the tipping of the stance feet relative to the ground. For the 115-N perturbation, the balancing time and maximum foot tipping angle were 1.66 s and 13.98 deg, respectively, for the hip–ankle controller and 0.96 s and 0.000024 deg, respectively, for the hip–knee–ankle controller. For the 115-N perturbation with the gyro balance feedback controller, the proposed stability region was able to determine that the robot was unbalanced before other criteria, such as the violation of the ZMP criterion and the onset of extreme upper body tilt angles, could detect the fall (Fig. 7).

Other than the gyro balance feedback controller, the push recovery controllers were able to reach states near but not beyond the stability region boundary. These results demonstrate that the stability regions characterize the full balance capability of the system and can be used to benchmark controllers. Typically, individual controllers cannot approach the boundary because of the extreme nature of the optimal trajectory solutions that correspond to the maximized (or minimized) initial COM *X*-velocities. However, the trajectories shown were able to do so because the simulation allowed for a limited degree of foot tipping while the computation of the balanced regions did not, due to the assumption of complete and full contact for both feet.

## 8 Conclusions

The proposed stability regions were computed with the consideration of full-order system dynamics, capabilities, and limitations in the sagittal plane. The regions obtained for walking were used to confirm and assess the contribution of swing leg momentum to balancing, and demonstrated that the steppable region significantly exceeds the instantaneous capture velocity boundary. The use of stability regions to represent the complete balance capability of a humanoid robot and to benchmark controllers was demonstrated for simple push recovery controllers. The computed stability regions also identified falling states earlier than did other criteria, such as the ZMP and torso tilt angle profile. While the addition of a knee subcontroller to an existing hip–ankle controller did not expand the balance capability of the robot, it improved balance performance by reducing balancing time and foot tipping.

## Acknowledgment

This work was supported in part by the U.S. National Science Foundation (CMMI-1436636 and IIS-1427193). The authors would like to thank Hyun Seok Shin for his help in data acquisition and processing.

## Conflict of Interest

There are no conflicts of interest.

## Data Availability Statement

The datasets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request. The authors attest that all data for this study are included in the paper.