Partition-Aware Stability Control for Humanoid Robot Push Recovery With Whole-Body Capturability

Balance stability control in biped robots presents a critical but difficult problem because of their high dimensionality, nonlinearity, and contact interactions with their environment [1]. Existing approaches for balance criteria can be categorized into data-driven, limit cycle, reference points, or space partition. Stability evaluation based on data-driven learning [2–5] is practically useful but lacks interpretability and requires a large number of tests for each system. In the limit cycle approaches [6–8], the system’s local stability in response to infinitesimal perturbations is evaluated with respect to specified trajectories such as periodic gait. Generally, these approaches are not suited for standing push recovery due to the presence of non-infinitesimal perturbations and aperiodicity. Reference point approaches consider bounding a defined point within a range on the ground plane, such as the zero-moment point (ZMP) [9] or its modifications [10–12] within the base of support (BOS) or its subsets, or in 3D space as thresholds for fall detection and prevention. Reference points are frequently based on reduced-order models or sensor measurements and can be implemented in real-time control with low computational effort. However, due to their general lack of complete system characteristics (e.g., kinematic and actuation limits) or contact interactions with the environment, most of them are neither necessary nor sufficient conditions to distinguish balance and fall [13–16].

Among the reference points, the capture point is used to indicate the proper foot placement on the ground required to come to a complete stop [17]. The instantaneous capture point is a special case of the capture point evaluated for a linear inverted pendulum model (LIPM) without consideration of swing leg dynamics, and is equivalent to the ground projection of the divergent component of motion [18,19] and the extrapolated center of mass [20]. The concept was enhanced with the allowance of offsets [21], extensions to 3D [19], and considerations of external forces and the limitations of the LIPM [22].

By contrast, space partition approaches derive a region of the state space of the system or its projection, such as center-of-mass (COM) state (position and velocity) or its augmentations [1,13,14,23,24], as a criterion for evaluating the balance stability of a given state for control. In particular, these are typically based on viability kernel, which is the set of all initial states of the system from which there exists a trajectory that never evolves to a failed or non-viable state [25]. Thus, a system state is viable if and only if it remains inside the system’s viability kernel. However, for a general biped robot, evaluation of the viability kernel is computationally intractable because of issues such as nonlinearity, high dimensionality, and redundancy [17,25]. To address this, the concept of capturability, which represents the ability of a robot to come to a complete stop (within a specified number of steps) without falling [17,26], provides a tractable form of the viability kernel. The N-step viable capture basin, defined as the set of all states that are capturable with N or fewer steps, is a partition of the system state space derived from the notion of the capture point. However, this and similar partitions [27] typically depend on simplifying assumptions or reduced-order models in order to avoid the issue of high dimensionality. While reduced-order models are often used to implement real-time control, they do not reflect whole-body system properties and dynamics, such as the full effects of linear and angular momenta, and thus cannot fully exploit the balance capability of the robots and may overly restrict the robot’s ability to perform general tasks.

Push recovery is an elementary task that seeks to fully explore and exploit a legged system’s balancing capability, in response to not only external disturbances but also internal or desired perturbations such as in gait. For this reason, it has been used to demonstrate several existing criteria [24,28], including the instantaneous capture point and its extensions [23,29] and optimal selection between ZMP feedback and footstep adjustment [30] or among strategies [31]. Push recovery strategies can be categorized into non-stepping and stepping. The non-stepping strategies can be divided according to their controlled joints: ankle, hip, and whole body. The ankle strategy only adjusts the ankle joint to modulate the COM and the center of pressure (COP), like in a simple inverted pendulum with a base link [1]. The hip and whole-body strategies include the contribution of the ankle joint, while also regulating the linear and angular momenta of the upper body through adjusting only the leg joints and all joints, respectively [32], like in an inverted-pendulum-based model with a flywheel [17,24,28] or multiple degrees-of-freedom (DOFs) [1,33]. The stepping strategy relies on changing the contact through foot placement to reduce the COM velocity and/or adjust the size or location of BOS. The effect of these strategies is to modulate the ground reaction forces [34]. For small perturbations, the limited COM or COP modulation of the ankle strategy is sufficient, while the hip or stepping strategy is needed for large perturbations. In general, the sequential employment of ankle, hip, and stepping strategies is used to maintain or recover balance and is superior to the employment of a single strategy [24,31].

In a prior work [14], partitions of augmented COM-state space as bipedal balance stability criteria during the single (SS) and double support (DS) contacts and stepping transitions were derived for whole-body systems based on the relevant definitions. From a balanced state, there exists a (controlled) trajectory such that the system does not ever need to change its contacts. From a steppable unbalanced state, the non-stance foot can reach a desired step before undesired contact(s) are made. While a balanced state is approximately 0-step capturable, a steppable state does not imply capturability, i.e., the subsequent recovery of balance, once that step is made. In this study, the formulation of the 1-step capturable boundary for whole-body humanoid robots is introduced. The boundary, along with those of balanced basins, is incorporated as explicit criteria to select among the ankle, hip, and captured stepping strategies in an advanced partition-aware push recovery controller (Fig. 1). Unlike those from the literature that are mainly based on reduced-order models [17,26,27,29], the balance maintenance and recovery criteria are obtained for a whole-body model with full-order nonlinear dynamics to accurately characterize the system-specific properties and capabilities. Standing push recovery from a zero step length is used to establish the criteria and evaluate the corresponding control. The significance of using the whole-body models when computing the stability boundaries is validated not only against a typical reduced-order model but also across the whole-body stability boundaries with different strategies and contacts. The use of the capturable and balanced state boundaries as criteria is verified in simulation experiments where the proposed control design enables the robot to fully exploit its balance capability through appropriate balancing actions in response to perturbations.

The DARwIn-OP humanoid robot (ROBOTIS Co., Ltd, South Korea) is chosen as the platform for implementation. The sagittal plane is selected as the plane of perturbation application during standing push recovery on level ground.

The humanoid robot is modeled in the sagittal plane as a planar whole-body system with a floating base attached at the torso. The total DOFs include the joint variables of the robot for in-plane rotation and additional fictitious joint variables, including two translational and one rotational DOFs, for the position and orientation of the floating base. The index p = 1, 2 identifies the stance and swing foot in SS and the rear and front stance foot in DS, respectively, and r_p = [r_p,xr_p,y]^T denotes the position vector of the origin of the pth foot-link local frame at the foot sole center with respect to the global frame {X, Y} with units in meters (Fig. 2). For notational simplicity without loss of generality, r₁ = 0 is assumed and Y = 0 represents level ground. The BOS lengths during SS and DS are the foot length fl and the sum of fl and the desired step length $Δrxdesired$⁠, respectively.

The range of motion of each joint was obtained as the most extreme positions of the servomotor measured by its encoder and verified against existing data [35]. The torque and velocity limits of the servomotor are modeled as a hexagonal region [13,14] (Fig. 2), denoting its desired operating range.

The formulations of balanced state boundary construction from prior works [13,14] are briefly revisited here for DS in the sagittal plane and refined for balancing with ankle and hip strategies. The boundaries are computed over the sampled domain of the task-augmented COM state space with an optimization-based framework. The domain is a set of the sampled COM positions $r¯sampled$ augmented with desired foot contacts.

The general constraints with desired contact(s) from the prior works [13,14] are used, representing ground penetration avoidance of the feet, system properties (i.e., joint kinematic and actuation limits including the zero forces and torques for the floating base), whole-body dynamics, and contact interactions in the absence of any external wrenches except for the desired foot contacts at all times. Only full foot-ground contact is considered here.

These constraints are implemented by fixing the corresponding joint angles to those of the reference pose with $r¯=[0.00.2]T$⁠, $Δrxdesired=0$⁠, arms straight down, bent knees, and upright torso. The initial COM positions $r¯sampled$ are sampled from the arc formed by the COM workspace with all joints fixed except for the ankle at the reference pose.

Initial joint angular velocity conditions, which would fully determine the initial COM velocity, are not imposed.

The initial and final COM Y-positions for the pre-computed DS balanced basin are that of the reference pose for the ankle and hip strategies. Since a balanced state of the robot is also 0-step capturable, the satisfaction of this condition at t = t_step ensures that full static equilibrium can be eventually reached without further stepping or other contact changes.

A partition-aware controller for balanced basins and steppability introduced in a prior work [14] is modified in this study for recovery of balance by incorporating the captured stepping strategy. The ankle- and hip-strategy balanced state boundaries are actively used as explicit criteria to select among the ankle, hip, and captured-stepping strategies given the current estimated COM state of the robot (Fig. 4).

Forward kinematics is used to estimate the current COM state and foot orientations through the measured joint angles q^measured and global orientation of the torso obtained from an inertial measurement unit (IMU). The controller consists of ankle, hip, and captured stepping subcontrollers, which represent the respective strategies. The ankle subcontroller is always engaged, but the hip and captured stepping subcontrollers are triggered only when the estimated COM state exits the ankle- and hip-strategy balanced basins, respectively.

The proposed hip subcontroller is triggered when the estimated COM state exits the ankle-strategy balanced basin. It generates bang-bang-like motion profiles for the hip joint control [3,14] that regulates the ground reaction force through the angular momentum of the upper body, and for the pelvis X-position control that keeps the COM X-position within the BOS during balancing. The pth hip pitch angle bias Δq_p,hip and the pelvis X-position bias Δr_pelvis,x with respect to the stance foot position are regulated as follows:

The captured stepping subcontroller achieves the desired step length $Δrxdesired$ as quickly as possible with a bang-bang-like motion profile to adjust the swing foot and pelvis position with respect to the stance foot position in two swing phases (S1 and S2):

In order to supplement the captured stepping action, the ankle and hip subcontrollers remain engaged with gains and parameters that are different from those without stepping. All the gains and parameters of the ankle, hip, and captured stepping subcontrollers were manually tuned through simulation experiments to minimize foot tipping, avoid self-collision during stepping, and ensure full foot-ground contact at the end of the step.

The ankle- and hip-strategy balanced state and whole-body 1-step capturable boundaries were obtained for the humanoid robot in the sagittal plane and incorporated into the proposed partition-aware controller. Due to the consideration of standing push recovery in response to forward and backward perturbations, the objective function is the X-component of the COM velocity and only the X-component of the COM-state space is of interest. For the COM state constraint at t = t_step of the whole-body 1-step capturable boundary, a pre-computed DS whole-body-strategy balanced boundary from a prior work [13] was incorporated into the optimization algorithm as a linearly approximated form and 3.98% of the reduction ratio in the COM X-velocity was used for the step impact according to the simple collision model [39]. For captured stepping, the desired step length is $Δrxdesired=0.057m$ and the range of allowable swing foot angle is $[Φ2LB,Φ2UB]=[−π/5,π/5]$⁠. Control simulations were conducted to demonstrate the proposed controller and validate the use of the stability boundaries as criteria.

The ankle- and hip-strategy balanced state boundaries were computed for a DS standing pose with a zero step length (Fig. 5). Within the COM X-position range of the BOS for zero step length, the balanced basin area of the hip strategy is 44.5% greater than that of the ankle strategy due to the added balance capability of the knee and hip joints.

The whole-body 1-step capturable boundary was computed within the BOS for zero step length (Fig. 5). Additionally, another capturable boundary was computed with a constraint that disallowed all upper-body motion, by imposing q_upper(t) = 0 (i.e., at the reference configuration for each joint) at all times, to quantify the contribution of the upper-body DOFs in recovering balance. The increase in the capturable boundaries relative to that of the hip strategy demonstrates the greater contribution of the stepping versus hip strategy due to the improved mobility and extended BOS dimension. The relatively low gap between the two capturable boundaries is due to the low mass of the arms relative to the total robot mass. The time to step decreases as the COM X-position shifts forward, which limits the effect of upper-body momenta prior to stepping and causes the gap to narrow. On the other hand, while stepping is generally understood as the dominant method of balancing [27], this expected effect is not fully reflected in the observed increase of the computed capturable boundary relative to the balanced state boundary of the hip strategy or the DS balanced with the desired step length due to several factors. The desired step length is relatively short compared to the COM Y-position, which limits the COM velocity reduction according to the simple collision model used in this study. Thus, the increase in stability from stepping is primarily from the extended BOS for the desired step length and not from step impact. Non-stepping strategies also benefit disproportionately from DARwIn-OP’s long feet relative to its height and desired step length; it results in a relatively large BOS for zero step length during standing while interfering with swing-foot ground clearance for stepping. The robot’s uniform torque limits across all joints including the ankle, which is typically weaker in humans and other humanoid robots, directly increases the balance capability of the ankle and hip strategies. Another factor is the difference in the initial poses to achieve the initial COM X-positions, specifically, by adjusting the ankle joint angle for the balanced state boundaries versus the pelvis position for the capturable boundary. Nonetheless, the overall attributes of the computed capturable boundary and its relationship with the balanced boundaries demonstrate the validity of the proposed approach as proof of concept.

For comparison, the common LIPM-based capturability [17] with the same desired step length was also calculated from the 1-step instantaneous capture velocity $x˙cap=(Δrxdesired+fl/2−x¯)g/y¯$ that corresponds to the instantaneous capture point at the forward edge $Δrxdesired+fl/2$ of the stepped BOS (Fig. 2). Here, $x˙cap$ can be regarded as the maximum velocity threshold for achieving a captured state within one step [17]. On average, the LIPM-based capturable boundary is 59.5% greater than that of the whole body (Fig. 5). This implies that the LIPM does not accurately represent the actual robot’s capturability due to its modeling simplifications, including the lack of realistic kinematic and actuation limits, swing leg dynamics, and consideration of foot clearance. In particular, the typical LIPM-based capturability [16,17] ignores foot scuffing collisions through the assumption of zero swing foot velocity at the instant of stepping. These complete system properties and dynamics are incorporated within the constraints used to compute the boundaries in the proposed whole-body approach, providing a proof of concept of a more accurate threshold than that based on the reduced-order model.

The proposed partition-aware controller is verified and demonstrated by applying impulsive force perturbations to the torso of the robot in the Webots simulation environment (Cyberbotics Ltd., Switzerland) to reach various initial COM states from which the robot can recover or maintain balance through different strategies. In each test, the force perturbations were applied with the simulation time-step duration of 8 ms and had manually-tuned magnitudes such that the initial (i.e., post-impact) COM X-velocities approached each stability boundary at $r¯=[0.00.2]T$⁠, corresponding to the reference joint configurations, for the ankle and hip strategies and at slightly varied $r¯=[0.020.2]T$ for the stepping strategy. In order to evaluate the stability only in the sagittal plane, the bounding objects of both feet were widened laterally, and the torque and velocity limits of out-of-plane DOFs (hip yaw, hip roll, and ankle roll) were substantially increased in the simulation environment. The control parameter values were K_D,ankle = 0.5 and $KP,ankleIMU=1.0$ for the ankle subcontroller; $Δqhipbb=0.7rad$⁠, $Δrpelvis,xbb=−0.07m$⁠, T_H1 = 0.2 s, and T_H2 = 0.4 s for the hip subcontroller; and r^bb₂ = [0.1 0.04]^T, $Φ2,S1bb=π/6rad$⁠, $Φ2,S2bb=π/12rad$⁠, T_S1 = 0.2 s, and T_S2 = 0.1 s for the captured stepping subcontroller.

When the perturbed initial COM state was within the ankle-strategy balanced basin (Fig. 6: Point A), only the ankle subcontroller was engaged to maintain balance. For the perturbed initial COM state outside of the ankle-strategy but within the hip-strategy balanced basin (Fig. 6: Point B), the hip subcontroller was activated and successfully maintained the balance of the robot. This comparison shows the benefit of actively incorporating the balanced state boundaries for different strategies into control.

The captured stepping-strategy control was demonstrated when the initial COM state with $x¯˙=0.31m/s$ was between the hip-strategy balanced state and whole-body 1-step capturable (with q_upper(t) = 0$∀t$⁠) boundaries (Fig. 6: Point C). The ankle and hip subcontrollers remained engaged with the gains of the ankle subcontroller set to K_D,ankle = 0.5 and $KP,ankleIMU=0.0$ and the parameters of the hip subcontroller set to $Δqhipbb=0.56rad$ and $Δrpelvis,xbb=−0.02m$⁠. In this case, the hip subcontroller was triggered twice before achieving a captured state. From this initially unbalanced COM state, the robot successfully came to a complete stop (i.e., captured) at the pelvis X-position of $Δrxdesired/2$ after making the desired step, indicating recovery of balance. This again verifies that the captured stepping strategy has an increased stabilizing capability than that of the hip strategy alone [27], in consistency with the difference in size between their respective basins. Note that the exit of the trajectory outside of the whole-body capturable boundary for some duration does not reflect the falling of those states due to their conditions such as joint and system (especially the swing foot) states and contact status (SS to DS) that are different from those of the boundary states. The COM states at and after the stepped instant lie within the DS balanced basin for the desired step length, verifying that the imposition of the corresponding COM state constraint ensures that the robot eventually achieves a captured state.

On the other hand, the robot failed to recover balance from the initial COM state (Fig. 6: Point D) that is outside of the whole-body 1-step capturable boundary with the same controller action, demonstrating the validity of the computed boundary. Similar to the captured case, the brief penetration of the trajectory inside the whole-body capturable boundary does not reflect the capturability of those states due to their system and contact conditions that are different from those of the boundary states. It should also be noted that this COM state is well within the LIPM-based capturable boundary, which indicates the inadequacy of balance criteria based on reduced-order dynamics. These results show the merit of the whole-body formulations in the accuracy of the boundary, due to the incorporation of system-specific aspects such as kinematic and actuation limits, linear and angular momenta, and swing leg dynamics.

Assumptions and conditions in the simulation environment, such as the allowance of minimal foot tipping and the use of constant torque and velocity limits, may have increased the balance capability of the robot in simulations relative to that in the computed stability basin results. In particular, due to the difficulty of maintaining full foot contact in the simulation environment, some degree of foot tipping occurred and was followed by small oscillations that converged to a static equilibrium with full contact at the end (Fig. 6). However, in the computed balanced basins, full foot-ground contact was assumed for all times. On the other hand, other factors, such as feedback characteristics (e.g., settling time), may have reduced the capability of the robot in simulations. These modeling discrepancies were kept to a minimum and monitored to limit their effects on the push recovery experiments, such that the simulated results reasonably demonstrate and validate the proposed controller as well as the computed boundaries. Moreover, the foot tipping during balancing illustrates that the ZMP condition is insufficient as a balance criterion because it was violated even when the state was balanced or 0-step capturable.

The capturability formulated with whole-body system properties and dynamics of a humanoid robot provided valid criteria for balance recovery by stepping, and the computed boundaries were used to quantify the relative contributions of different balancing strategies and contacts. The incorporation of the 1-step capturable boundary, along with the balanced state boundaries, as explicit criteria into the proposed partition-aware controller demonstrated their validity in fully exploiting a humanoid robot’s balancing capability in push recovery simulations. Although the boundary may be relatively conservative due to the simplified step impact dynamics used for this preliminary result, it outperformed that derived from the reduced-order LIPM overall. The proposed approach can be extended to balance control for general bipedal tasks beyond standing push recovery. For instance, the 1-step capturable boundary can be applied iteratively to analyze or control N steps in walking. The relatively high computational cost due to the whole-body system will be addressed in a future study by approximating the stability boundaries from pre-computed results for selected configurations.

This work was supported in part by the U.S. National Science Foundation (CMMI-1436636 and IIS-1427193) and a Mitsui USA Foundation scholarship.

There are no conflicts of interest.

The authors attest that all data for this study are included in the paper.