Stochastic Multiple Shooting Trajectory Optimization via Sequential Local Policy Evaluation
2026-08-04 • Robotics
Robotics
AI summaryⓘ
The authors introduce a new method that improves how robots plan their movements over longer periods by breaking the problem into shorter parts connected by feedback controls. This approach makes it easier and faster to meet end goals, especially when the robot's behavior model is uncertain or unknown. They also developed a way to estimate how the system reacts to changes without needing detailed math models. Their tests on tasks like balancing a pole and landing a drone show their method works better than existing ones in finding good solutions with fewer tries.
Stochastic optimizationModel Predictive Path Integral (MPPI)Multiple shootingTrajectory optimizationTerminal constraintsReinforcement learningBlack-box dynamicsJacobian estimationUnderactuated systemsNonlinear control
Authors
Ashwin Gupta, Joseph Moore
Abstract
Stochastic single shooting trajectory optimization methods such as Model Predictive Path Integral control (MPPI) have been widely adopted in robotics due to their ability to reason about probabilistic dynamics and provide solutions where model gradients are noisy, costly to evaluate, or unavailable. However, satisfaction of terminal constraints when shooting over long action sequences is often sample inefficient, requiring a large number of iterations for convergence. In this paper, we present a stochastic multiple shooting method that optimizes short control action sequences connected via local feedback policies to improve sample efficiency and convergence to a terminal set. Additionally, we show that we are able to synthesize approximate system Jacobians purely from rollouts, making the method suitable for model-based reinforcement learning with black-box dynamics. We demonstrate the algorithm has improved sample efficiency and terminal set convergence for three nonlinear, underactuated optimization problems: a classic cartpole swingup task with analytical dynamics, a cartpole swingup task with learned neural network dynamics, and a VTOL quadplane performing a high angle-of-attack, precision post-stall landing maneuver.