As orbital space resources become increasingly scarce, the problem of orbital pursuit-evasion games among spacecraft has attracted widespread attention. In multi-to-one pursuit-evasion scenarios, multiple pursuit spacecraft are required to cooperatively capture a non-cooperative evasive target under impulsive maneuver and fuel constraints, giving rise to an optimal control problem characterized by strong nonlinearity and complex constraints. Existing differential game methods suffer from high computational complexity and poor real-time performance; numerical optimization approaches are sensitive to initial guesses and struggle to handle long-duration constraints; and although deep reinforcement learning has demonstrated advantages in some scenarios, it lacks proactive prediction capability when confronting unknown evasion strategies, thereby limiting pursuit performance. Consequently, how to predict the actions of the evading spacecraft in multi-to-one scenarios and accordingly optimize the cooperative strategies of multiple pursuers has become a key challenge for improving pursuit efficiency in complex orbital games.
In a recent study published in Space: Science & Technology, the team led by Bai Chengchao from the School of Astronautics, Harbin Institute of Technology, proposed a pursuit strategy construction method that integrates evasive spacecraft action prediction with the Deep Deterministic Policy Gradient (DDPG) algorithm. Based on the Clohessy–Wiltshire equations, the study establishes a mathematical model for the multi-to-one impulsive orbital pursuit-evasion game. A centralized training with decentralized execution (CTDE) framework is adopted to train the pursuit policy network, in which all pursuers share network parameters and generate impulsive maneuver commands based on local observations. Simultaneously, using the global state information and the evader's true action data collected during training, a supervised learning approach is employed to train an evader action prediction network, which predicts the evader's velocity increment from the locally observable relative states of the pursuer, thereby providing supplementary information on the evader's future state for pursuit decision-making. Simulation results demonstrate that the pursuit policy integrated with action prediction converges faster and achieves higher rewards during training, with prediction errors remaining largely below 0.3 km, outperforming conventional methods such as Kalman filtering. When the number of pursuers is three, the capture success rate reaches 0.95, representing an improvement of approximately 0.3 over traditional approaches, effectively compensating for the disadvantage of insufficient pursuer numbers. This research provides an effective learning-based pursuit strategy construction method for multi-to-one orbital pursuit-evasion games, offering significant engineering reference value for enhancing autonomous decision-making capabilities in complex space confrontation scenarios.
First, this paper focuses on the multi-to-one orbital pursuit-evasion game problem and establishes a mathematical model incorporating impulsive maneuvers and fuel constraints. Against the backdrop of increasingly strained orbital resources, multiple pursuit spacecraft are required to cooperatively capture a single non-cooperative evasive spacecraft, while the evader attempts to maximize its distance from the pursuers. Based on the Clohessy–Wiltshire equations, the study develops a relative motion model for spacecraft in the Local Vertical–Local Horizontal (LVLH) frame, in which both sides execute impulsive velocity-increment maneuvers at fixed time intervals and are subject to constraints on both per-maneuver velocity increments and cumulative total velocity increments. As illustrated in Fig. 1, multiple pursuers cooperatively approach the evader through finite sequences of impulsive maneuvers, and the mission is considered successful when the distance between any pursuer and the evader falls below a preset capture threshold. In this scenario, communication range constraints exist among the pursuers, allowing information sharing only within a certain proximity. This mathematical model provides a foundational environment for subsequent policy learning.
Second, this paper proposes a pursuit strategy construction method that integrates an evader action prediction network with the Deep Deterministic Policy Gradient (DDPG) algorithm, adopting a centralized training with decentralized execution (CTDE) framework for training the policy network. As illustrated in the system architecture of Fig. 2, the observation space of each pursuer comprises the relative state with respect to the evader, the states of the two nearest pursuers within communication range, and the predicted future state of the evader at the next time step output by the prediction network, totaling 29 dimensions. The policy network maps the observation to a three-dimensional continuous impulsive velocity-increment command. All pursuers share the same policy network parameters; the network is trained using global information but relies solely on local observations during execution. The reward function is designed as the sum of a continuous reward for approaching the evader and a large constant reward for successful capture, with an additional penalty for fuel consumption. On this basis, the study additionally trains an evader action prediction network, whose input consists of the relative states of the three pursuers closest to the evader, and whose output is the predicted velocity increment of the evader. Fig. 3 presents the architectures of the policy network, the Q-value network, and the prediction network, all of which are four-layer fully connected networks; the first three layers use the ReLU activation function, while the output layer employs tanh. The prediction network is trained via supervised learning using the interaction data collected after the pursuit policy training is completed, without requiring additional data acquisition.
Finally, this paper validates the effectiveness of the proposed method through a series of simulations. Fig. 4A presents the reward curves of both the pursuer and evader policies over 200,000 training episodes; the reward trends of the two sides are generally opposite and tend to stabilize after approximately 125,000 episodes, indicating that both policies reach an approximate game equilibrium. Fig. 4B compares the training curves of the pursuit policy with and without evader action prediction; the incorporation of prediction significantly improves both the convergence speed and the reward values, and the loss curve of the prediction network converges after about 1,000 epochs. In the test cases, the pursuit–evasion trajectories under three prediction methods (the prediction network, Kalman filtering, and keeping the action of the previous time step) are presented, all of which achieve successful capture. A comparison of the position prediction errors of the three methods throughout the pursuit–evasion process shows that the error of the prediction network remains essentially below 0.3 km, with an average of 0.137 km, and gradually decreases in the later stage of the engagement, whereas the errors of the comparative methods approach 0.6 km in the later stage. In terms of statistical performance, Fig. 5 shows that when three pursuers are employed, the capture success rate of the method incorporating prediction reaches 0.95, representing an improvement of approximately 0.3 over the conventional DDPG approach; when the number of pursuers is insufficient, the prediction effectively compensates for the numerical disadvantage. Fig. 6 demonstrates that when the evader adopts a non-maneuvering strategy, the capture success rate can reach 0.99; however, performance degrades when confronting unseen random or periodic strategies, reflecting the bottleneck of generalization capability inherent in learning-based policies. The initial position distribution of the pursuers significantly affects the success rate, with an optimal initial distance of approximately 38 km and an optimal enclosure area of approximately 2,300 km². The study notes that the proposed method relies on prior action data of the evader, and future work will explore prediction methods that do not require prior data, as well as the multi-to-many pursuit–evasion game problem.
