Urban flooding is among the costliest natural disasters worldwide. Timely and effective rescue path planning is crucial for minimizing loss of life and property. However, current research on path planning often fails to adequately consider the need to assess area risk uncertainties and bypass complex obstacles in flood rescue scenarios, presenting significant challenges for developing optimal rescue paths. This study proposes a deep reinforcement learning (RL) algorithm incorporating four main mechanisms to address these issues. Dual-priority experience replays and backtrack punishment mechanisms enhance the precise estimation of area risks. Concurrently, random noisy networks and dynamic exploration techniques encourage the agent to explore unknown areas in the environment, thereby improving sampling and optimizing strategies for bypassing complex obstacles. The study constructed multiple grid simulation scenarios based on real-world rescue operations in major urban flood disasters. These scenarios included uncertain risk values for all passable areas and an increased presence of complex elements, such as narrow passages, C-shaped barriers, and jagged paths, significantly raising the challenge of path planning. The comparative analysis demonstrated that only the proposed algorithm could bypass all obstacles and plan the optimal rescue path across nine scenarios. This research advances the theoretical progress for urban flood rescue path planning by extending the scale of scenarios to unprecedented levels. It also develops RL mechanisms adaptable to various extremely complex obstacles in path planning. Additionally, it provides methodological insights into artificial intelligence to enhance real-world risk management.

Traditional spacecraft attitude control often relies heavily on the dimension and mass information of the spacecraft. In active debris removal scenarios, these characteristics cannot be known beforehand because the debris can take any shape or mass. Additionally, it is not possible to measure the mass of the combined system of satellite and debris object in orbit. Therefore, it is crucial to develop an adaptive satellite attitude control that can extract mass information about the satellite system from other measurements. The authors propose using deep reinforcement learning (DRL) algorithms, employing stacked observations to handle widely varying masses. The satellite is simulated in Basilisk software, and the control performance is assessed using Monte Carlo simulations. The results demonstrate the benefits of DRL with stacked observations compared to a classical proportional-integral-derivative (PID) controller for the spacecraft attitude control. The algorithm is able to adapt, especially in scenarios with changing physical properties.

With the development of science and technology and economy, UAV is used more and more widely. However, the existing UAV trajectory planning methods have the limitations of high cost and low intelligence. In view of this, grey Wolf algorithm is being used to achieve collaborative trajectory optimization of UAV groups. However, it is found that the Grey Wolf optimization algorithm (GWO) has the problem of weak cooperation. In this study, based on the traditional GWO pheromone factor is introduced to improve it.. Aiming at the problem of unstable performance of swarm intelligence optimization algorithm under dynamic threat, deep reinforcement learning is used to optimize the model. An unmanned aerial vehicle swarm trajectory planning model was constructed based on the improved grey wolf algorithm. Through experimental analysis, the optimal fitness value of the improved grey wolf algorithm was lower than 0.43 of the grey wolf algorithm. Compared with other algorithms, the fitness value of this algorithm is significantly reduced and the stability is higher. In complex scenarios, the improved grey wolf algorithm had a trajectory length of 70.51 km and a planning time of 5.92 s, which was clearly superior to other algorithms. The path length planned by the research and design model was 58.476 km, which was significantly smaller than the other three models. The planning time was 5.33 s and the number of path extension points was 46. The indicator values of the Unmanned Aerial Vehicle swarm trajectory planning model designed by this research were all smaller than the other three models. By analyzing the results, the model can achieve low-cost trajectory optimization, providing more reasonable technical support for unmanned aerial vehicle mission execution.

Robotic Mobile Fulfillment Systems (RMFSs) face challenges in handling large-scale orders and navigating complex environments, frequently encountering a series of intricate decision-making problems, such as order allocation, shelf selection, and robot scheduling. To address these challenges, this paper integrates Deep Reinforcement Learning (DRL) technology into an RMFS, to meet the needs of efficient order processing and system stability. This study focuses on three key stages of RMFSs: order allocation and sorting, shelf selection, and coordinated robot scheduling. For each stage, mathematical models are established and the corresponding solutions are proposed. Unlike traditional methods, DRL technology is introduced to solve these problems, utilizing a Genetic Algorithm and Ant Colony Optimization to handle decision making related to large-scale orders. Through simulation experiments, performance indicators-such as shelf access frequency and the total processing time of the RMFS-are evaluated. The experimental results demonstrate that, compared to traditional methods, our algorithms excel in handling large-scale orders, showcasing exceptional superiority, capable of completing approximately 110 tasks within an hour. Future research should focus on integrated decision-making modeling for each stage of RMFSs and designing efficient heuristic algorithms for large-scale problems, to further enhance system performance and efficiency.

Edge servers frequently manage their own offline digital twin (DT) services, in addition to caching online digital twin services. However, current research often overlooks the impact of offline caching services on memory and computation resources, which can hinder the efficiency of online service task processing on edge servers. In this study, we concentrated on service caching and task offloading within a collaborative edge computing system by emphasizing the integrated quality of service (QoS) for both online and offline edge services. We considered the resource usage of both online and offline services, along with incoming online requests. To maximize the overall QoS utility, we established an optimization objective that rewards the throughput of online services while penalizing offline services that miss their soft deadlines. We formulated this as a utility maximization problem, which was proven to be NP-hard. To tackle this complexity, we reframed the optimization problem as a Markov decision process (MDP) and introduced a joint optimization algorithm for service caching and task offloading by leveraging the deep Q-network (DQN). Comprehensive experiments revealed that our algorithm enhanced the utility by at least 14.01% compared with the baseline algorithms.

Portfolio management (PM) is a popular financial process that concerns the occasional reallocation of a particular quantity of capital into a portfolio of assets, with the main aim of maximizing profitability conditioned to a certain level of risk. Given the inherent dynamicity of stock exchanges and development for long-term performance, reinforcement learning (RL) has become a dominating solution for solving the problem of portfolio management in an automated and efficient manner. Nevertheless, the present RL-based PM methods just take into account the variations in prices of portfolio assets and the implications of price variations, while overlooking the significant relationships among different assets in the market, which are extremely valuable for managerial decisions. To close this gap, this paper introduces a novel deep model that combines two subnetworks; one to learn a temporal representation of historical prices using a refined temporal learner, while the other learns the relationships between different stocks in the market using a relation graph learner (RGL). Then, the above learners are integrated into the curriculum RL scheme for formulating the PM as a curriculum Markov Decision Process, in which an adaptive curriculum policy is presented to enable the agent to adaptively minimize risk value and maximize cumulative return. Proof-of-concept experiments are performed on data from three public stock indices (namely S&P500, NYSE, and NASDAQ), and the results demonstrate the efficiency of the proposed framework in improving the portfolio management performance over the competing RL solutions.

The reef ecosystem plays a vital role as a habitat for fish species with limited swimming capabilities, serving not only as a sanctuary and food source but also influencing their behavioral tendencies. Understanding the intricate mechanism through which fish adeptly navigate the moving targets within reef environments within complex water flow, all while evading obstacles and maintaining stable postures, has remained a challenging and prominent subject in the realms of fish behavior, ecology, and biomimetics alike. An integrated simulation framework is used to investigate fish predation problems within intricate environments, combining deep reinforcement learning algorithms (DRL) with high-precision fluid-structure interaction numerical methods-immersed boundary lattice Boltzmann method (lB-LBM). The Soft Actor-Critic (SAC) algorithm is used to improve the intelligent fish\'s capacity for random exploration, tackling the multi-objective sparse reward challenge inherent in real-world scenarios. Additionally, a reward shaping method tailored to its action purposes has been developed, capable of capturing outcomes and trend characteristics effectively. The convergence and robustness advantages of the method elucidated in this paper are showcased through two case studies: one addressing fish capturing randomly moving targets in hydrostatic flow field, and the other focusing on fish counter-current foraging in reef environments to capture drifting food. A comprehensive analysis was conducted of the influence and significance of various reward types on the decision-making processes of intelligent fish within intricate environments.

As autonomous driving may be the most important application scenario of the next generation, the development of wireless access technologies enabling reliable and low-latency vehicle communication becomes crucial. To address this, 3GPP has developed Vehicle-to-Everything (V2X) specifications based on 5G New Radio (NR) technology, where Mode 2 Side-Link (SL) communication resembles Mode 4 in LTE-V2X, allowing direct communication between vehicles. This supplements SL communication in LTE-V2X and represents the latest advancements in cellular V2X (C-V2X) with the improved performance of NR-V2X. However, in NR-V2X Mode 2, resource collisions still occur and thus degrade the age of information (AOI). Therefore, an interference cancellation method is employed to mitigate this impact by combining NR-V2X with Non-Orthogonal multiple access (NOMA) technology. In NR-V2X, when vehicles select smaller resource reservation intervals (RRIs), higher-frequency transmissions use more energy to reduce AoI. Hence, it is important to jointly considerAoI and communication energy consumption based on NR-V2X communication. Then, we formulate such an optimization problem and employ the Deep Reinforcement Learning (DRL) algorithm to compute the optimal transmission RRI and transmission power for each transmitting vehicle to reduce the energy consumption of each transmitting vehicle and the AoI of each receiving vehicle. Extensive simulations demonstrate the performance of our proposed algorithm.

This study investigates the dynamic deployment of unmanned aerial vehicles (UAVs) using edge computing in a forest fire scenario. We consider the dynamically changing characteristics of forest fires and the corresponding varying resource requirements. Based on this, this paper models a two-timescale UAV dynamic deployment scheme by considering the dynamic changes in the number and position of UAVs. In the slow timescale, we use a gate recurrent unit (GRU) to predict the number of future users and determine the number of UAVs based on the resource requirements. UAVs with low energy are replaced accordingly. In the fast timescale, a deep-reinforcement-learning-based UAV position deployment algorithm is designed to enable the low-latency processing of computational tasks by adjusting the UAV positions in real time to meet the ground devices\' computational demands. The simulation results demonstrate that the proposed scheme achieves better prediction accuracy. The number and position of UAVs can be adapted to resource demand changes and reduce task execution delays.

The segmented mirror co-phase error identification technique based on supervised learning methods has the advantages of simple application conditions, no dependence on custom sensors, a fast calculation speed, and low computing power requirements compared with other methods. However, it is often difficult to obtain a high accuracy in practical application situations with this method because of the difference between the training model and the actual model. The reinforcement learning algorithm does not need to model the real system when operating the system. However, it still retains the advantages of supervised learning. Thus, in this paper, we placed a mask on the pupil plane of the segmented telescope optical system. Moreover, based on the wide spectrum, point spread function, and modulation transfer function of the optical system and deep reinforcement learning-without modeling the optical system-a large-range and high-precision piston error automatic co-phase method with multiple-submirror parallelization was proposed. Finally, we carried out relevant simulation experiments, and the results indicate that the method is effective.