Methods for implementing quadrotors autonomy based on hybrid learning methods

I.P. Ramyk, Ya.M. Linder

Abstract


This paper reviews and analyzes methods for achieving quadcopter autonomy. It shows disadvantages and lim itations of the classical "Perception-Planning-Control" pipeline. A fundamental limitation of this approach is the inability of mathematical models to take into account all complex effects of the unpredictable environment. In return, the application of machine learning algorithms enables the implementation of control agents based on experience of interactions with real or simulated environments, significantly improving system adaptability to non-standard conditions. The core of this work compares machine learning methods applied to quadcopter au tonomy task. It provides a detailed overview of reinforcement learning. It is shown that model-free algorithms are able to outperform professional human pilots in specific tasks. However, they require significant amounts of data and training time. In return, model-based reinforcement learning improves training efficiency. During the training, the agent learns a world model that can be used to predict environment dynamics. The article also explores imitation learning and derived methods. An effective approach is to sequentially apply imitation learn ing and reinforcement learning, which combines the strengths of both approaches. The paper reviews works relying on physics-informed methods using differentiable simulators. Differentiable simulators are used to cal culate loss function gradients relative to control parameters. All discussed methods are analyzed regarding data efficiency, computational resource requirements, and fundamental limitations. The analysis results can be used to select quadcopter control architectures based on available computational resources and specific task require ments.

Problems in programming 2025; 4: 53-62


Keywords


quadrotors; autonomous flight; machine learning; reinforcement learning; imitation learning; companion computer; flight controller, differentiable simulation

References


Lukash Y., Prystavka P. A research platform for vision-based UAV autonomy: Architecture and implementation. Fourth International Conference on Cyber Hygiene & Conflict Management in Global Information Networks (CHCM 2025). 2025. Vol. 4024. pp. 250-259.

Faessler M., Fontana F., Forster C., Scaramuzza D. Automatic Re-Initialization and Failure Recovery for Aggressive Flight with a Monocular Vision-Based Quadrotor. 2015 IEEE International Conference on Robotics and Automation (ICRA). 2015. pp. 1722–1729.

Companion Computers | PX4 Guide (main). PX4 Documentation.

Ciccone F., Ceruti A. Real-Time Search and Rescue with Drones: A Deep Learning Approach for Small-Object Detection Based on YOLO. Drones. 2025. Vol. 9, no. 8. 514 P.

Jain R., Jones C., Lucas R., Siddiqui H. Autonomous Aerial Drone with Infrared Depth Tracking. University of Central Florida. 2020.

Liu X., Nardari G. V., Cladera Ojeda F., Tao Y., Zhou A., Donnelly T., Qu C., Chen S. W., Romero R. A. F., Taylor C. J., Kumar V. Large-scale Autonomous Flight with Real time Semantic SLAM under Dense Forest Canopy. arXiv. 2021.

Daspan A., Nimsongprasert A., Srichai P., Wiengchand P. Implementation of Robot Operating System in Raspberry Pi 4 for Autonomous Landing Quadrotor on ArUco Marker. International Journal of Mechanical Engineering and Robotics Research. 2023. Vol. 12, no. 4. pp. 210–217.

Xiao J., Zhang R., Zhang Y., Feroskhan M. Vision-based Learning for Drones: A Survey. arXiv preprint. 2023. arXiv:2312.05019.

George A., Koivumäki N., Hakala T., Suomalainen J., Honkavaara E. Visual-Inertial Odometry Using High Flying Altitude Drone Datasets. Drones. 2023. Vol. 7, no. 1. 36 P.

Richter C., Bry A., Roy N. Polynomial trajectory planning for aggressive quadrotor flight in dense indoor environments. In: Masayuki Inaba, Peter Corke (eds.) Robotics Research. The 16th International Symposium ISRR, 16–19 December 2013, Singapore. Springer Tracts in Advanced Robotics, vol. 114. Cham: Springer, 2016. pp. 649–666.

Mamo M. B. Trajectory tracking control of quadcopter by designing Third order SMC Controller. Global Scientific Journals. 2020. Vol. 8, no. 9. pp. 2100–2108.

Abu Ihnak M. S., Edardar M. M. Comparing LQR and PID Controllers for Quadcopter Control Effectiveness and Cost Analysis. 2023 International Conference on Systems and Control (ICSC). IEEE, 2023. pp. 770–775.

Mnih V., Kavukcuoglu K., Silver D., Rusu A. A., Veness J., Bellemare M. G., Graves A., Riedmiller M., Fidjeland A. K., Ostrovski G., Petersen S., Beattie C., Sadik A., Antonoglou I., King H., Kumaran D., Wierstra D., Legg S., Hassabis D. Human-level control through deep reinforcement learning. Nature. 2015. Vol. 518, no. 7540. pp. 529–533.

Olivares D., Fournier P., Vasishta P., Marzat J. Model-Free versus Model-Based Reinforcement Learning for Fixed-Wing UAV Attitude Control Under Varying Wind Conditions. arXiv preprint. 2024. arXiv:2409.17896.

Kaufmann E., Bauersfeld L., Loquercio A., Müller M., Koltun V., Scaramuzza D. Champion-level drone racing using deep reinforcement learning. Nature. 2023. Vol. 620, no. 7976. pp. 982–987.

Mengozzi S. Learning Agile Flight Using Massively Parallel Deep Reinforcement Learning: Master’s thesis. University of Bologna, Department of Electrical, Electronic, and Information Engineering "Guglielmo Marconi", 2022. 67 P.

Arranz R., Carramiñana D., de Miguel G., Besada J. A., Bernardos A. M. Application of Deep Reinforcement Learning to UAV Swarming for Ground Surveillance. Sensors. 2023. Vol. 23, no. 21. 8766 P.

Romero A., Shenai A., Geles I., Aljalbout E., Scaramuzza D. Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight. arXiv preprint. 2025. arXiv:2501.14377.

Chen D., Zhou B., Koltun V., Krähenbühl P. Learning by Cheating. arXiv preprint. 2019. arXiv:1912.12294.

Pfeiffer C., Wengeler S., Loquercio A., Scaramuzza D. Visual Attention Prediction Improves Performance of Autonomous Drone Racing Agents. arXiv preprint. 2022. arXiv:2201.02569.

Kaufmann E., Loquercio A., Ranftl R., Müller M., Koltun V., Scaramuzza D. Deep Drone Acrobatics. Robotics: Science and Systems (RSS), 2020.

Abusadeh R. Evaluation of Imitation Learning with Reinforcement Learning-Based Fine Tuning for Different Control Tasks. Master’s thesis. Czech Technical University in Prague, Faculty of Electrical Engineering, Department of Cybernetics, 2025.

Xing J., Romero A., Bauersfeld L., Scaramuzza D. Bootstrapping Reinforcement Learning with Imitation for Vision-Based Agile Flight. arXiv preprint. 2024. arXiv:2403.12203.

Nahrendra I. M. A., Tirtawardhana C., Yu B., Lee E. M., Myung H. Retro-RL: Reinforcing Nominal Controller With Deep Reinforcement Learning for Tilting-Rotor Drones. arXiv preprint. 2022. arXiv:2207.03124.

Zhang R., Zhang D., Mueller M. ProxFly: Robust Control for Close Proximity Quadcopter Flight via Residual Reinforcement Learning. arXiv preprint. 2024. arXiv:2409.13193.

Blukis V., Huber S., Wrona K. K., Büchler D., Muehlebach M., Krause A., Hanna J. P., Hennan D. D., Ansari S. S. P. Learning on the Fly: Rapid Policy Adaptation via Differentiable Simulation. arXiv preprint. 2025. arXiv:2508.21065.

Schnell P., Thuerey N. Stabilizing Backpropagation Through Time to Learn Complex Physics. International Conference on Learning Representations (ICLR), 2024. arXiv:2405.02041.

Ren J., YuC., Chen S., Ma X., Pan L., Liu Z. DiffMimic: Efficient Motion Mimicking with Differentiable Physics. arXiv preprint. 2023. arXiv:2304.03274.

Heeg J., Song Y., Scaramuzza D. Learning Quadrotor Control From Visual Features Using Differentiable preprint. Simulation. arXiv 2024. arXiv:2410.15979.


Refbacks

  • There are currently no refbacks.