Thesis subject

MSc thesis topic: Reinforcement Learning-Based Obstacle Avoidance and Imaging Optimization for UAV Tunnel Inspection in Simulated Environments

Autonomous UAVs offer great potential for enhancing tunnel inspection by reducing manual labour and improving safety. However, tunnel environments pose unique challenges for UAV navigation and data acquisition, including unpredictable obstacles and limited visibility. While digital twin simulations have enabled the testing of flight strategies in realistic scenarios, effective autonomous operation still depends on the UAV’s ability to avoid obstacles while ensuring comprehensive image coverage.

To address these challenges, this project focuses on two key components: developing reinforcement learning-based obstacle avoidance models and optimizing imaging strategies through camera angle and overlap rate adjustments. These components are tested within a simulated tunnel environment to evaluate their performance and identify best practices for real-world inspection tasks.

Background

UAVs are increasingly used for tunnel inspection, but challenges such as confined space, dynamic obstacles, and limited GPS signal make autonomous navigation and effective imaging difficult. Rigid flight paths often fail to adapt to sudden obstacles, while over-planning can lead to inefficient or redundant image capture. Reinforcement learning offers a way to train UAVs to avoid obstacles adaptively, but its application in tunnel-like environments remains underexplored. At the same time, camera angles and image overlap rates must be carefully optimized to balance inspection quality and efficiency. This project investigates both aspects using a digital twin simulation platform.

Relevance to research/projects at GRS

UAVs are a key tool for geoinformation data acquisition and widely used at GRS. The tools developed here are also relevant in areas of poor GPS reception outdoors.

Objectives and Research questions

Objectives, choose from:

  • To implement and train reinforcement learning algorithms (e.g., PPO, DDPG) for obstacle avoidance in tunnel-like simulated environments, and evaluate their performance under randomized obstacle scenarios.
  • To examine the influence of camera angle and image overlap rate on visual coverage and data redundancy during UAV-based tunnel inspection missions.
  • To propose optimized combinations of navigation strategies and imaging parameters that improve inspection efficiency while maintaining obstacle avoidance safety margins in simulation.

Potential research questions:

  • How can reinforcement learning be effectively applied to train UAVs for reliable obstacle avoidance in randomly generated tunnel scenarios?
  • What imaging strategies—specifically in terms of camera orientation and overlap rate—are most effective for ensuring comprehensive tunnel inspection coverage?
  • How do navigation strategies and imaging configurations interact to influence inspection accuracy and operational efficiency in simulated tunnel environments?

Requirements

  • Ability to work in Python or C++.
  • Have a foundation in Machine Learning (ML) and Deep Learning (DL) and be familiar with at least one major framework (TensorFlow or PyTorch).
  • Understand the core concepts of Reinforced Learning (RL) and be able to apply common RL algorithms (e.g. PPO) for implementation and training.
  • Ability to set up and experiment with simulation environments using or quickly learning AirSim and Unreal Engine.

Literature and information
Expected reading list before starting the thesis research

  • Assis, L. S., Daud Filho, A. C., Rocha, L., Vivaldini, K. C. T., Caurin, G. A. D. P., & Futai, M. M. Innovations in Tunnel Inspection Using Drones and Digital Twins for Geometric Survey. Available at SSRN 4798917.
  • Dong, Y. (2024). The design of autonomous uav prototypes for inspecting tunnel construction environment. arXiv preprint arXiv:2408.07286.
  • Li, J., Huang, D., & Yang, P. (2018). Inspection method of images' overlap of UAV photogrammetry based on features matching. In MATEC Web of Conferences (Vol. 173, p. 02022). EDP Sciences.
  • Xu, Z., Chen, B., Zhan, X., Xiu, Y., Suzuki, C., & Shimada, K. (2023). A vision-based autonomous UAV inspection framework for unknown tunnel construction sites with dynamic obstacles. IEEE Robotics and Automation Letters, 8(8), 4983-4990.
  • Xu, Z., Han, X., Shen, H., Jin, H., & Shimada, K. (2025). Navrl: Learning safe flight in dynamic environments. IEEE Robotics and Automation Letters.
  • Zhang, Y., & Wang, H. (2021, September). Adaptive interfered fluid dynamic system algorithm based on deep reinforcement learning framework. In International Conference on Autonomous Unmanned Systems (pp. 1388-1397). Singapore: Springer Singapore

Theme(s): Sensing & measuring; Modelling & visualisation