Combining LiDAR data, 360 imagery and game engines to advance forest monitoring

By Raphael Zürcher

Abstract
The evaluation of the demand in virtual reality based applications, specifically in forest based applications, is on the rise. Virtual reality applications include digital twin technology, a virtual representation serving as a real-time digital counterpart of the reality, which has been gaining increasing popularity, not only in forestry. Furthermore, including Light Detection and Ranging (LiDAR) as an emerging technology with big potential in forest management in building a digital twin, can help capture the forest very accurately, but can be described as rather abstract. These technologies, as well as drones are being developed and researched in a rapid manner, leading towards automated data acquisition. Additionally, through increased usage, technology is becoming cheaper and data is collected more frequently. These factors stimulate the demand for future developments in forest applications. With immersive virtual reality (VR) technology and game engines, such as Unity, these forestry data can be integrated in exploratory VR applications with potential in communication and collaboration. 360° images and videos can complement LiDAR data with a close to reality representation. Historically, LiDAR and forest parameter data sets have been analyzed on a 2D desktop application. The main objective was to assess whether the immersive forest experience, based on selected parameters evaluated heuristically by an expert panel, assisted in forest monitoring and management. This objective was defined due to the research gap in empirical evaluations of applications on usability, opportunities, and limitations. To achieve this objective, a Forest VR Application was developed with collected LiDAR data and immersive media. Terrestrial, airborne and mobile LiDAR data, as well as 360° images and videos were collected and forest sounds was recorded in the Eifel National Park in Germany. Measurement tools in VR on LiDAR data were integrated in a workbench, which allows the user to scale, rotate, select, translate trees or whole forests and then measure their height, diameter breast height (DBH), basal area or volume. Bing maps, as a spatial component, allows the user to explore the area surrounding the forest plot. Further, species as an indicator on biodiversity were explored with the PlantNet Species Recognition API integrated into the Forest VR Application. A heuristic expert evaluation with seven participants provided positive, but also critical feedback on the Forest VR Application’s user experience (UX), usability, ease of use and more criteria. For the measurement of tree height and DBH in VR, points were selected with the k-d tree algorithm. The tree height measurements (-3.1%) in VR resulted lower than the script extracted heights, whereas the DBH (+0.45%) measurements in VR resulted higher than the script extracted DBH. The questionnaire evaluation reveals that training, communication and education in forestry are interesting fields to implement the Forest VR Application. Additionally, multiple disciplines within forestry are of interest to advance forest monitoring with VR applications, such as thinning trainers, forest modelling, validation, sharing results or setting up a research plan. It is concluded that the exploration of the VR applications in forestry provides big benefits and fosters a collaborative future in forest management for forest stakeholders, and also the general public.

Keywords: Virtual Reality; Digital Twin; Forest Management; LiDAR; Immersive Media; Measurements; Game Engine; Heuristic Evaluation; UX