Labeling and segmentation of treepoint cloud in immersive virtual reality
Summary
Effective data annotation is crucial for the advancement of machine learning techniques for
autonomous labeling of point cloud datasets for environmental studies and other fields.
Traditional point cloud annotation workflows utilize 3D datasets on a 2D screen, which
degrades the user experience and negatively impacts performance. Concepts like embodied
digital twins and virtual reality may offer an alternative approach for annotating and exploring
datasets in 3D. However, there is a lack of research on determining which annotation
approaches offer the most optimal solution for labeling and segmenting components of
individual tree point clouds. Additionally, only a few studies refer to users’ evaluations of
usability and workload and compare different annotation approaches, as well as examine the
influence of participants individual differences. To address this gap, game engine technologies
were used to recreate a virtual annotating environment and an interactive experience to capture
the users’ experience and performance on two commonly used annotation approaches and their
combinations.
The first annotating method, namely the Sphere Selection approach, offered a 3D pointer with
adapting shapes for free-style detailed point selection. The second method, namely the
Container Creation approach, offered a solution based on control points and nodes for polygon
creation, offering a quick approach for massive point selection and annotation. The
combination of the two methods formed the third approach, namely, the combination of the
two approaches. The results indicate that the sphere selection approach is optimal in terms of
time and accuracy for annotating components of individual trees. Participants reported that this
approach was perceived as more effective, enjoyable, and the least physically and mentally
demanding annotation technique compared to the other two. The findings of this research
highlighted the potential of each approach for future environmental applications for labeling
3D point cloud data in virtual reality.
The findings of this research contribute to field data annotation and highlight the potential for
immersive technologies in environmental studies. Future researchers can focus on further
investigation of the potential of each annotation approach for labeling and segmenting
individual tree point clouds and further investigation of participant’s individual differences on
performance and usability. Finally, the findings of this thesis could be a starting point for the
design criteria for 3D VR tree annotation tools and applications and the participant application
usability study.