Full-Body Tracking Berbasis OpenCV dan MediaPipe untuk Interaksi Objek Virtual - Putricia Hendra, Ria Amelia Shinta; Prapatoni, Velian; Arryanto, Bahiskara Ananda; Sari, Anggraini Puspita (Journal of Multidisciplinary Inquiry in Science, Technology and Educational Research, 2024)
This research introduces a system that tracks full body movements in real-time to interact with virtual objects by combining OpenCV and MediaPipe in the Unity3D game engine. The system aims to overcome the drawbacks of current tracking solutions, which typically need unique hardware and are complicated, thus restricting their usability. The suggested method uses OpenCV for capturing and preparing images, while MediaPipe Pose is chosen for its precise and efficient real-time body landmark detection. The information on the user’s body position is sent to Unity3D through a named pipe system, allowing accurate management of a 3D character’s actions. Tests on two devices with varying hardware specifications indicated that the system successfully monitors body positions and movements in real-time, enabling interactive interaction with virtual objects in Unity3D. An examination of performance indicated that the processing speed, accuracy of landmark detection, and frame rate are all notably affected by the hardware specifications, especially the processor and GPU. Devices with higher specifications significantly offered a more seamless and speedy user experience. The study suggests that merging OpenCV and MediaPipe provides a precise and effective method for tracking full body movements, suitable for different interactive settings like virtual and augmented reality.