Full-Body Tracking Berbasis OpenCV dan MediaPipe untuk Interaksi Objek Virtual
Article Sidebar
Main Article Content
Abstract
This research introduces a system that tracks full body movements in real-time to interact with virtual objects by combining OpenCV and MediaPipe in the Unity3D game engine. The system aims to overcome the drawbacks of current tracking solutions, which typically need unique hardware and are complicated, thus restricting their usability. The suggested method uses OpenCV for capturing and preparing images, while MediaPipe Pose is chosen for its precise and efficient real-time body landmark detection. The information on the user's body position is sent to Unity3D through a named pipe system, allowing accurate management of a 3D character's actions. Tests on two devices with varying hardware specifications indicated that the system successfully monitors body positions and movements in real-time, enabling interactive interaction with virtual objects in Unity3D. An examination of performance indicated that the processing speed, accuracy of landmark detection, and frame rate are all notably affected by the hardware specifications, especially the processor and GPU. Devices with higher specifications significantly offered a more seamless and speedy user experience. The study suggests that merging OpenCV and MediaPipe provides a precise and effective method for tracking full body movements, suitable for different interactive settings like virtual and augmented reality.
Article Details
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
References
Anthes, C., García-Hernández, R. J., Wiedemann, M., & Kranzlmüller, D. (2016). State of the art of virtual reality technology. In 2016 IEEE Aerospace Conference (pp. 1-19). Big Sky, MT, USA: IEEE. https://doi.org/10.1109/AERO.2016.7500674
Aristidou, A., Lasenby, J., Chrysanthou, Y., & Shamir, A. (2018). Inverse kinematics techniques in computer graphics: A survey. Computer Graphics Forum, 37(6), 35-58. https://doi.org/10.1111/cgf.13310
Chung, J.-L., Ong, L.-Y., & Leow, M.-C. (2022). Comparative analysis of skeleton-based human pose estimation. Future Internet, 14(380). https://doi.org/10.3390/fi14120380
Clark, R. A., Pua, Y., Bryant, K., Hunt, P., & Clark, M. M. (2013). Validity and reliability of the Microsoft Kinect for providing lateral trunk lean feedback during gait retraining. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 21(4), 567-574. https://doi.org/10.1109/TNSRE.2013.2260472
Dewi, C., Chen, R.-C., Jiang, X., & Yu, H. (2022). Adjusting eye aspect ratio for strong eye blink detection based on facial landmarks. PeerJ Computer Science, 8, e943. https://doi.org/10.7717/peerj-cs.943
Ganeshsar. (2023). Multithreaded Unity Python MediaPipe Body/Pose. GitHub. https://github.com/ganeshsar/UnityPythonMediaPipeBodyPose
Josyula, R. (2021). Human pose estimation (Studi Independen Musim Panas). Northeastern University, Electrical and Computer Engineering Department, Augmented Cognition Laboratory (ACLab).
Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., ... & Chang, M. (2019). MediaPipe: A framework for building perception pipelines. Google Research. arXiv preprint arXiv:1906.08172.
Makahaube, S. S., Sambul, A. M., & Sompie, S. R. U. (2021). Implementation of gesture recognition technology for automated education service kiosk. Universitas Sam Ratulangi, 16(4), 465–472.
Merwan, C. (2024, 27 Januari). Named pipes in .NET (C#). Medium. https://medium.com/codenx/named-pipes-in-net-c-c0459e165371
Pranav, D. (2022, 1 Maret). MediaPipe: The ultimate guide to video processing. LearnOpenCV. https://learnopencv.com/introduction-to-mediapipe/
Rasyid, F., Mustafa, M., Mahersatillah, A., Rizal, M., Mushaf, M., & Arifin, A. (2023). Deteksi mata di video smartphone menggunakan mediapipe python. JOINTECS (Journal of Information Technology and Computer Science), 8(2), 49-56. https://doi.org/10.31328/jointecs.v8i2.4562
Sari, A. P., Suzuki, H., Kitajima, T., Yasuno, T., Prasetya, D. A., & Nachrowie. (2020). Prediction model of wind speed and direction using convolutional neural network - long short term memory. In 2020 IEEE International Conference on Power and Energy (PECon) (pp. 356-361). https://doi.org/10.1109/PECon48942.2020.9314474
Ullah, K., Ahmed, I., Ahmad, M., & Khan, I. (2019). Comparison of person tracking algorithms using overhead view implemented in OpenCV. In 2019 9th Annual Information Technology, Electromechanical Engineering and Microelectronics Conference (IEMECON) (pp. 15-20). Jaipur, India: IEEE. https://doi.org/10.1109/IEMECONX.2019.8877025
Vetter, M. (2019). A pattern for client/server communication via Named Pipes via C# [Kode komputer]. GitHub. https://github.com/malcomvetter/NamedPipes
Zulkhaidi, T. C. A.-S., Maria, E., & Yulianto, Y. (2020). Pengenalan pola bentuk wajah dengan OpenCV. Jurnal Rekayasa Teknologi Informasi, 3(2), 181. Samarinda: Politeknik Pertanian.