USE OF COMPUTER VISION METHODS FOR USER INTERFACE CONTROL OF INTERACTIVE SOFTWARE
DOI:
https://doi.org/10.24867/30SA05MilekicKeywords:
Computer vision, Python programming language, MediaPipe, PyAutoGUI, Unreal EngineAbstract
This paper investigates the application of computer vision to control the user interface, specifically the input devices of a computer, and management of a game created in Unreal Engine. Python, MediaPipe, and PyAutoGUI libraries were used. The controls are implemented in real-time through communication between a Python script and the game engine.
References
[1] Dragan Micić, Tončić Danilo – "Lokalizacija na osnovu markera", https://esveske.github.io/pdf/2017/PFE1701.pdf
[2] Python Documentation – https://docs.python.org/3/tutorial/
[3] Pajankar, Ashwin – "Naučite Python 3", Agencija Eho, 2022
[4] PyAutoGUI Documentation – https://pyautogui.readthedocs.io/en/latest/
[5] Dattanikaran – "Get Started with PyAutoGUI", Medium
[6] Google – MediaPipe Solutions Guide, https://ai.google.dev/edge/mediapipe/solutions/guide
[7] MediaPipe Hands Documentation – https://mediapipe.readthedocs.io/en/latest/solutions/hands.html
[8] MediaPipe Face Mesh Documentation – https://mediapipe.readthedocs.io/en/latest/solutions/face_mesh.html
[9] Ushaw G., Alanazi F. – "Improving Detection of DeepFakes through Facial Region Analysis", ResearchGate
[10] RisingStack – "Node Hero – Getting Started With Node.js Tutorial", https://blog.risingstack.com/node-hero-tutorial-getting-started-with-node-js/
[11] Bogomyakov E. – "How to connect Unreal Engine 4 to local server via WebSocket using blueprints?", Medium
