Proceeding of the Faculty of technical Sciences, Novi Sad

Engineering Animation

Vol. 40 No. 05 (2025): Proceedings of the Faculty of Technical Sciences

USE OF COMPUTER VISION METHODS FOR USER INTERFACE CONTROL OF INTERACTIVE SOFTWARE

  • Danijela Milekić
DOI:
https://doi.org/10.24867/30SA05Milekic
Submitted
May 9, 2025
Published
2025-12-10

Abstract

This paper investigates the application of computer vision to control the user interface, specifically the input devices of a computer, and management of a game created in Unreal Engine. Python, MediaPipe, and PyAutoGUI libraries were used. The controls are implemented in real-time through communication between a Python script and the game engine.

References

  1. [1] Dragan Micić, Tončić Danilo – "Lokalizacija na osnovu markera", https://esveske.github.io/pdf/2017/PFE1701.pdf
  2. [2] Python Documentation – https://docs.python.org/3/tutorial/
  3. [3] Pajankar, Ashwin – "Naučite Python 3", Agencija Eho, 2022
  4. [4] PyAutoGUI Documentation – https://pyautogui.readthedocs.io/en/latest/
  5. [5] Dattanikaran – "Get Started with PyAutoGUI", Medium
  6. [6] Google – MediaPipe Solutions Guide, https://ai.google.dev/edge/mediapipe/solutions/guide
  7. [7] MediaPipe Hands Documentation – https://mediapipe.readthedocs.io/en/latest/solutions/hands.html
  8. [8] MediaPipe Face Mesh Documentation – https://mediapipe.readthedocs.io/en/latest/solutions/face_mesh.html
  9. [9] Ushaw G., Alanazi F. – "Improving Detection of DeepFakes through Facial Region Analysis", ResearchGate
  10. [10] RisingStack – "Node Hero – Getting Started With Node.js Tutorial", https://blog.risingstack.com/node-hero-tutorial-getting-started-with-node-js/
  11. [11] Bogomyakov E. – "How to connect Unreal Engine 4 to local server via WebSocket using blueprints?", Medium