Robotická paže ovládaná lidskou rukou pomocí zpracování obrazu

Anoop Mohan Rode

Diplomová práce

Human Hand-Gesture Controlled Robotic Arm by Image Processing

Abstract:

This research presents an interdisciplinary methodology for the design, implementation and development of the robotic gesture control system integrating with computer vision, mechanical principles, and electronics circuitry. The interdisciplinary approach of the project involves the collaboration of mechanical, computer and electrical electronics domains. The circuit design architecture has generic components such as ESP32 WROOM32 Module and MG996R Servo Motor. This system architecture has been developed to obtain the precise angular control of servo motors with the help of PWM signals generated by the ESP32 module. The interfacing and configuration step of the ESP32 WROOM Module involves firmware development using Arduino IDE and leading communication with peripheral devices via UART protocol. Serial communication has been established between C++ running on ESP32 and Python script on Laptop. Computer Vision Algorithm allows us to detect and track the objects desired objects in footage. The Libraries used for gesture recognition by using MediaPipe & OpenCV Libraries. Over the pre-processed footage with the help of libraries, I have implemented a custom post-processing landmark detection technique for angular calculation via vector approach. The mechanical structure design and fabrication involves Modelling and 3D printing of the parts. After 3D printing involves the assembly of parts and calibrating with servo angular motion for operating precisely. The demonstrated methodology connects engineering principles with robotic control strategy to obtain a robust control algorithm using computer vision to operate finger movement precisely. This project can be the promising for the applications such as prosthetics Hands and HMI applications on unmanned fields.

Abstract:

This research presents an interdisciplinary methodology for the design, implementation and development of the robotic gesture control system integrating with computer vision, mechanical principles, and electronics circuitry. The interdisciplinary approach of the project involves the collaboration of mechanical, computer and electrical electronics domains. The circuit design architecture has generic components such as ESP32 WROOM32 Module and MG996R Servo Motor. This system architecture has been developed to obtain the precise angular control of servo motors with the help of PWM signals generated by the ESP32 module. The interfacing and configuration step of the ESP32 WROOM Module involves firmware development using Arduino IDE and leading communication with peripheral devices via UART protocol. Serial communication has been established between C++ running on ESP32 and Python script on Laptop. Computer Vision Algorithm allows us to detect and track the objects desired objects in footage. The Libraries used for gesture recognition by using MediaPipe & OpenCV Libraries. Over the pre-processed footage with the help of libraries, I have implemented a custom post-processing landmark detection technique for angular calculation via vector approach. The mechanical structure design and fabrication involves Modelling and 3D printing of the parts. After 3D printing involves the assembly of parts and calibrating with servo angular motion for operating precisely. The demonstrated methodology connects engineering principles with robotic control strategy to obtain a robust control algorithm using computer vision to operate finger movement precisely. This project can be the promising for the applications such as prosthetics Hands and HMI applications on unmanned fields.

Keywords

Computer Vision, ESP32 Wroom, 3D Printing, Hand Gesture Recognition, Bionic Hand Motion and Motion Control, HMI

Jazyk práce: angličtina

Datum vytvoření / odevzdání či podání práce: 14. 5. 2024

Obhajoba závěrečné práce

• Vedoucí: Ing. Miroslav Holada, Ph.D.

Technická univerzita v Liberci

Fakulta mechatroniky, informatiky a mezioborových studií

Magisterský studijní program / obor:
Mechatronics / Mechatronics