عنوان انگلیسی مقاله:
Fingertip detection and tracking for recognition of air-writing in videos
ترجمه فارسی عنوان مقاله:
تشخیص و ردیابی اثر انگشت برای تشخیص هوا-نوشتاری در فیلم ها
Sciencedirect - Elsevier - Expert Systems With Applications, 136 (2019) 217-229: doi:10:1016/j:eswa:2019:06:034
Sohom Mukherjee a , ∗, Sk. Arif Ahmed b , Debi Prosad Dogra c , Samarjit Kar b , Partha Pratim Roy
Air-writing is the process of writing characters or words in free space using finger or hand movements without the aid of any hand-held device. In this work, we address the problem of mid-air finger writ- ing using web-cam video as input. In spite of recent advances in object detection and tracking, accurate and robust detection and tracking of the fingertip remains a challenging task, primarily due to small di- mension of the fingertip. Moreover, the initialization and termination of mid-air finger writing is also challenging due to the absence of any standard delimiting criterion. To solve these problems, we pro- pose a new writing hand pose detection algorithm for initialization of air-writing using the Faster R-CNN framework for accurate hand detection followed by hand segmentation and finally counting the num- ber of raised fingers based on geometrical properties of the hand. Further, we propose a robust finger- tip detection and tracking approach using a new signature function called distance-weighted curvature entropy. Finally, a fingertip velocity-based termination criterion is used as a delimiter to mark the com- pletion of the air-writing gesture. Experiments show the superiority of the proposed fingertip detection and tracking algorithm over state-of-the-art approaches giving a mean precision of 73.1% while achiev- ing real-time performance at 18.5 fps, a condition which is of vital importance to air-writing. Character recognition experiments give a mean accuracy of 96.11% using the proposed air-writing system, a result which is comparable to that of existing handwritten character recognition systems.
Keywords: Air-writing | Hand pose detection | Fingertip detection and tracking | Handwritten character recognition | Human-computer interaction (HCI)