Magestro: Gamification of the Data Collection Process for Development of the Hand Gesture Recognition Technology

The work presented in this paper, explores the enhancement of the processes of data collection and data annotation via gamification. The need for structured data is vast as intelligent systems are highly dependant on machine-readable information. As seen in nowadays, many existing systems address this issue by relying on users to provide the data they need and also annotate them. However, a major flaw in this process is the lack of motive in the action which in turn lowers the volume of data. Gamification can potentially be an aid to the aforementioned issue as it is a known method for providing the motive needed through entertainment. On the other hand, the amusement of such a setting could lower the quality of the collected information. As such, for the use case of Hand Tracking (HT) and Gesture Recognition (GR) we have conducted an experiment in order to measure the effectiveness of gamification on this task. The relevant data are binary images which contain white values for the area in which the hand is detected as well as the classification of the gesture that is being performed. The tool used for the generation of this information is a computer vision solution, operating on a single RGB input, provided by ManoMotion AB. The hypothesis is that a gamified version of the task would provide a lower quality data in comparison with a non gamified task. Therefore, we conducted an experiment with 10 participants using both methods in order to collect the annotated data. 1288 of total images were collected from both applications. These images were later on used as an input for evaluation coming from external observation. The analysis of the result revealed that in the case of the gamified version, both the quality of the collected images and accuracy of classification was higher. As such, the hypothesis was not confirmed and in turn point out towards a potential benefit of gamification for the task. Based on that, this study suggests that a gamified version of a task is more likely to provide with quality information.