BeyondVision: An EMG-driven Micro Hand Gesture Recognition Based on Dynamic Segmentation
Authors: Nana Wang, Jianwei Niu, Xuefeng Liu, Dongqin Yu, Guogang Zhu, Xinghao Wu, Mingliang Xu, Hao Su
IJCAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Subjective and objective experimental results show that our approach achieves over 95% average recognition rate, 2000Hz sampling frequency, and real-time micro gesture recognition. Our technique has been applied in a commercially available product, introduced at: https://github.com/tyc333/No Barriers. Moreover, Experiments show that Beyond Vision can achieve an impressive average accuracy rate of over 95%, and is effective in MHGs recognition, which satisfies the basic requirements of human-computer interaction, and has the potential for further commercial application. We conduct an experiment using the widely played Subway Surfer game to evaluate our real-time recognition efficiency. Subway Surfer requires quick and accurate gesture inputs, which makes it a suitable platform to evaluate our technology. The game operations include four key commands: left swipe, right swipe, jump, and crouch. Our approach can effectively map these game commands to intuitive MHGs, that is, using the thumb and the first two joints of the index finger. The experiment shows that users can effectively operate the game by Beyond Vision (we show the detailed experiment processing in the demo video of our supplementary materials). Moreover, Figure 10 shows some application samples of our designed MHG control command of Human-Computer interaction, including clicking or releasing a button, moving and dragging the cursor, inputting special commands, and so on. Each of the above control functions has been tested and verified on a real machine. |
| Researcher Affiliation | Collaboration | Nana Wang1,3 , Jianwei Niu1,2 , Xuefeng Liu1 , Dongqin Yu3 , Guogang Zhu1 , Xinghao Wu1 , Mingliang Xu2 , Hao Su2, 1State Key Lab of VR Technology and System, School of CSE, Beihang University, Beijing, China 2School of Computer and Artificial Intelligence, Zhengzhou University, Zhengzhou, China 3No Barriers.ai Technology, Hangzhou, China |
| Pseudocode | Yes | Algorithm 1 Weight segmentation algorithm Input: S , w, s, t Output: C |
| Open Source Code | Yes | Our technique has been applied in a commercially available product, introduced at: https://github.com/tyc333/No Barriers. |
| Open Datasets | No | Collection. We collect data from 20 participants, including 12 males and 8 females. Each participant underwent roughly 500 data collection sessions, with each session lasting 10 seconds. Following data processing and filtering, we allocate one-ninth of the total data as our test set. Each category has at least 5,000 samples, with certain categories containing up to approximately 12,000 samples. The diverse samples ensure our dataset is both comprehensive and robust for our evaluation purposes. |
| Dataset Splits | No | Following data processing and filtering, we allocate one-ninth of the total data as our test set. |
| Hardware Specification | No | Device parameter. Our device weighs 70g, and has a 500m Ah battery, 3-hour operational time, and 24-hour standby time. Our processing is performed on a computer, and the device transmits EMG to the computer via Wi Fi. |
| Software Dependencies | No | The paper describes the proposed BV-Net as a lightweight CNN architecture and mentions components like convolutional layers, batch normalization, and Mish operation, but does not provide specific version numbers for any software libraries, frameworks (e.g., TensorFlow, PyTorch), or programming languages used for implementation. |
| Experiment Setup | No | The paper describes the network architecture (BV-Net) and the loss function used (focal loss with L2 regularization) but does not provide specific hyperparameter values such as learning rate, batch size, number of epochs, or details about the optimizer used, which are critical for reproducing the experimental setup. |