[1]周 祺,张 帅.基于手部姿态识别的MIDI体感交互系统设计[J].湖北工业大学学报,2023,(2):27-32.
　ZHOU Qi,ZHANG Shuai. Design of MIDI Somatosensory Interaction System based on Hand Posture Recognition[J].,2023,(2):27-32.
点击复制

基于手部姿态识别的MIDI体感交互系统设计()
分享到：

《湖北工业大学学报》[ISSN:1003-4684/CN:42-1752/Z]

卷:
期数:
2023年第2期
页码:
27-32
栏目:
湖北工业大学学报
出版日期:
2023-04-30

文章信息/Info

Title:
 Design of MIDI Somatosensory Interaction System based on Hand Posture Recognition
文章编号:
1003-4684(2023)02-0027-06
作者:
周 祺;  张 帅
湖北工业大学 工业设计学院，湖北 武汉 430068
Author(s):
 ZHOU Qi;  ZHANG Shuai
 School of Industrial Design, Hubei Univ. of Tech., Wuhan 430068, China
关键词:
MIDI;  树莓派;  手势识别;  ShuffleNet v2;  体感交互
Keywords:
 ; MIDI;  Raspberry Pi;  gesture recognition;  ShuffleNet v2;  somatosensory interaction
分类号:
TP 271
文献标志码:
A
摘要:
为摆脱MIDI音乐设备在交互方式上的限制，同时针对目前体感交互系统便携性差、价格昂贵等问题，将边缘计算设备与轻量级网络结合，提出了一种基于手部姿态识别的音乐体感交互系统。系统上位机依托于树莓派 4b，通过单目摄像头获取数据进而识别手部关键点，调用舵机追踪目标，并通过nRF24L01通信模块将动作指令发送给下位机，从而实现中远距离控制音乐演奏。通过在轻量级网络ShuffleNet v2中嵌入SENet通道注意力机制并进行网络瘦身操作，系统可在200 ms内完成体感控制任务，能满足用户手势控制、设备协同等需求。
Abstract:
 In order to get rid of the limitation of MIDI music equipment in the way of interaction, and to solve the problems of poor portability and high price of the current somatosensory interaction system, combining edge computing equipment with a lightweight network, a music somatosensory interaction system based on hand gesture recognition is proposed. The host computer of the system relies on the Raspberry Pi 4b, obtains data through the monocular camera to identify the key points of the hand, calls the steering gear to track the target, and sends the action command to the lower computer through the nRF24L01 communication module, so as to realize the medium and long-distance control of music performance. By embedding the SENet channel attention mechanism in the lightweight network ShuffleNet v2 and performing the network slimming operation, the system can complete the somatosensory control task within 200ms, which can meet the needs of user gesture control, device coordination, etc., and the system is convenient and efficient. It provides a reference for deploying somatosensory interactive applications on mobile devices.

参考文献/References:

[1] LUTHER L, TIBERIUS V, BREM A. User Experience (UX) in business, management, and psychology: A bibliometric mapping of the current state of research[J]. Multimodal Technologies and Interaction, 2020, 4(02): 18.
[2] NUNEZ J C, CABIDO R, PANTRIGO J J, et al. Convolutional neural networks and long short-term memory for skeleton-based human activity and hand gesture recognition[J]. Pattern Recognition, 2018, 76: 80-94.
[3] SZEGEDY C, LIU W, JIA Y, et al. Going deeper with convolutions[C].∥Proceedings of the IEEE conference on computer vision and pattern recognition, 2015:1-9.
[4] Howard A G, Zhu M, Chen B, et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications[EB/OL]. (2017-04-17)[2021-12-30]. https://arxiv.53yu.com/abs/1704.04861.
[5] MA N, ZHANG X, ZHENG H T, et al. Shufflenet v2: Practical guidelines for efficient cnn architecture design[C].∥Proceedings of the European conference on computer vision (ECCV), 2018: 116-131.
[6] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C].∥Proceedings of the IEEE conference on computer vision and pattern recognition,2018: 7132-7141.
[7] 林景栋,吴欣怡,柴毅,等.卷积神经网络结构优化综述[J].自动化学报,2020,46(01):24-37.
[8] LIU Z, LI J, SHEN Z, et al. Learning efficient convolutional networks through network slimming[C].∥Proceedings of the IEEE international conference on computer vision,2017: 2736-2744.
[9] BALAJI V, BALAJI M, CHANDRASEKARAN M, et al. Optimization of PID control for high speed line tracking robots[J]. Procedia Computer Science, 2015, 76: 147-154.
[10] DE OLIVEIRA H M,DE OLIVEIRA R C. Understanding MIDI: A Painless Tutorial on Midi Format[EB/OL]. (2017-05-15)[2021-12-30]. https:∥arxiv.53yu.com/abs/1705.05322.

备注/Memo

备注/Memo:
［收稿日期］ 2022-01-19
［第一作者］ 周 祺（1990-），湖北武汉人，湖北工业大学教授，研究方向为产品创新设计
［通信作者］ 张 帅（1996-），河北唐山人，湖北工业大学硕士研究生，研究方向为体感交互设计
更新日期/Last Update: 2023-04-26