Bootstrap Sample

Abstract

We present the MyoTilt target selection method for smartwatches, which employs a combination of a tilt operation and electromyography (EMG). First, a user tilts his/her arm to indicate the direction of cursor movement on the smartwatch; then s/he applies forces on the arm. EMG senses the force and moves the cursor to the direction where the user is tilting his/her arm to manipulate the cursor on the smartwatch with minimal effort, by tilting the arm and applying force to it. We conducted an experiment to investigate its performance and to understand its usability. Result showed that participants selected small targets with an accuracy greater than 93.89%. In addition, performance significantly improved compared to previous tilting operation methods. Likewise, its accuracy was stable as targets became smaller, indicating that the method is unaffected by the “fat finger problem”.  

Summary Video

Presented Slides


 

Challenges

Touch operation on mobile devices is affected by a fat finger problem, in which displays are occluded by a user’s finger, making precise operations difficult. As a result, there have been researched many alternative method; this research focuses on the “tilting operation”. Tilting operation shows a significant level of performance, but there are also significant limitations: it requires large movements of a user’s arms, and the display is on the backside of the arm, making it difficult to see. In addition, a user needs to move his/her elbow to move the cursor, which is sometimes physically difficult.  

Concept of our method

In our method, to reduce the angle of inclination on smartwatches, we use EMG (electromyography) sensor to determine the amount of operation. In general, in tilting manipulation methods as used in smartwatches, both the direction and the amount of operation are determined from the value of the tilt sensor. So, to obtain a large amount of operation and movement of cursor, it is necessary to greatly tilt the smartwatch. The above problem can be solved by reducing the angle of inclination in the tilting operation on smartwatches. In this research, we will consider a method to determine the amount of operation from the values of other sensors in the tilting operation.  

Implementation

We use Myo gesture control armband (EMG sensor). In MyoTilt, the operation direction is determined by tilting the device, which is obtained from the acceleration sensor. The unit vector in the operation direction is calculated based on the strain in the arm, measured using EMG. To calculate this value, the source signal must be acquired from Myo at 100Hz for eight channels; we use the RMS (Root Mean Square) value during 100 data and the total value is taken as the “myoelectric potential.” MyoTilt measures myoelectric potential for 10 seconds. Average value will be used as a minmum myoelectric potential below the threshold value, the value is 0 px. And at the maximum myoelectric potential (127×8=1016), the value is 240 px (moved from the center of the screen to the screen edge). Within this range, the value corresponds to the muscle power applied.  

Publications

[1] Hiroki Kurosawa, Daisuke Sakamoto and Tetsuo Ono. MyoTilt: A Target Selection Method for Smartwatches using the Tilting Operation and Electromyography. In Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI ’18), ACM, Article No.43, 2018. [link]

[2] 黒澤紘生, 坂本大介, 小野哲雄. スマートウォッチの傾きと筋電情報の組み合わせによるポインティング手法. インタラクション2018(プレミアム発表), 情報処理学会, pp.926-931, 2018. [link]

[3] 黒澤紘生, 坂本大介, 小野哲雄. スマートウォッチの傾きと筋電情報の組み合わせによるポインティング手法. 情報処理学会論文誌, 特集論文インタラクションの理解および基盤・応用技術, Vol.60, No.2, 2019.

[4] Hiroki Kurosawa. MyoTilt: A Target Selection Method for Smartwatches using the Tilting Operation and Electromyography. Hokkaido University–Seoul National University Joint Symposium 2018 International Workshop on New Frontiers in Convergence Science and Technology (Poster), 2018. [link]

 

Acknowledgements

This work was supported by JSPS KAKENHI (Grant-in-Aid for Scientific Research on Innovative Areas) Grant Number 26118006.

 

Authors

Hiroki Kurosawa

is a graduate student at Hokkaido University. He leaded this research project. His interests are Interaction Design and Bio signals.

Daisuke Sakamoto

is an Associate Professor at Hokkaido University. He was a Project Lecturer at Hokkaido University when this research was conducted.

Tetsuo Ono

is a Professor at Hokkaido University. His research interests include robot informatics, ambient intelligence and smart environment, human-robot interaction, human-agent interaction, a computational model of emotion, evolution of languages and the theory of mind.