Accordingly, the general claim of this thesis is that understanding how users move their fingers during input will allow increasing the expressiveness of the interaction techniques we can create for wearable devices with limited resources. Furthermore, this thesis supports the general claim with a series of scenarios for wearable devices by investigating finger input movements in the air.
Wearables
Several developments in technology and digital platform markets have contributed to the popularization of VR HMDs. This is a key reason why many wearable devices still rely on other large form factor devices for input and interaction.
Input on smartwatches
The availability of the high bandwidth tasks is essential for smartwatches to be successful standalone devices. Many previous works have proposed new interaction techniques to achieve the high bandwidth tasks with additional finger-mounted wearables [68] or the sequence of tapping [69], swiping [52], or zooming [70] on the touchscreen.
Input on HMDs
Therefore, this thesis argues that new interaction techniques for achieving high bandwidth tasks should consider both the general input skills and accessible sensors in smartwatches. Consequently, this thesis argues that new bare-hand interaction techniques for high-bandwidth tasks must consider both the unconstrained finger movements and the available hand tracking systems in commercial HMDs.
Properties of Finger for input
Much previous work has focused on the properties of finger gestures on the large form factor devices. Previous research has investigated the comfort during touch input on the surface of large form factor devices [101].
Claim
In short, finger movements during input have the potential to generate several new input features and modalities. In particular, understanding the finger movements for the specialized form factors of wearable devices can enable unique input functions to be identified with the reliable built-in sensors.
Outline, Contribution, and Conclusion
The keyboard for complex typing tasks shows increased expressiveness of touch input on the small touchscreen of smart watches. This thesis explores the properties of thumb motion during thumb insertion into small targets around the hands.
Input Techniques on Smartwatches
Touching with a finger on a small touchscreen remains a problem of screen occlusion during touch. Meanwhile, many researchers have explored the input around the skin-touch device to expand the input area.
Input Techniques on HMDs
Gugenheimeret al.[207] investigated the touch interaction on the side and front of VR HMD. After detection, a depth camera estimates the location of the finger touch in the air on the virtual keyboard.
Summary
Meanwhile, the latest HMDs with the built-in or plug-in hand tracking system can enable thumb-to-finger interaction with bare hands. Solimanet al.[266] used a shoulder-mounted depth camera to record discrete and continuous thumb-to-finger input.
Properties of Finger Movements in Comfort Angle of Touch Input
Abstract
Introduction
Therefore, this paper collects and contributes baseline data on data comfort in smartwatches from a pair of studies that combine and compare traditional time and accuracy metrics with touch comfort ratings. The results highlight the stable and comfortable range for angular input for each of the three finger regions.
Related Work
Second, this paper considers three different touch input techniques to specify input angles: touch on the side and flat of the index finger and that by the pair of index and middle fingers. In this way, this paper complements previous work by improving our understanding of the ease of touch input on a new form factor of wearable devices and with a larger set of finger stretches.
Sensing Rotations
In addition, a detailed critique by Mayer et al. [284] highlights improvements in accuracy compared to Xiaoet et al. [94] oscillation accuracy, it is not clear that these apply to the situation where the fingers are flat on the touch screen surface (low height). The performance of the heuristic algorithm Xiaoet al.[94] is optimal in this setting and variations in the reporting in Mayeret al.[284] hard to judge if their algorithm offers improvements when fingers are flat.
Static Angle Study
Within each condition, participants completed two blocks of eight randomly ordered sets of trials, one for each angle considered in the study. This suggests that we can be confident in the validity of the data and that the participants' subjectively experienced comfort was consistent throughout the study.
Dynamic Angle Study
Finally, to validate the logistic regression model that distinguished between flat and lateral touches in the static study, we tested it with data from the dynamic study. This led to an accuracy of 95.6%, almost exactly the same as in the static study.
Design Recommendations
For initial touches, a minimum target angle width would be 20◦ as recommended in the static study. In the static study, compared to the two other conditions studied, the side region provided a modestly greater range of comfortable entry angles.
Interaction Techniques and Applications
A menu with a dynamic corner on the top and left side of the watch could be called up with a prolonged (>500ms hold) touch of the side area. Finger-controlled flat orientation rotations while rotating with side-adjusted zoom.
Limitations
Conclusions
Properties of Finger Movements in Finger Identification
Abstract
Introduction
Our analysis describes the touches and discusses the efficiency and accuracy of user input and recognition rate for finger identification in these scenarios. Third, this paper contributes to a discussion of how finger identification can realistically be implemented in actual wearable devices and the types of interaction it can support.
Related work
In terms of design, most suggestions are variations on the idea that specific fingers can be used to access different functions. The goal of this paper is to build on these results and ideas to create a robust finger identification system that works with data from currently available touchscreen technology.
Performance Study
Thumb (left), index (middle) and middle finger (right) touch the screen in the exaggerated positions. These include using standard reporting methods in the operating system [112], using specialized hardware such as fingerprint scanners [96], constructing custom sensor grids [66], and modifying touch drivers on existing smart devices [94, 281].
Di-type and Tri-type Keyboards
These figures are somewhat slower than those recorded in the initial sessions of Gupta and Balakrishnan [68]. Looking at the data in detail, we observed a disproportionate number of inch misfinger errors in the bottom row of keys.
Recommendations and Designs
To be reliable, fingerprint identification systems must therefore rely on techniques such as residential thresholds before starting classification processes. This kind of icon fits our design recommendations as they are relatively large, sporadically accessible and usually located away from the extreme edges of the screen.
Discussion and Conclusion
Rather than as completely new contributions, we present these designs as customized versions of existing concepts that fit the capabilities of the functional finger recognition system proposed in this paper.
Properties of Finger movements in In-air Finger Stroking
Abstract
Introduction
This allows us to define fundamental requirements and guidelines for designing in-air typing systems in VR. This analysis provides quantitative data that can support the development of in-air typing systems for VR based on unconstrained finger motion.
Throughout this paper, we compare the data we report with existing data captured in air-constrained typing (ATK [47]) settings. The design of airborne typing systems can also draw on the extensive literature on unconstrained text input on flat surfaces such as mobile phones and tablets.
This system loads the 3D position data from all finger stroking movements and provides a visual environment to scroll through 3D representations of typing actions and manually label each action with the appropriate character (eg the character that should have been typed). . Before the end point, the finger moves at speed, so we simply categorized starting points at the origin of these essentially ballistic movements - the closest local speed minima.
General description: in-air typing
We therefore hypothesized that air typing would exhibit differences in typing strategies based on the different global hand movement patterns reported in previous work. Since increased dynamic palm motion will affect finger stroke properties in general, we identify writing strategy as a critical factor in understanding unconstrained air typing behavior.
Finger Kinematics
Dark blue indicates unrestricted writing from current study and light blue indicates limited writing from ATK. Since finger stroking in unconstrained typing leads to these distinctive movements of the palms, we suggest that travel distance may be particularly salient as a key feature for detecting finger strokes in unconstrained typing.
Correlated Movement of Fingers
In contrast, the recovery ratio was lower for limited typing, as it clearly emphasizes individual keystrokes rather than continuous typing. We speculate that the higher recovery ratio may help support clearer finger stroke segmentation and finger recognition.
Inter-keystroke Relationship
We note that this difference may be due to passive finger movements that occurred as a result of forces applied as a fortuitous but inevitable consequence of using more dynamic whole-hand movements during typing. Due to the shorter inter-test stroke intervals, FINGER group produced longer overlap time between successive finger strokes than that of HAND group.
Individual in-air keys
The data were further analyzed using the Mann-Whitney U test on the typing strategy variable. This data shows that finger presses became faster and deeper towards the bottom of the keyboard.
Feasibility of Finger Classification
When participants performed a finger press on the keys toward the center of the keyboard (around the F and J keys), the directions of the fingertip presses were nearly vertical. Typically, the thumb was pressed with a stroke that closely matched the forward direction of the participants.
General Discussion
Detecting a finger stroke in unconstrained in-air typing should consider other factors as features besides just amplitudes in finger flexion. Understanding finger movements during the unrestricted in-air typing will be crucial for future designs.
Conclusions
In addition, future work should apply approaches from bio-mechanics, or models of human movement, such as Fitts' law [327], to finger-pressing movements in the air. These patterns can provide additional insight into the relationship between the users' finger movements and the air keys they intend to select.
Properties of Finger Movement in In-air Thumb Typing
Abstract
Introduction
First, the design of a two-handed, unencumbered thumb text input system for a commercially available HMD. The second contribution is two assessments of keyboard design and layout in repeated (once) and phrase (twice) text input tasks.
Related Work
One approach that has the potential to achieve a more familiar experience is in-air typing. As such, while in-air typing is a promising approach for HMD-based text input, we note that current solutions are not yet practical.
Platform and Environment
Range of Motion Study
This was 2.5 cm beyond the outer edge of the target grid, aligned with the center of the grid. Additionally, we validated these target locations against the reported detection resolution of the Quest HMD used in this work.
Bigram Study
Using this criterion enabled the collection of the most diverse set of valid targeting movements possible. In a largely similar study involving pairs of thumb (and finger) tapping on the nails of the same hand, Lee et al.[262].
Keyboard Layout Selection
Because of the challenges in contextualizing the entanglement results, we further calculated the decomposition results of Gonget al.[160] for all appearances. In Gonget al.'s WrisText [160], for example, the top 1% score for the layout they develop is 85.9%; for the rendering selected here it is 93.3%, an increase that we believe will lead to significantly better word suggestion performance.
Typing study: evaluating the performance of ThumbAir
Data per participant are included to highlight the variability in the word repetition task across different users. Data per participant is included to highlight the variability in the sentence typing task across different users.
Comparison Study: Comparing ThumbAir against a Baseline
This suggests that participants corrected the vast majority of typos that occurred in the study. Participants reported higher levels of perceived exertion at the shoulder (p-value = 0.049) in the baseline condition.
Discussion and Conclusion
It would also be important to expand the studies reported in this article with work examining a range of more diverse settings. Immediate future work will need to redesign or integrate a wide range of features into our system so that users can actually type.
Appendix
Summary
The third scenario examined the characteristics of in-air finger strokes during unconstrained in-air typing tasks. Then, through the analysis of finger kinematics, the unconstrained in-air finger strokes showed faster speed (762 mm/s), shorter duration (322 ms) and lower amplitude (49 mm) than in constrained writing tasks with specific behavioral instruction [47] .
Design Considerations on finger input techniques for wearables
The results of Chapter V contributed to the in-depth analysis of in-air finger strokes. This data will be useful for developing mid-air thumb touch techniques for virtual objects.
Limitation and Future Work
The airborne finger entry techniques in the current studies also used specific finger movements. To achieve this, this thesis (Chapter V) described the properties of finger movements during casual air typing.
Conclusion
Brown, “Tactile feedback for mobile interactions,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ser. Zhai, “Performance of touch screen soft buttons,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ser.
