Have you enjoyed watching the sci-fi movie “Ready player one” about the dynamic action story with virtual reality (VR)? If yes, then you would have interest in our research that helps to realize the amazing VR technology in the movie to real world.
People perceive various stimuli from the surrounding environment by using the sensory receptors of the body. This allows them to comprehend their situation and respond according to the surrounding environment, which facilitates survival in nature. With respect to VR and augmented reality (AR), various types of wearable gear embedding sensors and actuators have been developed to provide stimulus from the virtual or augmented environment to users in the real world.
To realize physical interaction with virtual objects, three major feedback displays have been employed in glove-type devices: tactile, haptic, and thermal. Among those three feedbacks, the thermal perception is essential for the survival and daily activities of people. Our vital system is homoeothermic and its efficiency is influenced by temperature. As a result, we feel comfortable in an optimal temperature range and try to avoid extreme temperature environment inducing thermal damage. Thus, it is desirable to realize thermal feedback stimulation for improving the sense of realism in VR for users.
For thermal stimulus, conventional systems utilize liquid circulation with bulky external sources or thermoelectric devices (TEDs) on rigid structures with big heatsinks. However, these systems are difficult to apply to compact wearable gear used for complex hand motions to interact with VR. Furthermore, generating a rapid temperature difference, especially cooling, in response to a thermal stimulus in real-time is challenging for the conventional systems. To overcome this challenge and enhance wearability, we developed an untethered real-time thermal display glove. The concept of the thermal display glove is shown in Figure 1.
Figure 1. Illustration of the operation of thermal display glove system linked to the virtual environment and physiological process. The person wears the proposed thermal display glove and grab the cold cup in the virtual space. The person receives two different stimuli from the VRs: visual and thermal ones. From two external stimulations, the brain as if it were a real situation and the user can feel realism from the virtual environment.
This glove comprised piezoelectric sensors enabling hand motion sensing and flexible TEDs for bidirectional thermal stimulus on skin. The customized flexible TEDs can decrease the temperature by 10 °C at room temperature in less than 0.5 s. Moreover, they have sufficiently high durability to withstand over 5,000 bends and high flexibility under a bending radius of 20 mm. In a user test with 20 subjects, the correlation between thermal perception and the displayed object’s color was verified, and a survey result showed that the thermal display glove provided realistic and immersive experiences to users when interacting with VR.
Figure 2. The demonstration of the thermal display glove system. (a) Taking a coke can out of a refrigerator, (b) petting a dog freely, and (c) grasping a chicken in a microwave. (d-f) show the screen of the thermal imaging camera corresponding to (a-c) at the same time. (g) Legend of temperature displayed by the thermal imaging camera. The range of temperature is between 15 ℃ and 40 ℃. The motion of the thermal display glove, the virtual environment, and the real-time thermal feedback were recorded as supplementary movies (Movie S2- S4).
The background of this study is the fusion of specialized research results in two different research fields. The robotics research group in Korea Institute of Science and Technology (KIST) had been working for the research project on human-centered interaction for coexistent space introducing soft robotics, human-robot interaction technology, and VR technology. In this project, a motion sensing glove that can be interacting with VR was developed but this glove could not display thermal feedback due to the aforementioned challenging issues. Thus, we were looking for a novel solution of thermal actuation to enhance the realism of user experience with the VR interactive glove. Meanwhile, the electric engineering research group in Korea Advanced Institute of Science and Technology (KAIST) has been working to develop a novel technology of flexible TEDs with high performance, and they launched a start-up company “TEGWAY” to expand their applications based on the technology. The KIST group has the strength to integrate robot systems for human interacting applications. On the other hand, the KAIST group has the strength in developing novel electrical components. Combining each strengths, we can finally develop the novel thermal display glove. We expect that such collaborating work would expand the potential of various applications interacting with VR.
For more details about this study, please see our publication.
Seung-Won Kim et al. Thermal display glove for interacting with virtual reality. Scientific Reports (2020), https://doi.org/10.1038/s41598-020-68362-y.