Deep Learning Enabled Smart Mats as A Scalable Floor Monitoring System
We demonstrate a smart floor monitoring system through the integration of self-powered triboelectric sensing, large-scale screen printing, and deep learning-based data analytics.
Toward the realization of smart buildings and smart homes, floor as one of our most frequently interactive interfaces can be embedded with sensor arrays to extract the abundant sensory information and avoid using the security cameras with the privacy concern. Until now, the previously developed floor sensors normally exhibit the following constraints: low scalability for the large-area floor sensing, high implementation cost, large power consumption, and significantly increased systemic complication for large arrays. More importantly, the data analytics approach is based on the acquired signal magnitude and/or frequency, which may lose some important features hidden in the sensing signals such as the identity information.
Since 2012, the triboelectric nanogenerator (TENG) technology has shown unprecedented potentials in the diverse applications of energy harvesting and self-powered sensing, based on the coupling effect of contact electrification and electrostatic induction. By using Maxwell’s displacement current as the thrust to effectively convert mechanical energy into electricity, TENG exhibits superior advantages of self-generated signals, cost effectiveness, easy fabrication, and broad availability of materials. When combined with large-scale screen printing technique, the resultant TENG based floor sensors could have high potentials for large-scale, low-cost, and self-powered floor sensing technology.
Over the past few years, artificial intelligence has demonstrated the outstanding capability to extract the full sensory information from sensors in a monitoring system, through the machine learning assisted data analytics. In this study, we develop a revolutionized smart floor monitoring system, by the integration of self-powered triboelectric sensing, large-scale screen printing, and deep learning-based data analytics (see Figure 1). The combination of triboelectric sensing and low-cost screen printing technology results in a large-scale floor sensing solution for the applications in smart buildings/homes. In addition, this floor mats (i.e., sensors) are fabricated with unique “identity” electrode patterns to enable the parallel connection of them in an array configuration, minimizing the output terminals and systemic complication.
After integrating the triboelectric floor mat array with the data acquisition module and deep learning-based data analytics, the smart floor monitoring system is realized. According to the time-domain signal analytics, the real-time stepping position and activity status of a person on the floor mat array can be detected. Moreover, benefiting by the deep learning-based data analytics, the identity information of the person can also be recognized by his unique walking gaits. For a 10-person deep learning model with 1000 data samples, the average recognition accuracy is 96%, offering high accuracy in the practical applications. In a demonstrated smart building scenario, the acquired position information of the person can be used to auto-control the corresponding lights, while the recognized identity information can be adopted to auto-control the door access based on whether the person is a valid user or not. With the excellent capability of position sensing, activity monitoring and identity recognition, the developed smart floor technology can pave the way for using floor as the functional interface in diverse applications of smart buildings/homes, e.g., home automation, healthcare, and security.
Figure 1: the smart floor monitoring system with the capability of position sensing, activity monitoring and identity recognition.
Please check out our recent work published on Nature Communications: “Deep learning enabled smart mats as a scalable floor monitoring system” at the link: https://doi.org/10.1038/s41467-020-18471-z.