A memory efficient novel deep learning architecture enabling diverse feature extraction on wearable motion sensor data

Abstract

Extracting representative features to recognize human activities through the use of wearables is an area of on-going research. We propose a novel hybrid net-work architecture to recognize human activities through the use of wearable motion sensors and deep learning techniques. The long short-term memory (LSTM) and the 2D convolutional neural network (CNN) branches of the model that run in parallel receive the raw signals and their spectrograms, respectively. We compare the classification performance of the proposed network with five commonly used network architectures: 1D CNN, 2D CNN, LSTM, standard 1D CNN-LSTM, and an alternative 1D CNN-LSTM model. We tune the hyper-parameters of all six models using Bayesian optimization and test the models on two publicly available datasets. The proposed 2D CNN-LSTM architecture achieves the highest aver-age accuracies of 95.66% and 92.95% on the two datasets, which are, respectively, 2.45% and 3.18% above those of the 2D CNN model that ranks the second. User identification is another problem that we have addressed in this thesis. Firstly, we use binary classifier models to detect activity signals that are useful for the user identity recognition task. Useful signals are transmitted to the next module and used by the proposed deep learning model for user identity recognition. Moreover, we investigate feature transfer between the human activity and user identity recognition tasks which enables shortening the training processes by 8.7 to 17 times without a significant degradation in classification accuracies. Finally, we elaborate on reducing the model sizes of the proposed models for human activity and user identity recognition problems. By using transfer learning, pooling layers, and eight-bit weight quantization methods, we have reduced the model sizes by 17–116 times without a significant degradation in classification accuracies.