Browsing by Subject "Recurrent Neural Network (RNN)"

Now showing 1 - 3 of 3

Open Access
Multimodal video-based personality recognition using Long Short-Term Memory and convolutional neural networks
(2019-07) Aslan, Süleyman
Personality computing and affective computing, where recognition of personality traits is essential, have gained increasing interest and attention in many research areas recently. The personality traits are described by the Five-Factor Model along five dimensions: openness, conscientiousness, extraversion, agreeableness, and neuroticism. We propose a novel approach to recognize these five personality traits of people from videos. Personality and emotion affect the speaking style, facial expressions, body movements, and linguistic factors in social contexts, and they are affected by environmental elements. For this reason, we develop a multimodal system to recognize apparent personality traits based on various modalities such as the face, environment, audio, and transcription features. In our method, we use modality-specific neural networks that learn to recognize the traits independently and we obtain a final prediction of apparent personality with a feature-level fusion of these networks. We employ pre-trained deep convolutional neural networks such as ResNet and VGGish networks to extract high-level features and Long Short-Term Memory networks to integrate temporal information. We train the large model consisting of modality-specific subnetworks using a two-stage training process. We first train the subnetworks separately and then fine-tune the overall model using these trained networks. We evaluate the proposed method using ChaLearn First Impressions V2 challenge dataset. Our approach obtains the best overall “mean accuracy” score, averaged over five personality traits, compared to the state-of-the-art.
Open Access
Online learning with recurrent neural networks
(2018-07) Ergen, Tolga
In this thesis, we study online learning with Recurrent Neural Networks (RNNs). Particularly, in Chapter 2, we investigate online nonlinear regression and introduce novel regression structures based on the Long Short Term Memory (LSTM) network, i.e., is an advanced RNN architecture. To train these novel LSTM based structures, we introduce highly e cient and e ective Particle Filtering (PF) based updates. We also provide Stochastic Gradient Descent (SGD) and Extended Kalman Filter (EKF) based updates. Our PF based training method guarantees convergence to the optimal parameter estimation in the Mean Square Error (MSE) sense. In Chapter 3, we investigate online training of LSTM architectures in a distributed network of nodes, where each node employs an LSTM based structure for online regression. We rst provide a generic LSTM based regression structure for each node. In order to train this structure, we introduce a highly e ective and e cient Distributed PF (DPF) based training algorithm. We also introduce a Distributed EKF (DEKF) based training algorithm. Here, our DPF based training algorithm guarantees convergence to the performance of the optimal centralized LSTM parameters in the MSE sense. In Chapter 4, we investigate variable length data regression in an online setting and introduce an energy e cient regression structure build on LSTM networks. To reduce the complexity of this structure, we rst replace the regular multiplication operations with an energy e cient operator. We then apply factorizations to the weight matrices so that the total number of parameters to be trained is signi cantly reduced. We then introduce online training algorithms. Through a set of experiments, we illustrate signi cant performance gains and complexity reductions achieved by the introduced algorithms with respect to the state of the art methods.
Open Access
Recurrent neural network learning with an application to the control of legged locomotion
(2015) Çatalbaş, Bahadır
Use of robots for real life applications has an increasing trend in today's industry and military. The robot platforms are capable of performing dangerous and difficult tasks, which are not efficient when carried out by human beings. Most of these tasks require high motion ability. There are various robotic platform and locomotion algorithms which may solve a given task. Among these, the biped robot platforms promise high performance in realizing difficult maneuver due to their morphological similarity to legged animals. Thus, legged locomotion is highly desirable in order to perform difficult maneuvers in rough terrain environments. However both modeling and control of such structures are quite difficult due to highly nonlinear structure of the resulting equations of motion and computational load of inverse kinematic equations. Central nervous systems and spinal cords of animals take role in control of locomotion of animals together. For controlling such biped robotic platforms frequently used control algorithms are based on so-called Central Pattern Generators (CPG). The controllers based on CPGs can be realized in different ways which includes the utilization of neural networks. However CPG is only capable of imitating spinal cord type of reflex-based motions in locomotion because of their restricted parameter space to sustain stable oscillation. Fully recurrent neural networks have capability of controlling locomotion with a higher conscious level such as central nervous system, hence motion space can be enlarged. Unfortunately, training of recurrent neural networks (RNN) takes long time. Moreover, their behaviors may be unpredictable against untrained inputs and training process may encounter with instability related problems easily. In order to solve these problems, various acceleration and regularization techniques are tested in the neural network training and their successes were compared with each other. Furthermore, time constant and error gradient limitation methods are employed to sustain stable training and their benefits are discussed. Finally leg angles of walking biped robot are taught to a group of RNNs with different configurations by benefiting from training stability enhancing methods. The resulting RNNs are then used in biped locomotion by using a classical PD controller. After that, performance of resulting RNNs and their stable locomotion generation capabilities are evaluated and effects of configuration parameters are discussed in detail.