Intelligent sensing for robot mapping and simultaneous human localization and activity recognition

Abstract

We consider three different problems in two different sensing domains, namely ultrasonic sensing and inertial sensing. Since the applications considered in each domain are inherently different, this thesis is composed of two main parts. The approach common to the two parts is that raw data acquired from simple sensors is processed intelligently to extract useful information about the environment. In the first part, we employ active snake contours and Kohonen’s selforganizing feature maps (SOMs) for representing and evaluating discrete point maps of indoor environments efficiently and compactly. We develop a generic error criterion for comparing two different sets of points based on the Euclidean distance measure. The point sets can be chosen as (i) two different sets of map points acquired with different mapping techniques or different sensing modalities, (ii) two sets of fitted curve points to maps extracted by different mapping techniques or sensing modalities, or (iii) a set of extracted map points and a set of fitted curve points. The error criterion makes it possible to compare the accuracy of maps obtained with different techniques among themselves, as well as with an absolute reference. We optimize the parameters of active snake contours and SOMs using uniform sampling of the parameter space and particle swarm optimization. A demonstrative example from ultrasonic mapping is given based on experimental data and compared with a very accurate laser map, considered an absolute reference. Both techniques can fill the erroneous gaps in discrete point maps. Snake curve fitting results in more accurate maps than SOMs because it is more robust to outliers. The two methods and the error criterion are sufficiently general that they can also be applied to discrete point maps acquired with other mapping techniques and other sensing modalities. In the second part, we use body-worn inertial/magnetic sensor units for recognition of daily and sports activities, as well as for human localization in GPSdenied environments. Each sensor unit comprises a tri-axial gyroscope, a tri-axial accelerometer, and a tri-axial magnetometer. The error characteristics of the sensors are modeled using the Allan variance technique, and the parameters of lowand high-frequency error components are estimated. Then, we provide a comparative study on the different techniques of classifying human activities that are performed using body-worn miniature inertial and magnetic sensors. Human activities are classified using five sensor units worn on the chest, the arms, and the legs. We compute a large number of features extracted from the sensor data, and reduce these features using both Principal Components Analysis (PCA) and sequential forward feature selection (SFFS). We consider eight different pattern recognition techniques and provide a comparison in terms of the correct classification rates, computational costs, and their training and storage requirements. Results with sensors mounted on various locations on the body are also provided. The results indicate that if the system is trained by the data of an individual person, it is possible to obtain over 99% correct classification rates with a simple quadratic classifier such as the Bayesian decision method. However, if the training data of that person are not available beforehand, one has to resort to more complex classifiers with an expected correct classification rate of about 85%. We also consider the human localization problem using body-worn inertial/ magnetic sensors. Inertial sensors are characterized by drift error caused by the integration of their rate output to get position information. Because of this drift, the position and orientation data obtained from inertial sensor signals are reliable over only short periods of time. Therefore, position updates from externally referenced sensors are essential. However, if the map of the environment is known, the activity context of the user provides information about position. In particular, the switches in the activity context correspond to discrete locations on the map. By performing activity recognition simultaneously with localization, one can detect the activity context switches and use the corresponding position information as position updates in the localization filter. The localization filter also involves a smoother, which combines the two estimates obtained by running the zero-velocity update (ZUPT) algorithm both forward and backward in time. We performed experiments with eight subjects in an indoor and an outdoor environment involving “walking,” “turning,” and “standing” activities. Using the error criterion in the first part of the thesis, we show that the position errors can be decreased by about 85% on the average. We also present the results of a 3-D experiment performed in a realistic indoor environment and demonstrate that it is possible to achieve over 90% error reduction in position by performing activity recognition simultaneously with localization.