Browsing by Author "Albaba, Berat Mert"

Now showing 1 - 5 of 5

Open Access
A 3D game theoretical framework for the evaluation of unmanned aircraft systems airspace integration concepts
(Elsevier, 2021-10-23) Albaba, Berat Mert; Musavi, Negin; Yıldız, Yıldıray
Predicting the outcomes of integrating Unmanned Aerial System (UAS) into the National Airspace System (NAS) is a complex problem, which is required to be addressed by simulation studies before allowing the routine access of UAS into the NAS. This paper focuses on providing a 3-dimensional (3D) simulation framework using a game-theoretical methodology to evaluate integration concepts using scenarios where manned and unmanned air vehicles co-exist. In the proposed method, the human pilot interactive decision-making process is incorporated into airspace models which can fill the gap in the literature where the pilot behavior is generally assumed to be known a priori. The proposed human pilot behavior is modeled using a dynamic level-k reasoning concept and approximate reinforcement learning. The level-k reasoning concept is a notion in game theory and is based on the assumption that humans have various levels of decision making. In the conventional “static” approach, each agent makes assumptions about his or her opponents and chooses his or her actions accordingly. On the other hand, in the dynamic level-k reasoning, agents can update their beliefs about their opponents and revise their level-k rule. In this study, Neural Fitted Q Iteration, which is an approximate reinforcement learning method, is used to model time-extended decisions of pilots with 3D maneuvers. An analysis of UAS integration is conducted using an Example 3D scenario in the presence of manned aircraft and fully autonomous UAS equipped with sense and avoid algorithms.
Open Access
Driver modeling through deep reinforcement learning and behavioral game theory
(Institute of Electrical and Electronics Engineers, 2021-05-05) Albaba, Berat Mert; Yıldız, Yıldıray
In this work, a synergistic combination of deep reinforcement learning and hierarchical game theory is proposed as a modeling framework for behavioral predictions of drivers in highway driving scenarios. The modeling framework presented in this work can be used in a high-fidelity traffic simulator consisting of multiple human decision-makers. This simulator can reduce the time and effort spent for testing autonomous vehicles by allowing safe and quick assessment of self-driving control algorithms. To demonstrate the fidelity of the proposed modeling framework, game-theoretical driver models are compared with real human driver behavior patterns extracted from two different sets of traffic data.
Open Access
Modeling cyber-physical human systems via an interplay between reinforcement learning and game theory
(Elsevier, 2019) Albaba, Berat Mert; Yıldız, Yıldıray
Predicting the outcomes of cyber-physical systems with multiple human interactions is a challenging problem. This article reviews a game theoretical approach to address this issue, where reinforcement learning is employed to predict the time-extended interaction dynamics. We explain that the most attractive feature of the method is proposing a computationally feasible approach to simultaneously model multiple humans as decision makers, instead of determining the decision dynamics of the intelligent agent of interest and forcing the others to obey certain kinematic and dynamic constraints imposed by the environment. We present two recent exploitations of the method to model (1) unmanned aircraft integration into the National Airspace System and (2) highway traffic. We conclude the article by providing ongoing and future work about employing, improving and validating the method. We also provide related open problems and research opportunities.
Open Access
Predicting human behavior using static and dynamic models
(2021-08) Albaba, Berat Mert
Modeling human behavior is a challenging problem and it is necessary for the safe integration of autonomous systems into daily life. This thesis focuses on modeling human behavior through static and dynamic models. The ﬁrst contribution of this thesis is a stochastic modeling framework, which is a synergistic combination of a static iterated reasoning approach and deep reinforcement learning. Using statistical goodness of ﬁt tests, the proposed approach is shown to accurately predict human driver behavior in highway scenarios. Although human driver behavior are modeled successfully with the static model, the scope of interactions that can be modeled with this approach is limited to short duration interactions. For interactions that are long enough to induce adaptive behavior, we need models that incorporate learning. The second contribution of this thesis is a learning model for time extended human-human interactions. Through a hierarchical reasoning solution approach, equilibrium concepts are combined with Gaussian Processes to predict the learning behavior. As a result, a novel bounded rational learning model is proposed.
Open Access
SyNet: an ensemble network for object detection in UAV images
(IEEE, 2021-05-05) Albaba, Berat Mert; Özer, Sedat
Recent advances in camera equipped drone applications and their widespread use increased the demand on vision based object detection algorithms for aerial images. Object detection process is inherently a challenging task as a generic computer vision problem, however, since the use of object detection algorithms on UAVs (or on drones) is relatively a new area, it remains as a more challenging problem to detect objects in aerial images. There are several reasons for that including: (i) the lack of large drone datasets including large object variance, (ii) the large orientation and scale variance in drone images when compared to the ground images, and (iii) the difference in texture and shape features between the ground and the aerial images. Deep learning based object detection algorithms can be classified under two main categories: (a) single-stage detectors and (b) multi-stage detectors. Both single-stage and multi-stage solutions have their advantages and disadvantages over each other. However, a technique to combine the good sides of each of those solutions could yield even a stronger solution than each of those solutions individually. In this paper, we propose an ensemble network, SyNet, that combines a multi-stage method with a single-stage one with the motivation of decreasing the high false negative rate of multi-stage detectors and increasing the quality of the single-stage detector proposals. As building blocks, CenterNet and Cascade R-CNN with pretrained feature extractors are utilized along with an ensembling strategy. We report the state of the art results obtained by our proposed solution on two different datasets: namely MS-COCO and visDrone with %52.1 mAP IoU=0.75 is obtained on MS-COCO val2017 dataset and %26.2 mAP IoU=0.75 is obtained on VisDrone test - set. Our code is available at: https://github.com/mertalbaba/SyNet.