Online anomaly detection with minimax optimal density estimation in nonstationary environments

Simple item page
buir.contributor.authorKozat, Süleyman Serdar
dc.citation.epage1227en_US
dc.citation.issueNumber5en_US
dc.citation.spage1213en_US
dc.citation.volumeNumber66en_US
dc.contributor.authorGokcesu, K.en_US
dc.contributor.authorKozat, Süleyman Serdaren_US
dc.date.accessioned2019-02-21T16:06:00Z
dc.date.available2019-02-21T16:06:00Z
dc.date.issued2018en_US
dc.departmentDepartment of Electrical and Electronics Engineeringen_US
dc.description.abstractWe introduce a truly online anomaly detection algorithm that sequentially processes data to detect anomalies in time series. In anomaly detection, while the anomalous data are arbitrary, the normal data have similarities and generally conforms to a particular model. However, the particular model that generates the normal data is generally unknown (even nonstationary) and needs to be learned sequentially. Therefore, a two stage approach is needed, where in the first stage, we construct a probability density function to model the normal data in the time series. Then, in the second stage, we threshold the density estimation of the newly observed data to detect anomalies. We approach this problem from an information theoretic perspective and propose minimax optimal schemes for both stages to create an optimal anomaly detection algorithm in a strong deterministic sense. To this end, for the first stage, we introduce a completely online density estimation algorithm that is minimax optimal with respect to the log-loss and achieves Merhav's lower bound for general nonstationary exponential-family of distributions without any assumptions on the observation sequence. For the second stage, we propose a threshold selection scheme that is minimax optimal (with logarithmic performance bounds) against the best threshold chosen in hindsight with respect to the surrogate logistic loss. Apart from the regret bounds, through synthetic and real life experiments, we demonstrate substantial performance gains with respect to the state-of-the-art density estimation based anomaly detection algorithms in the literature.
dc.description.provenanceMade available in DSpace on 2019-02-21T16:06:00Z (GMT). No. of bitstreams: 1 Bilkent-research-paper.pdf: 222869 bytes, checksum: 842af2b9bd649e7f548593affdbafbb3 (MD5) Previous issue date: 2018en
dc.description.sponsorshipManuscript received February 8, 2017; revised July 7, 2017, September 10, 2017, October 20, 2017, and November 16, 2017; accepted November 24, 2017. Date of publication December 18, 2017; date of current version January 26, 2018. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Jorge F. Silva. This work was supported in part by the Turkish Academy of Sciences Outstanding Researcher Programme and the Scientific and Technological Research Council of Turkey under Contract 113E517. (Corresponding author: Kaan Gokcesu.) The authors are with the Department of Electrical and Electronics Engineering, Bilkent University, Ankara 06800, Turkey (e-mail: gokcesu@ee.bilkent. edu.tr; kozat@ee.bilkent.edu.tr).
dc.identifier.doi10.1109/TSP.2017.2784390
dc.identifier.issn1053-587X
dc.identifier.urihttp://hdl.handle.net/11693/50286
dc.language.isoEnglish
dc.publisherInstitute of Electrical and Electronics Engineers
dc.relation.isversionofhttps://doi.org/10.1109/TSP.2017.2784390
dc.relation.projectBilkent Üniversitesi - Türkiye Bilimsel ve Teknolojik Araştirma Kurumu, TÜBITAK: 113E517
dc.source.titleIEEE Transactions on Signal Processingen_US
dc.subjectAnomaly detectionen_US
dc.subjectDensity estimationen_US
dc.subjectMinimax optimalen_US
dc.subjectOnline learningen_US
dc.subjectTime seriesen_US
dc.titleOnline anomaly detection with minimax optimal density estimation in nonstationary environmentsen_US
dc.typeArticleen_US

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Online_anomaly_detection_with_minimax_optimal_density_estimation_in_nonstationary_environments.pdf
Size:
1.41 MB
Format:
Adobe Portable Document Format
Description:
Full printable version
Download

Collections

Scholarly Publications - Electrical and Electronics Engineering