Master Assignment
Benchmarking facebook's prophet, pelt and twitter's anomaly etection and automated deployment to cloud
Type: Master M-BIT
Location: University of Twente
Period: Oct 2018 - Mar 2019
Student: Srivastava, S. (Siddhartha, Student M-CS)
Date final project: March 18, 2019
Supervisors
Abstract:
We perform benchmark and do a comparison of three algorithms, namely Facebook's Prophet and PELT, which are changepoint detection algorithms, and Twitter's Anomaly Detection, which is an anomaly detec tion algorithm, to see which one is better. The benchmarking is done over synthetic and real datasets, and they have been chosen to accommodate as many real world cases as possible. The metric chosen to compare them are Accuracy, Error rate, Specficity, Precision, Recall and F-measure. Out of these metrics, Precision, Recall and F-measure have been given more weight because they are dependent on number of true positives de tected, the points that actually are anomalies/changepoint, which is what we are interested in. Less importance is given to Accuracy, Error rate and Specficity as they are dependent on number of true negatives, the points that are not changepoints/anomalies, which we are less interested in. Run time of the algorithms is also taken into account. We found that PELT is better than Prophet in terms of Precision, Recall and F-measure, and also it is faster than Prophet. Twitter's Anomaly Detection works best on real data. One of the algorithms, PELT, was deployed to cloud over Kubernetes, a container orchestration engine.