Anomalous Model-Driven-Telemetry Network-Stream BGP Detection

There is a growing demand for real-time analysis of network data streams. In recent years, Model Driven Telemetry (MDT) has been developed-in place of conventional methods such as Simple Network Management Protocol (SNMP), Syslog and CLI commands-to provide a fine-grain holistic view of a network at the control, data and management planes. High-frequency MDT data streams generated from network devices enable new ways of designing Network Operation and Management (OAM) solutions, laying the foundation for future "self-driving"networks.In this paper we study anomaly detection using MDT data streams in a data center environment. In many commercial data centers, BGP is re-purposed for (policy-driven, path-based) intra-routing (as opposed to inter-domain routing that it was originally designed for) to take advantage of rich path diversity. Several vendors have developed MDT data models using YANG that allow routers/switches to express and stream various BGP features for (centralized) network OAM operations. We develop a systematic MDT data processing and feature selection framework that is portable to multiple MDT vendors. Furthermore, we advance NetCorDenstream that builds and improves upon OutlierDenStream proposed in [10] for real-time detection of streamed anomalous MDT data. We show that NetCorDenstream achieves a 59% reduction in alarms raised when compared with OutlierDenStream, thereby reducing the (attention) burden placed on network operators. In particular, it increases alarm detection precision significantly while decreasing false alarms at the expense of a slightly delayed response time.