Microservices Observatory

LegoSDN logo.

Microservices Observatory is a collection of projects meant to disentangle and illuminate the complexity of effectively managing high-performance and available cloud-native applications. Currently, our focus revolves around several distinct thrusts:

  • Kubernetes Operators: These encapsulate administrator knowledge about managing the life-cycle of a microservices. Our focus is to understand and quantify the availability and security implications of automation provided by ``operators''. Motivated by our observations, we are developing tools to safeguard and protect production networks from emerging issues.
  • Digitial Clone for AIOps (KubeKlone): A recent trend involves employing machine learning techniques to automate configuration-related Operational tasks (often referred to as AIOPs). Often such automation is geared towards discovering configuration choices that minimize cost, increase revenue, or improve performance. Today, AIOPs techniques are costly and time-consuming because the ML techniques require significant amounts of data, which is challenging to collect at scale. We are developing a digital twin of cloud-based cloud-native infrastructure. We plan to explore the feasibility of design AIOps for emerging cloud-native infrastructure deployments, e.g., 5G and edge computing.
  • eBPF Observability: Extended Berkeley Packet Filters (eBPF) are an increasingly popular technology used to support networking and monitoring for dynamic microservices environments. This is due to the flexibility and performance improvements that come with the ability to dynamically verify and insert programs into the kernel. However, this programmability makes it challenging to understand the system state at runtime. For example, suppose network packets are being anomalously dropped or corrupted; the root cause could be eBPF programs that process network packets, but there's currently no means to trace the eBPF programs that altered a packet, or to measure the resource consumption of these eBPF programs. To reduce time to insight for such systems, we are developing an observability framework to manage eBPF-based distributed systems.

Read our Vision Paper on Microservices and associated challenges »

Projects

KubeKlone

A digital clone of a microservices infrastructure provides integration of several learning-based AIOPs (specifically those based on Bayesian optimization and Reinforcement ).

Survey results.

Comprehensive analysis Kubernete Operator

A first look at open issues of over 40 operators from one of the most prominent operator frameworks (Operator Hub).

Suture Framework

Suture

A framework that turbo-charges existing Kubernetes operators to support virtualization, finer-grained security, and defense capabilities.

bpfscope

BPFScope

An observability framework to manage eBPF-based distributed systems.


Talks

@CoNEXT '20 Student workshop

We presented our vision for Suture at the ACM Conference on emerging Networking EXperiments and Technologies (CoNEXT) virtual event on December & 15, 2020.


News

  [October 2021] We are attending KubeCon'21. Please help us by filling this survey and feel free to get in touch if you are attending !

  [September 2021] Theo, received a $250K grant from VMWare for research on ``Human in the Loop AIOps'' for Microservices management.

  [April 2021] Theo, received a $50K grant from NEC to work on data-driven approaches for improving web performance on 5G.


Source Code

Coming soon ....


People

Akshat Mahajan

Akshat Mahajan

Akshat is a second year MSc student in the Computer Science Department.

Personal Web Page

Ayush Bhardwaj

Ayush Bhardwaj

Ayush is a first year PhD student in the Computer Science Department.

Personal Web Page

Desmond

Desmond Cheong

Desmond is a fourth year undergraduate student in the Computer Science Department.

Personal Web Page

Theophilus Benson

Theophilus Benson

Theo is an Assistant Professor in the Computer Science Department.

Personal Web Page