{"id":32339,"date":"2024-03-19T09:00:00","date_gmt":"2024-03-19T16:00:00","guid":{"rendered":"https:\/\/azure.microsoft.com\/en-us\/blog\/?p=32339"},"modified":"2024-03-15T14:16:15","modified_gmt":"2024-03-15T21:16:15","slug":"microsoft-open-sources-retina-a-cloud-native-container-networking-observability-platform","status":"publish","type":"post","link":"https:\/\/azure.microsoft.com\/en-us\/blog\/microsoft-open-sources-retina-a-cloud-native-container-networking-observability-platform\/","title":{"rendered":"Microsoft open sources Retina: A cloud-native container networking observability platform"},"content":{"rendered":"\n

The Microsoft Azure Container Networking team is excited to announce Retina, a cloud-native container networking observability platform that enables Kubernetes users, admins, and developers to visualize, observe, debug, and analyze Kubernetes\u2019 workload traffic irrespective of Container Network Interface (CNI), operating system (OS), and cloud. We are excited to release Retina as an open-source repository<\/a> that helps with DevOps and SecOps related networking cases for your Kubernetes clusters and we invite the open-source community to innovate along with us.<\/p>\n\n\n

\n\t
\n\t\t
\n\n\t\t\t\t\t\t\t
\n\t\t\t\t\t\"Photagraphy\t\t\t\t<\/div>\n\t\t\t\n\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\n\t\t\t\t\t

Retina<\/h2>\n\n\t\t\t\t\t
\n\t\t\t\t\t\t

A cloud-native container networking observability platform<\/p>\n\t\t\t\t\t<\/div>\n\n\t\t\t\t\t\t\t\t\t\t\t

\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\tRead more<\/span>\n\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t<\/div>\n\n\t\t\t\t\t<\/div>\n\t<\/div>\n<\/div>\n\n\n\n

<\/p>\n\n\n\n

Embracing and advancing open-source software<\/h2>\n\n\n\n

Cloud native technologies like Kubernetes have made building applications that can run anywhere, easier. At the same time, many applications have become more complex, and managing them in the cloud is increasingly difficult. As companies build cloud-native applications composed of interconnected services and then deploy them to multiple public clouds as well as their private infrastructure, network related observability, troubleshooting, and debugging has become increasingly difficult.<\/p>\n\n\n\n

With the power of extended Berkley Packet Filter (eBPF)<\/a>, it is now possible to offer actionable network insights including how containerized micro-services interact and do so in non-intrusive ways without any change in the applications itself\u2014that\u2019s exactly what Retina sets out to achieve. Retina will help democratize network observability and troubleshooting by bringing new focus to the experience of application developers. Retina provides developers with simple ways to observe and troubleshoot their applications for issues such as packet drops and latency without worrying about the complexities of the underlying network infrastructure and transformations.<\/p>\n\n\n\n

Based on our positive experience in the community with eBPF and Cilium, we are excited to build on this relationship and engage both more closely and with more communities. We believe that by opening Retina to the community, we can benefit from informed feedback, innovative ideas, and collaborative efforts that will help enhance and expand Retina\u2019s capabilities.<\/p>\n\n\n\n

Retina solutions and capabilities<\/h2>\n\n\n\n

Drawing from our extensive experience managing multiple container networking services for the Azure Kubernetes Service (AKS)<\/a>, we identified critical gaps in network monitoring, the collection of network metrics and traces from Kubernetes clusters. Retina is a cutting-edge solution that closes these gaps and is designed to tackle the complex challenges of managing and supporting Kubernetes networks providing infrastructure- and site-reliability engineers comprehensive insights into cluster networking. Retina also provides deep traffic analysis with Kubernetes-specific context, translating metrics into either industry-standard Prometheus or network flow logs.<\/p>\n\n\n\n

Existing open-source solutions are often tightly coupled with specific CNI’s, OS, or data planes, thereby limiting their versatility and use. For this reason, Retina has been designed and developed to be a highly versatile, adaptable, and extensible framework of plugins capable of working seamlessly with any CNI, OS, or cloud provider\u2014making it a valuable addition to any existing toolset. Retina supports both Linux and Windows data planes, ensuring it meets the diverse needs of infrastructure- and site-reliability engineers, while maintaining a minimal memory and CPU footprint on the cluster\u2014this remains true even at scale. Retina\u2019s pluggability<\/strong> design ethos helps us easily extend and adapt to address new use cases without depending on any specific CNI, OS, or data plane.<\/p>\n\n\n

\"Architecture
Figure 1: Architecture overview of Retina<\/figcaption><\/figure>\n\n\n\n

One of Retina\u2019s key features provides deep network traffic insights that include Layer 4 (L4) metrics, Domain Name System (DNS) metrics, and distributed packet captures. It seamlessly integrates the Kubernetes app model offering pod-level metrics with detailed context. It emits actionable networking observability data into industry-standard Prometheus metrics providing node-level metrics (for example, forward, drop, Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and Linux utility) and pod-level metrics (such as basic metrics, DNS, and API server latency.)<\/p>\n\n\n\n

Retina’s distributed packet captures are label-driven\u2014allowing users to specify what, where, and who to capture packets from. Additionally, it provides historical context of network flow logs and advanced debugging capabilities that enhance network troubleshooting and performance optimization.<\/p>\n\n\n\n

Our vision for Retina<\/h2>\n\n\n\n

Many enterprises are multi-cloud and want solutions that work well not just on Microsoft Azure<\/a>, but on other clouds as well as on-premises. Retina is open-source and multi-cloud from day one. By open-sourcing Retina, we aim to share our knowledge and vision for Kubernetes networking observability with the broader cloud-native ecosystem. Our hope is that Retina will evolve and grow through collaboration with other developers and organizations who share similar experiences and goals in this field.<\/p>\n\n\n\n

In terms of architecture, extensibility was key from the outset and will remain going forward. Retina offers extensibility in data collection\u2014allowing users to easily add new metrics and insights. It also offers extensibility in exporters\u2014enabling users to integrate with other monitoring systems and tools. This flexibility ensures that Retina can adapt to different use cases and environments, making it a versatile and powerful platform for Kubernetes networking observability. In conclusion, we envision Retina as a platform allowing anyone to contribute, extend, and innovate on ultimately creating a robust, purpose-built, and comprehensive solution for Kubernetes networking observability.<\/p>\n\n\n\n

Getting started<\/h2>\n\n\n\n

We hope this blog post has given you an overview of Retina, its features, and benefits. Retina is a project that we are very excited about, and we welcome your feedback, suggestions, and contributions. Whether you are a network engineer, a DevOps engineer, a site reliability engineer, or a cloud-native enthusiast, we invite you to try out Retina and see how it can help you gain deeper visibility into your Kubernetes networks.<\/p>\n\n\n\n