{"id":33016,"date":"2024-04-08T08:00:00","date_gmt":"2024-04-08T15:00:00","guid":{"rendered":"https:\/\/azure.microsoft.com\/en-us\/blog\/?p=33016"},"modified":"2024-04-16T08:44:21","modified_gmt":"2024-04-16T15:44:21","slug":"advancing-memory-leak-detection-with-aiops-introducing-resin","status":"publish","type":"post","link":"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/","title":{"rendered":"Advancing memory leak detection with AIOps\u2014introducing RESIN"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">\u201c<em>Operating a cloud infrastructure at global scale is a large and complex task, particularly when it comes to service standard and quality. In a <a href=\"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-azure-service-quality-with-artificial-intelligence-aiops\/\">previous blog<\/a>, we shared how AIOps was leveraged to improve service quality, engineering efficiency, and customer experience. In this blog, I\u2019ve asked Jian Zhang, Principal Program Manager from the AIOps Platform and Experiences team to share how AI and machine learning is used to automate memory leak detection, diagnosis, and mitigation for service quality.<\/em>\u201d<em>\u2014<\/em>Mark Russinovich, Chief Technology Officer, Azure.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\" \/>\n\n\n\n<p class=\"wp-block-paragraph\"><em>This post includes contributions from Principal Data Scientist Manager Cong Chen and Partner Data Scientist Manager Yingnong Dang of Azure AIOps Platform and Experience team, Senior Data Scientist Vivek Ramamurthy, Principal Data Scientist Manager Ze Li, and Partner Group Software Engineering Manager Murali Chintalapati of Azure Core team<\/em>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In the ever-evolving landscape of cloud computing, memory leaks represent a persistent challenge\u2014affecting performance, stability, and ultimately, the user experience. Therefore, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/resin-a-holistic-service-for-dealing-with-memory-leaks-in-production-cloud-infrastructure\/\">memory leak detection<\/a> is important to cloud service quality. Memory leaks happen when memory is allocated but not released in a timely manner unintentionally. It causes potential performance degradation of the component and possible crashes of the operation system (OS). Even worse, it often affects other processes running on the same machine, causing them to be slowed down or even killed.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Given the impact of memory leak issues, there are many studies and solutions for memory leak detection. Traditional detection solutions fall into two categories: static and dynamic detection. The static leak detection techniques analyze software source code and deduce potential leaks whereas the dynamic method detects leak through instrumenting a program and tracks the object references at runtime.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">However, these conventional techniques for detecting memory leaks are not adequate to meet the needs of leak detection in a cloud environment. The static approaches have limited accuracy and scalability, especially for leaks that result from cross-component contract violations, which need rich domain knowledge to capture statically. In general, the dynamic approaches are more suitable for a cloud environment. However, they are intrusive and require extensive instrumentations. Furthermore, they introduce high runtime overhead which is costly for cloud services.<\/p>\n\n\n<div class=\"wp-block-msxcm-cta-block\" data-moray data-bi-an=\"CTA Block\">\n\t<div class=\"card d-block mx-ng mx-md-0\">\n\t\t<div class=\"row no-gutters\">\n\n\t\t\t\t\t\t\t<div class=\"col-md-4\">\n\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"575\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/02\/Azure_Blog_Abstract-07_1260x708-1-1024x575.jpg\" class=\"card-img img-object-cover\" alt=\"A decorative abstract green and blue pattern\" srcset=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/02\/Azure_Blog_Abstract-07_1260x708-1-1024x575.jpg 1024w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/02\/Azure_Blog_Abstract-07_1260x708-1-300x169.jpg 300w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/02\/Azure_Blog_Abstract-07_1260x708-1-768x432.jpg 768w, https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/02\/Azure_Blog_Abstract-07_1260x708-1.jpg 1260w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/>\t\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"d-flex col-md\">\n\t\t\t\t<div class=\"card-body align-self-center p-4 p-md-5\">\n\t\t\t\t\t\n\t\t\t\t\t<h2>RESIN<\/h2>\n\n\t\t\t\t\t<div class=\"mb-3\">\n\t\t\t\t\t\t<p>Designed to address memory leaks in production cloud infrastructure<\/p>\n\t\t\t\t\t<\/div>\n\n\t\t\t\t\t\t\t\t\t\t\t<div class=\"link-group\">\n\t\t\t\t\t\t\t<a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/resin-a-holistic-service-for-dealing-with-memory-leaks-in-production-cloud-infrastructure\/\" class=\"btn btn-link text-decoration-none p-0\" target=\"_blank\">\n\t\t\t\t\t\t\t\t<span>Explore the research<\/span>\n\t\t\t\t\t\t\t\t<span class=\"glyph-append glyph-append-chevron-right glyph-append-xsmall\"><\/span>\n\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t<\/div>\n\n\t\t\t\t\t<\/div>\n\t<\/div>\n<\/div>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"introducing-resin\">Introducing RESIN<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Today, we are introducing RESIN, an end-to-end memory leak detection service designed to holistically address memory leaks in large cloud infrastructure. RESIN has been used in <a href=\"http:\/\/azure.microsoft.com\">Microsoft Azure<\/a> production and demonstrated effective leak detection with high accuracy and low overhead.<\/p>\n\n\n<div class=\"wp-block-msxcm-kicker-container\">\n\t<div class=\" wp-block-msxcm-kicker-block wp-block-msxcm-kicker--align-left\" data-bi-an=\"Kicker Left\">\n\t\t<p class=\"wp-block-msxcm-kicker__title small text-neutral-400 text-uppercase\">\n\t\t\tREsin: a holistic service for memory leaks\t\t<\/p>\n\t\t<a\n\t\t\tclass=\"wp-block-msxcm-kicker__cta btn btn-link p-0 text-decoration-none\"\n\t\t\thref=\"https:\/\/www.usenix.org\/system\/files\/osdi22-lou-resin.pdf\"\n\t\t\ttarget=\"_blank\"\t\t>\n\t\t\t<span>Read the report<\/span> <span class=\"glyph-append glyph-append-xsmall wp-block-msxcm-kicker__glyph glyph-append-go\"><\/span>\n\t\t<\/a>\n\t<\/div>\n<\/div>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"resin-system-workflow\">RESIN system workflow<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A large cloud infrastructure could consist of hundreds of software components owned by different teams. Prior to RESIN, memory leak detection was an individual team\u2019s effort in Microsoft Azure. As shown in Figure 1, RESIN utilizes a centralized approach, which conducts leak detection in multi-stages for the benefit of low overhead, high accuracy, and scalability. This approach does not require access to components\u2019 source code or extensive instrumentation or re-compilation.<\/p>\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/04\/RESIN-1.webp\" alt=\"flowchart of RESIN workflow\" class=\"wp-image-33024 webp-format\" data-orig-src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/04\/RESIN-1.webp\"><figcaption class=\"wp-element-caption\"><strong>Figure 1<\/strong>: RESIN workflow<\/figcaption><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">RESIN conducts low-overhead monitoring using monitoring agents to collect memory telemetry data at host level. A remote service is used to aggregate and analyze data from different hosts using a bucketization-pivot scheme. When leaking is detected in a bucket, RESIN triggers an analysis on the process instances in the bucket. For highly suspicious leaks identified, RESIN performs live heap snapshotting and compares it to regular heap snapshots in a reference database. After generating multiple heap snapshots, RESIN runs diagnosis algorithm to localize the root cause of the leak and generates a diagnosis report to attach to the alert ticket to assist developers for further analysis\u2014ultimately, RESIN automatically mitigates the leaking process.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"detection-algorithms\">Detection algorithms<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">There are unique challenges in memory leak detection in cloud infrastructure:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">Noisy memory usage caused by changing workload and interference in the environment results in high noise in detection using static threshold-based approach.<\/li>\n\n\n\n<li class=\"wp-block-list-item\">Memory leak in production systems are usually fail-slow faults that could last days, weeks, or even months and it can be difficult to capture gradual change over long periods of time in a timely manner.<\/li>\n\n\n\n<li class=\"wp-block-list-item\">At the scale of Azure global cloud, it\u2019s not practical to collect fine-grained data over long period of time.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">To address these challenges, RESIN uses a two-level scheme to detect memory leak symptoms: A global bucket-based pivot analysis to identify suspicious components and a local individual process leak detection to identify leaking processes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">With the bucket-based pivot analysis at component level, we categorize raw memory usage into a number of buckets and transform the usage data into summary about number of hosts in each bucket. In addition, a severity score for each bucket is calculated based on the deviations and host count in the bucket. Anomaly detection is performed on the time-series data of each bucket of each component. The bucketization approach not only robustly represents the workload trend with noise tolerance but also reduces computational load of the anomaly detection.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">However, detection at component level only is not sufficient for developers to investigate the leak efficiently because, normally, many processes run on a component. When a leaking bucket is identified at the component level, RESIN runs a second-level detection scheme at the process granularity to narrow down the scope of investigation. It outputs the suspected leaking process, its start and end time, and the severity score.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"diagnosis-of-detected-leaks\">Diagnosis of detected leaks<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Once a memory leak is detected, RESIN takes a snapshot of live heap, which contains all memory allocations referenced by running application, and analyzes the snapshots to pinpoint the root cause of the detected leak. This makes memory leak alert actionable.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">RESIN also leverages Windows heap manager\u2019s snapshot capability to perform live profiling. However, the heap collection is expensive and could be intrusive to the host\u2019s performance. To minimize overhead caused by heap collection, a few considerations are considered to decide how snapshots are taken.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">The heap manager only stores limited information in each snapshot such as stack trace and size for each active allocation in each snapshot.<\/li>\n\n\n\n<li class=\"wp-block-list-item\">RESIN prioritizes candidate hosts for snapshotting based on leak severity, noise level, and customer impact. By default, the top three hosts in the suspected list are selected to ensure successful collection.<\/li>\n\n\n\n<li class=\"wp-block-list-item\">RESIN utilizes a long-term, trigger-based strategy to ensure the snapshots capture the complete leak. To facilitate the decision regarding when to stop the trace collection, RESIN analyzes memory growth patterns (such as steady, spike, or stair) and takes a pattern-based approach to decide the trace completion triggers.<\/li>\n\n\n\n<li class=\"wp-block-list-item\">RESIN uses a periodical fingerprinting process to build reference snapshots, which is compared with the snapshot of suspected leaking process to support diagnosis.<\/li>\n\n\n\n<li class=\"wp-block-list-item\">RESIN analyzes the collected snapshots to output stack traces of the root.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"mitigation-of-detected-leaks\">Mitigation of detected leaks<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">When a memory leak is detected, RESIN attempts to automatically mitigate the issue to avoid further customer impact. Depending on the nature of the leak, a few types of mitigation actions are taken to mitigate the issue. RESIN uses a rule-based decision tree to choose a mitigation action that minimizes the impact.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If the memory leak is localized to a single process or Windows service, RESIN attempts the lightest mitigation by simply restarting the process or the service. OS reboot can resolve software memory leaks but takes a much longer time and can cause virtual machine downtime and as such, is normally reserved as the last resort. For a non-empty host, RESIN utilizes solutions such as <a href=\"https:\/\/azure.microsoft.com\/en-us\/blog\/improving-azure-virtual-machines-resiliency-with-project-tardigrade\/\">Project Tardigrade<\/a>, which skips hardware initialization and only performs a kernel soft reboot, after <a href=\"https:\/\/azure.microsoft.com\/en-us\/blog\/improving-azure-virtual-machine-resiliency-with-predictive-ml-and-live-migration\/\">live virtual machine migration,<\/a> to minimize user impact. A full OS reboot is performed only when the soft reboot is ineffective.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">RESIN stops applying mitigation actions to a target once the detection engine no longer considers the target leaking.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"result-and-impact-of-memory-leak-detection\">Result and impact of memory leak detection<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">RESIN has been running in production in Azure since late 2018 and to date, it has been used to monitor millions of host nodes and hundreds of host processes daily. Overall, we achieved 85% precision and 91% recall with RESIN memory leak detection,<sup>1<\/sup> despite the rapidly growing scale of the cloud infrastructure monitored.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The end-to-end benefits brought by RESIN are clearly demonstrated by two key metrics:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li class=\"wp-block-list-item\"><strong>Virtual machine unexpected reboots<\/strong>: the average number of reboots per one hundred thousand hosts per day due to low memory.<\/li>\n\n\n\n<li class=\"wp-block-list-item\"><strong>Virtual machine allocation error<\/strong>: the ratio of erroneous virtual machine allocation requests due to low memory.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">Between September 2020 and December 2023, the virtual machine reboots were reduced by nearly 100 times, and allocation error rates were reduced by over 30 times. Furthermore, since 2020, no severe outages have been caused by Azure host memory leaks.<sup>1<\/sup><\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"learn-more-about-resin\">Learn more about RESIN<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">You can improve the reliability and performance of your cloud infrastructure, and prevent issues caused by memory leaks through RESIN\u2019s end-to-end memory leak detection capabilities designed to holistically address memory leaks in large cloud infrastructure. To learn more, <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/publication\/resin-a-holistic-service-for-dealing-with-memory-leaks-in-production-cloud-infrastructure\/\">read the publication<\/a>.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\" \/>\n\n\n\n<p class=\"wp-block-paragraph\"><sup>1<\/sup> <strong>RESIN: A Holistic Service for Dealing with Memory Leaks in Production Cloud Infrastructure<\/strong>, Chang Lou,\u202fJohns Hopkins University;\u202fCong Chen,\u202fMicrosoft Azure;\u202fPeng Huang,\u202fJohns Hopkins University;\u202fYingnong Dang,\u202fMicrosoft Azure;\u202fSi Qin,\u202fMicrosoft Research;\u202fXinsheng Yang,\u202fMeta;\u202fXukun Li,\u202fMicrosoft Azure;\u202fQingwei Lin,\u202fMicrosoft Research;\u202fMurali Chintalapati,\u202fMicrosoft Azure, OSDI\u201922.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>We are introducing RESIN, an end-to-end memory leak detection service designed to holistically address memory leaks in large cloud infrastructure.<\/p>\n","protected":false},"author":42,"featured_media":33029,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"ms_queue_id":[],"ep_exclude_from_search":false,"_classifai_error":"","_classifai_text_to_speech_error":"","_alt_title":"","footnotes":"","msx_community_cta_settings":[]},"categories":[1482],"tags":[1505],"audience":[3054,3053],"content-type":[1465],"product":[1535],"tech-community":[],"topic":[],"coauthors":[21],"class_list":["post-33016","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-management-and-governance","tag-advancing-reliability","audience-business-decision-makers","audience-it-decision-makers","content-type-announcements","product-microsoft-azure-portal","review-flag-1-1680286581-825","review-flag-fall-1680286584-980","review-flag-machi-1680286585-314","review-flag-partn-1680286579-901"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.2 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Advancing memory leak detection with AIOps\u2014introducing RESIN | Microsoft Azure Blog<\/title>\n<meta name=\"description\" content=\"We are introducing RESIN, an end-to-end memory leak detection service designed to holistically address memory leaks in large cloud infrastructure.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Advancing memory leak detection with AIOps\u2014introducing RESIN | Microsoft Azure Blog\" \/>\n<meta property=\"og:description\" content=\"We are introducing RESIN, an end-to-end memory leak detection service designed to holistically address memory leaks in large cloud infrastructure.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/\" \/>\n<meta property=\"og:site_name\" content=\"Microsoft Azure Blog\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/microsoftazure\" \/>\n<meta property=\"article:published_time\" content=\"2024-04-08T15:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-04-16T15:44:21+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/04\/Azure_Hero_Cylinder_Blue_GreenGrad_cropped.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1260\" \/>\n\t<meta property=\"og:image:height\" content=\"708\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Mark Russinovich\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/04\/Azure_Hero_Cylinder_Blue_GreenGrad_cropped.png\" \/>\n<meta name=\"twitter:creator\" content=\"@azure\" \/>\n<meta name=\"twitter:site\" content=\"@azure\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Mark Russinovich\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/\"},\"author\":[{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/mark-russinovich\/\",\"@type\":\"Person\",\"@name\":\"Mark Russinovich\"}],\"headline\":\"Advancing memory leak detection with AIOps\u2014introducing RESIN\",\"datePublished\":\"2024-04-08T15:00:00+00:00\",\"dateModified\":\"2024-04-16T15:44:21+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/\"},\"wordCount\":1507,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/04\/Azure_Hero_Cylinder_Blue_GreenGrad_cropped.webp\",\"keywords\":[\"Advancing reliability\"],\"articleSection\":[\"Management and governance\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/\",\"name\":\"Advancing memory leak detection with AIOps\u2014introducing RESIN | Microsoft Azure Blog\",\"isPartOf\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/04\/Azure_Hero_Cylinder_Blue_GreenGrad_cropped.webp\",\"datePublished\":\"2024-04-08T15:00:00+00:00\",\"dateModified\":\"2024-04-16T15:44:21+00:00\",\"description\":\"We are introducing RESIN, an end-to-end memory leak detection service designed to holistically address memory leaks in large cloud infrastructure.\",\"breadcrumb\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/#primaryimage\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/04\/Azure_Hero_Cylinder_Blue_GreenGrad_cropped.webp\",\"contentUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/04\/Azure_Hero_Cylinder_Blue_GreenGrad_cropped.webp\",\"width\":1260,\"height\":708,\"caption\":\"abstract image\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Blog home\",\"item\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Management and governance\",\"item\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/category\/management-and-governance\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Advancing memory leak detection with AIOps\u2014introducing RESIN\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#website\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/\",\"name\":\"Microsoft Azure Blog\",\"description\":\"Get the latest Azure news, updates, and announcements from the Azure blog. From product updates to hot topics, hear from the Azure experts.\",\"publisher\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization\",\"name\":\"Microsoft Azure Blog\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp\",\"contentUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp\",\"width\":512,\"height\":512,\"caption\":\"Microsoft Azure Blog\"},\"image\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/microsoftazure\",\"https:\/\/x.com\/azure\",\"https:\/\/www.instagram.com\/microsoftdeveloper\/\",\"https:\/\/www.linkedin.com\/company\/16188386\",\"https:\/\/www.youtube.com\/user\/windowsazure\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/person\/b2603da1afac705823964361ce9072c0\",\"name\":\"Kristin Gallagher\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/295fa37b6bb2bbf59603c38b6ac7a7b4b86cd0f736387182fa9d0117f52cdf5e?s=96&d=mm&r=gb83eb8c5c3f8feea9763b473dabe8524\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/295fa37b6bb2bbf59603c38b6ac7a7b4b86cd0f736387182fa9d0117f52cdf5e?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/295fa37b6bb2bbf59603c38b6ac7a7b4b86cd0f736387182fa9d0117f52cdf5e?s=96&d=mm&r=g\",\"caption\":\"Kristin Gallagher\"},\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/kristingallagher\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Advancing memory leak detection with AIOps\u2014introducing RESIN | Microsoft Azure Blog","description":"We are introducing RESIN, an end-to-end memory leak detection service designed to holistically address memory leaks in large cloud infrastructure.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/","og_locale":"en_US","og_type":"article","og_title":"Advancing memory leak detection with AIOps\u2014introducing RESIN | Microsoft Azure Blog","og_description":"We are introducing RESIN, an end-to-end memory leak detection service designed to holistically address memory leaks in large cloud infrastructure.","og_url":"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/","og_site_name":"Microsoft Azure Blog","article_publisher":"https:\/\/www.facebook.com\/microsoftazure","article_published_time":"2024-04-08T15:00:00+00:00","article_modified_time":"2024-04-16T15:44:21+00:00","og_image":[{"width":1260,"height":708,"url":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/04\/Azure_Hero_Cylinder_Blue_GreenGrad_cropped.png","type":"image\/png"}],"author":"Mark Russinovich","twitter_card":"summary_large_image","twitter_image":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/04\/Azure_Hero_Cylinder_Blue_GreenGrad_cropped.png","twitter_creator":"@azure","twitter_site":"@azure","twitter_misc":{"Written by":"Mark Russinovich","Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/#article","isPartOf":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/"},"author":[{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/mark-russinovich\/","@type":"Person","@name":"Mark Russinovich"}],"headline":"Advancing memory leak detection with AIOps\u2014introducing RESIN","datePublished":"2024-04-08T15:00:00+00:00","dateModified":"2024-04-16T15:44:21+00:00","mainEntityOfPage":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/"},"wordCount":1507,"commentCount":0,"publisher":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization"},"image":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/#primaryimage"},"thumbnailUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/04\/Azure_Hero_Cylinder_Blue_GreenGrad_cropped.webp","keywords":["Advancing reliability"],"articleSection":["Management and governance"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/","name":"Advancing memory leak detection with AIOps\u2014introducing RESIN | Microsoft Azure Blog","isPartOf":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/#primaryimage"},"image":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/#primaryimage"},"thumbnailUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/04\/Azure_Hero_Cylinder_Blue_GreenGrad_cropped.webp","datePublished":"2024-04-08T15:00:00+00:00","dateModified":"2024-04-16T15:44:21+00:00","description":"We are introducing RESIN, an end-to-end memory leak detection service designed to holistically address memory leaks in large cloud infrastructure.","breadcrumb":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/#primaryimage","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/04\/Azure_Hero_Cylinder_Blue_GreenGrad_cropped.webp","contentUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/04\/Azure_Hero_Cylinder_Blue_GreenGrad_cropped.webp","width":1260,"height":708,"caption":"abstract image"},{"@type":"BreadcrumbList","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/advancing-memory-leak-detection-with-aiops-introducing-resin\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Blog home","item":"https:\/\/azure.microsoft.com\/en-us\/blog\/"},{"@type":"ListItem","position":2,"name":"Management and governance","item":"https:\/\/azure.microsoft.com\/en-us\/blog\/category\/management-and-governance\/"},{"@type":"ListItem","position":3,"name":"Advancing memory leak detection with AIOps\u2014introducing RESIN"}]},{"@type":"WebSite","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#website","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/","name":"Microsoft Azure Blog","description":"Get the latest Azure news, updates, and announcements from the Azure blog. From product updates to hot topics, hear from the Azure experts.","publisher":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/azure.microsoft.com\/en-us\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization","name":"Microsoft Azure Blog","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp","contentUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp","width":512,"height":512,"caption":"Microsoft Azure Blog"},"image":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/microsoftazure","https:\/\/x.com\/azure","https:\/\/www.instagram.com\/microsoftdeveloper\/","https:\/\/www.linkedin.com\/company\/16188386","https:\/\/www.youtube.com\/user\/windowsazure"]},{"@type":"Person","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/person\/b2603da1afac705823964361ce9072c0","name":"Kristin Gallagher","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/295fa37b6bb2bbf59603c38b6ac7a7b4b86cd0f736387182fa9d0117f52cdf5e?s=96&d=mm&r=gb83eb8c5c3f8feea9763b473dabe8524","url":"https:\/\/secure.gravatar.com\/avatar\/295fa37b6bb2bbf59603c38b6ac7a7b4b86cd0f736387182fa9d0117f52cdf5e?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/295fa37b6bb2bbf59603c38b6ac7a7b4b86cd0f736387182fa9d0117f52cdf5e?s=96&d=mm&r=g","caption":"Kristin Gallagher"},"url":"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/kristingallagher\/"}]}},"msxcm_display_generated_audio":false,"msxcm_animated_featured_image":null,"distributor_meta":false,"distributor_terms":false,"distributor_media":false,"distributor_original_site_name":"Microsoft Azure Blog","distributor_original_site_url":"https:\/\/azure.microsoft.com\/en-us\/blog","push-errors":false,"_links":{"self":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/posts\/33016","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/users\/42"}],"replies":[{"embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/comments?post=33016"}],"version-history":[{"count":0,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/posts\/33016\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/media\/33029"}],"wp:attachment":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/media?parent=33016"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/categories?post=33016"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/tags?post=33016"},{"taxonomy":"audience","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/audience?post=33016"},{"taxonomy":"content-type","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/content-type?post=33016"},{"taxonomy":"product","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/product?post=33016"},{"taxonomy":"tech-community","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/tech-community?post=33016"},{"taxonomy":"topic","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/topic?post=33016"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/coauthors?post=33016"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}