{"id":2572,"date":"2018-06-27T00:00:00","date_gmt":"2018-06-27T07:00:00","guid":{"rendered":"https:\/\/azure.microsoft.com\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight"},"modified":"2023-08-01T16:52:34","modified_gmt":"2023-08-01T23:52:34","slug":"enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight","status":"publish","type":"post","link":"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/","title":{"rendered":"Enterprises get deeper insights with Hadoop and Spark updates on Azure HDInsight"},"content":{"rendered":"<p>Azure HDInsight is one of the most popular services amongst enterprise for open source Hadoop &#038; Spark analytics on Azure. With the <a href=\"https:\/\/azure.microsoft.com\/en-us\/blog\/azure-hdinsight-announcements-significant-price-reduction-and-amazing-new-capabilities\/\" target=\"_blank\" rel=\"noopener\">plus 50 percent price cut on HDInsight<\/a>, customers moving to the cloud are reaping more savings than ever.<\/p>\n<p><em>\u201c<\/em><a href=\"https:\/\/customers.microsoft.com\/en-us\/story\/pros\" target=\"_blank\" rel=\"noopener\"><em>PROS<\/em><\/a><em> is a pioneer in using machine learning to give companies an accurate and profitable pricing. PROS Guidance product runs enormously complex pricing calculations based on variables that comprise multiple terabytes of data. In Azure HDInsight, a process that formerly took several days now takes just a few minutes.\u201d<\/em>&#8211; Ed Gonzalez, Product Manager, PROS<\/p>\n<p>Today we are announcing updates to Apache Spark, Apache Kafka, ML Services, Azure Data Lake Storage Gen2 and enhancements to Enterprise Security Package. These new capabilities will continue to drive savings for many of our customers. In addition to this, Microsoft is continuing to deepen its commitment to the Apache Hadoop ecosystem and has extended its partnership with Hortonworks to bring the best of Apache Hadoop and the open source big data analytics to the Cloud.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" alt=\"PartnerLogos\" height=\"198\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/06\/7ecd4fe7-bc84-44f2-ad6f-9fdab7c78ea0.webp\" title=\"PartnerLogos\" width=\"1225\"><\/p>\n<h2>Continued investment in Open Source for new capabilities and reliability<\/h2>\n<h3>Reliable Open Source<\/h3>\n<p>Microsoft\u2019s is contributing to Apache Hadoop ecosystem and also ensuring Azure is the most reliable place to run this ecosystem.\u00a0 The ecosystem has dependencies on many open source projects which need to be configured together for the stack to work. HDInsight provides pre-tuned clusters out of the box for the best performance. Today we are enabling updates to Apache Hadoop, <a href=\"https:\/\/www.apache.org\/dist\/kafka\/1.0.0\/RELEASE_NOTES.html\" target=\"_blank\" rel=\"noopener\">Apache Spark 2.3, Apache Kafka 1.0<\/a> and 2000 plus bug fixes across 20 plus Open Source frameworks that are part of HDInsight.\u00a0<\/p>\n<p><em>\u201cHDInsight is the best place to run open source frameworks for big data. It meets key enterprise needs allowing them to easily modernize their on-premise solutions on a fully managed HDInsight cloud service.\u201d<\/em> &#8211; Rohan Kumar, Corporate Vice President, Microsoft<\/p>\n<h2>Enabling data scientists with Machine Learning Services 9.3<\/h2>\n<p>Today, we are excited to announce the general availability of Machine Learning (ML) Services 9.3 on Azure HDInsight. With this release, we are providing data scientists and engineers with the best of open source enhanced with algorithmic innovations and ease of operationalization, all available in their preferred language with the speed of Apache Spark. This release expands upon the capabilities offered in R Server with added support for Python, leading to the cluster name change from R Server to ML Services. <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/hdinsight\/r-server\/r-server-get-started\" target=\"_blank\" rel=\"noopener\">Get started with ML Services on HDInsight<\/a>.<\/p>\n<h2>Introducing an all new Azure Data Lake Storage Gen2, the Data Lake for everyone<\/h2>\n<h3>Gen2 Storage<\/h3>\n<p>Today\u2019s Data Lake options require customers to choose between scalability, availability, and cost with the security they require. Azure Data Lake Storage provides a no-compromise foundation for building scalable, and highly performant big data analytics solutions without trading off security and cost. <a href=\"https:\/\/azure.microsoft.com\/en-us\/blog\/azure-simplifies-cloud-analytics\/\" target=\"_blank\" rel=\"noopener\">Microsoft is announcing a preview of Azure Data Lake Storage Gen2<\/a>, a globally available HDFS filesystem to store and analyze petabyte-size files and trillions of objects. Today we are excited to announce a preview of <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/storage\/data-lake-storage\/quickstart-create-connect-hdi-cluster\" target=\"_blank\" rel=\"noopener\">HDInsight with Azure Data Lake Storage Gen2<\/a>.<\/p>\n<h2>Uncompromising on security: Virtual Network Service Endpoints<\/h2>\n<h3>Service Endpoints<\/h3>\n<p>Today we are enhancing HDInsight to include support for <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/virtual-network\/virtual-network-service-endpoints-overview\" target=\"_blank\" rel=\"noopener\">Virtual Network Service Endpoints<\/a> which allows customers to securely connect to Azure Blob Storage, Azure Data Lake Storage Gen2, Cosmos DB and SQL databases. By enabling a Service Endpoint for Azure HDInsight, traffic flows through a secured route from within the Azure data center.<\/p>\n<h2>Secure and compliant<\/h2>\n<p>HDInsight brings <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/hdinsight\/domain-joined\/apache-domain-joined-introduction\" target=\"_blank\" rel=\"noopener\">enterprise grade protection<\/a> of your data with encryption, virtual networks, Active Directory based authentication, role-based authorization, fine grained access control, <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/hdinsight\/hdinsight-hadoop-oms-log-analytics-tutorial\" target=\"_blank\" rel=\"noopener\">single pane of glass for monitoring<\/a> and more. The service is globally available in 20 plus regions including sovereign clouds in US, Germany and China and meets key compliance standards such has HIPPA, PCI, ISO and more.<\/p>\n<h2>Delivering cost-effectiveness<\/h2>\n<h3>50 percent price cut<\/h3>\n<p>With pay per use billing, <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/hdinsight\/hdinsight-hadoop-create-linux-clusters-adf\" target=\"_blank\" rel=\"noopener\">on demand clusters<\/a>, <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/hdinsight\/hdinsight-administer-use-portal-linux#scale-clusters\" target=\"_blank\" rel=\"noopener\">scale up or down<\/a> and separation of compute and storage customers are able to run their big data jobs more efficiently. We reduced the price of HDInsight by <a href=\"https:\/\/azure.microsoft.com\/en-us\/blog\/azure-hdinsight-announcements-significant-price-reduction-and-amazing-new-capabilities\/\" target=\"_blank\" rel=\"noopener\">plus 50 percent<\/a>, so you can enjoy even more cost savings.<\/p>\n<h2>Try HDInsight now<\/h2>\n<p>We hope you take full advantage of today\u2019s announcements and we are excited to see what you will build with Azure HDInsight. <a href=\"https:\/\/azure.microsoft.com\/en-us\/blog\/azure-hdinsight-training-resources-learn-about-big-data-using-open-source-technologies\/\" target=\"_blank\" rel=\"noopener\">Read this developer guide<\/a> and follow the quick start guide to learn more about implementing these pipelines and architectures on Azure HDInsight. Stay up-to-date on the latest Azure HDInsight news and features by following us on Twitter <a href=\"https:\/\/twitter.com\/search?q=%23HDInsight&#038;src=typd\" target=\"_blank\" rel=\"noopener\">#HDInsight<\/a> and <a href=\"https:\/\/twitter.com\/AzureHDInsight\" target=\"_blank\" rel=\"noopener\">@AzureHDInsight<\/a>. For questions and feedback, please reach out to <a href=\"mailto:AskHDInsight@microsoft.com\">AskHDInsight@microsoft.com<\/a>.<\/p>\n<h2>About HDInsight<\/h2>\n<p>Azure HDInsight is Microsoft\u2019s premium managed offering for running open source workloads on Azure. Today, we are excited to announce several new capabilities across a wide range of OSS frameworks.<\/p>\n<p>Azure HDInsight powers some of the top customer\u2019s mission critical applications ranging in a wide variety of sectors including, manufacturing, retail education, nonprofit, government, healthcare, media, banking, telecommunication, insurance and many more industries ranging in use cases from ETL to Data Warehousing, from Machine Learning to IoT and many more.<\/p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Azure HDInsight is Microsoft\u2019s premium managed offering for running open source workloads on Azure. Today, we are excited to announce several new capabilities across a wide range of OSS frameworks.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"ms_queue_id":[],"ep_exclude_from_search":false,"_classifai_error":"","_classifai_text_to_speech_error":"","_alt_title":"","footnotes":"","msx_community_cta_settings":[]},"categories":[1474,1491],"tags":[],"audience":[3054,3057,3053,3056],"content-type":[],"product":[1610,2895],"tech-community":[],"topic":[],"coauthors":[97],"class_list":["post-2572","post","type-post","status-publish","format-standard","hentry","category-analytics","category-storage","audience-business-decision-makers","audience-data-professionals","audience-it-decision-makers","audience-it-implementors","product-azure-data-lake-storage","product-azure-hdinsight-on-azure-kubernetes-service-aks"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.2 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Enterprises get deeper insights with Hadoop and Spark updates on Azure HDInsight | Microsoft Azure Blog<\/title>\n<meta name=\"description\" content=\"Azure HDInsight is Microsoft\u2019s premium managed offering for running open source workloads on Azure. Today, we are excited to announce several new capabilities across a wide range of OSS frameworks.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Enterprises get deeper insights with Hadoop and Spark updates on Azure HDInsight | Microsoft Azure Blog\" \/>\n<meta property=\"og:description\" content=\"Azure HDInsight is Microsoft\u2019s premium managed offering for running open source workloads on Azure. Today, we are excited to announce several new capabilities across a wide range of OSS frameworks.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/\" \/>\n<meta property=\"og:site_name\" content=\"Microsoft Azure Blog\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/microsoftazure\" \/>\n<meta property=\"article:published_time\" content=\"2018-06-27T07:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-08-01T23:52:34+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/06\/7ecd4fe7-bc84-44f2-ad6f-9fdab7c78ea0.webp\" \/>\n<meta name=\"author\" content=\"Microsoft Azure\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@azure\" \/>\n<meta name=\"twitter:site\" content=\"@azure\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Microsoft Azure\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/\"},\"author\":[{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/microsoft-azure\/\",\"@type\":\"Person\",\"@name\":\"Microsoft Azure\"}],\"headline\":\"Enterprises get deeper insights with Hadoop and Spark updates on Azure HDInsight\",\"datePublished\":\"2018-06-27T07:00:00+00:00\",\"dateModified\":\"2023-08-01T23:52:34+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/\"},\"wordCount\":843,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/06\/7ecd4fe7-bc84-44f2-ad6f-9fdab7c78ea0.webp\",\"articleSection\":[\"Analytics\",\"Storage\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/\",\"name\":\"Enterprises get deeper insights with Hadoop and Spark updates on Azure HDInsight | Microsoft Azure Blog\",\"isPartOf\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/06\/7ecd4fe7-bc84-44f2-ad6f-9fdab7c78ea0.webp\",\"datePublished\":\"2018-06-27T07:00:00+00:00\",\"dateModified\":\"2023-08-01T23:52:34+00:00\",\"description\":\"Azure HDInsight is Microsoft\u2019s premium managed offering for running open source workloads on Azure. Today, we are excited to announce several new capabilities across a wide range of OSS frameworks.\",\"breadcrumb\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/#primaryimage\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/06\/7ecd4fe7-bc84-44f2-ad6f-9fdab7c78ea0.webp\",\"contentUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/06\/7ecd4fe7-bc84-44f2-ad6f-9fdab7c78ea0.webp\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Blog home\",\"item\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Analytics\",\"item\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/category\/analytics\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Enterprises get deeper insights with Hadoop and Spark updates on Azure HDInsight\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#website\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/\",\"name\":\"Microsoft Azure Blog\",\"description\":\"Get the latest Azure news, updates, and announcements from the Azure blog. From product updates to hot topics, hear from the Azure experts.\",\"publisher\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization\",\"name\":\"Microsoft Azure Blog\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp\",\"contentUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp\",\"width\":512,\"height\":512,\"caption\":\"Microsoft Azure Blog\"},\"image\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/microsoftazure\",\"https:\/\/x.com\/azure\",\"https:\/\/www.instagram.com\/microsoftdeveloper\/\",\"https:\/\/www.linkedin.com\/company\/16188386\",\"https:\/\/www.youtube.com\/user\/windowsazure\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/person\/c702e5edd662b328b49b7e1180cab117\",\"name\":\"shakir\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g7664e653ea371ce16eaf75e9fa8952c4\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g\",\"caption\":\"shakir\"},\"sameAs\":[\"https:\/\/azure.microsoft.com\"],\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/shakir\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Enterprises get deeper insights with Hadoop and Spark updates on Azure HDInsight | Microsoft Azure Blog","description":"Azure HDInsight is Microsoft\u2019s premium managed offering for running open source workloads on Azure. Today, we are excited to announce several new capabilities across a wide range of OSS frameworks.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/","og_locale":"en_US","og_type":"article","og_title":"Enterprises get deeper insights with Hadoop and Spark updates on Azure HDInsight | Microsoft Azure Blog","og_description":"Azure HDInsight is Microsoft\u2019s premium managed offering for running open source workloads on Azure. Today, we are excited to announce several new capabilities across a wide range of OSS frameworks.","og_url":"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/","og_site_name":"Microsoft Azure Blog","article_publisher":"https:\/\/www.facebook.com\/microsoftazure","article_published_time":"2018-06-27T07:00:00+00:00","article_modified_time":"2023-08-01T23:52:34+00:00","og_image":[{"url":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/06\/7ecd4fe7-bc84-44f2-ad6f-9fdab7c78ea0.webp","type":"","width":"","height":""}],"author":"Microsoft Azure","twitter_card":"summary_large_image","twitter_creator":"@azure","twitter_site":"@azure","twitter_misc":{"Written by":"Microsoft Azure","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/#article","isPartOf":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/"},"author":[{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/microsoft-azure\/","@type":"Person","@name":"Microsoft Azure"}],"headline":"Enterprises get deeper insights with Hadoop and Spark updates on Azure HDInsight","datePublished":"2018-06-27T07:00:00+00:00","dateModified":"2023-08-01T23:52:34+00:00","mainEntityOfPage":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/"},"wordCount":843,"commentCount":0,"publisher":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization"},"image":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/#primaryimage"},"thumbnailUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/06\/7ecd4fe7-bc84-44f2-ad6f-9fdab7c78ea0.webp","articleSection":["Analytics","Storage"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/","name":"Enterprises get deeper insights with Hadoop and Spark updates on Azure HDInsight | Microsoft Azure Blog","isPartOf":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/#primaryimage"},"image":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/#primaryimage"},"thumbnailUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/06\/7ecd4fe7-bc84-44f2-ad6f-9fdab7c78ea0.webp","datePublished":"2018-06-27T07:00:00+00:00","dateModified":"2023-08-01T23:52:34+00:00","description":"Azure HDInsight is Microsoft\u2019s premium managed offering for running open source workloads on Azure. Today, we are excited to announce several new capabilities across a wide range of OSS frameworks.","breadcrumb":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/#primaryimage","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/06\/7ecd4fe7-bc84-44f2-ad6f-9fdab7c78ea0.webp","contentUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/06\/7ecd4fe7-bc84-44f2-ad6f-9fdab7c78ea0.webp"},{"@type":"BreadcrumbList","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/enterprises-get-deeper-insights-with-hadoop-and-spark-updates-on-azure-hdinsight\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Blog home","item":"https:\/\/azure.microsoft.com\/en-us\/blog\/"},{"@type":"ListItem","position":2,"name":"Analytics","item":"https:\/\/azure.microsoft.com\/en-us\/blog\/category\/analytics\/"},{"@type":"ListItem","position":3,"name":"Enterprises get deeper insights with Hadoop and Spark updates on Azure HDInsight"}]},{"@type":"WebSite","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#website","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/","name":"Microsoft Azure Blog","description":"Get the latest Azure news, updates, and announcements from the Azure blog. From product updates to hot topics, hear from the Azure experts.","publisher":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/azure.microsoft.com\/en-us\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization","name":"Microsoft Azure Blog","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp","contentUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp","width":512,"height":512,"caption":"Microsoft Azure Blog"},"image":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/microsoftazure","https:\/\/x.com\/azure","https:\/\/www.instagram.com\/microsoftdeveloper\/","https:\/\/www.linkedin.com\/company\/16188386","https:\/\/www.youtube.com\/user\/windowsazure"]},{"@type":"Person","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/person\/c702e5edd662b328b49b7e1180cab117","name":"shakir","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g7664e653ea371ce16eaf75e9fa8952c4","url":"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g","caption":"shakir"},"sameAs":["https:\/\/azure.microsoft.com"],"url":"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/shakir\/"}]}},"msxcm_display_generated_audio":false,"msxcm_animated_featured_image":null,"distributor_meta":false,"distributor_terms":false,"distributor_media":false,"distributor_original_site_name":"Microsoft Azure Blog","distributor_original_site_url":"https:\/\/azure.microsoft.com\/en-us\/blog","push-errors":false,"_links":{"self":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/posts\/2572","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/comments?post=2572"}],"version-history":[{"count":0,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/posts\/2572\/revisions"}],"wp:attachment":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/media?parent=2572"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/categories?post=2572"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/tags?post=2572"},{"taxonomy":"audience","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/audience?post=2572"},{"taxonomy":"content-type","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/content-type?post=2572"},{"taxonomy":"product","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/product?post=2572"},{"taxonomy":"tech-community","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/tech-community?post=2572"},{"taxonomy":"topic","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/topic?post=2572"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/coauthors?post=2572"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}