{"id":1394,"date":"2019-04-24T00:00:00","date_gmt":"2019-04-24T00:00:00","guid":{"rendered":"https:\/\/azure.microsoft.com\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services"},"modified":"2025-06-20T12:49:00","modified_gmt":"2025-06-20T19:49:00","slug":"dear-spark-developers-welcome-to-azure-cognitive-services","status":"publish","type":"post","link":"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/","title":{"rendered":"Dear Spark developers: Welcome to Azure Cognitive Services"},"content":{"rendered":"\n<figure class=\"wp-block-image has-custom-border\"><img decoding=\"async\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/04\/dd10b978-1bcc-4d0a-84ee-b1d2223cfa85.webp\" alt=\"Azure Cognitive Services on Spark\" style=\"border-radius:0px\" title=\"Azure Cognitive Services on Spark\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><i>This post was co-authored by Mark Hamilton, Sudarshan Raghunathan, Chris Hoder, and the MMLSpark contributors.<\/i><\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"integrating-the-power-of-azure-cognitive-services-into-your-big-data-workflows-on-apache-spark\">Integrating the power of Azure Cognitive Services into your big data workflows on <a href=\"https:\/\/databricks.com\/spark\/about\">Apache Spark\u2122<\/a><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Today at Spark + AI Summit 2019, we&#8217;re excited to introduce a new set of models in the SparkML ecosystem that make it easy to leverage the Azure Cognitive Services at terabyte scales. With only a few lines of code, developers can embed cognitive services within your existing distributed machine learning pipelines in Spark ML. Additionally, these contributions allow Spark users to chain or <i>Pipeline<\/i> services together with deep networks, gradient boosted trees, and any SparkML model and apply these hybrid models in elastic and serverless distributed systems.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">From image recognition to object detection using speech recognition, translation, and text-to-speech, Azure Cognitive Services makes it easy for developers to add intelligent capabilities to their applications in any scenario. To this date, more than a million developers have already discovered and tried Cognitive Services to accelerate breakthrough experiences in their application.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"azure-cognitive-services-on-apache-spark\">Azure Cognitive Services on Apache Spark\u2122<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Cognitive Services on Spark enable working with Azure\u2019s Intelligent Services at massive scales with the Apache Spark<b>\u2122<\/b> distributed computing ecosystem. The Cognitive Services on Spark are compatible with any <b>Spark 2.4 cluster such as <\/b><a href=\"https:\/\/azure.microsoft.com\/en-us\/services\/databricks\/\"><b>Azure Databricks<\/b><\/a>, <a href=\"https:\/\/github.com\/Azure\/aztk\">Azure Distributed Data Engineering Toolkit<\/a>&nbsp;<a href=\"https:\/\/github.com\/Azure\/aztk\">(AZTK) on Azure Batch<\/a>, <a href=\"https:\/\/cloudblogs.microsoft.com\/sqlserver\/2018\/09\/24\/sql-server-2019-preview-combines-sql-server-and-apache-spark-to-create-a-unified-data-platform\/\">Spark in SQL Server<\/a>, and Spark clusters on <a href=\"https:\/\/azure.microsoft.com\/en-us\/services\/kubernetes-service\/\">Azure Kubernetes Service<\/a>. Furthermore, we provide idiomatic bindings in PySpark, Scala, Java, and R (Beta).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Cognitive Services on Spark allows users to embed general purpose and continuously improve intelligent models directly into their Apache Spark\u2122 and SQL computations. This contribution aims to liberate developers from low-level networking details, so they can focus on creating intelligent, distributed applications. Each Cognitive Service is a SparkML transformer, so users can add services to existing SparkML pipelines. We also introduce a new type of API to the SparkML framework that allows users to parameterize models by either a single scalar, or a column of a distributed spark DataFrame. This API yields a succinct, yet powerful fluent query language that offers a full distributed parameterization without clutter. For more information, check out <a href=\"https:\/\/databricks.com\/sparkaisummit\/north-america\/sessions-single-2019?id=124\">our session<\/a>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"use-azure-cognitive-services-on-spark-in-these-3-simple-steps\">Use Azure Cognitive Services on Spark in these 3 simple steps:<\/h3>\n\n\n\n<ol class=\"wp-block-list\">\n<li class=\"wp-block-list-item\"><a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/cognitive-services\/cognitive-services-apis-create-account\">Create an Azure Cognitive Services Account<\/a><\/li>\n\n\n\n<li class=\"wp-block-list-item\"><a href=\"https:\/\/mmlspark.blob.core.windows.net\/website\/index.html#install\">Install MMLSpark on your Spark Cluster<\/a><\/li>\n\n\n\n<li class=\"wp-block-list-item\"><a href=\"https:\/\/github.com\/Azure\/mmlspark\/blob\/master\/notebooks\/samples\/AzureSearchIndex%20-%20Met%20Artworks.ipynb\">Try our example notebook<\/a><\/li>\n<\/ol>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"low-latency-high-throughput-workloads-with-the-cognitive-service-containers\">Low-latency, high-throughput workloads with the cognitive service containers<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The cognitive services on Spark are compatible with services from any region of the globe, however many scenarios require low or no-connectivity and ultra-low latency. To tackle these with the cognitive services on Spark, we have recently <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/cognitive-services\/cognitive-services-container-support\">released several cognitive services as docker containers<\/a>. These containers enable running cognitive services locally or directly on the worker nodes of your cluster for ultra-low latency workloads. To make it easy to create Spark Clusters with embedded cognitive services, we have <a href=\"https:\/\/github.com\/helm\/charts\/pull\/11285\">created a Helm Chart<\/a> for deploying a Spark clusters onto the popular container orchestration platform Kubernetes. Simply point the Cognitive Services on Spark at your container\u2019s URL to go local!<\/p>\n\n\n\n<figure class=\"wp-block-image has-custom-border\"><img decoding=\"async\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/04\/ed28fc76-d6ed-4f3d-9d5d-01b44376446c.webp\" alt=\"Cognitive Services in Containers\" style=\"border-radius:0px\" title=\"Cognitive Services in Containers\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"add-any-web-service-to-apache-spark-with-http-on-spark\">Add <em>a<\/em><i><em>n<\/em>y<\/i> web service to Apache Spark\u2122 with HTTP on Spark<\/h2>\n\n\n\n<figure class=\"wp-block-image has-custom-border\"><img decoding=\"async\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/04\/20304e6d-12ea-40fe-b5fd-cb4bab7b28a0.webp\" alt=\"Http on Spark.\" style=\"border-radius:0px\" title=\"\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The Cognitive Services are just one example of using networking to share software across ecosystems. The web is full of HTTP(S) web services that provide useful tools and serve as one of the standard patterns for making your code accessible in any language. Our goal is to allow Spark developers to tap into this richness from within their existing Spark pipelines.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">To this end, we present HTTP on Spark, an integration between the entire HTTP communication protocol and Spark SQL. HTTP on Spark allows Spark users to leverage the parallel networking capabilities of their cluster to integrate any local, docker, or web service. At a high level, HTTP on Spark provides a simple and principled way to integrate any framework into the Spark ecosystem.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">With HTTP on Spark, users can create and manipulate their requests and responses using SQL operations, maps, reduces, filters, and any tools from the Spark ecosystem. When combined with SparkML, users can chain services together and use Spark as a distributed micro-service orchestrator. HTTP on Spark provides asynchronous parallelism, batching, throttling, and exponential back-offs for failed requests so that you can focus on the core application logic.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"real-world-examples\">Real world examples<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"the-metropolitan-museum-of-art\">The Metropolitan Museum of Art<\/h3>\n\n\n\n<figure class=\"wp-block-image has-custom-border\"><img decoding=\"async\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/04\/1aa534dd-136b-45e8-a85c-bda74364178d.webp\" alt=\"Azure Search on Spark for The MET\" style=\"border-radius:0px\" title=\"Azure Search on Spark for The MET\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">At Microsoft, we use HTTP on Spark to power a variety of projects and customers. Our <a href=\"https:\/\/www.ailab.microsoft.com\/experiments\/gen-studio\">latest project<\/a> uses the Computer Vision APIs on Spark and Azure Search on Spark to create a searchable database of Art for The Metropolitan Museum of Art (The MET). More Specifically, we load <a href=\"https:\/\/www.metmuseum.org\/about-the-met\/policies-and-documents\/open-access\">The MET\u2019s Open Access catalog<\/a> of images, and use the Computer Vision APIs to annotate these images with searchable descriptions in parallel. We also used CNTK on Spark, and <a href=\"https:\/\/spark.apache.org\/docs\/2.1.0\/ml-features.html#locality-sensitive-hashing\">SparkML\u2019s Locality Sensitive Hash<\/a> implementation to futurize these images and create a custom reverse image search engine. For more information on this work, check out our <a href=\"https:\/\/www.aka.ms\/aiart\">AI Lab<\/a> or our <a href=\"https:\/\/github.com\/Microsoft\/GenStudio\">Github<\/a>.<\/p>\n\n\n\n<figure class=\"wp-block-image has-custom-border\"><img decoding=\"async\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/04\/f8fdc817-eb91-4865-a01d-06318c8e8a19.webp\" alt=\"Architecture of Azure Search on Spark\" style=\"border-radius:0px\" title=\"Architecture of Azure Search on Spark\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"the-snow-leopard-trust\">The Snow Leopard Trust<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">We partnered with <a href=\"https:\/\/www.snowleopard.org\/\">the Snow Leopard Trust<\/a> to help track and understand the endangered Snow Leopard population using the Cognitive Services on Spark. We began by creating a fully labelled training dataset for leopard classification by pulling snow leopard images from Bing on Spark. We then used CNTK and Tensorflow on Spark to train a deep classification system. Finally, we interpreted our model using <a href=\"https:\/\/aka.ms\/spark\">LIME on Spark<\/a> to refine our leopard classifier into a leopard detector without drawing a single bounding box by hand! For more information, you can check out our <a href=\"https:\/\/blogs.technet.microsoft.com\/machinelearning\/2018\/10\/03\/deep-learning-without-labels\/\">blog post<\/a>.<\/p>\n\n\n\n<figure class=\"wp-block-image has-custom-border\"><img decoding=\"async\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/04\/976fb667-4e3e-42be-abe8-ec84a2425f4c.webp\" alt=\"Architecture of Snow Leopard classification\" style=\"border-radius:0px\" title=\"Architecture of Snow Leopard classification\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"conclusion\">Conclusion<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">With only a few lines of code you can start integrating the power of Azure Cognitive Services into your big data workflows on Apache Spark<b>\u2122<\/b>. The Spark bindings offer high throughput and run anywhere you run Spark. The Cognitive Services on Spark fully integrate with containers for high performance, on premises, or low connectivity scenarios. Finally, we have provided a general framework for working with any web service on Spark. You can start leveraging the Cognitive Services for your project with our open source initiative <a href=\"https:\/\/aka.ms\/spark\">MMLSpark on Azure Databricks<\/a>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"learn-more\">Learn more<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/aka.ms\/spark\">Web<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/aka.ms\/mmlspark\">Github<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a>Email: <\/a>mmlspark-support@microsoft.com<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Today at Spark AI Summit 2019, we&#8217;re excited to introduce a new set of models in the SparkML ecosystem that make it easy to leverage the Azure Cognitive Services at terabyte scales.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"ms_queue_id":[],"ep_exclude_from_search":false,"_classifai_error":"","_classifai_text_to_speech_error":"","_alt_title":"","footnotes":"","msx_community_cta_settings":[]},"categories":[1454],"tags":[],"audience":[3057,3055,3056],"content-type":[1460],"product":[3164],"tech-community":[],"topic":[],"coauthors":[112],"class_list":["post-1394","post","type-post","status-publish","format-standard","hentry","category-ai-machine-learning","audience-data-professionals","audience-developers","audience-it-implementors","content-type-events","product-microsoft-foundry","review-flag-1680286581-364","review-flag-2-1680286581-601","review-flag-3-1680286581-173","review-flag-4-1680286581-250","review-flag-anywh-1680286580-635","review-flag-integ-1680286579-214","review-flag-lever-1680286579-649","review-flag-machi-1680286585-314","review-flag-ml-1680286585-776","review-flag-new-1680286579-546","review-flag-on-pr-1680286585-571"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.2 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Dear Spark developers: Welcome to Azure Cognitive Services | Microsoft Azure Blog<\/title>\n<meta name=\"description\" content=\"Today at Spark AI Summit 2019, we&#039;re excited to introduce a new set of models in the SparkML ecosystem that make it easy to leverage the Azure Cognitive Services at terabyte scales.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Dear Spark developers: Welcome to Azure Cognitive Services | Microsoft Azure Blog\" \/>\n<meta property=\"og:description\" content=\"Today at Spark AI Summit 2019, we&#039;re excited to introduce a new set of models in the SparkML ecosystem that make it easy to leverage the Azure Cognitive Services at terabyte scales.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/\" \/>\n<meta property=\"og:site_name\" content=\"Microsoft Azure Blog\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/microsoftazure\" \/>\n<meta property=\"article:published_time\" content=\"2019-04-24T00:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-06-20T19:49:00+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/04\/dd10b978-1bcc-4d0a-84ee-b1d2223cfa85.webp\" \/>\n<meta name=\"author\" content=\"Anand Raman\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@azure\" \/>\n<meta name=\"twitter:site\" content=\"@azure\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Anand Raman\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/\"},\"author\":[{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/anand-raman\/\",\"@type\":\"Person\",\"@name\":\"Anand Raman\"}],\"headline\":\"Dear Spark developers: Welcome to Azure Cognitive Services\",\"datePublished\":\"2019-04-24T00:00:00+00:00\",\"dateModified\":\"2025-06-20T19:49:00+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/\"},\"wordCount\":1028,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/04\/dd10b978-1bcc-4d0a-84ee-b1d2223cfa85.webp\",\"articleSection\":[\"AI + machine learning\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/\",\"name\":\"Dear Spark developers: Welcome to Azure Cognitive Services | Microsoft Azure Blog\",\"isPartOf\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/04\/dd10b978-1bcc-4d0a-84ee-b1d2223cfa85.webp\",\"datePublished\":\"2019-04-24T00:00:00+00:00\",\"dateModified\":\"2025-06-20T19:49:00+00:00\",\"description\":\"Today at Spark AI Summit 2019, we're excited to introduce a new set of models in the SparkML ecosystem that make it easy to leverage the Azure Cognitive Services at terabyte scales.\",\"breadcrumb\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/#primaryimage\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/04\/dd10b978-1bcc-4d0a-84ee-b1d2223cfa85.webp\",\"contentUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/04\/dd10b978-1bcc-4d0a-84ee-b1d2223cfa85.webp\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Blog home\",\"item\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"AI + machine learning\",\"item\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/category\/ai-machine-learning\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Dear Spark developers: Welcome to Azure Cognitive Services\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#website\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/\",\"name\":\"Microsoft Azure Blog\",\"description\":\"Get the latest Azure news, updates, and announcements from the Azure blog. From product updates to hot topics, hear from the Azure experts.\",\"publisher\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization\",\"name\":\"Microsoft Azure Blog\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp\",\"contentUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp\",\"width\":512,\"height\":512,\"caption\":\"Microsoft Azure Blog\"},\"image\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/microsoftazure\",\"https:\/\/x.com\/azure\",\"https:\/\/www.instagram.com\/microsoftdeveloper\/\",\"https:\/\/www.linkedin.com\/company\/16188386\",\"https:\/\/www.youtube.com\/user\/windowsazure\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/person\/c702e5edd662b328b49b7e1180cab117\",\"name\":\"shakir\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g7664e653ea371ce16eaf75e9fa8952c4\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g\",\"caption\":\"shakir\"},\"sameAs\":[\"https:\/\/azure.microsoft.com\"],\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/shakir\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Dear Spark developers: Welcome to Azure Cognitive Services | Microsoft Azure Blog","description":"Today at Spark AI Summit 2019, we're excited to introduce a new set of models in the SparkML ecosystem that make it easy to leverage the Azure Cognitive Services at terabyte scales.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/","og_locale":"en_US","og_type":"article","og_title":"Dear Spark developers: Welcome to Azure Cognitive Services | Microsoft Azure Blog","og_description":"Today at Spark AI Summit 2019, we're excited to introduce a new set of models in the SparkML ecosystem that make it easy to leverage the Azure Cognitive Services at terabyte scales.","og_url":"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/","og_site_name":"Microsoft Azure Blog","article_publisher":"https:\/\/www.facebook.com\/microsoftazure","article_published_time":"2019-04-24T00:00:00+00:00","article_modified_time":"2025-06-20T19:49:00+00:00","og_image":[{"url":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/04\/dd10b978-1bcc-4d0a-84ee-b1d2223cfa85.webp","type":"","width":"","height":""}],"author":"Anand Raman","twitter_card":"summary_large_image","twitter_creator":"@azure","twitter_site":"@azure","twitter_misc":{"Written by":"Anand Raman","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/#article","isPartOf":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/"},"author":[{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/anand-raman\/","@type":"Person","@name":"Anand Raman"}],"headline":"Dear Spark developers: Welcome to Azure Cognitive Services","datePublished":"2019-04-24T00:00:00+00:00","dateModified":"2025-06-20T19:49:00+00:00","mainEntityOfPage":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/"},"wordCount":1028,"commentCount":0,"publisher":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization"},"image":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/#primaryimage"},"thumbnailUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/04\/dd10b978-1bcc-4d0a-84ee-b1d2223cfa85.webp","articleSection":["AI + machine learning"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/","name":"Dear Spark developers: Welcome to Azure Cognitive Services | Microsoft Azure Blog","isPartOf":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/#primaryimage"},"image":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/#primaryimage"},"thumbnailUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/04\/dd10b978-1bcc-4d0a-84ee-b1d2223cfa85.webp","datePublished":"2019-04-24T00:00:00+00:00","dateModified":"2025-06-20T19:49:00+00:00","description":"Today at Spark AI Summit 2019, we're excited to introduce a new set of models in the SparkML ecosystem that make it easy to leverage the Azure Cognitive Services at terabyte scales.","breadcrumb":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/#primaryimage","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/04\/dd10b978-1bcc-4d0a-84ee-b1d2223cfa85.webp","contentUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2019\/04\/dd10b978-1bcc-4d0a-84ee-b1d2223cfa85.webp"},{"@type":"BreadcrumbList","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/dear-spark-developers-welcome-to-azure-cognitive-services\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Blog home","item":"https:\/\/azure.microsoft.com\/en-us\/blog\/"},{"@type":"ListItem","position":2,"name":"AI + machine learning","item":"https:\/\/azure.microsoft.com\/en-us\/blog\/category\/ai-machine-learning\/"},{"@type":"ListItem","position":3,"name":"Dear Spark developers: Welcome to Azure Cognitive Services"}]},{"@type":"WebSite","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#website","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/","name":"Microsoft Azure Blog","description":"Get the latest Azure news, updates, and announcements from the Azure blog. From product updates to hot topics, hear from the Azure experts.","publisher":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/azure.microsoft.com\/en-us\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization","name":"Microsoft Azure Blog","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp","contentUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp","width":512,"height":512,"caption":"Microsoft Azure Blog"},"image":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/microsoftazure","https:\/\/x.com\/azure","https:\/\/www.instagram.com\/microsoftdeveloper\/","https:\/\/www.linkedin.com\/company\/16188386","https:\/\/www.youtube.com\/user\/windowsazure"]},{"@type":"Person","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/person\/c702e5edd662b328b49b7e1180cab117","name":"shakir","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g7664e653ea371ce16eaf75e9fa8952c4","url":"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g","caption":"shakir"},"sameAs":["https:\/\/azure.microsoft.com"],"url":"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/shakir\/"}]}},"msxcm_display_generated_audio":false,"msxcm_animated_featured_image":null,"distributor_meta":false,"distributor_terms":false,"distributor_media":false,"distributor_original_site_name":"Microsoft Azure Blog","distributor_original_site_url":"https:\/\/azure.microsoft.com\/en-us\/blog","push-errors":false,"_links":{"self":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/posts\/1394","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/comments?post=1394"}],"version-history":[{"count":3,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/posts\/1394\/revisions"}],"predecessor-version":[{"id":42734,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/posts\/1394\/revisions\/42734"}],"wp:attachment":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/media?parent=1394"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/categories?post=1394"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/tags?post=1394"},{"taxonomy":"audience","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/audience?post=1394"},{"taxonomy":"content-type","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/content-type?post=1394"},{"taxonomy":"product","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/product?post=1394"},{"taxonomy":"tech-community","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/tech-community?post=1394"},{"taxonomy":"topic","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/topic?post=1394"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/coauthors?post=1394"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}