{"id":2894,"date":"2018-04-02T00:00:00","date_gmt":"2018-04-02T00:00:00","guid":{"rendered":"https:\/\/azure.microsoft.com\/blog\/operationalize-azure-databricks-notebooks-using-data-factory"},"modified":"2025-07-01T08:18:40","modified_gmt":"2025-07-01T15:18:40","slug":"operationalize-azure-databricks-notebooks-using-data-factory","status":"publish","type":"post","link":"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/","title":{"rendered":"Ingest, prepare, and transform using Azure Databricks and Data Factory"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Today\u2019s business managers depend heavily on reliable data integration systems that run complex ETL\/ELT workflows (extract, transform\/load and load\/transform data). These workflows allow businesses to ingest data in various forms and shapes from different on-prem\/cloud data sources; transform\/shape the data and gain actionable insights into data to make important business decisions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">With the <a href=\"https:\/\/azure.microsoft.com\/en-us\/blog\/azure-databricks-industry-leading-analytics-platform-powered-by-apache-spark\/\">general availability<\/a> of Azure Databricks comes support for doing ETL\/ELT with <a href=\"https:\/\/azure.microsoft.com\/en-us\/services\/data-factory\">Azure Data Factory<\/a>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This integration allows you to operationalize ETL\/ELT workflows (including analytics workloads in Azure Databricks) using data factory pipelines that do the following:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li class=\"wp-block-list-item\"><a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/data-factory\/copy-activity-overview\" target=\"_blank\" rel=\"noopener\">Ingest data <\/a>at scale using 70+ on-prem\/cloud data sources<\/li>\n\n\n\n<li class=\"wp-block-list-item\">Prepare and transform (clean, sort, merge, join, etc.) the ingested data in Azure Databricks as a <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/data-factory\/transform-data-databricks-notebook\" target=\"_blank\" rel=\"noopener\">Notebook<\/a> activity step in data factory pipelines<\/li>\n\n\n\n<li class=\"wp-block-list-item\"><a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/data-factory\/monitor-visually\">Monitor and manage<\/a> your E2E workflow<\/li>\n<\/ol>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/04\/29515ffa-a917-4ebb-b326-f6655cab3bda.webp\" alt=\"image\" title=\"image\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Take a look at a sample data factory pipeline where we are ingesting data from Amazon S3 to Azure Blob, processing the ingested data using a Notebook running in Azure Databricks and moving the processed data in Azure SQL Datawarehouse.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/04\/59dffc8c-ffee-4198-a5d0-a056c93fb19f.webp\" alt=\"image\" title=\"image\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">You can parameterize the entire workflow (folder name, file name, etc.) using rich <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/data-factory\/control-flow-expression-language-functions\" target=\"_blank\" rel=\"noopener\">expression support<\/a> and operationalize by defining a <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/data-factory\/how-to-create-schedule-trigger\" target=\"_blank\" rel=\"noopener\">trigger<\/a> in data factory.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"get-started-today\">Get started today!<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">We are excited for you to try Azure Databricks and Azure Data Factory integration and let us know your feedback.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Get started by clicking the <strong>Author &amp; Monitor<\/strong> tile in your provisioned v2 data factory blade.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/04\/4f457049-78f6-4aa9-a866-163db29b4ce9.webp\" alt=\"image\" title=\"image\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Click on the Transform data with Azure Databricks tutorial and learn step by step how to operationalize your ETL\/ELT workloads including analytics workloads in Azure Databricks using Azure Data Factory.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/04\/6679559a-6e3d-459f-b447-e780b75e5929.webp\" alt=\"image\" title=\"image\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">We are continuously working to add new features based on customer feedback. Get more information and detailed steps for using the <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/data-factory\/transform-data-using-databricks-notebook\">Azure Databricks and Data Factory integration<\/a>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/data-factory\/quickstart-create-data-factory-portal\">Get started<\/a> building pipelines easily and quickly using Azure Data Factory. If you have any feature requests or want to provide feedback, please visit the <a href=\"https:\/\/feedback.azure.com\/forums\/270578-azure-data-factory\">Azure Data Factory forum<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Azure Databricks general availability was announced on March 22, 2018. Now Azure Databricks is fully integrated with Azure Data Factory (ADF).<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"ms_queue_id":[],"ep_exclude_from_search":false,"_classifai_error":"","_classifai_text_to_speech_error":"","_alt_title":"","footnotes":"","msx_community_cta_settings":[]},"categories":[1454,1474,1473],"tags":[48],"audience":[3054,3057,3055,3053,3056],"content-type":[],"product":[1821,1544],"tech-community":[],"topic":[],"coauthors":[617],"class_list":["post-2894","post","type-post","status-publish","format-standard","hentry","category-ai-machine-learning","category-analytics","category-databases","tag-big-data","audience-business-decision-makers","audience-data-professionals","audience-developers","audience-it-decision-makers","audience-it-implementors","product-azure-data-factory","product-azure-databricks","review-flag-1680286581-295","review-flag-1680286581-56","review-flag-1680286581-364","review-flag-gener-1680286584-335","review-flag-new-1680286579-546","review-flag-on-pr-1680286585-789"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.2 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Ingest, prepare, and transform using Azure Databricks and Data Factory | Microsoft Azure Blog<\/title>\n<meta name=\"description\" content=\"Azure Databricks general availability was announced on March 22, 2018. Now Azure Databricks is fully integrated with Azure Data Factory (ADF).\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Ingest, prepare, and transform using Azure Databricks and Data Factory | Microsoft Azure Blog\" \/>\n<meta property=\"og:description\" content=\"Azure Databricks general availability was announced on March 22, 2018. Now Azure Databricks is fully integrated with Azure Data Factory (ADF).\" \/>\n<meta property=\"og:url\" content=\"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/\" \/>\n<meta property=\"og:site_name\" content=\"Microsoft Azure Blog\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/microsoftazure\" \/>\n<meta property=\"article:published_time\" content=\"2018-04-02T00:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-07-01T15:18:40+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/04\/29515ffa-a917-4ebb-b326-f6655cab3bda.webp\" \/>\n<meta name=\"author\" content=\"Gaurav Malhotra\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@azure\" \/>\n<meta name=\"twitter:site\" content=\"@azure\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Gaurav Malhotra\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/\"},\"author\":[{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/gaurav-malhotra\/\",\"@type\":\"Person\",\"@name\":\"Gaurav Malhotra\"}],\"headline\":\"Ingest, prepare, and transform using Azure Databricks and Data Factory\",\"datePublished\":\"2018-04-02T00:00:00+00:00\",\"dateModified\":\"2025-07-01T15:18:40+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/\"},\"wordCount\":336,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/04\/29515ffa-a917-4ebb-b326-f6655cab3bda.webp\",\"keywords\":[\"Big Data\"],\"articleSection\":[\"AI + machine learning\",\"Analytics\",\"Databases\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/\",\"name\":\"Ingest, prepare, and transform using Azure Databricks and Data Factory | Microsoft Azure Blog\",\"isPartOf\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/04\/29515ffa-a917-4ebb-b326-f6655cab3bda.webp\",\"datePublished\":\"2018-04-02T00:00:00+00:00\",\"dateModified\":\"2025-07-01T15:18:40+00:00\",\"description\":\"Azure Databricks general availability was announced on March 22, 2018. Now Azure Databricks is fully integrated with Azure Data Factory (ADF).\",\"breadcrumb\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/#primaryimage\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/04\/29515ffa-a917-4ebb-b326-f6655cab3bda.webp\",\"contentUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/04\/29515ffa-a917-4ebb-b326-f6655cab3bda.webp\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Blog home\",\"item\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"AI + machine learning\",\"item\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/category\/ai-machine-learning\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Ingest, prepare, and transform using Azure Databricks and Data Factory\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#website\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/\",\"name\":\"Microsoft Azure Blog\",\"description\":\"Get the latest Azure news, updates, and announcements from the Azure blog. From product updates to hot topics, hear from the Azure experts.\",\"publisher\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization\",\"name\":\"Microsoft Azure Blog\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp\",\"contentUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp\",\"width\":512,\"height\":512,\"caption\":\"Microsoft Azure Blog\"},\"image\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/microsoftazure\",\"https:\/\/x.com\/azure\",\"https:\/\/www.instagram.com\/microsoftdeveloper\/\",\"https:\/\/www.linkedin.com\/company\/16188386\",\"https:\/\/www.youtube.com\/user\/windowsazure\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/person\/c702e5edd662b328b49b7e1180cab117\",\"name\":\"shakir\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g7664e653ea371ce16eaf75e9fa8952c4\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g\",\"caption\":\"shakir\"},\"sameAs\":[\"https:\/\/azure.microsoft.com\"],\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/shakir\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Ingest, prepare, and transform using Azure Databricks and Data Factory | Microsoft Azure Blog","description":"Azure Databricks general availability was announced on March 22, 2018. Now Azure Databricks is fully integrated with Azure Data Factory (ADF).","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/","og_locale":"en_US","og_type":"article","og_title":"Ingest, prepare, and transform using Azure Databricks and Data Factory | Microsoft Azure Blog","og_description":"Azure Databricks general availability was announced on March 22, 2018. Now Azure Databricks is fully integrated with Azure Data Factory (ADF).","og_url":"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/","og_site_name":"Microsoft Azure Blog","article_publisher":"https:\/\/www.facebook.com\/microsoftazure","article_published_time":"2018-04-02T00:00:00+00:00","article_modified_time":"2025-07-01T15:18:40+00:00","og_image":[{"url":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/04\/29515ffa-a917-4ebb-b326-f6655cab3bda.webp","type":"","width":"","height":""}],"author":"Gaurav Malhotra","twitter_card":"summary_large_image","twitter_creator":"@azure","twitter_site":"@azure","twitter_misc":{"Written by":"Gaurav Malhotra","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/#article","isPartOf":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/"},"author":[{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/gaurav-malhotra\/","@type":"Person","@name":"Gaurav Malhotra"}],"headline":"Ingest, prepare, and transform using Azure Databricks and Data Factory","datePublished":"2018-04-02T00:00:00+00:00","dateModified":"2025-07-01T15:18:40+00:00","mainEntityOfPage":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/"},"wordCount":336,"commentCount":0,"publisher":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization"},"image":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/#primaryimage"},"thumbnailUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/04\/29515ffa-a917-4ebb-b326-f6655cab3bda.webp","keywords":["Big Data"],"articleSection":["AI + machine learning","Analytics","Databases"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/","name":"Ingest, prepare, and transform using Azure Databricks and Data Factory | Microsoft Azure Blog","isPartOf":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/#primaryimage"},"image":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/#primaryimage"},"thumbnailUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/04\/29515ffa-a917-4ebb-b326-f6655cab3bda.webp","datePublished":"2018-04-02T00:00:00+00:00","dateModified":"2025-07-01T15:18:40+00:00","description":"Azure Databricks general availability was announced on March 22, 2018. Now Azure Databricks is fully integrated with Azure Data Factory (ADF).","breadcrumb":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/#primaryimage","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/04\/29515ffa-a917-4ebb-b326-f6655cab3bda.webp","contentUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2018\/04\/29515ffa-a917-4ebb-b326-f6655cab3bda.webp"},{"@type":"BreadcrumbList","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/operationalize-azure-databricks-notebooks-using-data-factory\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Blog home","item":"https:\/\/azure.microsoft.com\/en-us\/blog\/"},{"@type":"ListItem","position":2,"name":"AI + machine learning","item":"https:\/\/azure.microsoft.com\/en-us\/blog\/category\/ai-machine-learning\/"},{"@type":"ListItem","position":3,"name":"Ingest, prepare, and transform using Azure Databricks and Data Factory"}]},{"@type":"WebSite","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#website","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/","name":"Microsoft Azure Blog","description":"Get the latest Azure news, updates, and announcements from the Azure blog. From product updates to hot topics, hear from the Azure experts.","publisher":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/azure.microsoft.com\/en-us\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization","name":"Microsoft Azure Blog","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp","contentUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp","width":512,"height":512,"caption":"Microsoft Azure Blog"},"image":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/microsoftazure","https:\/\/x.com\/azure","https:\/\/www.instagram.com\/microsoftdeveloper\/","https:\/\/www.linkedin.com\/company\/16188386","https:\/\/www.youtube.com\/user\/windowsazure"]},{"@type":"Person","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/person\/c702e5edd662b328b49b7e1180cab117","name":"shakir","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g7664e653ea371ce16eaf75e9fa8952c4","url":"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g","caption":"shakir"},"sameAs":["https:\/\/azure.microsoft.com"],"url":"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/shakir\/"}]}},"msxcm_display_generated_audio":false,"msxcm_animated_featured_image":null,"distributor_meta":false,"distributor_terms":false,"distributor_media":false,"distributor_original_site_name":"Microsoft Azure Blog","distributor_original_site_url":"https:\/\/azure.microsoft.com\/en-us\/blog","push-errors":false,"_links":{"self":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/posts\/2894","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/comments?post=2894"}],"version-history":[{"count":1,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/posts\/2894\/revisions"}],"predecessor-version":[{"id":44767,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/posts\/2894\/revisions\/44767"}],"wp:attachment":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/media?parent=2894"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/categories?post=2894"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/tags?post=2894"},{"taxonomy":"audience","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/audience?post=2894"},{"taxonomy":"content-type","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/content-type?post=2894"},{"taxonomy":"product","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/product?post=2894"},{"taxonomy":"tech-community","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/tech-community?post=2894"},{"taxonomy":"topic","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/topic?post=2894"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/coauthors?post=2894"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}