{"id":3245,"date":"2017-11-27T00:00:00","date_gmt":"2017-11-27T00:00:00","guid":{"rendered":"https:\/\/azure.microsoft.com\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code"},"modified":"2025-06-26T08:34:08","modified_gmt":"2025-06-26T15:34:08","slug":"run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code","status":"publish","type":"post","link":"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/","title":{"rendered":"Run your PySpark Interactive Query and batch job in Visual Studio Code"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">We are excited to introduce the integration of HDInsight PySpark into Visual Studio Code (VSCode), which allows developers to easily edit Python scripts and submit PySpark statements to HDInsight clusters. For PySpark developers who value productivity of Python language, VSCode HDInsight Tools offer you a quick Python editor with simple getting started experiences, and enable you to submit PySpark statements to HDInsight clusters with interactive responses. This interactivity brings the best properties of Python and Spark to developers and empowers you to gain faster insights.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"key-customer-benefits\">Key customer benefits<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">Interactive responses brings the best properties of Python and Spark with flexibility to execute one or multiple statements.<\/li>\n\n\n\n<li class=\"wp-block-list-item\">Built-in Python language service, such as IntelliSense auto-suggest, autocomplete, and error marker, among others.<\/li>\n\n\n\n<li class=\"wp-block-list-item\">Preview and export your PySpark interactive query results to CSV, JSON, and Excel formats.<\/li>\n\n\n\n<li class=\"wp-block-list-item\">Integration with Azure for HDInsight cluster management and query submissions.<\/li>\n\n\n\n<li class=\"wp-block-list-item\">Link with Spark UI and Yarn UI for further troubleshooting.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"how-to-start-hdinsight-tools-for-vscode\">How to start HDInsight Tools for VSCode<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Simply open your Python files in your HDInsight workspace and connect to Azure. You can then start to author Python script or Spark SQL to query your data.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">Run Spark Python interactive<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2017\/11\/a7af96ad-720c-47bb-985a-a36f4427aeb7.webp\" alt=\"80d4b7a9-1031-4834-b5a6-3e133aab258d\" title=\"80d4b7a9-1031-4834-b5a6-3e133aab258d\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2017\/11\/492b76ba-eaea-4b62-be85-86f34fd0b268.webp\" alt=\"35fc205e-3743-422b-8ed4-3c8bbc778b2f\" title=\"35fc205e-3743-422b-8ed4-3c8bbc778b2f\" \/><\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">Run Spark SQL interactive<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2017\/11\/c4a7ab33-6836-45ad-bade-4ea81133fb42.webp\" alt=\"ba3e6ffe-f0fc-4538-aaaa-d33b813bd792\" title=\"ba3e6ffe-f0fc-4538-aaaa-d33b813bd792\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"how-to-install-or-update\">How to install or update<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">First, install Visual Studio Code and download Mono 4.2.x (for Linux and Mac). Then get the latest HDInsight Tools by going to the VSCode Extension repository or the VSCode Marketplace and searching \u201c<strong>HDInsight Tools for VSCode<\/strong>\u201d.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2017\/11\/b9c84107-ffd7-44c0-91f5-20a4ebf6b0e2.webp\" alt=\"a4991f2e-c132-4794-8a4c-08b3cf68b328\" title=\"a4991f2e-c132-4794-8a4c-08b3cf68b328\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">For more information about Azure Data Lake Tool for VSCode, please use the following resources:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">User Manual: <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/hdinsight\/hdinsight-for-vscode\">HDInsight Tools for VSCode<\/a><\/li>\n\n\n\n<li class=\"wp-block-list-item\">User Manual: <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/hdinsight\/set-up-pyspark-interactive-environment?branch=pr-en-us-26060\">Set Up PySpark Interactive Environment<\/a><\/li>\n\n\n\n<li class=\"wp-block-list-item\">Demo Video: <a href=\"https:\/\/youtu.be\/RLjVNP8ufuw\">HDInsight for VSCode Video<\/a><\/li>\n\n\n\n<li class=\"wp-block-list-item\">Hive LLAP: <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/hdinsight\/hdinsight-hadoop-use-interactive-hive\">Use Interactive Query with HDInsight<\/a><\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Learn more about today\u2019s announcements on the <a href=\"https:\/\/azure.microsoft.com\/en-us\/blog\/\">Azure blog<\/a> and <a href=\"https:\/\/blogs.msdn.microsoft.com\/azuredatalake\">Big Data blog<\/a>. Discover more on the <a href=\"https:\/\/azure.microsoft.com\/en-us\/updates\/\">Azure service updates<\/a> page.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If you have questions, feedback, comments, or bug reports, please use the comments below or send a note to <a>hdivstool@microsoft.com<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>We are excited to introduce the integration of HDInsight PySpark into Visual Studio Code (VSCode), which allows developers to easily edit Python scripts and submit PySpark statements to HDInsight clusters. This interactivity brings the best properties of Python and Spark to developers and empowers you to gain faster insights.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"ms_queue_id":[],"ep_exclude_from_search":false,"_classifai_error":"","_classifai_text_to_speech_error":"","_alt_title":"","footnotes":"","msx_community_cta_settings":[]},"categories":[1551],"tags":[48],"audience":[3055],"content-type":[],"product":[],"tech-community":[],"topic":[],"coauthors":[663],"class_list":["post-3245","post","type-post","status-publish","format-standard","hentry","category-developer-tools","tag-big-data","audience-developers","review-flag-2-1680286581-601","review-flag-4-1680286581-250"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.2 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Run your PySpark Interactive Query and batch job in Visual Studio Code | Microsoft Azure Blog<\/title>\n<meta name=\"description\" content=\"We are excited to introduce the integration of HDInsight PySpark into Visual Studio Code (VSCode), which allows developers to easily edit Python scripts and submit PySpark statements to HDInsight clusters. This interactivity brings the best properties of Python and Spark to developers and empowers you to gain faster insights.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Run your PySpark Interactive Query and batch job in Visual Studio Code | Microsoft Azure Blog\" \/>\n<meta property=\"og:description\" content=\"We are excited to introduce the integration of HDInsight PySpark into Visual Studio Code (VSCode), which allows developers to easily edit Python scripts and submit PySpark statements to HDInsight clusters. This interactivity brings the best properties of Python and Spark to developers and empowers you to gain faster insights.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/\" \/>\n<meta property=\"og:site_name\" content=\"Microsoft Azure Blog\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/microsoftazure\" \/>\n<meta property=\"article:published_time\" content=\"2017-11-27T00:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-06-26T15:34:08+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2017\/11\/a7af96ad-720c-47bb-985a-a36f4427aeb7.webp\" \/>\n<meta name=\"author\" content=\"Jenny Jiang\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@azure\" \/>\n<meta name=\"twitter:site\" content=\"@azure\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Jenny Jiang\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/\"},\"author\":[{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/jenny-jiang\/\",\"@type\":\"Person\",\"@name\":\"Jenny Jiang\"}],\"headline\":\"Run your PySpark Interactive Query and batch job in Visual Studio Code\",\"datePublished\":\"2017-11-27T00:00:00+00:00\",\"dateModified\":\"2025-06-26T15:34:08+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/\"},\"wordCount\":335,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2017\/11\/a7af96ad-720c-47bb-985a-a36f4427aeb7.webp\",\"keywords\":[\"Big Data\"],\"articleSection\":[\"Developer tools\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/\",\"name\":\"Run your PySpark Interactive Query and batch job in Visual Studio Code | Microsoft Azure Blog\",\"isPartOf\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2017\/11\/a7af96ad-720c-47bb-985a-a36f4427aeb7.webp\",\"datePublished\":\"2017-11-27T00:00:00+00:00\",\"dateModified\":\"2025-06-26T15:34:08+00:00\",\"description\":\"We are excited to introduce the integration of HDInsight PySpark into Visual Studio Code (VSCode), which allows developers to easily edit Python scripts and submit PySpark statements to HDInsight clusters. This interactivity brings the best properties of Python and Spark to developers and empowers you to gain faster insights.\",\"breadcrumb\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/#primaryimage\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2017\/11\/a7af96ad-720c-47bb-985a-a36f4427aeb7.webp\",\"contentUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2017\/11\/a7af96ad-720c-47bb-985a-a36f4427aeb7.webp\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Blog home\",\"item\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Developer tools\",\"item\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/category\/developer-tools\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Run your PySpark Interactive Query and batch job in Visual Studio Code\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#website\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/\",\"name\":\"Microsoft Azure Blog\",\"description\":\"Get the latest Azure news, updates, and announcements from the Azure blog. From product updates to hot topics, hear from the Azure experts.\",\"publisher\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization\",\"name\":\"Microsoft Azure Blog\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp\",\"contentUrl\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp\",\"width\":512,\"height\":512,\"caption\":\"Microsoft Azure Blog\"},\"image\":{\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/microsoftazure\",\"https:\/\/x.com\/azure\",\"https:\/\/www.instagram.com\/microsoftdeveloper\/\",\"https:\/\/www.linkedin.com\/company\/16188386\",\"https:\/\/www.youtube.com\/user\/windowsazure\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/person\/c702e5edd662b328b49b7e1180cab117\",\"name\":\"shakir\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g7664e653ea371ce16eaf75e9fa8952c4\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g\",\"caption\":\"shakir\"},\"sameAs\":[\"https:\/\/azure.microsoft.com\"],\"url\":\"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/shakir\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Run your PySpark Interactive Query and batch job in Visual Studio Code | Microsoft Azure Blog","description":"We are excited to introduce the integration of HDInsight PySpark into Visual Studio Code (VSCode), which allows developers to easily edit Python scripts and submit PySpark statements to HDInsight clusters. This interactivity brings the best properties of Python and Spark to developers and empowers you to gain faster insights.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/","og_locale":"en_US","og_type":"article","og_title":"Run your PySpark Interactive Query and batch job in Visual Studio Code | Microsoft Azure Blog","og_description":"We are excited to introduce the integration of HDInsight PySpark into Visual Studio Code (VSCode), which allows developers to easily edit Python scripts and submit PySpark statements to HDInsight clusters. This interactivity brings the best properties of Python and Spark to developers and empowers you to gain faster insights.","og_url":"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/","og_site_name":"Microsoft Azure Blog","article_publisher":"https:\/\/www.facebook.com\/microsoftazure","article_published_time":"2017-11-27T00:00:00+00:00","article_modified_time":"2025-06-26T15:34:08+00:00","og_image":[{"url":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2017\/11\/a7af96ad-720c-47bb-985a-a36f4427aeb7.webp","type":"","width":"","height":""}],"author":"Jenny Jiang","twitter_card":"summary_large_image","twitter_creator":"@azure","twitter_site":"@azure","twitter_misc":{"Written by":"Jenny Jiang","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/#article","isPartOf":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/"},"author":[{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/jenny-jiang\/","@type":"Person","@name":"Jenny Jiang"}],"headline":"Run your PySpark Interactive Query and batch job in Visual Studio Code","datePublished":"2017-11-27T00:00:00+00:00","dateModified":"2025-06-26T15:34:08+00:00","mainEntityOfPage":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/"},"wordCount":335,"commentCount":0,"publisher":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization"},"image":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/#primaryimage"},"thumbnailUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2017\/11\/a7af96ad-720c-47bb-985a-a36f4427aeb7.webp","keywords":["Big Data"],"articleSection":["Developer tools"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/","name":"Run your PySpark Interactive Query and batch job in Visual Studio Code | Microsoft Azure Blog","isPartOf":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/#primaryimage"},"image":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/#primaryimage"},"thumbnailUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2017\/11\/a7af96ad-720c-47bb-985a-a36f4427aeb7.webp","datePublished":"2017-11-27T00:00:00+00:00","dateModified":"2025-06-26T15:34:08+00:00","description":"We are excited to introduce the integration of HDInsight PySpark into Visual Studio Code (VSCode), which allows developers to easily edit Python scripts and submit PySpark statements to HDInsight clusters. This interactivity brings the best properties of Python and Spark to developers and empowers you to gain faster insights.","breadcrumb":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/#primaryimage","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2017\/11\/a7af96ad-720c-47bb-985a-a36f4427aeb7.webp","contentUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2017\/11\/a7af96ad-720c-47bb-985a-a36f4427aeb7.webp"},{"@type":"BreadcrumbList","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/run-your-pyspark-interactive-query-and-batch-job-in-visual-studio-code\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Blog home","item":"https:\/\/azure.microsoft.com\/en-us\/blog\/"},{"@type":"ListItem","position":2,"name":"Developer tools","item":"https:\/\/azure.microsoft.com\/en-us\/blog\/category\/developer-tools\/"},{"@type":"ListItem","position":3,"name":"Run your PySpark Interactive Query and batch job in Visual Studio Code"}]},{"@type":"WebSite","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#website","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/","name":"Microsoft Azure Blog","description":"Get the latest Azure news, updates, and announcements from the Azure blog. From product updates to hot topics, hear from the Azure experts.","publisher":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/azure.microsoft.com\/en-us\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#organization","name":"Microsoft Azure Blog","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp","contentUrl":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-content\/uploads\/2024\/06\/microsoft_logo.webp","width":512,"height":512,"caption":"Microsoft Azure Blog"},"image":{"@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/microsoftazure","https:\/\/x.com\/azure","https:\/\/www.instagram.com\/microsoftdeveloper\/","https:\/\/www.linkedin.com\/company\/16188386","https:\/\/www.youtube.com\/user\/windowsazure"]},{"@type":"Person","@id":"https:\/\/azure.microsoft.com\/en-us\/blog\/#\/schema\/person\/c702e5edd662b328b49b7e1180cab117","name":"shakir","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g7664e653ea371ce16eaf75e9fa8952c4","url":"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/9342c7c05bb16548741bc5cd3a3e3b7ee0c8e746844ad2cc582db5beb5514c6f?s=96&d=mm&r=g","caption":"shakir"},"sameAs":["https:\/\/azure.microsoft.com"],"url":"https:\/\/azure.microsoft.com\/en-us\/blog\/author\/shakir\/"}]}},"msxcm_display_generated_audio":false,"msxcm_animated_featured_image":null,"distributor_meta":false,"distributor_terms":false,"distributor_media":false,"distributor_original_site_name":"Microsoft Azure Blog","distributor_original_site_url":"https:\/\/azure.microsoft.com\/en-us\/blog","push-errors":false,"_links":{"self":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/posts\/3245","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/comments?post=3245"}],"version-history":[{"count":1,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/posts\/3245\/revisions"}],"predecessor-version":[{"id":43836,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/posts\/3245\/revisions\/43836"}],"wp:attachment":[{"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/media?parent=3245"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/categories?post=3245"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/tags?post=3245"},{"taxonomy":"audience","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/audience?post=3245"},{"taxonomy":"content-type","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/content-type?post=3245"},{"taxonomy":"product","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/product?post=3245"},{"taxonomy":"tech-community","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/tech-community?post=3245"},{"taxonomy":"topic","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/topic?post=3245"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/azure.microsoft.com\/en-us\/blog\/wp-json\/wp\/v2\/coauthors?post=3245"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}