To provide more authentication options, HDInsight Tools for VSCode now can be connected to HDInsight cluster through Ambari for job submissions. You can easily link (HDInsight: Link a cluster) or unlink (HDInsight: Unlink a cluster) a normal cluster by using Ambari managed username and password, which is independent of your Azure signing process. The Ambari connection applies to Spark and Hive clusters in all the Azure environments which host HDInsight services.
To support HDInsight Enterprise Secure Package (in preview), you can also connect to the secured cluster through domain username (e.g. firstname.lastname@example.org). This connection is applicable for both traditional blob storage (WASB) or Azure Data Lake Storage (ADLS) as underlying storage. Once you connect to the secured HDInsight cluster, you can use the signed in domain credentials for all you job submissions.
This addition grants you more flexibilities to connect to your HDInsight clusters in addition to your Azure subscriptions and greatly simplify your experiences in submitting your Hive and Spark jobs.
How to link a cluster
- Open the command palette by selecting CTRL+SHIFT+P, and then enter HDInsight: Link a cluster.
- Enter HDInsight cluster URL -> input Username -> input Password -> select cluster type – –> Link success confirmation message. Now you can submit your hive / spark query or job to this linked cluster.
Please note that you can use either Ambari username / pwd or Secure Hadoop domain username / pwd to connect.
- Use command HDInsight: List cluster if you want to see more information on the Linked cluster.
- You also can unlink a cluster via command HDInsight: Unlink a cluster from command palette.
How to install or update
First, install Visual Studio Code and download Mono 4.2.x (for Linux and Mac). Then get the latest HDInsight Tools by going to the VSCode Extension repository or the VSCode Marketplace and searching “HDInsight Tools for VSCode”.
For more information about HDInsight Tools for VSCode, please use the following resources:
- User Manual: HDInsight Tools for VSCode
- User Manual: Set Up PySpark Interactive Environment
- Demo Video: HDInsight for VSCode Video
- Hive LLAP: Use Interactive Query with HDInsight
If you have questions, feedback, comments, or bug reports, please use the comments below or send a note to email@example.com.