We are pleased to announce the December updates of HDInsight Tools for IntelliJ & Eclipse. The HDInsight Tools for IntelliJ & Eclipse serve the open source community and will be of interest to HDInsight Spark developers. The tools run smoothly in Linux, Mac and Windows. The recent release focuses on users’ feedback to ensure a smooth user experiences on project creation and submission. The release also covers a couple of new features including Spark 2.0 support, local run, and a refined Job View & Job Graph.
Support Spark 2.0
The HDInsight Tools for IntelliJ & Eclipse now is fully compatible Spark 2.0. It allows you to enjoy the cool features from Spark 2.0 including API usability, SQL 2003 support, performance improvements, structured streaming, R UDF support, as well as operational improvements.
Local Run – Use the HDInsight Tools for IntelliJ with the Hortonworks Sandbox
With this feature, the HDInsight Tools for IntelliJ can work with generic Hadoop clusters in addition to submitting Spark jobs to HDInsight clusters. Using the Hortonworks Sandbox allows you to work with Hadoop locally on your development environment. Once you have developed a solution and want to deploy it at scale, you can then move to an HDInsight cluster.
Connect to local sandbox for local run and debug
Job View & Job Graph
The updated Job View provides you a slick UI to view your jobs list, job summary, and details for a selected job. The job graph also allows you to view the execution details, task summary, and executors view for a job.
Job List and Job Summary
Job Graph
Task Summary
Executors View
Installation – IntelliJ
User can get the latest bits by going to IntelliJ repository, and searching “Azure Toolkit.” IntelliJ will also prompt users for latest update if user has already installed the plugin.
Installation – Eclipse
User can get the latest bits by going to the Eclipse repository and searching “Azure Toolkit for Java”. Eclipse will also prompt you for the latest updates if you have previously installed the Azure Toolkit for Java plugin.
For more information, check out the following:
- IntelliJ HDInsight Spark Local Run: Use HDInsight Tools for IntelliJ with Hortonworks Sandbox
- IntelliJ Remote Debug: Use HDInsight Tools in Azure Toolkit for IntelliJ to debug Spark applications remotely on HDInsight Spark Linux cluster
Create Spark Applications:
- IntelliJ User Guide: Use HDInsight Tools in Azure Toolkit for IntelliJ to create Spark applications for HDInsight Spark Linux cluster
- Video: Introducing HDInsight Tools for IntelliJ for Spark Development
- Eclipse User Guide: Use HDInsight Tools in Azure Toolkit for Eclipse to create Spark applications
- Video: Use HDInsight Tool for Eclipse to create Spark applications
Learn more about today’s announcements on the Azure blog and Big Data blog.
Discover more Azure service updates.
If you have questions, feedback, comments, or bug reports, please use the comments below or send a note to hdivstool@microsoft.com.