HDInsight tools for IntelliJ & Eclipse December Updates

1月 10, 2017 に投稿済み

Principal Program Manager, Big Data Team

We are pleased to announce the December updates of HDInsight Tools for IntelliJ & Eclipse. The HDInsight Tools for IntelliJ & Eclipse serve the open source community and will be of interest to HDInsight Spark developers. The tools run smoothly in Linux, Mac and Windows. The recent release focuses on users’ feedback to ensure a smooth user experiences on project creation and submission. The release also covers a couple of new features including Spark 2.0 support, local run, and a refined Job View & Job Graph.

Support Spark 2.0

The HDInsight Tools for IntelliJ & Eclipse now is fully compatible Spark 2.0. It allows you to enjoy the cool features from Spark 2.0 including API usability, SQL 2003 support, performance improvements, structured streaming, R UDF support, as well as operational improvements.

Pic1

Local Run - Use the HDInsight Tools for IntelliJ with the Hortonworks Sandbox

With this feature, the HDInsight Tools for IntelliJ can work with generic Hadoop clusters in addition to submitting Spark jobs to HDInsight clusters. Using the Hortonworks Sandbox allows you to work with Hadoop locally on your development environment. Once you have developed a solution and want to deploy it at scale, you can then move to an HDInsight cluster.

Connect to local sandbox for local run and debug

Pic2

Job View & Job Graph

The updated Job View provides you a slick UI to view your jobs list, job summary, and details for a selected job. The job graph also allows you to view the execution details, task summary, and executors view for a job.

Job List and Job Summary

Pic3

Job Graph

Pic4

Task Summary

Pic5

Executors View

Pic6

Installation – IntelliJ

User can get the latest bits by going to IntelliJ repository, and searching “Azure Toolkit.” IntelliJ will also prompt users for latest update if user has already installed the plugin.

IntelliJ Installation

 

Installation – Eclipse

User can get the latest bits by going to the Eclipse repository and searching “Azure Toolkit for Java”. Eclipse will also prompt you for the latest updates if you have previously installed the Azure Toolkit for Java plugin.

Eclipse Installation

 

For more information, check out the following:

        Create Spark Applications:

Learn more about today’s announcements on the Azure blog and Big Data blog.

Discover more Azure service updates.

If you have questions, feedback, comments, or bug reports, please use the comments below or send a note to hdivstool@microsoft.com.