Spark

Understanding HDInsight Spark jobs and data through visualizations in the Jupyter Notebook

29. april 2019

The Jupyter Notebook on HDInsight Spark clusters is useful when you need to quickly explore data sets, perform trend analysis, or try different machine learning models. Not being able to track the status of Spark jobs and intermediate data can make it difficult for data scientists to monitor and optimize what they are doing inside the Jupyter Notebook.

Senior Program Manager, Big Data Team

Azure HDInsight integration with Data Lake Storage Gen2 preview - ACL and security update

10. december 2018

Today we are sharing an update to the Azure HDInsight integration with Azure Data Lake Storage Gen 2. This integration will enable HDInsight customers to drive analytics from the data stored in Azure Data Lake Storage Gen 2 using popular open source frameworks such as Apache Spark, Hive, MapReduce, Kafka, Storm, and HBase in a secure manner.

Principal Program Manager, Azure HDInsight