Visual Studio Code Gets HDInsight Tools for Big Data Analytics -- Visual Studio Magazine

Visual Studio Code Gets HDInsight Tools for Big Data Analytics

By David Ramel
01/25/2019

Microsoft announced HDInsight Tools for Visual Studio Code is now generally available, letting coders do Big Data analytics right from within the cross-platform, open source code editor.

Specifically, Azure HDInsight Tools for Visual Studio Code is an extension in the Visual Studio Code Marketplace "for developing Hive Interactive Query, Hive Batch Job and PySpark Job against Microsoft HDInsight." The company describes Azure HDInsight as an enterprise-grade service for open source analytics.

The Azure HDInsight service supports popular open source Apache products including Hadoop, Spark, Kafka, HBase, Live LLAP and Storm, as well as including machine learning functionality. The VSCode extension, however focuses on developing HDInsight PySpark and Hive batch jobs and interactive query.

"For PySpark developers who value the productivity Python enables, HDInsight Tools for VSCode offer a quick Python editor with simple getting started experiences, and allow you to submit PySpark statements to HDInsight clusters with interactive responses," said Jenny Jiang, principal program manager, Big Data Team, in a blog post Wednesday (Jan. 23). "This interactivity brings the best properties of Python and Spark to developers and empowers you to gain faster insights.

"For Hive developers, HDInsight tools for VSCode offer great data warehouse query experiences for Big data and helpful features in querying log files and gaining insights."

Key benefits to developers in the tool include:

Integration with Azure worldwide environments for Azure sign-in and HDInsight cluster management
HDInsight Hive and Spark job submission with integration with Spark UI and Yarn UI
Interactive responses with the flexibility to execute one or multiple selected Hive and Python scripts
Preview and export your interactive query results to CSV, JSON, and Excel format
Built-in Hive language services such as IntelliSense auto-suggest, autocomplete, and error marker, among others
Supports HDInsight ESP Cluster and Ambari connection
Simplified cluster and Spark job configuration management

The post also detailed several improvements that were made to the tool since it was released in preview, including: connectivity to more Azure environments; support for the HDInsight Enterprise Security Package; simpler clusters and Spark job configuration management; and integration with VSCode Azure Account and HDInsight Explorer.

The extension lists more than 15,500 installs and more than 52,819 downloads, earning an average 4.1 rating (0-5 scale) from seven developers who reviewed it.

About the Author

David Ramel is an editor and writer at Converge 360.

Printable Format

comments powered by Disqus

Featured

Compare New GitHub Copilot Free Plan for Visual Studio/VS Code to Paid Plans

The free plan restricts the number of completions, chat requests and access to AI models, being suitable for occasional users and small projects.
Diving Deep into .NET MAUI

Ever since someone figured out that fiddling bits results in source code, developers have sought one codebase for all types of apps on all platforms, with Microsoft's latest attempt to further that effort being .NET MAUI.
Copilot AI Boosts Abound in New VS Code v1.96

Microsoft improved on its new "Copilot Edit" functionality in the latest release of Visual Studio Code, v1.96, its open-source based code editor that has become the most popular in the world according to many surveys.
AdaBoost Regression Using C#

Dr. James McCaffrey from Microsoft Research presents a complete end-to-end demonstration of the AdaBoost.R2 algorithm for regression problems (where the goal is to predict a single numeric value). The implementation follows the original source research paper closely, so you can use it as a guide for customization for specific scenarios.
Versioning and Documenting ASP.NET Core Services

Building an API with ASP.NET Core is only half the job. If your API is going to live more than one release cycle, you're going to need to version it. If you have other people building clients for it, you're going to need to document it.