Visual Studio Code Gets HDInsight Tools for Big Data Analytics

Microsoft announced HDInsight Tools for Visual Studio Code is now generally available, letting coders do Big Data analytics right from within the cross-platform, open source code editor.

Specifically, Azure HDInsight Tools for Visual Studio Code is an extension in the Visual Studio Code Marketplace "for developing Hive Interactive Query, Hive Batch Job and PySpark Job against Microsoft HDInsight." The company describes Azure HDInsight as an enterprise-grade service for open source analytics.

The Azure HDInsight service supports popular open source Apache products including Hadoop, Spark, Kafka, HBase, Live LLAP and Storm, as well as including machine learning functionality. The VSCode extension, however focuses on developing HDInsight PySpark and Hive batch jobs and interactive query.

HDInsight Tools for Visual Studio Code
[Click on image for larger view.] HDInsight Tools for Visual Studio Code (source: Microsoft).

"For PySpark developers who value the productivity Python enables, HDInsight Tools for VSCode offer a quick Python editor with simple getting started experiences, and allow you to submit PySpark statements to HDInsight clusters with interactive responses," said Jenny Jiang, principal program manager, Big Data Team, in a blog post Wednesday (Jan. 23). "This interactivity brings the best properties of Python and Spark to developers and empowers you to gain faster insights.

"For Hive developers, HDInsight tools for VSCode offer great data warehouse query experiences for Big data and helpful features in querying log files and gaining insights."

Key benefits to developers in the tool include:

  • Integration with Azure worldwide environments for Azure sign-in and HDInsight cluster management
  • HDInsight Hive and Spark job submission with integration with Spark UI and Yarn UI
  • Interactive responses with the flexibility to execute one or multiple selected Hive and Python scripts
  • Preview and export your interactive query results to CSV, JSON, and Excel format
  • Built-in Hive language services such as IntelliSense auto-suggest, autocomplete, and error marker, among others
  • Supports HDInsight ESP Cluster and Ambari connection
  • Simplified cluster and Spark job configuration management

The post also detailed several improvements that were made to the tool since it was released in preview, including: connectivity to more Azure environments; support for the HDInsight Enterprise Security Package; simpler clusters and Spark job configuration management; and integration with VSCode Azure Account and HDInsight Explorer.

The extension lists more than 15,500 installs and more than 52,819 downloads, earning an average 4.1 rating (0-5 scale) from seven developers who reviewed it.

About the Author

David Ramel is an editor and writer for Converge360.

comments powered by Disqus


  • C# Slides in Usage Ranking of Programming Languages

    "The fact that C# lost three places in the ranking of language communities during the last three years is mostly explained by its slower growth compared to C/C++ and PHP."

  • Telerik UI for Blazor Updated

    Progress announced an update to its Telerik UI for Blazor components, targeting Microsoft's open source Blazor framework that lets C# coders create web apps without having to rely upon JavaScript.

  • Infragistics Unveils UI Components for Blazor

    Infragistics, specializing in third-party UI/UX controls and tools, unveiled a new offering targeting Blazor, Microsoft's red-hot open source framework that allows for C#-based web development instead of traditional mainstay JavaScript.

  • AWS Open Sources Tool for Porting .NET Framework Apps to .NET Core

    Leading cloud computing platform Amazon Web Services open sourced the it announced in July for helping users port old .NET Framework applications to the new .NET Core framework.

  • Uno Platform Ports Windows Calculator to Linux

    Uno Platform has ported the famed Windows Calculator, open sourced last year, to Linux as part of a continuing "proof point" effort to demonstrate the reach of what it describes as the sole UI offering available to target Windows, WebAssembly, iOS, macOS, Android and Linux with single-codebase applications coded in C# and XAML.

Upcoming Events