Enabling Enterprise AI for Everyone with SQL Server

Microsoft believes artificial intelligence is so powerful it should be "democratized" so organizations and developers of all types can use it to transform and improve their business practices.

In fact, as mysterious and arcane as AI may seem to the mainstream, Microsoft believes it should be a primary tool used by not only data scientists, but also developers, business analysts, end users and just about everyone else in an organization.

"AI is for everyone and my goal is to showcase some of the ways we are trying to make AI accessible to all kinds of developers," explained Pranav Rastogi yesterday in a keynote address titled "Enabling Enterprise Developers in AI - How Microsoft is Doing It" at the .NET-centric Live! 360 conference in Orlando.

"AI is for everyone and my goal is to showcase some of the ways we are trying to make AI accessible to all kinds of developers."

Pranav Rastogi, Program Manager, Microsoft

It turns out Microsoft is doing it with a lot of help from its data services, primarily its flagship RDBMS, SQL Server, and many related services hosted in the Azure cloud.

Rastogi, a Microsoft program manager, was on hand to provide the keynote presentation for the Artificial Intelligence Live! and SQL Server Live! segments of the multi-part tech conference, showing how tightly AI and SQL Server are intertwined at the company.

SQL Server 2019
[Click on image for larger view.] SQL Server 2019 (source: Microsoft).

In fact, as we reported, the preview of the upcoming SQL Server 2019 bakes in Big Data capabilities to provide the data-based bedrock of AI analysis.

The company said SQL Server 2019 creates a unified data platform by packaging Apache Spark and Hadoop Distributed File System (HDFS) in with the SQL Server database engine, helping data developers seamlessly ingest, store and analyze vast amounts of data.

Combining Big Data with SQL Server, Rastogi said, lets .NET-centric developers work with popular Hadoop-related technologies such as Apache Spark without having to move data from one platform to another. "Data becomes a lot more simpler with this edition," he said. For example, data from many different kinds of disparate sources can be combined for operation such as running Spark on SQL clusters and doing SQL queries on HDFS-stored data. "It's about integrating all your data sources" and managing the data together, Rastogi said.

Pranav Rastogi at Live! 260
[Click on image for larger view.] Pranav Rastogi at Live! 260

Other popular tools can be used also, including those from the open source camp such as Jupyter notebooks and the Python and Scala programming languages that developers can use in familiar environments.

Another part of an enterprise data modernization effort involves moving to the Azure cloud, with a typical scenario seeing SQL Server run on-premises and cloud services such as Azure SQL Database taking over for migrations to the sky. Azure SQL Database provide managed instances that make it easy to move enterprise data systems to the cloud.

Rastogi noted that Microsoft, recognizing the popularity of open source tools and wanting to give developers choices, also lets them use other relational database engines to run on Azure, such as PostgreSQL, MySQL, MariaDB and so on.

For organizations starting out by building cloud-native applications in the cloud, one service that is popular is Azure Cosmos DB, Microsoft's globally distributed, multi-model database service "for managing data at planet-scale." Rastogi said it provides a layer where developers can build multiple applications on the stack, using database choices such as Cassandra, MongoDB and so on.

Once data of all kinds is collected, prepped and combined into a data lake, the AI functionality comes into play to provide actionable business insights.

Big Data Clusters Provide 'A Complete AI Platform'
[Click on image for larger view.] Big Data Clusters Provide 'A Complete AI Platform' (source: Microsoft).

Some of the tools used here mentioned by Rastogi around both sides of this effort include the following, with official company descriptions:

  • Azure Data Factory: "Hybrid data integration at global scale."
  • Azure Databricks: "Fast, easy, and collaborative Apache Spark–based analytics service."
  • Azure SQL Data Warehouse: "Lets you quickly implement a high-performance, globally available, and secure cloud data warehouse. You can independently scale compute and storage, while pausing and resuming your data warehouse within minutes through a massively parallel processing architecture designed for the cloud."
  • Power BI: "Power BI is a business analytics solution that lets you visualize your data and share insights across your organization, or embed them in your app or website. Connect to hundreds of data sources and bring your data to life with live dashboards and reports."
  • Azure Data Explorer: "Perform ad-hoc queries on terabytes of data with Azure Data Explorer -- a lightning-fast indexing and querying service to help you build near real-time and complex analytics solutions. Azure Data Explorer allows you to quickly identify trends, patterns, or anomalies in all data types inclusive of structured, semi structured and unstructured data."
  • Azure Machine Learning: "Simple, scalable, cutting edge. Welcome to Machine Learning Studio, the Azure Machine Learning solution you've grown to love. Machine Learning Studio is a powerfully simple browser-based, visual drag-and-drop authoring environment where no coding is necessary. Go from idea to deployment in a matter of clicks."
  • Azure Cognitive Services: "Infuse your apps, websites and bots with intelligent algorithms to see, hear, speak, understand and interpret your user needs through natural methods of communication. Transform your business with AI today."
  • ML.NET: "An open source and cross-platform machine learning framework."

"Microsoft has provided all these services to take care of key concepts around: reliability, scalability and deployment in a secure, enterprise manner, so you can start using these services very, very quickly," Rastogi said.

Machine learning is a particularly hot area of AI, and Rastogi noted that developers in the .NET/Azure camp can leverage prebuilt ML models or create their own.

"So you can leverage Azure and deploy all these models on CPUs or GPUs, or FPGAs as well, and then you can complete your entire CI/CD loop from a modern operations organization's perspective," Rastogi said. "Azure Machine Learning allows you to train and deploy models across all these form factors. And then, behind the scenes, it has telemetry built in which allows you to track how model performance is doing over time, and then you can sort of manage these models into production."

Rastogi summarized his presentation from four different perspectives of users who Microsoft wants to empower with AI capabilities.

"From a business/end-user perspective, using Power BI you can easily discover and explore insights with AI in minutes," Rastogi said. "You can apply Azure Cognitive Services or you can apply models that were made using your data science team.

"From a business analyst perspective, you can easily do scenarios like recommendations or sentiment analysis by exploring your data in Power BI and using PowerQuery, and then you can add insights to your data set.

"From a BI pro perspective, you can do self-service ML. And from a data science perspective you can use the Azure AI platform to build custom models that you can deploy using Azure ML, and from a business analyst's perspective, all you have to do is consume those models."

Finally, Rastogi concluded: "The idea is to democratize AI for each and every employee, so that it's available and employees can use it to transform their own businesses."

The next full Visual Studio Live! conference will be held in Las Vegas March 3-8, 2019.

About the Author

David Ramel is an editor and writer for Converge360.

comments powered by Disqus


  • New .NET 9 Templates for Blazor Hybrid, .NET MAUI

    Microsoft's fifth preview of .NET 9 nods at AI development while also introducing new templates for some of the more popular project types, including Blazor Hybrid and .NET MAUI.

  • What's Next for ASP.NET Core and Blazor

    Since its inception as an intriguing experiment in leveraging WebAssembly to enable dynamic web development with C#, Blazor has evolved into a mature, fully featured framework. Integral to the ASP.NET Core ecosystem, Blazor offers developers a unique combination of server-side rendering and rich client-side interactivity.

  • Nearest Centroid Classification for Numeric Data Using C#

    Here's a complete end-to-end demo of what Dr. James McCaffrey of Microsoft Research says is arguably the simplest possible classification technique.

  • .NET MAUI in VS Code Goes GA

    Visual Studio Code's .NET MAUI workload, which evolves the former Xamarin.Forms mobile-centric framework by adding support for creating desktop applications, has reached general availability.

  • Visual Studio Devs Quick to Sound Off on Automatic Updates: 'Please No'

    A five-year-old Visual Studio feature request for automatic IDE updates is finally getting enacted by Microsoft amid a lot of initial developer pushback, seemingly misplaced.

Subscribe on YouTube