News

Enabling Enterprise AI for Everyone with SQL Server

• By David Ramel
• 12/05/2018

Microsoft believes artificial intelligence is so powerful it should be "democratized" so organizations and developers of all types can use it to transform and improve their business practices.

In fact, as mysterious and arcane as AI may seem to the mainstream, Microsoft believes it should be a primary tool used by not only data scientists, but also developers, business analysts, end users and just about everyone else in an organization.

"AI is for everyone and my goal is to showcase some of the ways we are trying to make AI accessible to all kinds of developers," explained Pranav Rastogi yesterday in a keynote address titled "Enabling Enterprise Developers in AI - How Microsoft is Doing It" at the .NET-centric Live! 360 conference in Orlando.

It turns out Microsoft is doing it with a lot of help from its data services, primarily its flagship RDBMS, SQL Server, and many related services hosted in the Azure cloud.

Rastogi, a Microsoft program manager, was on hand to provide the keynote presentation for the Artificial Intelligence Live! and SQL Server Live! segments of the multi-part tech conference, showing how tightly AI and SQL Server are intertwined at the company.

In fact, as we reported, the preview of the upcoming SQL Server 2019 bakes in Big Data capabilities to provide the data-based bedrock of AI analysis.

The company said SQL Server 2019 creates a unified data platform by packaging Apache Spark and Hadoop Distributed File System (HDFS) in with the SQL Server database engine, helping data developers seamlessly ingest, store and analyze vast amounts of data.

Combining Big Data with SQL Server, Rastogi said, lets .NET-centric developers work with popular Hadoop-related technologies such as Apache Spark without having to move data from one platform to another. "Data becomes a lot more simpler with this edition," he said. For example, data from many different kinds of disparate sources can be combined for operation such as running Spark on SQL clusters and doing SQL queries on HDFS-stored data. "It's about integrating all your data sources" and managing the data together, Rastogi said.

Other popular tools can be used also, including those from the open source camp such as Jupyter notebooks and the Python and Scala programming languages that developers can use in familiar environments.

Another part of an enterprise data modernization effort involves moving to the Azure cloud, with a typical scenario seeing SQL Server run on-premises and cloud services such as Azure SQL Database taking over for migrations to the sky. Azure SQL Database provide managed instances that make it easy to move enterprise data systems to the cloud.

Rastogi noted that Microsoft, recognizing the popularity of open source tools and wanting to give developers choices, also lets them use other relational database engines to run on Azure, such as PostgreSQL, MySQL, MariaDB and so on.

For organizations starting out by building cloud-native applications in the cloud, one service that is popular is Azure Cosmos DB, Microsoft's globally distributed, multi-model database service "for managing data at planet-scale." Rastogi said it provides a layer where developers can build multiple applications on the stack, using database choices such as Cassandra, MongoDB and so on.

Once data of all kinds is collected, prepped and combined into a data lake, the AI functionality comes into play to provide actionable business insights.

Some of the tools used here mentioned by Rastogi around both sides of this effort include the following, with official company descriptions:

• Azure Data Factory: "Hybrid data integration at global scale."
• Azure Databricks: "Fast, easy, and collaborative Apache Spark–based analytics service."
• Azure SQL Data Warehouse: "Lets you quickly implement a high-performance, globally available, and secure cloud data warehouse. You can independently scale compute and storage, while pausing and resuming your data warehouse within minutes through a massively parallel processing architecture designed for the cloud."
• Power BI: "Power BI is a business analytics solution that lets you visualize your data and share insights across your organization, or embed them in your app or website. Connect to hundreds of data sources and bring your data to life with live dashboards and reports."
• Azure Data Explorer: "Perform ad-hoc queries on terabytes of data with Azure Data Explorer -- a lightning-fast indexing and querying service to help you build near real-time and complex analytics solutions. Azure Data Explorer allows you to quickly identify trends, patterns, or anomalies in all data types inclusive of structured, semi structured and unstructured data."
• Azure Machine Learning: "Simple, scalable, cutting edge. Welcome to Machine Learning Studio, the Azure Machine Learning solution you've grown to love. Machine Learning Studio is a powerfully simple browser-based, visual drag-and-drop authoring environment where no coding is necessary. Go from idea to deployment in a matter of clicks."
• Azure Cognitive Services: "Infuse your apps, websites and bots with intelligent algorithms to see, hear, speak, understand and interpret your user needs through natural methods of communication. Transform your business with AI today."
• ML.NET: "An open source and cross-platform machine learning framework."

"Microsoft has provided all these services to take care of key concepts around: reliability, scalability and deployment in a secure, enterprise manner, so you can start using these services very, very quickly," Rastogi said.

Machine learning is a particularly hot area of AI, and Rastogi noted that developers in the .NET/Azure camp can leverage prebuilt ML models or create their own.

"So you can leverage Azure and deploy all these models on CPUs or GPUs, or FPGAs as well, and then you can complete your entire CI/CD loop from a modern operations organization's perspective," Rastogi said. "Azure Machine Learning allows you to train and deploy models across all these form factors. And then, behind the scenes, it has telemetry built in which allows you to track how model performance is doing over time, and then you can sort of manage these models into production."

Rastogi summarized his presentation from four different perspectives of users who Microsoft wants to empower with AI capabilities.

"From a business/end-user perspective, using Power BI you can easily discover and explore insights with AI in minutes," Rastogi said. "You can apply Azure Cognitive Services or you can apply models that were made using your data science team.

"From a business analyst perspective, you can easily do scenarios like recommendations or sentiment analysis by exploring your data in Power BI and using PowerQuery, and then you can add insights to your data set.

"From a BI pro perspective, you can do self-service ML. And from a data science perspective you can use the Azure AI platform to build custom models that you can deploy using Azure ML, and from a business analyst's perspective, all you have to do is consume those models."

Finally, Rastogi concluded: "The idea is to democratize AI for each and every employee, so that it's available and employees can use it to transform their own businesses."

The next full Visual Studio Live! conference will be held in Las Vegas March 3-8, 2019.