Q&A

Build Your First AI Applications with Local AI

• By David Ramel
• 12/05/2024

Developing AI applications using local models -- often referred to as "local AI" -- involves running AI algorithms directly on your local infrastructure, no cloud-based services needed. Many developers believe this approach offers enhanced privacy, reduced latency, and greater control over their data.

Considerations when building local AI applications include:

• Hardware requirements: While many local AI tools are optimized for consumer-grade hardware, performance can vary based on your device's specifications.
• Model selection: Choose models that are compatible with local deployment and align with your application's requirements.
• Data management: Ensure that your data is organized and accessible for local processing, keeping in mind storage limitations.

So getting started building your first local AI application might seem daunting, which is why Craig Loewen, senior product manager at Microsoft, will explain exactly how do do that in the aptly named session "Build Your First AI Applications with Local AI" at the big Visual Studio Live! developer conference coming to Las Vegas in March.

"AI right now feels like a vast space which can be hard to jump into," Loewen says. "This session will be boiling it down to hands on ways to get started by running AI models locally on your machine to develop your first application."

For that, he promises attendees will learn:

• An overview of what "AI" means in terms of what's available for developers in the market
• How to use AI models locally
• How to build a sample app with a local AI model

We caught up with Loewen to get a sneak peek at his session, and he shared some insights on the current state of AI development and what developers can expect from his presentation.

VisualStudioMagazine: What inspired you to present a session on this topic?
Loewen: At the specific team I work at within Microsoft, our goal is to make development awesome on Windows! And as part of that, we took a look at how we could utilize AI to help supercharge some developer workflows, which is how we have created features like Terminal Chat in Windows Terminal, and Advanced Paste in PowerToys. As part of that exploration I learned a lot about using local AI, and wanted to share those learnings with a broader audience.

What are the primary advantages of developing AI applications locally compared to utilizing cloud-based solutions?
The big advantages are cost, and not having to send anything to a server. If you can run something locally on your machine, then you can develop without a strong network connection, and without requiring to pay for cloud usage. This doesn't just apply to development either, as your end users might run in a network-less environment, or may be interacting with sensitive data that they don't want leaving their machine.

Could you elaborate on one or two of the key challenges developers might face when building AI applications on local machines?
I find one of the biggest challenges is how to get started. There's Chat GPT, GPT-4o, RAG, and so many more acronyms, providers, platforms and different models. This talk will aim to simplify that and scope it down to a practical example to get you started, and get you the knowledge of what those terms refer to!

What hardware specifications are recommended for efficient local AI development?
Essentially, the better the hardware, the better answers you will get. But you can run the examples I'm going to show on a machine with 16 GB of RAM, and on CPU. Having a GPU will make the process faster.

Is it difficult integrate locally developed AI applications into larger, cloud-based systems if needed?
In general no! Since this session will be focusing on language models, they are just using natural language processing, and it is possible to build a larger scale cloud model and use that instead.

Note: Those wishing to attend the conference can save hundreds of dollars by registering early, according to the event's pricing page. "Save $500 when you Register by the Cyber week deadline of Dec. 6," said the organizer of the event, which is presented by the parent company of Visual Studio Magazine.