News

Microsoft Improves Natural Language UI for LLMs with Open Source TypeChat

"No more prompt engineering! Types are all you need."

• By David Ramel
• 07/25/2023

Microsoft announced an open source project called TypeChat to help developers create better natural language UI for working with the large language models (LLMs) in advanced AI-powered applications.

As its name suggests, it leverages the company's TypeScript programming language, which builds on vanilla JavaScript by adding the ability to use types.

The idea behind TypeChat is to provide structured type-safe data -- in a form such as JSON -- that machine language systems can better work with, as opposed to data typically delivered in the form of a casual human conversation. So it's much easier to write code to present the data in a structured way.

Some example apps were published on GitHub to help developers get started creating apps that could use functionality including:

For a simplified example, TypeChat can be used in a chatbot app to respond with type-safe data to the user query (or prompt): "Provide 3 suggestions for specific places to go to in Seattle on a rainy day."

Rather than asking the app to work with a natural language response like, "Here are three specific places you can go to in Seattle on a rainy day," it can work with more structured data -- to present in a table for example -- like:

{
  "data": [
    {
      "venue": "Museum of Pop Culture",
      "description": "Explore the fascinating world of popular culture through interactive exhibits, music memorabilia, and multimedia displays at the Museum of Pop Culture."
    },
    {
      "venue": "Seattle Art Museum",
      "description": "Indulge in the diverse collection of artworks from around the world at the Seattle Art Museum, featuring both contemporary and classical pieces."
    },
    {
      "venue": "Pike Place Market",
      "description": "Enjoy the vibrant atmosphere of Pike Place Market, where you can wander through covered stalls offering fresh produce, crafts, and delicious food from local vendors."
    }
  ]
}

However, Microsoft said, examples like the above might help guide structure, but they don't define what an AI should return extensively, or provide anything to be validated against.

TypeScript and its compiler can do those types of things.

Types Are All You Need

"TypeScript has shown that it's well-suited to precisely describe JSON. But what happens when a language model stumbles and makes up a response that doesn't conform to our types?" Microsoft asked in a July 20 blog post.

"Well because these types are valid TypeScript code, we can validate the response against them using the TypeScript compiler itself! In fact, the error feedback from the compiler can even be used to guide repairs. When put together, we can get a robust process for getting well-typed responses that our apps can further massage, validate with a user, etc."

That works to the open source project's goal: "TypeChat was created with the purpose of increasing safety in natural language interfaces."

TypeChat works to go beyond TypeScript compiler capabilities by adding things like:

• Constructing a prompt to the LLM using types.
• Validating the LLM response conforms to the schema. If the validation fails, repair the non-conforming output through further language model interaction.
• Summarizing succinctly (without use of a LLM) the instance and confirm that it aligns with user intent.

A FAQ says: "TypeChat was created with the purpose of increasing safety in natural language interfaces.

We believe TypeChat has three key primary benefits when working with large language models:

• Accurate: Large language models do a great job matching user intent to scoped types. TypeChat's validation and repair cleans up the rest!
• Approachable: No more prompt engineering! Types are all you need. You probably have them already lying around.
• Safety: Types constrain domain and model uncertainty. Repeating back the instance confirms that it aligns with user intent before taking action."

For developers who really want to get started with seeing what TypeChat can do, Microsoft published app examples ranging from Sentiment to Restaurant to Music on GitHub.

Also, Microsoft's TypeChat GitHub.io site provides links to documentation, blog and a readymade npm install typechat command for installing the NuGet package.