In-Depth

Bard vs. Bing: AI Search Bots Answer Questions About Visual Studio and .NET

• By David Ramel
• 03/22/2023

With Google recently releasing a generative AI-powered search bot called Bard to rival Microsoft's "new Bing" search experience, we put both to the test, feeding them identical questions about Visual Studio and .NET.

Note that, at this stage, it's a bit of an apples-to-oranges comparison, as Google was slightly later to the game and heavily emphasizes the "experiment" focus of the release, designed to help the company solicit feedback and iteratively improve the experience. While Bard was just made available yesterday, the new Bing was announced some six weeks earlier.

Both machine language constructs use cutting-edge large language models (LLM), GPT-4 for Bing, created by Microsoft partner OpenAI (creator of ChatGPT), and the homegrown LaMDA (Language Model for Dialogue Applications) for Bard.

While neither LLM has been optimized for coding (like OpenAI's Codex, which powers GitHub Copilot), both can generate code, producing nearly identical results when asked to generate a C# bubble sort function (admittedly the most basic of examples):

We decided to just feed it ordinary questions of interest to readers of Visual Studio Magazine.

Note that both bots provide different answers to the same questions on different days, or even to the same question immediately asked again. So the following represents only a snapshot in time -- your mileage will vary, with different users almost certainly receiving different answers.

But the comparison does give a general idea of how each bot works, with Bard being more of a "just-the-facts" conversational partner, while Bing is more chatty and friendly, in an AI sort of way (at least using the "More Balanced" option, as opposed to "More Creative" and "More Precise"). Bard also pays much more attention to formatting, often presenting answers in easy-to-read bullet-point lists, while Bing tends to mostly just lump its responses together, though it does sometime use lists. Also, Bing fully documents its work, providing links to the sources it consulted in generating its responses, though it has an odd habit of providing different footnote link numbers (like, 1, 2 and 3) that all go to the same source. Bard, on the other hand, only provided a linked source in one response.

In fact, there are many other differences between the two, so we asked both about those differences:

What are the differences between Google's new Bard search experience and Microsoft's new Bing search experience, both powered by AI?

• Bard:

  Google's new Bard search experience and Microsoft's new Bing search experience, both powered by AI, have a number of similarities and differences.

  Similarities:

  • Both search engines use AI to generate more relevant and helpful results.
  • Both search engines allow users to ask questions in natural language.
  • Both search engines provide a variety of features, such as image search and video search.

  Differences:

  • Google's Bard search experience is more focused on providing comprehensive and informative answers to questions, while Microsoft's Bing search experience is more focused on providing quick and easy access to information.
  • Google's Bard search experience is more personalized, while Microsoft's Bing search experience is more generic.
  • Google's Bard search experience is more integrated with other Google products, such as Google Assistant and Google Maps, while Microsoft's Bing search experience is more isolated.

  Ultimately, the best search experience for you will depend on your individual needs and preferences. If you are looking for a search engine that provides comprehensive and informative answers to questions, Google's Bard search experience is a good option. If you are looking for a search engine that provides quick and easy access to information, Microsoft's Bing search experience is a good option.

• Bing:

  Google's new AI chatbot Bard is designed to help users brainstorm and answer queries(1). Unlike Bing Chat, Bard does not look up search results --all the information it returns is generated by the model itself(1).

  Microsoft's new Bing Chat experience is powered by AI and uses Microsoft's GPT-3 model(2). The main difference between Bard and BingGPT is that Google's bot is noticeably more dry and uncontroversial than BingGPT(2). When Microsoft's BingGPT came out in early February, it quickly revealed an unhinged side(2).

Note that Bing was pretty hard on itself here, citing the "unhinged" source. It also got its backing model wrong, citing GPT-3 instead of the correct answer, GPT-4.

The Q&A
Keeping all of the above in mind (and noting that each construct sometimes provides wrong answers or simply makes stuff up -- known as hallucinating), here's how each search bot answered identical questions about Visual Studio and .NET.

What are the highlights of Visual Studio 2022 v17.6 Preview 2?

• Bard:

  Visual Studio 2022 v17.6 Preview 2 is a major release that includes a number of new features and improvements. Here are some of the highlights:

  • Improved IntelliSense for C# and Visual Basic: IntelliSense now provides more detailed information about the members of a class or namespace, including their return types and parameters.
  • Better support for .NET Core: Visual Studio now includes a number of features that make it easier to develop and deploy applications on .NET Core, such as support for the new .NET Core SDK and the ability to debug .NET Core applications in Visual Studio.
  • New tools for game developers: Visual Studio now includes a number of new tools that make it easier to develop and debug games, such as the ability to profile game performance and the ability to generate source code from game assets.
  • Enhancements to the C++ compiler: The C++ compiler in Visual Studio has been enhanced to improve the performance of C++ applications.
  • New support for Azure: Visual Studio now includes a number of features that make it easier to develop and deploy applications on Azure, such as support for the Azure SDK and the ability to debug Azure applications in Visual Studio.

  These are just a few of the highlights of Visual Studio 2022 v17.6 Preview 2. To learn more about all of the new features and improvements, please visit the Visual Studio website.

• Bing:

  Visual Studio 2022 v17.6 Preview 2 was released on March 14th, 2023(1). The release notes mention that it has added support for .NET 7.0.202 + 6.0.407 SDKs and improved readability and usability for IDE status bar(1). It also includes new features and improvements for game developers, mobile developers, C++ developers and more(2).

Observations: Both Bard and Bing dropped the ball on this one. It looks like Bard just generated responses concerning generic Visual Studio rather than Preview 2. For example, its first bullet point mentions IntelliSense, which isn't discussed at all in the Preview 2 blog post or release notes. Bing, meanwhile, generated Preview 2-specific info, but primarily based its response on the release notes for the Mac version of the IDE, not the much different and much more widely used Windows version.

What are the highlights of .NET 8?

• Bard:

  .NET 8 is the latest version of the .NET Framework, a software framework that allows developers to build and run applications on Windows, macOS, Linux, and other platforms. .NET 8 includes a number of new features and improvements, including:

  • Support for C# 10: C# 10 is the latest version of the C# programming language, and it includes a number of new features, such as pattern matching, record types, and improved nullability support.
  • Support for .NET Core 3.1: .NET Core 3.1 is a major update to the .NET Core framework, and it includes a number of new features, such as support for Windows Forms and WPF applications.
  • Improved performance: .NET 8 includes a number of performance improvements, such as a new garbage collector and improved JIT compilation.
  • Improved security: .NET 8 includes a number of security improvements, such as support for .NET Core's new security model and improved code signing support.
  • Improved developer experience: .NET 8 includes a number of improvements to the developer experience, such as improved IntelliSense and better support for Visual Studio's new features.

  These are just a few of the highlights of .NET 8. For more information, please visit the .NET website.

• Bing:

  .NET 8 introduces several new features and improvements(1). Some of these include:

  • 'dotnet publish' and 'dotnet pack' commands have been improved with new options(2).
  • System.Text.Json serialization has been improved with new features(2).
  • New methods for working with randomness have been added(2).
  • Performance-focused types such as FrozenDictionary<TKey,TValue> and FrozenSet<T> have been added(1).
  • Native AOT (Ahead-of-Time) compilation is now supported(3).
  • Code generation has been improved(1).
  • .NET container images have been updated(2).

  You can find more details about these features and more in Microsoft's official documentation (2)(1)(4).

Observations: Bard slipped again, for example, inexplicably citing support for the really old .NET Core 3.1. Cases like this show the value of Bing's linked sources, allowing users to more easily fact-check the sometimes wild responses. Also, C# 10 is not the latest version of C#, as v11 was released last November. Bing did much better, generating its response primarily from Microsoft's "what's new" post about .NET 8.

What is "Blazor United"?

• Bard:

  Blazor United is a project that aims to unify the different ways of building web applications with Blazor. It will allow developers to use a single Blazor-based architecture for all their web UI scenarios, including:

  • Plain HTML rendering
  • Full interactivity either server-side or on WebAssembly
  • Streaming Rendering for fast loads from the server
  • Progressively enhance sites to make navigation and form submissions smoother
  • Adopt a full-blown SPA approach for parts of an app

  Blazor United is still under development, but it has the potential to make Blazor even more powerful and flexible. Sources - Learn more jonhilton.net/blazor-united/

• Bing:

  Blazor United is a feature in ASP.NET Core roadmap for .NET 8(1)(2)(3). It will enable you to use a single Blazor-based architecture for server-side rendering and full client-side interactivity with Blazor Server or WebAssembly(3)(4). That's all within a single project with the ability to easily switch between different rendering modes and even mix them in the same page(3)(4).

Observations: Both did well on this question, but Bing gets the edge because it sourced Visual Studio Magazine 😊.

Conclusion
As expected in this early experimental stage, Bard is more likely to bungle its responses. Testing that bungling is the whole point of the experimental preview, after all, with users asked to give each response a thumbs-up or thumbs-down. And those planned iterative improvements will likely source its responses better with links so they can more easily be fact checked.

Bard, however, was just as fast as Bing or sometimes even faster in generating its responses, and as noted, formats responses better. It will be interesting to revisit this test in a few months or however long it takes for improved iterations to take hold. Stay tuned.