Redmond Diary

By Andrew J. Brust

Blog archive

NoSQL, No Peace

After several months of research, review and revision, a white paper I wrote for the SQL Azure team, "NoSQL and the Windows Azure Platform", has been published by Microsoft. If you go to http://www.microsoft.com/windowsazure/whitepapers and do a find within the page for "NoSQL" you'll see a link for it. If you'd rather download the PDF directly, you can do so by clicking here. The 25-page (not including cover and TOC) paper provides an introduction to NoSQL database technology, and its major subcategories, for those new to the subject; an examination of NoSQL technologies available in the cloud using Windows Azure and SQL Azure; and a critical discussion of the NoSQL and relational database approaches, including the suitability of each to line-of-business application development.

As I conducted my research for the paper, and read material written by members of the NoSQL community, I found a consistent sentiment toward, and desire for, cleaning the database technology slate. NoSQL involves returning to a basic data storage and retrieval approach. Many NoSQL databases, including even Microsoft's Azure Table Storage, are premised on basic key-value storage technology -- in effect, the same data structure used in collections, associative arrays and cache products.

I couldn't help thinking that the recent popularity of NoSQL is symptomatic of a generational and cyclical phenomenon in computing. As product categories (relational databases in this case) mature, products within them load up on features and create a barrier to entry for new, younger developers. The latter group may prefer to start over with a fresh approach to the category, rather than learn the wise old ways of products whose market presence predates their careers -- sometimes by a couple of decades.

The new generation may do this even at the risk of regression in functionality. In the case of NoSQL databases, that regression may include loss of "ACID" (transactional) guarantees; declarative query (as opposed to imperative inspection of collections of rows); comprehensive tooling; and wide availability of trained and experienced professionals. Existing technologies have evolved in response to the requirements, stress tests, bug reports, and user suggestions accumulated over time. And sometimes old technologies can even be used in ways equivalent to the new ones. Two cases in point: the old SQL Server Data Services was a NoSQL store, and its underlying implementation used SQL Server. Even the developer fabric version of Azure Table Storage is implemented using SQL Server Express Edition's XML columns.

So if older technologies are proven technologies, and if they can be repurposed to function like some of the newer ones, what causes such discomfort with them? Is it mere folly of younger developers? Are older developers building up barriers of vocabulary, APIs and accumulated, sometimes seldom used, features in their products, to keep their club an exclusive one?

In other engineering disciplines, evolution in technology is embraced, built upon, made beneficial to consumers, and contributory to overall progress. But the computing disciplines maintain a certain folk heroism in rejecting prior progress as misguided. For some reason, we see new implementations of established solutions as elegant and laudable. And virtually every player in the industry is guilty of this. I haven't figured out why this phenomenon exists, but I think it's bad for the industry. It allows indulgence to masquerade as enlightenment, and it holds the whole field back.

Programming has an artistic element to it; it's not mere rote science. That's why many talented practitioners are attracted to the field, and removing that creative aspect of software work would therefore be counter-productive. But we owe it to our colleagues, and to our customers, to conquer fundamentally new problems, rather than provide so many alternative solutions to the old ones. There's plenty of creativity involved in breaking new ground too, and I dare say it brings more benefit to the industry, even to society.

NoSQL is interesting technology and its challenge to established ways of thinking about data does have merit and benefit. Nevertheless, I hope the next disruptive technology to come along says yes to conquering new territory. At the very least, I hope it doesn't start with "No."

Posted by Andrew J. Brust on 04/25/2011


comments powered by Disqus

Featured

  • Creating Reactive Applications in .NET

    In modern applications, data is being retrieved in asynchronous, real-time streams, as traditional pull requests where the clients asks for data from the server are becoming a thing of the past.

  • AI for GitHub Collaboration? Maybe Not So Much

    No doubt GitHub Copilot has been a boon for developers, but AI might not be the best tool for collaboration, according to developers weighing in on a recent social media post from the GitHub team.

  • Visual Studio 2022 Getting VS Code 'Command Palette' Equivalent

    As any Visual Studio Code user knows, the editor's command palette is a powerful tool for getting things done quickly, without having to navigate through menus and dialogs. Now, we learn how an equivalent is coming for Microsoft's flagship Visual Studio IDE, invoked by the same familiar Ctrl+Shift+P keyboard shortcut.

  • .NET 9 Preview 3: 'I've Been Waiting 9 Years for This API!'

    Microsoft's third preview of .NET 9 sees a lot of minor tweaks and fixes with no earth-shaking new functionality, but little things can be important to individual developers.

  • Data Anomaly Detection Using a Neural Autoencoder with C#

    Dr. James McCaffrey of Microsoft Research tackles the process of examining a set of source data to find data items that are different in some way from the majority of the source items.

Subscribe on YouTube