Data Driver

Blog archive

Microsoft Shows Off BI "Engine Of The Devil"

Microsoft hinted at some amazing new capabilities coming to its BI products in yesterday’s Business Intelligence Conference keynote, including an "engine of the devil" that allows instant analysis of more than 2 billion rows of data.

Microsoft BI engineer Amir Netz joined senior exec Ted Kummert to demo new technology the company is working on to go beyond current capabilities of its recently released PowerPivot "BI for the masses" application. I previously wrote about a demo I attended that showed how easily PowerPivot could handle 44 million rows of data.

Netz had upped the ante Monday at Microsoft’s TechEd conference in New Orleans with a 100-million-row example. Yesterday, using a server with more memory than the desktops used before, he seamlessly tackled the 2-billion-row demo with the same storage technology used in PowerPivot. "You know, we talked about wicked fast yesterday," he said. "This is beyond wicked fast; this is the engine of the devil, right?"

The crowd loved it, but Kummert wasn’t so sure about the characterization. Microsoft senior leadership probably isn’t too keen on branding its products with references to Satan. "You said that, I didn’t," Kummert said, before joking about Netz’s upcoming performance review.

Netz said the company had received great feedback on PowerPivot, but some BI pros wanted more capabilities and functionality. His demo connected the PowerPivot technology that can be used in Excel to SQL Server Analysis Services for some incredible performance metrics.

He explained the demo: "We have 10 widgets on the screen, six slicers and four charts. And each one of those sends two queries to the data source in order to render itself. So we have about 20 queries being sent. Every one of those queries is a full table-scan of the 2 billion rows. So all together whenever I click, we are scanning 40 billion rows, and it takes about two and a half seconds to do that. So if you just do the math in your head, we are seeing here a scan rate of a trillion rows per minute. That's kind of what we're talking about."

Then he jokingly countered Kummert by saying "Performance review!"

Netz also shows some striking PowerPivot data visualization capabilities that reportedly will be available in the next 30 days.

In another tidbit, Kummert said the company was working on bringing the full capabilities of SQL Server to SQL Azure, its cloud-based "set of relational features oriented toward application development scenarios." That mismatch of features has been a point of contention in the SQL Server community.

"We're hard at work on that now," Kummert said. "I'm not announcing the specific release timeframe for that, but this is something you're going to hear from us shortly in the future."

Check out the video of the keynote and weigh in with your thoughts in the comment section below or send me an e-mail.

Posted by David Ramel on 06/09/2010


comments powered by Disqus

Featured

  • Creating Reactive Applications in .NET

    In modern applications, data is being retrieved in asynchronous, real-time streams, as traditional pull requests where the clients asks for data from the server are becoming a thing of the past.

  • AI for GitHub Collaboration? Maybe Not So Much

    No doubt GitHub Copilot has been a boon for developers, but AI might not be the best tool for collaboration, according to developers weighing in on a recent social media post from the GitHub team.

  • Visual Studio 2022 Getting VS Code 'Command Palette' Equivalent

    As any Visual Studio Code user knows, the editor's command palette is a powerful tool for getting things done quickly, without having to navigate through menus and dialogs. Now, we learn how an equivalent is coming for Microsoft's flagship Visual Studio IDE, invoked by the same familiar Ctrl+Shift+P keyboard shortcut.

  • .NET 9 Preview 3: 'I've Been Waiting 9 Years for This API!'

    Microsoft's third preview of .NET 9 sees a lot of minor tweaks and fixes with no earth-shaking new functionality, but little things can be important to individual developers.

  • Data Anomaly Detection Using a Neural Autoencoder with C#

    Dr. James McCaffrey of Microsoft Research tackles the process of examining a set of source data to find data items that are different in some way from the majority of the source items.

Subscribe on YouTube