Data Driver

Blog archive

Microsoft Shows Off BI "Engine Of The Devil"

Microsoft hinted at some amazing new capabilities coming to its BI products in yesterday’s Business Intelligence Conference keynote, including an "engine of the devil" that allows instant analysis of more than 2 billion rows of data.

Microsoft BI engineer Amir Netz joined senior exec Ted Kummert to demo new technology the company is working on to go beyond current capabilities of its recently released PowerPivot "BI for the masses" application. I previously wrote about a demo I attended that showed how easily PowerPivot could handle 44 million rows of data.

Netz had upped the ante Monday at Microsoft’s TechEd conference in New Orleans with a 100-million-row example. Yesterday, using a server with more memory than the desktops used before, he seamlessly tackled the 2-billion-row demo with the same storage technology used in PowerPivot. "You know, we talked about wicked fast yesterday," he said. "This is beyond wicked fast; this is the engine of the devil, right?"

The crowd loved it, but Kummert wasn’t so sure about the characterization. Microsoft senior leadership probably isn’t too keen on branding its products with references to Satan. "You said that, I didn’t," Kummert said, before joking about Netz’s upcoming performance review.

Netz said the company had received great feedback on PowerPivot, but some BI pros wanted more capabilities and functionality. His demo connected the PowerPivot technology that can be used in Excel to SQL Server Analysis Services for some incredible performance metrics.

He explained the demo: "We have 10 widgets on the screen, six slicers and four charts. And each one of those sends two queries to the data source in order to render itself. So we have about 20 queries being sent. Every one of those queries is a full table-scan of the 2 billion rows. So all together whenever I click, we are scanning 40 billion rows, and it takes about two and a half seconds to do that. So if you just do the math in your head, we are seeing here a scan rate of a trillion rows per minute. That's kind of what we're talking about."

Then he jokingly countered Kummert by saying "Performance review!"

Netz also shows some striking PowerPivot data visualization capabilities that reportedly will be available in the next 30 days.

In another tidbit, Kummert said the company was working on bringing the full capabilities of SQL Server to SQL Azure, its cloud-based "set of relational features oriented toward application development scenarios." That mismatch of features has been a point of contention in the SQL Server community.

"We're hard at work on that now," Kummert said. "I'm not announcing the specific release timeframe for that, but this is something you're going to hear from us shortly in the future."

Check out the video of the keynote and weigh in with your thoughts in the comment section below or send me an e-mail.

Posted by David Ramel on 06/09/2010


comments powered by Disqus

Featured

  • Mastering Blazor Authentication and Authorization

    At the Visual Studio Live! @ Microsoft HQ developer conference set for August, Rockford Lhotka will explain the ins and outs of authentication across Blazor Server, WebAssembly, and .NET MAUI Hybrid apps, and show how to use identity and claims to customize application behavior through fine-grained authorization.

  • Linear Support Vector Regression from Scratch Using C# with Evolutionary Training

    Dr. James McCaffrey from Microsoft Research presents a complete end-to-end demonstration of the linear support vector regression (linear SVR) technique, where the goal is to predict a single numeric value. A linear SVR model uses an unusual error/loss function and cannot be trained using standard simple techniques, and so evolutionary optimization training is used.

  • Low-Code Report Says AI Will Enhance, Not Replace DIY Dev Tools

    Along with replacing software developers and possibly killing humanity, advanced AI is seen by many as a death knell for the do-it-yourself, low-code/no-code tooling industry, but a new report belies that notion.

  • Vibe Coding with Latest Visual Studio Preview

    Microsoft's latest Visual Studio preview facilitates "vibe coding," where developers mainly use GitHub Copilot AI to do all the programming in accordance with spoken or typed instructions.

  • Steve Sanderson Previews AI App Dev: Small Models, Agents and a Blazor Voice Assistant

    Blazor creator Steve Sanderson presented a keynote at the recent NDC London 2025 conference where he previewed the future of .NET application development with smaller AI models and autonomous agents, along with showcasing a new Blazor voice assistant project demonstrating cutting-edge functionality.

Subscribe on YouTube