SQL Server Developers Worried as Netflix Chooses NoSQL

Netflix is hiring, but SQL Server gurus need not apply.

The company that transformed from a clunky DVD-by-mail delivery system to cutting-edge video streaming from the cloud has some SQL Server developers worried about their future job prospects as Netflix embraces the NoSQL movement. Few things are scarier than wondering if you backed the wrong horse in your career choice.

"This sort of questions are freaking me out," wrote one blogger about a forum post that expressed concern about the Netflix decision. "Can you imagine how panicked programmers are, knowing that not only tens of programming languages are out there (plus C/C++ is dead, Java is dead, etc.), but also a couple of different programming paradigms?"

The post he was referring to appeared on a SQLServerCentral forum and referenced a Netflix blog discussing the move to the NoSQL camp. "Is this something relational database guys has to be concerned about? I am concerned on what future holds for SQL-Developers and SQL-server DBAs," said forum member sqlcool.

The blog posting that stirred all this up was titled "NoSQL at Netflix" and was written a couple weeks ago by Yury Izrailevsky, director of Cloud and Systems Infrastructure at Netflix.

"While it is not easy to re-architect your systems to not run join queries, or not rely on read-after-write consistency (hey, just cache the value in your app!), we have found ourselves braving the new frontier of NoSQL distributed databases," Izrailevsky said.

The Neflix exec explained: "the reasons behind our choice of three ... NoSQL tools: SimpleDB, Hadoop/HBase and Cassandra."

After discussing each one, he admitted:

Adopting the non-relational model in general is not easy, and Netflix has been paying a steep pioneer tax while integrating these rapidly evolving and still maturing NoSQL products. There is a learning curve and an operational overhead. Still, the scalability, availability and performance advantages of the NoSQL persistence model are evident and are paying for themselves already, and will be central to our long-term cloud strategy.

There has been plenty of other discussion about Netflix and its embrace of NoSQL, including a technical white paper by Netflix software architect "Sid" Anand. He’s going to talk more about it in a meeting next week titled "NoSQL @ Netflix."

Meanwhile, as Izrailevsky concluded: "For those technology superstars out there: Netflix is hiring."

Ouch.

Are you a SQL Server guy worried about this kind of stuff? Please share your thoughts by commenting here or drop me a line.

[Editor's note: This article was updated from its original posting, with corrected information pointed out by the commenter referenced in paragraph 3 and 4.]

Posted by David Ramel on 02/10/2011 at 1:15 PM4 comments


Simplified SQL Azure Development? Nope...

SQL Azure developers are at a disadvantage compared to Windows Azure coders because of a lack of desktop emulators, a Microsoft blog indicated.

The Windows Azure SDK includes emulators that run on your PC so you don't have to subscribe to the online service to code against it, noted Buck Woody in an earlier posting. Yesterday he said that posting brought up the question of whether there was such a thing for SQL Azure.

"The short answer is that there isn't one," Woody said in a Carpe Datum blog posting titled "Where is the SQL Azure Development Environment." (I realize that, being an editor, I'm overly picky about such things, but isn't that headline screaming for a closing question mark?)

He explains how to develop with SQL Azure without emulators ("you can simply treat it as another edition of SQL Server") and it doesn't sound that onerous, but I wonder about the scripting part: "you can script out the database when you're done as a SQL Azure script." Things may well have improved since I tried a SQL Server-to-SQL Azure database conversion script in the early days of the cloud-based offering, but I was pulling my hair out having to make all kinds of manual changes.

Anyway, Woody concluded:

"Will all this change? Will SSMS, 'Data Dude' and other tools change to include SQL Azure? Well, I don't have a specific roadmap for those tools, but we're making big investments on Windows Azure and SQL Azure, so I can say that as time goes on, it will get easier."

What do you think? Are you database guys getting the short shrift, or is it no big deal? Please comment here or drop me a line.

Posted by David Ramel on 02/04/2011 at 1:15 PM0 comments


Microsoft Targets BI in New SQL Server Training Kit

It's no coincidence that business intelligence is the focus of Microsoft's brand-new update to the SQL Server 2008 R2 training kit.

The very first item listed in the announcement for the SQL Server 2008 R2 Update For Developers January 2011 Update (seriously, guys: get some professional help for naming your products!) is "Build Your First Microsoft BI Solution with SQL Server 2008 R2."

The Redmond software giant has championed "self-service BI" in everything from the Business Decision Appliance to the project "Crescent" data visualization tool to the PowerPivot plug-ins for Excel and SharePoint.

Microsoft data and storage VP Ted Kummert seems to have made it his personal crusade. In case there's any doubt about that, a search for "Ted Kummert" and "self service BI" garners 3,380 hits in Google and and 1,190 in Bing.

That full-court press just continues with the new training kit update. It includes an extremely comprehensive set of offerings with 11 units that feature (as listed in the announcement):

  • 31 presentations (PowerPoint slides, videos and transcripts included)
  • 27 demos (installer scripts, videos and transcripts included)
  • 12 hands on labs (installer scripts included)

Let's hear it from you professional BI gurus and vendors out there: is Microsoft trying to put you out of business, like some are hinting? Comment here or drop me a line.

Posted by David Ramel on 01/26/2011 at 1:15 PM1 comments


Script Error, Not SQL Server, Caused Hotmail Outage

Last week the Data Driver wondered if SQL Server was behind the recent Hotmail outage.

The answer is: No.

It was a script error, according to a Windows Blog post by Mike Schackwitz, pointed out to me by a helpful member of Microsoft's PR firm.

To sum up, a script meant to delete temporary accounts used for automated testing accidentally deleted some real users' directory records. This supposedly didn't delete any actual data, but a lot of users responded to the post and disputed that, with many saying they lost years' worth of messages and that some accounts were still unavailable. The complaints were still rolling in as of today.

Microsoft representatives were helping some of those users, but there were many comments along the lines of "Disgusted!!!!" and "Shameful!!" and "IT IS CERTAINLY NOT FIXED."

If you're still missing data, the blog lists several links that might be able to help you recover it.

Are you still having Hotmail data issues? Comment here or drop me a line.

Posted by David Ramel on 01/12/2011 at 1:15 PM7 comments


Was SQL Server Behind Hotmail Outage?

Microsoft is providing scant details about last weekend's Windows Live Hotmail outage, but the Data Driver blog's interest was piqued by a Microsoft announcement in a blog post that customers "temporarily lost the contents of their mailbox through the course of mailbox load balancing between servers."

And, as has been reported many times, Hotmail runs on SQL Server. A December 2009 blog posting that offered "A peek behind the scenes at Hotmail" indicated the service is "the largest SQL Server 2008 deployment in the world (we monitor and manage many thousands of SQL servers)." (Coincidentally, both aforementioned posts were in the Windows Team Blog.)

Of course, after the outage was fixed, the anti-Microsoft camp soon weighed in with some pointed digs, such as these two found in comments on a Yahoo News article:

  • "Wow! Microsoft is blaming their service failure on a Microsoft server. Wow!"
  • "Maybe someone should tell them about Linux?"

But interestingly, according to Yahoo Answers, "Before Microsoft bought out Hotmail, their servers were all Linux based, and Microsoft left them that way for many years until the bad publicity forced them to use their own operating systems."

Of course, all computer systems will suffer failures, and I think Microsoft is as serious about reliability and security as anyone. But coming close on the heels of the cloud-based Business Productivity Online Suite data breach, the Hotmail outage opens the company up to yet more criticism as it seeks to transition to "live" and cloud-based services. Ironically, that "peek behind the scenes at Hotmail" posting mentioned earlier included this information:

We make significant investments in engineering and infrastructure to help keep Hotmail up and running 24 hours a day, day in and day out, year after year. You will rarely hear about these efforts – you will only read about them on the rare occasion that something goes wrong and our service has run into an issue.

Too true.

Do you have any inside info on how Hotmail uses SQL Server or more details on the nature of the outage? Please comment here or drop me a line.

Posted by David Ramel on 01/05/2011 at 1:15 PM2 comments


PHP Meets SQL Server -- Converting MySQL Queries

As an addendum to last week's post about a recent SQL Server camp held for PHP developers, Microsoft's Brian Swan has published some new nuts-and-bolts details for converting MySQL queries.

Swan, who blogs on the "At the Intersection of PHP and Microsoft" page, said he learned the technique at the JumpIn! camp, where Microsoft hosted some 17 prominent PHP developers in order to share ideas and educate them about the process of supporting SQL Server and SQL Azure in their PHP applications.

He said it was "an excellent two-way learning experience" and one of the things he learned was that the SQL Server Migration Assistant for MySQL 1.0 (SSMA) can convert specific database queries in addition to its main job of converting databases.

Swan explains the relatively straightforward process by using SSMA to convert the MySQL query:

SELECT post_title, post_date FROM wp_posts ORDER BY post_date LIMIT 5 OFFSET 5;

to the SQL Server query:

SELECT TOP (5) ssma$sub1.post_title, ssma$sub1.post_date
FROM
  (
    SELECT wp_posts.post_title, wp_posts.post_date, ROW_NUMBER() OVER (
      ORDER BY wp_posts.post_date) AS ssma$rownum
    FROM dbo.wp_posts
   ) AS ssma$sub1
WHERE ssma$sub1.[ssma$rownum] > 5
ORDER BY ssma$sub1.[ssma$rownum]
GO

(Whew! If I'm a noob evaluating the two on the basis of query simplicity, it's pretty clear which direction I'd lean.)

Swan noted that all queries won't translate successfully (if they contain functions specific to MySQL, for example), but even if it doesn't work, it can provide clues to help write the translated query yourself.

Do you have experience in supporting SQL Server in PHP apps? Is MySQL really that much simpler than SQL Server? Please share your thoughts here or drop me a line.

Posted by David Ramel on 12/08/2010 at 1:15 PM3 comments


PHP Developers Get SQL Server, SQL Azure Insights

Microsoft isn't letting up on its full-court press to woo PHP developers into the SQL Server world.

The company earlier this month hosted 17 PHP developers at a SQL Server JumpIn! Camp, where they learned how to incorporate SQL Server support into their apps with the help of Microsoft experts, according to a post yesterday in the Microsoft SQL Server Driver for PHP Team Blog.

Program manager Ashay Chaudhary reported that the 17 developers represented 10 PHP projects from several countries. They were treated to dinners, entertainment and more at the five-day camp, where the goal was to "listen to and learn from the PHP community and educate them about our technologies, SQL Server, SQL Azure, Windows Azure & IIS, and help them understand how they can leverage them best in the applications."

The 10 projects included content management systems, a profiling utility, an e-commerce solution, a Web application framework and more. Chaudhary said eight of the projects completed enough coding to demo their apps with SQL Server, while four actually managed to publish builds during the camp. Four projects reported improved their performance on IIS using the WinCache extension, four tested successfully on SQL Azure and three used the new PDO driver for SQL Server support.

"By adding support for SQL Server, these projects stand to gain from the broad customer base that already has SQL Server in their organizations," Chaudhary said.

"We now have a much clearer understanding about what we need to do to better support PHP applications on Windows/IIS/SQL Server and the Azure platform," he said. "The PHP participants now have a better understanding of how SQL Server works and how PHP applications can best utilize its capabilities, and understand how we are trying to add value to their applications."

Last April, Microsoft held a similar camp in Zurich, Switzerland. Chaudhary said he will write more about the November camp as he has time to reflect on the event, but he didn't indicate that any future camps were scheduled.

"For now," he said, "you can look forward to Windows, IIS, and SQL Server support in several great PHP applications with many more to come over the next few months."

One of those apps will be ImpressCMS, a content management system. Marc-Andre, CEO and founder of ImpressCMS, blogged that his goal at the camp was to "implement PDO and MSSQL support" into his product and reported it was a success. "We are still missing a few things in the installer to make our CMS install properly on MSSQL, but we will get there!" he said.

What do you think about Microsoft's overtures to the open source community? Comment here or drop me a line and share your thoughts.

Posted by David Ramel on 11/30/2010 at 1:15 PM1 comments


SQL Server Developers Get Taste of 'Juneau' Goodies

Here's good news if you're a database developer who doesn't like working with SQL Server Management Studio (SSMS): You may not have to use it too much longer.

At the recent PASS Summit, Microsoft showed a packed roomful of database developers how the next version of SQL Server--code-named "Denali"--will include a unified development environment based on Visual Studio.

Officially called SQL Server Developer Tools code-named "Juneau," the new bells and whistles were demonstrated by Microsoft's Don Box during a PASS keynote address by Quentin Clark.

"What we're trying to do with Juneau is really advance the state of the art of database development," Box said as he demonstrated the new functionality in a Visual Studio shell. "What we're doing is, we're looking at all the stuff we've done in the past, all the stuff that's been done in Visual Studio, around doing things like .NET, C++, C<--we're trying to bring that goodness to the database development world," Box said.

Working in the Visual Studio shell, Box said, lets developers "take advantage of the new shell, the new WPF-based text editor, new language services. All those accrue value to the SQL Server product. And this also accrues value to business intelligence, so the BIDS assets are going to be in the same shell as our relational database assets."

With a nod to those who happen to like SSMS, Box demonstrated the "connected experience that an SSMS user is going to be used to, inside of Juneau."

Box showed how Juneau lets developers use the Server Explorer to drill down into a database and get the same preview that SSMS provides. "I can say new query and I get the new text editor with new language services on top of T-SQL based on the database I'm deployed against," he said. He showed a simple Execute command and said, "I also can do execute with debugging, so basically anywhere I see SQL text in a text editor, I can select it and either execute it directly against the database, or I can execute under debugging and just start doing step into or step over, anyplace I see text."

A new table designer was also demoed that used the Visual Studio style panes of code, design or a split between the two. When changes were made to the design pane, they were immediately reflected in the text editor pane, and vice versa.

Also, Box noted that the table designer lets developers easily see subordinate objects such as check constraints, primary constraints and indices. As he clicked through these items, the relevant T-SQL code was brought up in the text editor, with the affected columns highlighted.

And in some ways, Juneau will surpass SSMS. When using a CREATE view, Box noted that it's not idempotent, so any changes made would normally involve transcribing them into ALTER commands. "One of the things that we do in the tool which is an advance of what I have in SSMS today, is I have the ability to take all these pending changes which I've been accruing and say 'figure out the ALTER script for me.' So if I say commit all to database, we actually do analysis of all the source text that you've now accrued vs. the actual catalog in the database and we figure out what needs to be done to make this so."

Plenty of other cool things were shown, also, including FileTable file storage, semantic searching and version-awareness and edition-awareness, which means Juneau enforces language constraints depending on your target, be it Denali, SQL Server 2008 or SQL Azure. And speaking of Azure and the cloud, while Box didn't address it directly, one of the aims of Juneau is to "is to make developing for SQL Azure and on-site SQL exactly the same," according to a recent interview with Clark by The Register.

You can register and see the Box demo yourself--along with other presentations--at the PASS Web site.

While the first Community Technology Preview (CTP) of Denali was released at the Summit, developers will have to wait a bit to get their hands on the Juneau technology--it's scheduled to be included in the next CTP, with no expected timeline provided.

Posted by David Ramel on 11/15/2010 at 1:15 PM1 comments


New SQL Server: Scanning 1 Trillion Rows Per Minute

There were plenty of "oohs" and "aahs" from the audience at the PASS Summit during Ted Kummert's keynote address Tuesday as he demonstrated new capabilities of the next version of SQL Server, called "Denali."

One notable demonstration was with Microsoft's Amir Netz, who ran a side-by-side comparison of a database query using some 2 billion records, with one side of the screen showing the query being run with regular technology and the other side showing the same query running many times faster with the VertiPaq technology borrowed from PowerPivot. Netz said the blinding speed of VertiPaq would equate to a theoretical processing rate of 1 trillion rows per minute.

The VertiPaq technology--using columnar storage as opposed to rows--is now available to parts of the new SQL Server, such as the new BI Development Studio running with Visual Studio 2010.

Netz also gathered applause by showing off "Crescent," the code name for "a new Web-based data visualization and reporting tool that extends PowerPivot," as described by Microsoft. Netz showed in a flashy demo how huge amounts of data can be manipulated to "tell a story" to others.

While the Community Technology Preview of Denali was released at PASS, Microsoft hasn't said when a final product is expected to ship.

Other enhancements introduced by Microsoft include:

  • A SQL Server 2008 R2 Parallel Data Warehouse appliance released with hardware partner HP (with other choices available in the future). "Through deep interoperability with the Microsoft Business Intelligence (BI) platform, customers can deliver managed self-service BI solutions to everyone in the organization," Microsoft said in a statement.
  • An updated Microsoft Certified Master (MCM) program for SQL Server that "allows SQL Server experts to achieve MCM certification with more flexibility and at a lower cost compared with the previous program."
  • Code-name "Atlanta," a configuration service hosted in the cloud that will "help IT proactively avoid configuration problems and rapidly resolve identified issues through step-by-step guidance."

The PASS Summit continues through Thursday.

Posted by David Ramel on 11/09/2010 at 1:15 PM1 comments


New DataMarket, Improved SQL Azure Services Announced

Microsoft last week announced that its portal to buy and sell data services has launched as DataMarket, part of the Windows Azure Marketplace.

Formerly called Project Codename "Dallas," the market serves as a repository "that makes it easy to discover, purchase and consume data through a variety of different sources, both public domain and commercial," said program manager Steve Yi in a PDC10 session.

DataMarket has a new Web site and new information sources available for subscription, with many more on the way. The site lists 13 data categories such as Financial, News, Statistics and Weather, along with 28 publishers and more than 70 individual data feeds.

The feeds are typically presented in the XML-based OData format and can be easily integrated into applications built with the .NET Framework, for example, or directly into PowerPivot and Tableau data visualization projects.

Microsoft is rolling out new paid services through Nov. 22, with a variety of pricing and consumption options. Users can also request early access to the paid subscriptions.

You can learn more about DataMarket here, and Microsoft has also put up several YouTube videos, dealing with topics such as discovering and subscribing to data and using DataMarket with PowerPivot.

In other data-centric news, the company announced several enhancements to the cloud-based SQL Azure, including a Community Technology Preview (CTP) of SQL Azure Reporting, bringing the same capabilities of SQL Server Reporting Services to the cloud.

A SQL Azure Data Sync CTP 2 was also announced, along with a lightweight, Silverlight-based database manager, formerly called "Project Codename Houston."

You can learn more by watching the following PDC10 video presentations:

Introduction to Windows Azure Marketplace DataMarket

What’s New in SQL Azure

Code First Development with Entity Framework

Building Engaging Apps with Windows Azure Marketplace DataMarket

Introduction to SQL Azure Data Sync

Introduction to SQL Azure Reporting

Introduction to Database Manager

What do you think about the new DataMarket and SQL Azure enhancements? Comment here or drop me a line.

Posted by David Ramel on 11/03/2010 at 1:15 PM0 comments


SQL Connectivity: Give Microsoft Your Two Cents' Worth

Even though SQL Server just underwent a major upgrade with 2008 R2, Microsoft is seeking customer input about new features you'd like to see added to the database technology.

The company this week posted the SQL Connectivity Customer Survey (Fall 2010), which will be up until Oct. 25.

The survey asks respondents to rate the importance of possible enhancements such as improved network performance (via tabular data stream compression), easier setup and configuration (through a richer GUI), "reducing the number of new connections that need to be opened" (through connection pooling improvements) and several others.

It was exactly one year ago that the SQL Connectivity team started "interacting on a regular basis with the developers and users in the form of surveys," said program manager Raghu Ram in the survey introduction.

"During the last 12 months, we completed surveys that focused on the broad SQL Connectivity components, including ODBC, ADO.NET, JDBC and PHP," Ram said. "These surveys provide us with an ability to validate some of the requests we have got from developers, users and partners." He said the roadmap for SQL Server has evolved based on this survey feedback.

Indeed, last summer saw the release of SQL Server Driver for PHP 2.0, which for the first time supports PHP Data Objects code.

So here's your chance to possibly influence the next upgrade. In addition to the possible enhancements listed above, the survey asks respondents to rate the importance of better support for multi-core CPUs and multiple network interface cards, better diagnostics and troubleshooting, transparent failover and new authentication types.

What's at the top of your list for SQL Server improvements? Comment here or drop me a line.

Posted by David Ramel on 10/20/2010 at 1:15 PM3 comments


Yup, No SQL Server CE in Windows Phone 7

Yesterday's Windows Phone 7 launch extravaganza renewed the months-long debate among developers about database options--specifically, whether the new mobile OS should come with persistent local storage such as built-in SQL Server Compact Edition.

Microsoft's answer, of course, is go to the cloud. And if you don't want to do that, you can opt for a local storage alternative such as XML files, isolated storage or third-party embedded solutions such as Perst.

Besides the "The cloud is the answer. What's the question?" mentality in Redmond, many (even Microsoft people) have pointed out that Win Phone 7 targets consumers more than enterprises, so there is less need of any SQL Server.

Still, considering all the integration with other Microsoft technologies such as Xbox, Office and so on, it seems strange there's no stated intention to provide SQL Server in the future, like they're doing with copy-and-paste functionality. Surely Microsoft isn't going to ignore the enterprise market, and developers in the enterprise market have clearly stated their preference. Check out these comments from the debate mentioned above on an MSDN forum:

  • No database in WM7 Phones? This is ridiculous! 
  • With this I may stick into iPhone development.
  • Without real database support, a phone is just a portal device.  It will have very little value in itself, and will be easily replaced with the next shiny new portal device with a flashy UI.
  • This is indeed a step backward from WM 6.5. Why not implement Sql CE Compact? You cannot write real business application without database support and cannot presume that we live in an always connected world. Developers need database support ... Are you really considering not to implement it?
  • This doesn't make any sense, surely it can't be a huge undertaking to incorporate SQL Server Compact with its managed wrapper into Windows Phone 7?
  • We just started our project (2 months ago) to move to WM ... THE one deciding factor was SqlCe ... with that gone we will definitely regroup and probably move to Droid or iPhone before we throw any more resources after WM with no stated intentions!
  • I concur with everyone else...developers NEED database support on Windows Phone 7!

It's pretty obvious what mobile developers want. Is Microsoft listening?

What do you think? We'd love to hear your arguments, pro and con. Has anybody seen definitive indications that SQL Server CE is coming to Windows Phone 7? Comment here or drop me a line.

Posted by David Ramel on 10/12/2010 at 1:15 PM27 comments


.NET Insight

Sign up for our newsletter.

I agree to this site's Privacy Policy.