Why Not Review Crummy Products?
I sometimes worry that, looking back at the reviews we do here, that I seem unremittingly positive about most of the products that we review. There's a couple of reasons for that.
First, generally speaking, we pick out products that we think have real benefit for developers and that developers are interested in. There's not much point in reviewing products that we know aren't great and that no one's buying. We do dig around looking for products to review that developers may overlook but, even then, we don't pick a product unless it we think it has real merit. Life is too short to beat up on products that no one cares about.
Second, when we review a product, we review it "on its merits." In other words, I look at the surrounding marketing material and talk to the company producing the product about what the product's goals are. My goal is to measure how well the product lives up to its goals and ferret out what developers might reasonably expect from a product that has those goals.
It seems foolish to me, for instance, in a review of a set of server-side only controls to complain about the lack of AJAX support. If I thought that developers expect that all controls should have AJAX support I wouldn't review the product -- it would be a waste of everyone's time. If I thought developers would value "server-side only" controls but might assume that the controls included AJAX support, then I would make clear that these controls don't have the AJAX support. But I wouldn't criticize a product for not having AJAX support: It wasn't part of the product's goals. I would criticize the product if the controls didn't work well as "server-side only" controls, though.
And I can also critique the product's goals: Lots of products seem to be built to do things that just don't seem to need to be done.
Even if a product does live up to the company's intent, I think it's fair for me to critique products on the basis of "value for money". If a product is delivering $50 worth of value for $200 worth of cost... well, that's just wrong.
That's not to say that if the product achieves goals that the company doesn't list that we can't assess how well the product achieves those "other" goals. It's the 'Shania Twain' effect. In an interview in a magazine, Ms. Twain was asked why her skin looked so great (and, you have to admit, the woman has great skin). She said it was the result of using Bag Balm, a product originally developed to deal with chapped cow udders (my wife speaks highly of the product also). I won't speak to Bag Balm's other uses (apparently some people eat it, believing it extends their lives and improves their health) but if I was reviewing Bag Balm -- and after seeing how it actually worked on cow's udders -- I might consider its effect on human skin.
What I can do, as a practicing .NET developer, is talk about how well the product actually works for a real developer building real applications for real clients in the world that we all live in.
Posted by Peter Vogel on 06/29/2010