Reviewing reviews


Probably one of my favorite kinds of things to read online is reviews. I don’t quite know why, but I find something about them — app reviews, gear reviews, gadget reviews, it doesn’t really matter — really enjoyable. I don’t think I’m alone: there are a lot of sites out there that feature reviews.

Heck, some sites exist purely to publish reviews. One of my favorites is a site called Tools and Toys, which “is a daily collection of items for the pickiest of gadget geeks, software aficionados, snowboard junkies, music lovers, writers, coffee nuts, and all around collectors of fine paraphernalia.”

I’ve been getting a bit more intentional about doing reviews on this site (I have a big backlist of cool things I really want to write about as soon as I get time) and so I’ve been trying to develop a method for writing them, and examining what exactly makes a good one. Why do I enjoy Tools and Toys so much? Why is it that I don’t particularly enjoy Consumer Reports, but I look forward with anticipation to John Siracusa’s mammoth screed on the latest version of OS X?

In really digging into what makes a good review, I’ve noticed that there’s a definite movement in some quarters towards higher quality reviews, and a backlash against the kind of pageview-centric tech journalism that currently reigns.

Less haste, more honesty

I have to admit, I’ve occasionally fallen into the trap of trying to get a review out as soon as possible after the product launch. It’s easy to fall into — after all, if you’re not first then your review will quickly become redundant and you won’t be able to add anything to the conversation. By the time you finally post it it’ll be old news.

Well, maybe. But if you’re rushing a review then — in my experience — you run the risk of falling into one of two traps:

  • Your review is essentially just a glorified feature-list,1 or

  • You manufacture some insights — something that sounds legitimate — and pass your experience of the product off as much more in-depth than it is.

The latter is extremely dangerous. As Dr. Drang noted:

Two weeks? You can’t decide something is the best in two weeks, no matter how much experience you have with similar products.

But so many reviews are written on a much flimsier basis, after much less time than that. Of course, depending on the review you don’t need to use the product for years before you pass judgment. However, it’s easy to go a little overboard in lip-service of something you haven’t really tried, and don’t know quite as well as you pretend.

Anyway, that’s why I’m giving up on writing BREAKING NEWS reviews.2

Revisitation

Not only is it important to spend some time with something in order to review it honestly, I think it’s also very important to revisit it at some point in time to decide whether or not you’re still recommending it. After all, someone looking for information on the Flurglewharfer 9000 might find your review as the first search item, read that you absolutely adore it, and rush out to buy one. Meanwhile you’ve moved on to the Flinkleranker XL, after your Flurglewharfer succumbed to a little-known manufacturing defect and imploded in the living room taking your pet corgi with it.

Some reviewers check back in on Twitter about products they’ve reviewed, after some time has passed. I think that’s a great idea, and I’m going to start doing it. Despite the fact that they’re generally much shorter, a “2 years later” review is much more valuable.

Conclusion

To me it seems like the ideal review has to have these qualities:

  • Honest (did it really change your life?)

  • Comparative (why does it stand out against the competition?)

  • Unique (what can you say about it that others won’t?)

The reviews that hit all of those are the ones that I really enjoy, and the kind that I’d like to write.

I’d love to hear what you think makes up a good review. Drop me an email, or find me as @sidoneill on Twitter, or @it on ADN.


  1. Which is fine, if that’s what you’re aiming for. 

  2. And also why my review of the Tom Bihn Smart Alec is still very much a work in progress…