This week Amazon (AMZN) finally banned its controversial practice of allowing free stuff in exchange for reviews—a process that, not surprisingly, skewed ratings and ruined their effectiveness. By some reports, as much as 20% of all reviews contain language that indicates product changed hands in exchange for a review, usually with a hard-to-believe line saying that the reviewer exercised the most impartial of judgment.
As Consumerist noted, this doesn’t necessarily mean that a product’s star rating will simply skew 20% higher, due to the positivity that free stuff brings. Instead, the breakdown is far more uneven, with some products having almost no incentivized reviews and some having almost all paid-for reviews. In some of those cases, like this umbrella, the paid-for reviews mismatched the buyer feedback by an average of two whole stars.
So now that Amazon’s new community guidelines prohibit vendors from giving away free products in exchange for reviews, what’s happening with the massive amount of untrustworthy reviews already published on the site?
According to an Amazon, those reviews aren’t going anywhere. “Reviews that were received prior to this policy change are being retroactively removed only if they are excessive or do not comply with the then policy,” said a spokesperson.
The word “excessive” gives Amazon some latitude, making it unlikely that many reviews will be pulled; many for-profit reviewers are careful – they give five stars to skew the ratings and ranking in Amazon search results yet write neutrally and critically in the text itself. In the game of rankings and numbers, those stars mean a whole lot more than the review copy itself.
For consumers, this means Amazon is still more or less littered with paid reviews, and should exercise the usual caveat emptor in the face of high star ratings. However, two simple tools make things easier. ReviewMeta.com allows consumers to effectively sort out the paid reviews, looking for language that suggests a product-review exchange. The other is a lot simpler: Looking at the age of the product listing, since old policy grandfathers in the paid reviews. The newer the product, the more trustworthy the reviews.
Ethan Wolff-Mann is a writer at Yahoo Finance focusing on consumerism, tech, and personal finance. Follow him on Twitter @ewolffmann.