Craft Beer, Fine Wine, Artisan Spirits, and Mouthgasmic Food.

Monday, April 12, 2010

Yelp's New Transparency Bad New for Businesses who write shill reviews

Yelp is trying to turn things around.  Public distrust.  Angry business owners.  Three class-action law suits.  Yelp has been on many people's naughty list lately, and they have implemented something new to turn that all around.  They have allowed previously "filtered" or "suppressed" reviews to be viewed by anyone and everyone.  Here's several links to the details on that, and I'll avoid rehashing all that information.  

Yelp's Official Blog Post about Filtering
Mashable's Blog post about Filtering

However, I spent a large chunk of time studying Austin business reviews, and here's what it means to users and to businesses (from my limited information that is).  I noticed that some businesses has only 10% of reviews filtered while others had more filtered reviews than reviews visible (i.e. 28 reviews visible, 47 filtered).  The percentiles were highly variable.  I haven't seen any patterns to them yet.  A few other things I gathered included:

1. Yelp has increased it's transparency, and that is bad news for sneaky businesses out there.  With the filtered reviews exposed, I easily found many businesses who wrote themselves shill reviews either using their own names or only writing positive reviews for other businesses they owned.  It appears that even if reviews are flagged by the community as being fake (i.e. the master debater's thread) they are either filtered or deleted due to violation of terms of service (TOS).  While I can't be absolutely certain due to the limits of information I have, I think they are deleted and placed below the filtered reviews.  You can't read the reviews anymore, but you can see who wrote them.

2. One thing I do notice is that the filtering system is fairly ineffective.  Sometimes it filters out people with 20 reviews, yet it leaves people with only one review unfiltered. Sometimes it filters out people who have spent a significant amount of time writing a thoughtful review, yet it leaves people with one-line reviews visible on the business page. That's the problem with having an algorithm do your work. 

3. The users who have filtered reviews (if everything is truly computer run numbers) should have all their reviews filtered.  I did find this to be true for the few people I researched on Yelp, but I can't be absolutely sure that this is true for all users who have filtered reviews.  Also, on the user(s)' profile page, it does not show whether or not their other reviews have been filtered.  It is also unlikely that any of these users know that their reviews have been filtered.  Theoretically, there should be a waiting period for all new users who have to work their way out of the filter by crossing whatever threshold the algorithm uses.  Yelp does not inform new users about that they are being filtered, which can be good or bad.  New users probably have a difficult time understanding the concept of filtering, and it might scare off new users from joining if they have to read a bunch of rules and regulations.  However, I think that if new Yelp users were informed of this filtering mechanism, Yelp might see more engaged new users.  That's all speculation though.  I can't think of any psychology studies examining commitment to a social community if there was a heavy time investment upfront without immediate reward. 

4. It appears that some administratively removed reviews are never shown in the deleted reviews.  I noticed very recently that Yelp had administratively deleted all reviews for a business if they were written before the business officially opened (i.e. Urban An American Grill and Lick it Bite it or Both).  After the business opened, those reviews did not reappear nor did they appear in the filtered or deleted review section.  It is my understanding that Yelp administrators would send an email to each of the users informing them that the review was removed.  However, there have also been some claims about missing reviews without an email from administration.  It does appear that all other administratively removed reviews that were flagged by the community do appear in the deleted review section.  

My advice for businesses is to scroll through your filtered reviewsIf they were written by yourself, your employees, or your friends, delete them as soon as possible.  These reviews might have been previously filtered and not visible to the Yelp community.  Now that they are visible to the Yelp community, users might feel as if you were trying to fool them with fake reviews and retaliate.  You might have escaped the master debater's thread previously, but now you can't hide those filtered reviews.  Even if users flag them now as shill reviews, they might still end up in the deleted section.  Deleting them now from the accounts you've (or employees or friends) created is the only way that they will disappear.  Additionally, reviews that are deleted are not searchable and do not come up in Google searches.  Threads, however, have no delete or edit button.  If you are called out on a thread, it is there to stay forever.  *Google can find quotes from filtered reviews though. 

Yelp has taken steps to be more transparent, but that also means that the businesses must also be more transparent.  You can run and hide, but not for long. 


  1. This is really frustrating for me since I've had my business on Yelp for a few months only. My customers want to write genuine reviews of my business, but they're filtered out because they themselves are also new to Yelp. I have 6 5-star reviews from real customers, but only 1 shows up. What's the point?!

  2. Hi Paloma,

    I'm sorry you're frustrated by Yelp. I'm not an employee of them nor do I think they are a perfect website, but I do understand why they would want to employ a filter on reviews.

    I looked through your filtered reviews, and unfortunately all of them look like they fit the factors on why reviews are filtered. None of your reviewers have photos, none of your reviewers have completed their profiles, and none of them appear to be interested in contributing to the Yelp community (very few have written other reviews). The filtering mechanism was put up so that reviews that are written by someone who wants to only promote one (or a few businesses, typically because they have interest in that business) are not shown.

    When the community sees reviews like this, they either 1. Assume that the business is asking people to give them Yelp reviews or 2. the business is creating fake accounts and reviewing themselves.

    It may not be fair, and it may not seem right. However, those are Yelp rules, users and businesses must play by them (as long as they aren't illegal). My goal in this blog post was to understand and decipher the outcomes of Yelp's changes.