Reading between the lines of Yelp Reviews - I'm a data geek.

Reviews are not always what they appear, especially on Yelp.  This post won't get into the nitty gritty about filtering, shill reviews, and other shenanigans, but this post will give a different view of how you can read between the reviews on Yelp.

*Note: This data generally applies to Austin-area restaurants (trailer or brick and mortar).  This does not apply to other types of businesses or mail order food products.  I have not looked at the data concerning those types of businesses. I have not looked at data in other cities. If you have a restaurant, don't fret if your review behavior doesn't line up with what generally happens.  Your restaurant might just be an outlier.  This will not apply.  

*In less webbie (or techie) locations or places with a smaller population, I find Yelp to be only marginally useful. 

When I use Yelp for search, the first two questions I have are:

• 1. What is the average star rating?
• 2. How many ratings? 

In Austin, you would expect a successful places to have a relatively large number (more than 75) of reviews after a year.  If the location has very few reviews after being open for a year or so, my next questions are:

• 1. Who wrote the reviews?
• 2. Is there anything that makes this place an outlier (difficult to reach location, odd hours, unavailable to the public, special events only, or otherwise)?
• 3.When I Google this location, is there anything odd on the web?  Is the owner an axe-murderer?

After making sure that there was nothing that made this place a horrendous outlier, I would think read the actual reviews for the qualitative data.  This is typically a laborious process just to figure out where I should eat. 


After going through this routine of trying to interpret the data, I found that the review behavior is much more telling than the reviews themselves.  Any place can have a high average star rating as it is fairly easy manipulate.  A restaurant can have only four ratings of five stars each, geee...... that restaurant looks perfect doesn't it?  I would beg to differ, the low number of ratings could be because people really dislike the restaurant and avoid it, or it could be that some Yelp users just don't write about negative experiences.  In any case, a low number of reviews after being open for business for a significant amount of time is a red warning flag.

Being the big data geek that I am, I've looked at review behavior for many restaurants over a long period of time.  One thing that stuck out was that successful restaurant consistently garnered reviews, positive and negative.  Not so successful restaurants would stagnate after an initial burst of reviews, garnering maybe one review every month or so.  In my interpretation of that behavior, a restaurant with a four-star average with a consistently growing number of reviews is a much better choice than a restaurant with perhaps a 4.5 star average, only a handful of reviews, and the latest review was two months ago.  Personally, I would rather eat at a restaurant with a lower average rating that had consistent growth in their reviews rather than a restaurant with a high average rating but no new reviews.  That's just my interpretation of the data, and my opinion. 

I will also add that public relations effort does have an impact on the review behavior, but only significantly in the early days of a restaurant's opening.  There will inevitably be a spurt of reviews (positive or negative) when a restaurant first opens, but if the restaurant fails to perform, the reviews stop coming in.  Additionally, if the restaurant participates in branding and publicity efforts, that might cause other spurts of reviews.  However, it doesn't change the review behavior significantly in the long term.

The chart above is something I created to model the review behavior, and it is based off of many Austin restaurants.  I will not list the ones that are unsuccessful, but I'll point out one that is successful.  The Noble Pig is a great example of a successful restaurant that did not engage in a large public relations effort.  They have been open for only about six months, and they are located a 40 minute drive away from downtown; yet they already have 69 reviews to-date with a 4.5 star rating.  Even though the location makes them an outlier, they are consistently garnering reviews.  That's a place I would definitely visit again, and again, and again.  *Disclosure: I met the chef at several events, and I really like his sandwiches. 

This post is just a suggestion that reviews and star ratings should just be taken with a grain of salt.  Looking at review behavior can give you more in-depth look at a restaurant's success. I'm using the word "success" loosely.  You can define it however you like.

How to spot fake reviews

You're a business owner, and you're wondering how other businesses get so many reviews.  How did Uchi get 527 reviews with an average of 4.5 stars on Yelp (as of 1/1/11)?  How did Sugar Mama's get 314 reviews with an average of 4.5 stars on Yelp (as of 1/1/11)?  How come no one is talking about your business?  *I know that Uchi and Sugar Mama's do NOT engage in the behavior described below.  Thusly, I feel comfortable using their data.  The temptation sets in.  You want more people to know about you.  You think no one will find out.  You're antsy for people to click on your website link.  You make a fake account on Yelp.  You give yourself five stars.  You wait a few minutes.  That wasn't too bad.  So you make another fake account.  You give yourself another five stars.  That was easy.  On the Internet, no one knows you're a dog right?  So you do it again. And again. And again.  Now you have seven positive reviews.  Job well done!  You ask your employees to log onto Yelp and to write reviews.  You ask all your friends, suppliers, regulars, and family members to do it.  It isn't technically wrong, if you don't get caught.  What an easy way to increase your ratings and pump up those reviews.  WRONG. It isn't long before you get sniffed out.  You're slammed on the Master Debater's Threads, and Google Alerts show that you have been caught red-handed.  Moral of the story. Don't write fake reviews.  Social media and crowd sourced content sites already spark controversy and debate.  Throw in examples of fake reviews written by business owners or their rivals, and everyone goes bananas.  Transparency in social media has been the norm with demand for honesty ever growing.  Users want to be able to put a face with a social media voice, and rest assured that conflicts of interests are revealed.  No one wants to be tricked.    As a consumer, you do want to know which reviews are honest.  Here's some quick tips on sniffing them out. Photo and name (Lack of online presence).  No photos and fake names are often hallmarks (though not always) of fake reviews.  Some fake reviewers have become savvy and will use a stock photo on the profile, and a generic screen name.  Stock photos (often flowers, trees, or puppies) are pretty easy to spot. Names are more difficult to gauge, but there are several ways to investigate. You can Google the screen name along with the city (example: Jennie C. Austin or MisoHungry Austin).  If nothing turns up, then it can get suspicious.  On occasion, the business owners and employees will use their real names and photos when reviewing their own businesses.  That's a dead give away as it is easy to check websites for business owner names or to find via Linkedin.com.  Sometimes, they will also write the review in first person.  "I worked really hard to open this restaurant" doesn't seem unbiased to me.  Location listed on the profile.  If the location listed is not in same town that the businesses, look for other reviews in the same town.  Frequently, travelers review more than one business when visiting other cities.  If a person from Montana reviews just a single business in Austin, that would raise eyebrows.  It is unlikely (but not impossible) that an out-of-towner would review a business that isn't already well-known.  Depth and breadth of reviews.  The number of reviews, types of businesses reviewed, and the details of the reviews are also very telling.  Lack of depth and breadth generally means that the person doesn't invest much time in the reviews or only selects particular businesses to review for whatever reason.  Those types of reviews are typically vague, lack descriptions, and are generic.  Here's some examples of vague reviews (names removed):  ********* has small plates packed with big flavor. Perfect place to meet for a drink and gently roll into dinner. Appreciated the guidance of the staff with perfect pairings of both beer and wine for our dishes. We ended up ordering the entire menu and nothing disappointed! Welcome to the neighborhood,*******! Wow, the ****** are amazing!!!!! I love all the*********, my favorite was the *******. I also had the ******* which was pure, literally....and wonderful on the recent cloudy cool days.  On a warmer day, the ****** is heaven on earth! What a novelty, can't believe I've never seen ******* anywhere else. This place is off the hizzy. HmmmMmmm I know what good is and this place is good, ya hear?  Other contributions and timing of reviews.  If the reviews for the business were written before the business opened, you better believe that raises the red flag.  Also, contributing photos of "behind the scenes" or construction shots of a business is an indicator of conflict of interest with the business.  If you see the same shots uploaded to a review site that are on a photographer's portfolio, that is a dead give away.  *Anyone that has a relationship with the business or conflict of interest should either disclose it in a review or not review at all.  Narrow range of businesses reviewed and aspects revealed.  This is a completely hypothetical example, though it has happened before with other suppliers.  A supplier naturally wants their customers to flourish, so they might be tempted to write reviews for their customers.  You might find a Yelp user that reviews only sushi restaurants, and the reviews praises only the freshness of the fish, how it is handled, and how it is prepared.  The reviewer's profile might say something about fish or link to a fish wholesaler website.  It is highly likely that the profile might belong to the restaurants' fish supplier.  That should be disclosed OR reviews of the restaurants in question should be avoided.  I can understand that a supplier really wants to let everyone know about their great ingredients and food, but the behavior should still be avoided. There are other ways to let people know about the ingredients.  Sometimes reviewers will outright say that they "are employees, and that's why they know the quality of the food is so great."  Oops. Sometimes the only other reviews are for other businesses in the same family (same owners).  Positive reviews of every single store in a chain is a little less than subtle.    The same review (or clusters of reviews) appear on multiple websites, sometimes posted within minutes of each other.  That's waving the giant red flag of "someone on our marketing team was given the task of copying and pasting fake reviews to different websites this morning."  A cluster of positive reviews pop up right after a single negative review.  Sometimes businesses have fake profiles in waiting.  If they receive a negative review, they immediately unleash several positive reviews.  It is pretty obvious when they are lined up chronologically. Those are just some ways that fake reviewers are sniffed out.  The best defense against them is the time it takes to create believable fake profiles/online presence.  *I've been going back and forth on whether or not I should write this.  I thought that in writing this, shady businesses owners might find tricks to appear less fake.  Honestly though, to create an online presence to does not escape the vigilant detection of savvy internet users and the likes of the Google machine would take hours upon hours.  I would guess that it would take no less than 50-100 hours per fraudulent online presence.  The likelihood of a business investing that many hours into a single fake review is extremely low.  Those types of businesses wouldn't even invest the time in reading this post. I don't think this blog post will affect these behaviors at all.