Yelp Sucks

Following up on a conversation here last week. Someone made the point that the cost of a wrongly removed review depends on how many reviews the business has in total, which changed how I weight errors in a small model I'm building for a class.

What I still can't resolve: a genuinely furious customer and a paid competitor review can read almost identically. Short, one star, no specifics, emotional.

For anyone who reads a lot of reviews — is there anything in the text alone that reliably separates the two? Or does it only become obvious once you can see the account, the timing, or a cluster of similar reviews?

My model deliberately sees only the text and the rating, and I'm starting to suspect that constraint makes the task impossible rather than just hard. I'd rather find that out now.

I am building an automated review moderation agent that classifies new reviews using only the text and star rating. Since the true state (genuine vs. fake/sabotage) is hidden, the agent has to make decisions under uncertainty.

From a store owner's perspective:

1. Which mistake hurts your business more: a False Positive (the agent incorrectly hides a real customer's 5-star review) or a False Negative (the agent fails to hide a fake 1-star competitor review)?
2. What threshold of uncertainty should trigger sending a review to a human manager ("warn/review queue") rather than taking auto-action?

I visited an amazing hidden-gem restaurant with my partner yesterday, and the food was absolutely delicious. I was surprised to see that the restaurant didn’t have many reviews.

I checked the “Not Recommended” section and saw only 4 reviews in there. The restaurant doesn’t have a huge number of reviews overall, but the majority are overwhelmingly positive, with many people highlighting how authentic and delicious the dishes are. There was only 3 reviews from Elite for this restaurant.

Have you ever come across a restaurant with very few reviews that turned out to be incredible and deserves to be much more well-known? I’d love to hear your recommendations!