Hello there,

I thought it was interesting how the ratings for products are calculated. I seem to think that the ratings are biased, and not fair. That's because in almost all cases the reviewer is forced to give a 1 (one) star rating instead of 0 (zero). Many people complain about this saying they would have given zero stars if they could rather than 1 star. People do not feel comfortable giving a 1 star rating to something that is completely bad or terrible. In fact, some mention giving a negative star rating like -1, -2, etc.

As an example, I see these statistics on an actual web site just today 06/15/2024:

"2.9 out of 5 stars"
5 stars 163
4 stars 44
3 stars 26
2 stars 23
1 star 190

If we take away the 1 star ratings I think we get "2.5 out of 5 stars".
Maybe not that much difference?
If we change the 1 star ratings to -1 star, I think we get "1.8 out of 5 stars".
That's more of a difference.

I also have to wonder how many good reviews are there only because of people coerced into providing good reviews to help compensate for a lot of bad reviews.

What do you think?

 

The scale has 5 buckets. It's irrelevant what the numerical value of the lowest bucket is: If a reviewer puts it in the lowest possible bucket, that's all you need to know. You could get more precision with a 10-point scale but I don't think the end result would be any more useful. In your example, one can only wonder why most people either love it or hate it. That wouldn't go away with a more precise scale.

 

I took my elderly mother to her doctor's appointment, and whilst waiting I read the notices in the waiting room (no old magazines to read any more)
According to one notice, the practice had been awarded three stars for their ability to deal with patients with dementia.
Three stars out of three? Three stars out of five? Three stars out of ten? It didn't say. . .

 

One critical thing to keep in mind is that the reviews are NEVER representative. Only a tiny fraction of people that could submit a review did, so the representative view is held by people that didn't submit reviews. Those that did are highly likely to have had specific reasons for doing so -- they either really, really loved it or, more likely, they really, really hated it. A large fraction of "5-star" reviews come from people that are being compensated in some way for doing it -- either they are being paid outright, or they are getting a discount for submitting a stellar review. The pool is so polluted that any aggregate score is meaningless. You can only hope to glean useful information by reading many of the reviews themselves and quickly dismissing both good and bad reviews that give few, if any, specific details.

 

You also have to eliminate all those bad reviews that are a result of the carrier losing or damaging the parcel.

 

The scale has 5 buckets. It's irrelevant what the numerical value of the lowest bucket is: If a reviewer puts it in the lowest possible bucket, that's all you need to know. You could get more precision with a 10-point scale but I don't think the end result would be any more useful. In your example, one can only wonder why most people either love it or hate it. That wouldn't go away with a more precise scale.

Hi,

Well I guess my point was about how the overall rating, such as 2.9 out of 5 stars, affects the casual reader. If they see 1 out of 5 stars that's obviously a lot different than 3 out of 5 stars, and even more so than 5 out of 5 stars. Thus, how the 'average' comes about and how reducing that '1' to either zero or -1 (or even lower) would affect that average which was 2.9 for this real life example.

 

I took my elderly mother to her doctor's appointment, and whilst waiting I read the notices in the waiting room (no old magazines to read any more)
According to one notice, the practice had been awarded three stars for their ability to deal with patients with dementia.
Three stars out of three? Three stars out of five? Three stars out of ten? It didn't say. . .

Hi,

I am pretty sure that would mean 3 out of 100 stars

Yeah, that's even stranger. Maybe they assume people are used to a top score of 5 so it would be 3 out of 5.

 

One critical thing to keep in mind is that the reviews are NEVER representative. Only a tiny fraction of people that could submit a review did, so the representative view is held by people that didn't submit reviews. Those that did are highly likely to have had specific reasons for doing so -- they either really, really loved it or, more likely, they really, really hated it. A large fraction of "5-star" reviews come from people that are being compensated in some way for doing it -- either they are being paid outright, or they are getting a discount for submitting a stellar review. The pool is so polluted that any aggregate score is meaningless. You can only hope to glean useful information by reading many of the reviews themselves and quickly dismissing both good and bad reviews that give few, if any, specific details.

Yeah that's a great way to look at it.

I see a lot of lengthy reviews though that do sound like they were written by ai or something so I start to suspect them too. Maybe this brings up the question of are any of the reviews any good at all.
One thing I noticed is that a slew of bad ratings does seem to indicate a bad product, even if there are some very good ratings. I found this out recently that's part of what made me start this thread.

The product was La Choy Chicken Chow Mein, the double can. Many people complained that it tasted nothing like it used to and it was actually bad. Well, I found out the hard way, it certainly is not like it used to be and I'll never buy it again that's for sure. Terrible, nastily taste.
I also noticed some 'good' reviews that actually mocked out the people who gave bad reviews, saying that they were just complainers that complain about everything. Now I know that's not true.
It actually looks like beef chow mein now too, which is a turn off right from the start before even eating any. Brown sauce, dark brown chicken.

 

You also have to eliminate all those bad reviews that are a result of the carrier losing or damaging the parcel.

Oh right. Some of the reviews are not actually for the item itself but for something related to the item like that, which have nothing to do with the value of the product itself.

 

Oh right. Some of the reviews are not actually for the item itself but for something related to the item like that, which have nothing to do with the value of the product itself.

The supplier chooses the carrier. If the supplier chooses, for the sake of price, a carrier that repeatedly loses and destroys parcels then the supplier deserves a bad rating. The product probably doesn't.
Of course there are customers who give the product a bad rating, because they don't understand what it is supposed to do, they wire it up wrongly and blow it up or destroy it is some other way.
Over the years I have developed quite some expertise in customer stupidity.

 

The supplier chooses the carrier. If the supplier chooses, for the sake of price, a carrier that repeatedly loses and destroys parcels then the supplier deserves a bad rating. The product probably doesn't.
Of course there are customers who give the product a bad rating, because they don't understand what it is supposed to do, they wire it up wrongly and blow it up or destroy it is some other way.
Over the years I have developed quite some expertise in customer stupidity.

Yeah I know what you mean. There are a lot of angles to this which renders the ratings systems a little, shall I say, "moot"

 

I think that a percentage "number" rating is basically useless.
A extensive written-review might have some value though,
because it "can be" pretty obvious "sometimes", what the user's IQ is.

If the consumer can't write a concise review,
articulating exactly "WHY" they found the product lacking in some way,
what would make anyone believe that they could competently operate a possibly complex device,
and, of course, without reading the instructions first.

Operator incompetency almost guarantees a poor rating,
and easily ~10% of the population is dangerously stupid,
so stupid in fact,
that the US-Armed-Forces will not accept them in any capacity.
I think that this is one of the most significant problems that the World faces today.
And don't forget,
these same people are in charge of piloting ~2.5-ton projectiles at high speeds,
so keep an Eye out for them, they will get You killed.
.
.
.

 

I think that a percentage "number" rating is basically useless.
A extensive written-review might have some value though,
because it "can be" pretty obvious "sometimes", what the user's IQ is.

If the consumer can't write a concise review,
articulating exactly "WHY" they found the product lacking in some way,
what would make anyone believe that they could competently operate a possibly complex device,
and, of course, without reading the instructions first.

Operator incompetency almost guarantees a poor rating,
and easily ~10% of the population is dangerously stupid,
so stupid in fact,
that the US-Armed-Forces will not accept them in any capacity.
I think that this is one of the most significant problems that the World faces today.
And don't forget,
these same people are in charge of piloting ~2.5-ton projectiles at high speeds,
so keep an Eye out for them, they will get You killed.
.
.
.

Hi,

Yes it is becoming more obvious now that ratings are to be taken as garbage unless there is good reason to believe otherwise.

What some people do is rave about how good something worked without any real reason for it. That's just as bad as ranting about how bad something is without any real reason for it.

 

The Fahrenheit scale is biased because at 32 degrees it is actually freezing.

Why don’t we adopt the only unbiased scale, Kelvin? Wow, it is 293 degrees in here!

 

The Fahrenheit scale is biased because at 32 degrees it is actually freezing.

Why don’t we adopt the only unbiased scale, Kelvin? Wow, it is 293 degrees in here!

Hi,

Ha ha, yeah, that would be both cool and funny at the same time

 

The Fahrenheit scale is biased because at 32 degrees it is actually freezing.

Why don’t we adopt the only unbiased scale, Kelvin? Wow, it is 293 degrees in here!

The fact that 32 °F is freezing has nothing to do with what makes the Farhenheit scale biased. If the scale were redefined so that ~8.536 °F/°C (instead of 1.8 °F/°C), this would result in an unbiased (more properly known as a ratio) temperature scale, while keeping the freezing point of water at 32 °F.

How is the Rankine scale biased?

Also, the term "degrees" doesn't apply to either Kelvin or Rankine, as both are units of measure on ratio scales similar to meters or feet -- it is simply 293 kelvins.

 

Thanks, I’ll try to keep my to tongue-in-cheek posts more technically accurate in the future.

 

I don't have a problem with product ratings, and in fact I rely on them quite a bit. If you're comparing two seemingly identical items on Amazon and one has two thousand 5-star reviews while the other shows 40 ratings averaging 3.5, it's an easy choice. If you're in an unfamiliar town and have to choose between the 3-start restaurant and the 4.5-star one, it's also no-brainer. Well actually you have to look at the prices. The high-rated but cheapo hole-in-the-wall is the place you want.

There are a couple caveats. Any rating with less than, say, ten ratings is a question mark. It could all be friends and family. And I look for a bimodal distribution as in the first post. In that case you have to read a few of the 5-star and 1-star reviews to make judgement on the reason for the difference. If all the one-stars mention the product breaking, you know it's a fine product as long as it lasts but has a poor build. Sometimes people complain about restaurants because they didn't like the waitress or whatever. I couldn't care less and just care about the food.

But anyway the wisdom of the crowd is worth noting.

 

I don't have a problem with product ratings, and in fact I rely on them quite a bit. If you're comparing two seemingly identical items on Amazon and one has two thousand 5-star reviews while the other shows 40 ratings averaging 3.5, it's an easy choice. If you're in an unfamiliar town and have to choose between the 3-start restaurant and the 4.5-star one, it's also no-brainer. Well actually you have to look at the prices. The high-rated but cheapo hole-in-the-wall is the place you want.

There are a couple caveats. Any rating with less than, say, ten ratings is a question mark. It could all be friends and family. And I look for a bimodal distribution as in the first post. In that case you have to read a few of the 5-star and 1-star reviews to make judgement on the reason for the difference. If all the one-stars mention the product breaking, you know it's a fine product as long as it lasts but has a poor build. Sometimes people complain about restaurants because they didn't like the waitress or whatever. I couldn't care less and just care about the food.

But anyway the wisdom of the crowd is worth noting.

Hi,

Yes I suppose an item with 10000 ratings is harder to fake. Too many people to pay off

 

I don't have a problem with product ratings…

I fully agree with this post. Like so many other things, the information must be analyzed, not taken as gospel. I often see 1-star reviews where the reviewer basically misunderstood what the product was intended to do.