Subscribers to consumerreports.org have been able to add user reviews of vacuums for several years now. It's not uncommon to read a user taking CR to the woodshed for a shoddy product purchased due to CR's high ranking. Some of the highest ranked products have some of the worst reviews. I suspect CR readers are toughest on the highest ranked products and more lenient on lesser ranked products since expectations are lower. The disagreement many times is associated with CR's overall criteria not aligning itself with the needs of some consumers. For example, it is common to see a consumer complaining that the highly rated canister vacuum weighing in at 26 pounds is too difficult for them to handle. Many complaints seem to deal with reliability.
Rather surprisingly , the Rainbow which rates very poorly in the Consumer Reports ratings, gets 4.5 out of 5 stars from the users. I don't know if this is a direct result of the sales pitch that goes with the Rainbow, or a deficiency in the testing by CR. Dyson owners are especially testy. They always seem surprised that their vacuums don't perform better in CR's tests carpet cleaning tests.
It's difficult to know whether the user reviews will influence ratings in the future due to their inherent biases. Perhaps CR will do a better job of evaluating reliability data in the future. CR has reported that when they get a lot of user complaints about reliability, that they dig deeper into the members annual survey to look for problems. However, users with problems are more likely to publish reviews than users that are content.
I believe feedback from members has resulted in the tests on cleaning pet hair and more emphasis on lightweight and ease of use.
The smart tyrant writes his own story to ensure that it is favorable. The lazy will repeat lines from the book without fact checking.
