Vacuum Cleaners Discussions

Consumer Reports March issue has an updated review of vacuum cleaners and water extraction cleaners.  There is also a review of some cleaning solutions for spot cleaning.  I don't have my copy with me, but some of the results are interesting.     

One bit of curiosity:  the Miele Twist ($550) and Miele Bolero ($800?) are ranked 3 and 5 respectively.  The numerical score is one point higher for the less expensive Miele.  Otherwise the scores on individual tests are equivalent.   CR does not seem to mention whether price is a component in their scoring system, and if so how much influence it has.

Dyson's reliability has dropped slightly, but it is still in 2nd place for uprights.  Kirby is alone in first place.  The Dyson DC28 (13th) is actually ranked higher than the Kirby Sentria (15th); (however, I double checked and the numerical scores are the same - so perhaps it's more accurate to say that they are tied). 

One thing that stands out is that there isn't much difference in the overall scores.  The highest rated vacuums tend to have the best scores for carpet cleaning.  However, the overall scores reflect the tradeoffs inherent in picking a vacuum.  The best cleaning vacuums (with the exception of the 2nd rated Hoover lightweight Platinum) tend to be the heaviest - which tends to cause them to have lower handling scores. 

Another thing that stands out is the high cost of some bags.  One vacuum uses bags that are $7.50 each apparently. 

Regarding carpet cleaners, the best hands down is calling a Pro like Stanley Steemer.   Several Hoover models did well.  CR was not very complimentary about the Dyson Zorb powder.  I don't know if they used it incorrectly, but it also was the most expensive application. 

Just for fun, I've been playing with the CR upright vacuum score data.  I've coded CR component scores as excellent=5, very good=4, good=3, fair=2, and poor=1.   For the time being, I've assigned a score of 2.5 for tools to the Oreck, Riccar, and Koblenz models that don't take tools.  Obviously, I don't know what goes into CR's proprietary formula, but I can regress the component scores and other factors (brush off - coded 1=yes, 0=no) on the published scores and see what correlates.  Price is divided by 100 and treated as a linear factor. 

Based on this model:

predicted score = 2.27 + carpet_score*4.36 + floor_score*1.75 + tool_score*2 + noise_score*2 + 1.76*emissions + 2.3*handling + 1.34*pet_score + 0.22*price/100 + 3.47 (if brush roll can be turned off, +0 otherwise) + 2.27.

Since I only have the categorized subscores, I probably can't develop a much better prediction model.  I assume CR uses more precise subscores in their formula.  Overall score tends to increase with price, but the relationship isn't statistically significant.  Under $200 of so, I suspect price makes a big difference in score - you get more bang for your buck by spending more and adding features. Bagless tends to score about 4.3 points lower than bagged on average, but with the other factors in the model, bagless (as an indicator variable) is not a significant predictor of overall score.  

CR seems to be inconsistent in how it codes full bag/bin indicator.  All Dysons get credit for a full bin indicator.  Very few Hoovers do, for whatever reason.   

n=52, adj R-squared=0.895, R-squared=0.91
score	Coef.	Std. Err.	P>|t|
carpet	4.36	0.46	<0.0001
floor	1.75	0.39	<0.0001
tool	2.01	0.36	<0.0001
noise	2.03	0.63	0.002
emissions	1.76	0.59	0.005
handling	2.30	0.63	0.001
pet	1.34	0.42	0.003
price_100	0.22	0.17	0.196
brush_off	3.47	0.96	0.001
intercept	2.27	3.83	0.557

When I look at residuals by brand (difference in predicted and actual score), I don't see any obvious bias in CR's results (i.e. some brands scoring much higher or lower than expected based on the regression model).   

Hi SEVERUS:

That's very good.  What is the 'intercept' at the bottom of the features:  Bag indicator?

Carmine D.

An intercept is just a constant or baseline score.   For example, the equation for a line is y=a*x + b.  b is a constant or y-intercept.  I'm assuming that the CR overall score is a weighted linear combination of subscores.  they may also award points for vacuums having features such as a full bag indicator. 

Keep in mind that CR categorizes the scores that they publish (excellent, very good, good, fair, poor).  However, I assume that they use the exact numerical score when they calculate overall score.  For example, I believe excellent corresponds to a score of 80 to 100.  I have to assign the same score (5) to everyone in that category, since I don't the precise score.  

At least based on my model, it agrees with CR's contention that carpet cleaning is the most important task, since that coefficient is the biggest.   Given that price and features tend to increase to some extent, it makes it difficult to know if price enters into the overall score. 

As I was entering the data, it was interesting to the inconsistencies in "full bag/dust bin indicator".    Apparently if you have a line drawn on the dirt canister that indicates full, it meets the criteria.  I was disappointed that most Hoovers don't have a way to turn the brush roll off.  Kirby was given credit for a way to shut off the brush roll - is that by disengaging the belt?

Severus wrote:

Hi Severus,

Yes indeed.  The dial or lever at the front of Kirby's detachable cleaning heads is called the belt lifter and is used to disengage the brush roll when and if desired.  Meant to ask did you factor in price level?  Kirby usually loses points with CR due to its high price.  Also, how do machines recommended as "best buys" work into the scoring?

Best,

Venson

Venson wrote:

I don't have CR's proprietary formula, so I can only guess.  My guess is that CR picks best buys after the rankings are computed based on price.   I'm also not certain that price goes into the rankings.  In the regression, overall score seemed to increase with price.  If there was a penalty, I would expect price to have a negative coefficient.   Price and features tend to increase together, so it is difficult to untangle them in a regression model. 

Here's a simpler model - overall score = carpet score + floor score + tool score + noise score + emissions score + handling score + pet score.   So based on this we would calculate the overall score as:

score = 1.7 + 1.13*pet score + 0.61*handling score + 2.6*emissions score +2.9*noise +2.15*tool score +2.18*bare floor score + 4.33 *carpet score

score	Coef.	Std. Err.	P>|t|
carpet	4.336026	0.533405	0
floor	2.177052	0.459653	0
tool	2.149497	0.424873	0
noise	2.89352	0.716015	0
emmissions	2.602233	0.689959	0
handling	0.613601	0.641415	0.344
pet	1.131269	0.510868	0.032
_cons	1.718993	4.663754	0.714

residuals are calculated as actual - predicted.  Since the residual for kirby is -.43, this simple model predicts a score of 68.46 for kirby, but cr's actual score is a 67.  Granted I'm using heavily rounded scores as data going into my model, but I'm not seeing much of a penalty for Kirby due to high price.   CR will likely never call Kirby a "best buy" due to the high price, but I don't see any evidence of penalizing Kirby for high price.   Only CR knows the proprietary formula, so only they know if they penalize a high priced machine.  Here are average residuals by brand - based on this simple model.  Note that a large positive residual (actual score better than predicted) is likely due performance above average within several categories.  Whereas I use ranks of 1 to 5 in my model for each subscore, CR likely has scores ranging from something 0 to 100.  Consider a hypothetical (i.e. made up data):  Let's say that both a Dyson and a Kirby score excellent on bare floor cleaning.  Suppose CR  awards 95 points to Kirby and 85 points to the Dyson based on performance - both are in the range for a categorical score of excellent.  I'm stuck giving them both the same score of 5 (on a scale of 1 to 5), because I don't have the exact scores from CR.  I only know that both scored excellent on bare floors. 

Over	average residual by brand
bissell	-1.31062
dirt devil	-2.72875
dyson	0.796197
electrolux	-1.93962
eureka	-0.4353
gary	0.129353
hoover	0.048261
kenmore	3.264786
kirby	-0.43285
koblenz	-5.98061
miele	-0.45761
oreck	0.172512
panasonic	1.564734
riccar	1.588726
royal	4.281464

I can't say for sure either as to how CR does the math on this or if they do much at all. I do know, only using Kirby as an instance, that there are a number of reasons to pull points due to size, weight, price -- which CR highly stresses -- and convenience.  Kirby's belt lifter requires the user to stoop to the floor to kill the brushroll action leading me to believe that other good uprights with electric switch-offs OR foot operated mechanics to stop the brushroll would gain higher marks in that area.

The "best buys" have never ranked as be-all-end-alls but appear per the ratings to have enough all around ability and convenience to make them good deals despite the higher marks of the top-rated models.

Venson

Venson wrote:

The best buy Hoover Tempo is lacking a brush roll shut off - which would be a concern to me.   CR likely uses focus groups and surveys to help decide which factors are important to consumers - and weights them accordingly in the overall score.  However, factors that are important to you and me, may not be the same factors that are important to focus groups.  "Best buys" make for good television/radio recommendations.