It’s been the bane of many a computer salesman to try and explain why one model of computer is “better” than another. Customers get lost in the quagmire of processor speeds, front-side bus integrated versus dedicated graphics, amount and type of ram, etc. How can one really tell which is faster, model A or model B?
Many of us resorted to running benchmarking utilities to get the bottom-line score, but this was subject to change with new versions of the tool, and the interpretation of the score was up for debate (some were biases toward better graphics handling, some to better office application performance, etc.). But it served as a baseline by which to compare two dissimilar systems.
A point in case, I had a system that benchmarked 1800 points that a customer was comparing to another system at a store down the street. The one down the street was $100 less and “the same computer,” meaning it was using the same process, had the same amount of RAM and disk space. I went down to the other store, benchmarking software in-hand, loaded it up, and ran it through three loops. The score came back just under 900 points.
I advised the customer that the other computer wasn’t the same, it had different parts (the motherboard and onboard video chip, in particular), but it was still a decent little rig. The price, I advised, was a bit high, after all, if a computer is half as fast as another machine, shouldn’t it cost half as much? He agreed and we called the store down the street, they wouldn’t budge on the price, so he walked out with the system I was selling.
With Windows Vista, Microsoft decided to introduce the same kind of tool so OEMs (like Dell, HP, and the like) could compare their systems to other systems in their line, and to competitor’s systems in an unbiased, level playing ground. This became known as the Windows Experience Index.
The WEI is a tool on systems running Windows Vista that you can get to by clicking the Windows Orb, right-clicking Computer, and clicking Properties. In the System section you’ll either see your Windows Experience Index, or a link to perform the testing to figure out your rating.
Each component is individually tested and rated, and the composite system score is displayed as the lowest score of the individual components.
Not the best criteria for testing or scoring in the world, but it’s a common baseline for rating, which is a good thing.
Unfortunately, a friend of mine did some digging, and found out that the Windows Experience Index is hackable. Using a relatively simple technique anyone with access to the system can change the Windows Experience Index to display virtually any score they want! Worse yet: if you re-run the tests, the actual scores are not displayed, the hacked versions are.
Microsoft has been advised of this issue but as of this writing they have not committed to any kind of a fix.
What does this mean to you? You cannot trust the Windows Experience Index to report true numbers until Microsoft makes it unhackable, or at least enables the end-user to get accurate results after re-running the tests.