How we protect our benchmarks from cheating
Posted October 8, 2015 by RyanR in News
Have you been following the VW scandal? It’s a timely reminder to be wary of manufacturers’ claims.
Our benchmarks help people see their hardware’s true performance, but you might wonder how we ensure that the scores are honest and accurate? Our commitment to providing you with benchmarks you can trust means that:
- Futuremark has public rules for manufacturers. In short, a platform must treat the benchmark as if it were any other application.
- You can only get a valid 3DMark score for a Windows PC by using WHQL drivers that Futuremark has checked and approved.
- Futuremark was the first benchmark developer to delist and remove mobile devices with hidden optimizations from its performance rankings.
- All Futuremark benchmarks are backed by detailed technical guides that explain exactly what’s being measured and how the scores are calculated.
- Futuremark’s Hardware Channel and Best Mobile Devices lists only show publicly available models. Futuremark never lists or leaks unverified scores from pre-release hardware.
If you’ve been writing about PC hardware for more than a decade, you’ll remember the outcry when various vendors were caught making dubious driver optimizations. And more recently, some smartphone manufacturers have tried, and failed, to cheat their way to higher benchmark scores.
So if there is one thing to take away, it’s that the cheaters always get caught. Not only because it is easy to detect, even without the special tools we have in our test lab, but because there are hundreds of press reviewers using 3DMark and PCMark, and millions of people benchmarking at home, all watching intently for any funny business.
You can contact us any time with questions about our benchmarks, rules or policies. And if you ever find hardware with a suspicious score, share the details with us and we’ll investigate right away.