Cheating on Benchmarks – "Everyone is doing it", the practice is an "industry standard"

guest · Apr 9, 2020

MediaTek caught cheating on benchmarks, says everyone is doing it
April 9, 2020
https://www.techspot.com/news/84761-mediatek-caught-cheating-benchmarks-everyone-does.html

A hot potato: Here’s a story you’ve heard before: a company has been caught cheating on mobile benchmark software. Rather than being limited to phones from a particular firm, the culprit in this instance is chipset maker MediaTek, meaning multiple handsets from several vendors are engaging in the practice. Its excuse? Everyone does it.

Anandtech discovered the cheating after benchmarking the MediaTek Dimensity 1000-powered Oppo Reno3 (Chinese version), and the European version of the Oppo Reno3 Pro, which is powered by the older, lower-end Helio P95.
To confirm the cheating, the publication used an altered version of PCMark that can’t be identified as a benchmarking tool by systems. Not too surprisingly, the P95 scored lower: a 30 percent drop in the overall score, with up to a 75 percent difference in specific workloads.
Click to expand...

MediaTek emphasized that manufacturers have the final say on what optimizing they want to include in APKs, and that “benchmarking optimizations” are an industry standard. When Huawei was caught manipulating benchmark scores, it also used the “others do it” excuse.
Click to expand...

Log in or Sign up

Cheating on Benchmarks – "Everyone is doing it", the practice is an "industry standard"

guest Guest

Log in or Sign up

Cheating on Benchmarks – "Everyone is doing it", the practice is an "industry standard"

guest Guest

Useful Searches