MediaTek caught cheating in benchmark tests, accuses “key competitor” of doing the same

Taiwanese semiconductor manufacturer MediaTek is under the scanner for reportedly employing devious means to boost its benchmark test results. While cheating through benchmark testing is not new, this is the first time an OEM has been caught manufacturing SoCs with hardcoded benchmark-tampering functions in the firmware. The ploy was first spotted by Anandtech when a Helio P95-powered European OPPO Reno 3 Pro outperformed the Dimensity 1000-powered Chinese OPPO Reno 3. This came as a surprise at first since the dual-core P95 is two generations older than the quad-core Dimensity 1000.

MediaTek caught cheating in benchmark scores

Soon after, a stealth version of the PCMark benchmark utility was installed on the Reno 3 Pro, which immediately recorded a massive 30 percent drop in benchmark scores compared to the previous run. The publication further added that some results dropped by 75 percent when the benchmark tests were run using the stealth version of the program that the firmware cannot recognize. The same tests were also run on the Chinese variant of the Reno 3 Pro, which is powered by the Snapdragon 765G. This time the results were comparable with the original benchmark score, which proves the device responded without resorting to cheating.
Upon further investigation, Anandtech discovered references tying popular benchmark platforms like Geekbench, AnTuTu, PCMark, 3DBench, and a few Chinese platforms with ‘sports mode’. When in sports mode, the memory controller of the chipset operates at maximum frequency at all times. This allows CPU cores to ramp up performance at a much quicker rate and operate for longer durations compared to everyday use.
The OPPO Reno 3 Pro is not the only device to fail the stealth PCMark benchmark tests. The Sony Xperia XA1, Xiaomi Redmi Note 8 Pro, and the Realme C3 also displayed similar results. Since the Xperia XA1 launched back in 2017, it is possible that MediaTek has been resorting to unfair practices for a while now.

MediaTek publishes statement defending its practices

MediaTek was quick to publish a press release on its website trying to answer some of the questions raised during the debacle. The document titled “Why MediaTek Stands Behind Our Benchmarking Practices” stresses on MediaTek adhering to “industry standards in benchmarking”.
MediaTek claimed that manufacturers have the final say about the tweaks and optimizations for specific APK files. The company further noted that some brands occasionally offer performance-enhancing features like ‘sports mode’ or ‘monster mode’ in specific markets, while disabling the feature for the rest of the world. MediaTek further shifted the blame on Anandtech for trying to question its benchmark optimization practices, claiming that it follows all industry norms. The company further added that its “key competitor” follows a similar practice.
“If they were to test some of our rival’s chipsets, they would uncover a similar pattern”. MediaTek’s vague reply does not hint at any manufacturer in particular, although big brands like Samsung, and more recently Huawei, have been caught trying to enhance benchmarking results in the past.