Qualcomm and MediaTek Fight Over Benchmark Results and Cheating

• MediaTek admits that certain benchmark apps are whitelisted to run at special performance modes.
• The company stated that everybody in the field is doing the same thing, so it’s an acceptable practice.
• Qualcomm responded by calling this cheating, claiming that they’re not doing it with their chips.

Last week, AnandTech openly accused MediaTek of cheating on mobile benchmark tests to bump up the resulting performance test scores. Evidently, smartphone OEMs who are using MediaTek chips were whitelisting certain benchmark apps to enable a “sports mode” that would give the processor an extra boost. This mode keeps all cores running at max speed, sets the scheduler to aggressive tracking, sets the memory controller fixed at maximum speeds, and activates various software-level boosting mechanisms. MediaTek has responded with a statement that admits the practice, saying that this is a standard thing in the industry since every mobile chipset manufacturer does this.

MediaTek feels that optimizing power and performance at the moment of the benchmark yields accurate results that represent the true capabilities on the processors, so this sports mode isn’t cheating. As Qualcomm was indirectly included in the mention of “every chipset manufacturer,” the American silicon makers felt that they should respond with an official statement. As they told Android Authority, they are not engaging in whitelisting practices, and they don’t try to enhance the performance of their chips outside what the users would regularly enjoy. They believe this whole approach defeats the very purpose of a benchmark and is blatant cheating.

Of course, MediaTek may have referred to Huawei or Samsung, as these two are also chipmakers. Still, considering Qualcomm is its market archrival, it was natural to assume that they were the ones implied in the statement. To summarize, MediaTek doesn’t feel like it is inflating benchmark scores to present unrealistic results. Still, Qualcomm thinks this qualifies as cheating and claim that they’re using such practice themselves. So, where does this leave the end consumer then? If running these apps yourself will not yield accurate or realistic results, then how can one compare two chips objectively?

One way to do it would be to sandbox the benchmarks so that the phones won’t be able to set different performance modes. Secondly, one could clone the app and rename the package so that it won’t trigger the unique settings. Finally, you may review the whitelist and pick a benchmark tool that’s not found there, so no boosts will happen while running the app. From our perspective, this should be a case handled by consumer protection agencies since companies are marketing false promises and create unrealistic expectations for potential buyers.