HOW fast does a computer work? The answer has long been a matter of dispute. No two systems are identical, and many measures are subjective: scrolling feels sluggish or zippy, or a poorly written browser on Apple’s Macintosh makes it seem slower most of the time than an efficient Windows equivalent on otherwise identical hardware.
Debates are often solved by apples-to-apples (or windows-to-windows) benchmark testing. These are identical, automatic, repetitive and computationally intensive tasks. Companies have long tried to game these tests; Anandtech examined in depth how Android smartphone makers, Samsung in particular, are keen to win bragging rights as a result.
At one level, gaming the tests is trivial. Anandtech (as well as other techie websites) have long found that some operating systems had in fact been designed to check whether particular benchmark programs were running. If so, the system would fiddle with settings to inflate performance. But how can a system be able to make its hardware appear faster for a moment?
It comes down to “clock speed”, the basic measure of oomph since the dawn of central processing units (CPUs). This indicates how many instructions or operations per second a processor can carry out. As computing becomes more complex, so does the notion of a single “operation”—some processors, for instance, do prep work for subsequent instructions at the same time they execute the current one. But that has not stopped the measure being employed.
It was once a sport to see how fast one could “overclock” an off-the-shelf CPU, by squeezing as much out of it as physically possible. This often required swapping out the computer clock’s crystal (whose oscillations govern a processor’s speed, like a pacemaker keeping a heart beating at a certain rate) or make other fiddly hardware modifications. Later, some processors allowed twiddling clock speeds while booting up. Nowadays many mobile processors for smartphones and tablets, and some more robust ones used in laptops, dynamically adjust clock speed during operation. The user is none the wiser.
The faster a processor operates, the more energy it consumes and thus the more heat it produces. That heat must be dissipated as fast as it is created through heat sinks, fans and other gubbins to prevent the processor from malfunctioning or failing entirely. An overheated chip can give faulty answers or stall. Chip-makers and hardware manufacturers strike a balance for any given CPU model for the optimal trade-off between performance, reliability and power consumption. On a mobile device, battery life is paramount; a CPU would typically be set to perform well below its maximum capability. (Data centres, including those run by Facebook and Google, have made great hay out of running equipment hotter, but this requires redundancy, specialised racks and cases and the constant replacement of a subset of servers.)
Anandtech found that various smartphones could improve benchmarking results by forcing the processor to work at a higher clock rate during tests. Some multi-core devices engaged a processor that would otherwise sit idle. This giddied the gadgets up, but drained the battery.
What seems truly remarkable, however, is that handset-makers have put any effort into this at all. Anandtech notes that the overall improved performance is piffling, and detectable. Future tests for mobiles are bound to be less vulnerable to such gaming.
And consumers increasingly ignore the specs in favour of what they see. When devices were far slower and more primitive, specifications may have told a story. Geeks like Babbage may still pore over figures like bits, pixels and clock speeds. Now the “fastest smartphone in independent testing” will struggle to outcompete one with the display, software and price that a buyer is actually after.
Powered by Facebook Comments