How we test Smartphones

How we test Smartphones

The cool thing about the internet and social media is that you get a ton of input and various experiences from like-minded folks, and those with interests similar to you. And more opinions are always a good thing, especially if you've time to process them according to your usage needs and expectations. However, many people simply don't have the time to trawl around, and even if they do, they're quite likely to get opinions that may not exactly be relevant to Singapore or Southeast Asian readers. Market forces, availability, services, support, lifestyle,  socioeconomic considerations and prices are very big factors that shape the narrative of how a product fares.

This is exactly why HardwareZone started the site back in 1998 to convey opinions and experiences that are more in tune with local needs. That said, we can't run away from the basics of how we peg and compare products and part of that comes down to how we derive certain measurable metrics that are important from our point of view, as well as several intangibles that would greatly differ across different users such as design, handling, visual and audio delivery and more.

In this article, we detail how we test smartphones. It will be a work-in-progress article as we beef up the details and keep the article updated to reflect what we hold important when testing new mobile devices. To give some relevance to this information, we'll also use sample benchmark data from a recent review.

Phones compared for the sample data are available in this link.

Our Measurable Metrics - Why we use these benchmarks

JetStream 2.0

JetStream 2 is a combination of a variety of JavaScript and Web Assembly benchmarks, including benchmarks that came before like SunSpider and Octane. It primarily tests for a system’s and browser’s ability to deliver a good web experience. It runs a total of 64 subtests, each weighted equally, with multiple iterations, and takes the geometric mean to compute the overall score. The higher the score, the better.

Geekbench 5

Geekbench CPU is a cross-platform processor benchmark that tests both single-core and multi-core performance with workloads that simulate real-world usage. Geekbench 5 scores are calibrated against a baseline score of 1000, which is the score of an Intel Core i3-8100. The higher the score, the better.

Geekbench 6

Geekbench 6 benchmarks single-core and multi-core CPU performance. Geekbench 6 scores are calibrated against a baseline score of 2500, which is the score of an Intel Core i7-12700. 

3DMark Wild Life (Unlimited)

3DMark Wild Life is a cross-platform benchmark for Windows, Android and Apple iOS for measuring GPU performance. Its graphics test consists of multiple scenes with variations in the amount of geometry, lights and post-processing effects, mirroring mobile games that are based on short bursts of intense activity. Wild Life uses the Vulkan graphics API on Windows PCs and Android devices. On iOS devices, it uses Metal.

In Unlimited mode, the benchmark runs offscreen using a fixed time step between frames. Unlimited mode renders exactly the same frames in every run on every device, regardless of resolution scaling. The higher the score, the better.

PCMark for Android - Work 3.0 and Storage 2.0

PCMark for Android is a benchmark for testing the performance of Android phones and tablets. The Work 3.0 test checks how the device handles common productivity tasks such as browsing the web, editing videos, working with documents and data, and editing photos. Storage 2.0 checks write-in and read-out performance for internal storage, external storage (if applicable), and SQLite database management. Together, the benchmarks can clue us in on how capable a phone is at handling everyday use. Work 3.0 scores are above, while Storage 2.0 scores are immediately below for each device - the higher the score, the better.

Since this is a newly introduced benchmark in our reviews, we’re building up our database of PCMark scores for Android phones.

Battery Life

Our new battery benchmark uses PCMark for Android’s Work 3.0 Battery Life test to determine a modern Android-based smartphone's battery uptime in minutes. This controlled benchmark simulates real-world usage with a combination of both web and social media browsing, video and photo editing, parsing data with various file formats, writing (on documents), and more. 

To enhance consistency in battery results, we start the test at: 

• 100% of the phone’s battery capacity until 20% left
• A fixed display brightness is calibrated at 200cd/m² (200-lux) with the help of a luminance meter 
• Max resolution
• Refresh rates unlocked
• 0% audio, or completely silenced where possible
• Full bars of Wi-Fi connectivity, Bluetooth, and notifications enabled
• Only one other battery recording app opened in the background
• Scored in uptime minutes; the higher, the better

The new benchmark will succeed our outgoing Battery Life benchmarking, which has the following parameters:

• Looping a 720p video with screen brightness and volume at 100%
• Wi-Fi and Bluetooth connectivity turned on
• Constant data streaming through email

Our articles may contain affiliate links. If you buy through these links, we may earn a small commission.