Performance Comparison of Android Devices
Benchmark results ("Baselines") were gathered from users' submissions to the PassMark web site as well as from internal testing. PerformanceTest Mobile conducts 17 different tests spanning five standard test suites (CPU, Memory, Disk, 2D & 3D Graphics). The results are combined to determine the PassMark rating for a device. The PassMark rating takes into account the results from all five standard test suites and provides a general overall system rating.
The CPUMark, which is the result from the CPU test suite, is the biggest contributor to the overall PassMark rating. The CPUMark offers insight into the performance the CPU of the device being tested. Since the PassMark rating takes into account varied device setup (i.e. external memory cards), some user may find that the CPUMark can offer a better gauge of raw performance.
To ensure that the full CPU power of a device is realized, PerformanceTest Mobile runs each CPU test on all available CPUs/Cores. Specifically, PerformanceTest Mobile runs one simultaneous CPU test for every logical CPU; physical CPU core (dual core) or physical CPU package (multiple CPU chips). So hypothetically, if you have a device that has two CPUs, each with dual cores then PerformanceTest Mobile will run four simultaneous tests.
The following list describes each of the tests performed.
The Integer Math Test aims to measure how fast the CPU can perform mathematical integer operations. An integer is a whole number with no fractional part. This is a basic operation in all computer software and provides a good indication of 'raw' CPU throughput. The test uses a large sets of random 32-bit and 64-bit integers and adds, subtracts, multiplies and divides these numbers.
For each of the disk tests, a test file is created in the respective storage location. Each
test file is 90MB in size with read and write block size set to 32KB. If there is not enough
space on the device, the test is skipped. All file I/O operations are sequential. Due to the
variability seen for disk tests, its contribution to the System Rating is weighted less in the
overall score.
The write and read test uses the NDK C functions open(...), read(...), write(...) to access the
files. The test will try to commence without using cache (O_DIRECT) if possible. However, not all
devices support opening files directly, therefore devices with cache usage will report higher results.
This suite contains a number of tests that exercise the memory sub-system of the computer (Random Access Memory - RAM). All tests use a combination of 32-bit and 64-bit data when reading or writing from or to RAM.
Database Operations
This makes heavy use of C++ STL containers to test the performance of memory in maintaining large structures
of data like that in a database.
Read Cached
This test measures the time taken to read a small block of memory. The block is small enough to be held entirely
in cache (if one is present).
Read Uncached
This test measures the time taken to read a large block of memory. Measured in GB/s, larger values are better.
The test will uses a 256 MB block. The block is too large to be held in cache.
Write
This test measures the time taken to write information into memory. Measured in GB/s, larger values are better.
Similar to the read uncached test, except writing is performance instead. The test also uses a 256 MB block.
Latency
Measures the time it takes for a single byte of memory to be transferred to the CPU for processing. Measured in nanoseconds,
lower values are better. The test will allocate a 256 MB buffer that is then filled with pointers to other locations in the
buffer, essentially a looping linked list. The order of the links is randomized within certain 1024 byte ranges.
Threaded
Nearly identical operations to the Read Uncached test, however being performed by separate processes simultaneously
to test how well memory copes with multiple concurrent accesses.
The Solid Vectors test draws a rectangle filled with a gradient, a solid filled ellipse, a line, and Bezier and a solid filled star as many times as possible. Every 32nd cycle the colors are randomized.
The Transparent Vectors test is similar to the Solid Vectors test however all color fills are partially transparent.
The Complex Vectors test draws a jungle scene using a number of complex vector images of trees and animals. This scene will be drawn as many times as possible within the time limit and always finishes on a complete drawing of the scene.
On Android, this test uses the svg-android library to load the vector graphic assets.
The Image Rendering test measures the rate at which a bitmap image can be rendered to the screen at various sizes. The test always finishes on a complete cycle of rendering smallest to largest.
The Image Filters test applies built-in rotation, brightness and greyscale filters to an image. Counts the number of filters applied per second.
Simple Test
Two hundred bouncing balls are active in the scene. Each ball has one of 25 unique textures and
each ball object has 1260 vertices and 1257 triangular polygons. Lighting and Fog is enabled. Over
the test duration, the scene is rendered into upto 8 possible back buffers before being presented to
the screen. Back buffer size range from 1x (Native Screen Width x Native Screen Height), 1x (Half
Native Screen Width x Native Screen Height), 2x (Half Native Screen Width x Half Native Screen Height),
4x (Forth Native Screen Width x Forth Native Screen Height).
Complex Tests
Three helicopters travel around the scene featuring 500 trees with 3 different tree textures
on a terrain with water. The terrain is formed with 10,000 triangular polygons (30,000 vertices).
Above the cloud layer, 3 jets travel accross the scene and is composed of 1194 vertices each. The
trailing smoke is formed with 500 particles and updated on each frame. The trees and water consist
of 2 polygons (6 vertices) and each helicopter object consist of 1,120 triangular polygons
(3,360 vertices). Lighting, fog and alpha blending are enabled. On lower powered devices, the
terrain is formed with less vertices, lower number of elements and no smoke particle system. In
these cases, the 3D Graphics score is penalized.
OpenGL ES using Unity
This test will launch a scene created using Unity 5.5 with 2500 Asteroids, 71 Space ships (13 unique models)
and 1 Space station. Polygon count for each space ship is between 5,000-30,000 with 39 different textures.
The asteroids are based 3 level subdivieded icosphere with one of 10 unique textures. A particle system is
used for the smoke trail for the main chased ship. On lower powered devices, the 3D Graphics score may be
penalized.
Issues: On some lower memory (RAM) devices may not be able to run or run but return no results on the OpenGL ES
Test using Unity. Android will start closing programs that use too much memory which might cause the program
to crash/restart without notice.