Some Arm boards have become quite powerful, while hardware based on low power Intel processor has generally become cheaper with both architectures somewhat converging in terms of performance and price.
Piotr Maliski got interested and purchased some low cost (<$150) Intel hardware to compare to mid-range Arm boards, throwing a Raspberry Pi 3 B+ into the mix as well for comparison.
Those are the Intel test boards / computers:
- Qotom motherboard with Intel Atom Z3735F Bay Trail processor, 2GB RAM, 32GB flash – $74 + shipping on Aliexpress
- Piesia nano ITX board with Intel Celeron N2806 Bay Trail processor, DDR3 SO-DIMM socket, SATA / mSATA interfaces – Piotr found it for around $85 on Aliexpress, but the price now jumped to over $150 plus shipping, which does not make it very attractive
- Generic thin mini ITX motherboard based on Celeron N3160 “Braswell” processor, DDR3 SO-DIMM socket, SATA / mSATA interfaces. $62.68 shipped on Aliexpress.
- MSI E350DM-E33 motherboard with “old” AMD E-350 dual core processor, 2x DDR3 SO-DIMM sockets, 6x SATA. Bought second hand for around $25-$30
- Asrock J5005-ITX motherboard with Intel Pentium J5005 Gemini Lake processor, 2x DDR4 SO-DIMM sockets, 4x SATA III interface. Bought for around $135
Those have be pitted against three Arm SBCs:
- RockPro64 with Rockchip RK3399 processor, 4GB LPDDR4, micro SD card or eMMC flash module storage – Around $95.
- ODROID-XU4 with Samsung Exynos 5422, 2GB RAM, micro SD card or eMMC flash module storage – Around $95, which should include shipping and EU taxes.
- Raspberry Pi 3 B+ – Around $45
He installed Linux on all those platforms and ran several Phoronix benchmarks and GLMark for graphics. The test did not run properly on all platform, which explains why not all 8 boards are not always represented in results.
The Gemini Lake motherboard is bound to outperform all other competitors in those test since it’s the most recent and expensive. So it’s not a surprise that it comes on top by a wide margin. As expected RockPro64 is quite faster than the Raspberry Pi 3 board. However, there must be a serious problem with the Celeron N3160 motherboard, as it should be faster than Bay Trail processors, and almost as fast as an Atom x7-Z8700 processor, but performs really poorly in all benchmarks run by Piotr. Sp it’s probably a good idea to ignore results about this board for now.
C-Ray floating-point benchmark actually looks quite good on Arm platforms, with the Pentium J5005 “only” about twice as fast. It’s also surprising to see ODROID-XU4 almost equal Pine64’s RK3399 board.
When benchmarking it’s always important to pay attention to details, and it’s especially the case with OpenSSL benchmarks, as specific instructions are often used for crypto operations like AES. Not all processors support those, sometimes they are not implemented in the code, or compiler flags used do not enable them. For more insights about this, read the insightful comments thread about Arm vs Intel OpenSSL benchmarks in a previous post. Nevertheless, OpenSSL benchmark results for the top three boards look very much like the same as for C-Ray, just ODROID-XU4 is slightly ahead of RockPro64.
Piotr also attempted to test graphics performance, but it’s particularly hard to compare Arm and x86 SoC because the latter normally rely on OpenGL ES that is a subset of OpenGL, and OpenGL benchmarks may not run on Arm, and if they do they may rely on emulation for missing commands. The latter possibly explains why RockPro64 compares so poorly, beside the faster GPU in the Gemini Lake processor.
Comparison of Arm boards with GLMark2 may be more interesting here, as again ODROID-XU4 surprises by outperforming RockPro64. But there’s again something very odd about the scores, as other have reported Raspberry Pi 3 B+ achieving 101 points. Testing graphics performance get even more complicated since graphics “frameworks” differ e.g. framebuffer, X11, Wayland, etc…, and in some cases offscreen benchmarks come into play.
The Linux distributions names and kernel versions, and the compiler flags are important to analyz the results since bugs get fixed, and different compiler flags may yield to fairly different results. You should be able to find those in OpenBenchmarking.org with the full benchmarks results.
Beside the benchmarks, details about each board including photos taken with a thermal camera can be found on Piotr’s blog.
Jean-Luc started CNX Software in 2010 as a part-time endeavor, before quitting his job as a software engineering manager, and starting to write daily news, and reviews full time later in 2011.
Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress
While cpu testing is simple in comparison, gpu testing is a very tricky business — one *really* needs to know what they’re testing. For instance, it’s quite often that a GL(ES) stack might be bottlenecked by a (cpu-based) copy-full framebuffer presentation stage, taking away cycles from the game logic code (if it’s a game, or anything that does sizeable work on the cpu). Alas, the gpu-stack stituation in arm land has been a wild west. One of the reasons my daily arm drive is a chromebook is the predictability and streamlining of the GLES stack.
N3160 performance is low but it could be something with the board having custom TDP set for it. With passive cooling they could crippled it. As of GLMark2 – GPUs those two ARM SoC have according to notebookcheck align like in my test – Exynos has a stronger GPU than RK3399. Plus there had been some artifacts in this test – running OpenGL test won’t be the best case scenario. Better of with Android testing both boards.
Re gpus, that’s the point, though — they’re significantly more complex than cpus, and are utilized via *way* more complex and abstract APIs, so ‘according to this or that paper spec (read: raw figures)’ means nothing unless you know exactly where the bottlenecks in your benchmarks sit. IFF the bottleneck is in *compute*, and the test aligns well with the older (2nd-gen Midgard) T628MP6 architecture, rather than with the newer (4th-gen Midgard) T860MP4, then *perhaps* the older Midgard could outperform the newer Midgard by some good margin. Otherwise I have gpu benchmarks here where the rk3399 outperforms the exynos5422 by… Read more »
You can always do off-screen tests, they will show you the GPU performance in the absolute – but it will be useless because what good use is a powerful gpu that is crippled by a buggy display server?
True. And even when doing offscreen tests, desktop GL (less so GLES) allows client code to do some quite stupid things that could tax the cpu beyond the necessart and/or impede cpu/gpu parallelism. Of course one day we’ll all be doing vulkan, but today is not that day. BTW, once again, thank you for your XU4 gnome-wayland image — its’ been quite useful to me.
I have no idea about broblems with this board, but support for intel atom class power saving features can seriously cripple the performance. Usually i start testing this type of boards by disabling all power saving features to get a reliable reference point.
Why not use glmark2-es2 since OpenGL-ES2 is supported natively on all systems?
I also second @Memeka’s comment that “off-screen” tests are useless since an expected primary use of a GPU is to actually see the rendering.
Was limited with what test cases Phoronix had. If there is an ES version I’ll look into it separately.
The RK3399 and XU4 aren’t surprising, the XU4 has more “fast” cores, and more cores overall.
If you are going to spend more than $100 always pick Intel/AMD since those CPUs are much faster than any ARM offerings. This does assume that the x86 boards are working correctly. Under $100 Intel/AMD can’ compete since they don’t really offer anything in that price category other that older chips. My two cents, it you want a desktop/laptop with GPU just spend over $100 and get an x86. ARM wins by far for embedded systems that don’t need a huge CPU and battery powered mobile (phones/tablets). Servers are currently a battleground. x86 dominates but there are serious attempts by… Read more »
The Benchmark is not true, Kernel compile time on Raspberry Pi 3 is 6 Hours (21000 seconds, Raspbian)
And why so? Note that phoronix benchmark has a specific kernel it builds. It doesn’t build a fully working kernel for given platform or distro. Just compile the same amount of code wherever it is run.
> And why so?
Since kernel compilation is also an (random) IO benchmark (same with some ‘DB benchmarks’). Kitchen-sink benchmarks do not differentiate between IO and CPU and thereby produce only numbers without meaning.
Did you check at which clockspeeds the various CPUs were running? Did you check for memory bandwidth and latency?
It’s not really without meaning as it can be re-run to get the same result and thus showcasing system performance for one of possible workloads.
IMO you need to test individually especially on system where random IO performance can be slow as hell (SD card). That’s the main culprit with all these low-end system comparisons: Apple vs. Oranges since storage is not taken into account (this even heavily influences tasks like ‘running Android’ or ‘Desktop Linux’ — slow random IO write performance and overall performance sucks regardless of any CPU and GPU benchmark results). ODROID-XU4 with fast eMMC vs. RockPro64 with slow SD card: XU4 wins ‘kernel compilation’ competition easily. Run RockPro64 off a fast NVMe SSD and it’s the other way around. So you… Read more »
Yes, it’s good to know each subsystem capabilities and performance, however at a first glance board consumer will want to know if and how well it can run software he/she wants it to. Can it run a database?, a desktop fluently?, can it run Skyrim? some sort of imperfect software… after that it can be beneficial to go down to a lower level to see what’s the limit or where performance can be gained. I still have those boards and I’m waiting for eMMC module plus I already have PCIe/NVMe adapter that can be used with RockPro64. I will be… Read more »
> SanDisk … especially a A2 rated one
Based on my experiences not the best idea currently since A2 SanDisk cards are slower than their older A1 siblings: https://github.com/ThomasKaiser/Knowledge/blob/master/articles/A1_and_A2_rated_SD_cards.md
Phoronix kernel compile benchmark would likely vary a lot depending on the micro SD card. If you use a cheap micro SD card, it’s quite possible compile time is much longer due to slower random I/Os.
If you compile the kernel yourself, then as pointed out by Piotr, it depends which modules are enabled and so on.
Yes, a low performance SD card will affect the result. I had to use one SD card for all as the other one I had gave much worse numbers.
‘Same SD card’ is also pointless since the host interfaces differ. ODROID-XU4 for examples implements SDR104 which will positively affect also random IO performance: https://forum.armbian.com/topic/954-sd-card-performance/?do=findComment&comment=49811 (please note that there I constantly talk about DDR50 which is BS since it’s the old ‘HS’ mode and not DDR50) And if you run the same kitchen-sink benchmark that relies on both CPU and IO performance on an Intel thingy off a quality SATA SSD and the next time on an ARM board off a crappy or even a good SD card the results should not surprise (‘good’ means ‘high random IO performance —… Read more »
If performance of XU4 are nearly same as RK3399, then what will be the result of ODROID-N2 ? Other than hardware accelerated video decoding, it is pointless to buy RK3399?
You should really read the article above and the link to comments section where it’s discussed how meaningless some (most) of the Phoronix ‘benchmarks’ are (especially true for the OpenSSL ‘benchmark’). Then you should always keep in mind that you test software and not hardware even if kitchen-sink benchmarks like Phoronix try to create another impression. Numbers vary based on settings and versions and even ‘events’ (see Spectre/Meltdown mitigations and their performance impacts). Essentially you benchmark software and need a lot of more work to get an idea how that relates to the underlying hardware capabilities. In case of RockPro64… Read more »