Since I’ve just installed Ubuntu 17.10 on MeLE PCG35 Apo, I decided I should also run some benchmarks comparing with other ARM and x86 Linux platforms I’ve tested in the past.I was particularly interested to compare the performance of Intel Apollo Lake processors (Celeron J3455 in this case) against higher end ARM processors like Rockchip RK3399 (2x A72, 4x A53) since systems have a similar price (~$150+), as well as against the older Bay Trail processor to see the progress achieved over the last 2 to 3 years.
To do so, I used Phoronix Benchmark Suite against Videostrong VS-RK3399 results (RK3399 development board):
|
1 2 3 4 |
sudo apt install php-cli php-gd php-xml php-zip wget http://phoronix-test-suite.com/releases/repo/pts.debian/files/phoronix-test-suite_7.4.0_all.deb sudo dpkg -i phoronix-test-suite_7.4.0_all.deb phoronix-test-suite benchmark 1709271-TY-1704029RI26 |
The benchmark first issued a warning about “powersave” governor, but I still went ahead, and once completed I change it to “performance” governor:
|
1 2 |
sudo apt install cpufrequtils sudo cpufreq-set -r -g performance |
…and ran the tests again. All results are available on OpenBenchmarking.
Let’s address the governor results first. cpufreq-info reports that powersave governor can also switch between 800 MHz and 2.30 GHz (turbo freq).
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
sudo cpufreq-info cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009 Report errors and bugs to cpufreq@vger.kernel.org, please. analyzing CPU 0: driver: intel_pstate CPUs which run at the same hardware frequency: 0 CPUs which need to have their frequency coordinated by software: 0 maximum transition latency: 0.97 ms. hardware limits: 800 MHz - 2.30 GHz available cpufreq governors: performance, powersave current policy: frequency should be within 800 MHz and 2.30 GHz. The governor "powersave" may decide which speed to use within this range. current CPU frequency is 629 MHz. |
As we’ll see from the results below pitting “MeLE PCG35 Apo – Ubuntu 17.10” (with powersave) and “MeLE PCG35 Apo- Ubuntu 17.10 Performance” that the governor settings did not matter one bit on the results, at least for the six benchmarks I ran.
Note that “MeUbuntu 14.04.3” represents MeLE PCG02U TV stick running Ubuntu 14.04.3. Every platform runs a different OS and kernel, so keep in mind the results may differ slightly (up or down) with different version. But as we’ll see the differences in performance are large enough that it likely does not matter that much.
John the Ripper password cracker, a multi-threaded benchmark, shows the Apollo Lake processor is clearly ahead of Rockchip RK3399 hexa-core processor, and the fastest ARM platform, Banana Pi M3, is equipped with an Allwinner A83T octa-core Cortex A7 processor @ 2.0 GHz. The Bay Trail system is over twice as slow as the Apollo Lake one, also note the larg-ish standard deviation (+/- 83.72) due to some cooling problem in the small form factor.
C-Ray is another multi-threaded benchmark, and here Rockchip RK3399 SoC does fairly well, but still but quite as well as the Celeron J3455.
Smallpt, yet another multi-threaded benchmark, does not really change the order with MeLE PCG35 Apo well ahead.
Himeno, a linear solver of pressure Poisson, must be using some x86 specific instructions or optimizations, as Intel platforms are well ahead, with Celeron J3455 about 2.5x faster than Rockchip RK3399 board.
OpenSSL is the domain of Intel platforms likely benefiting from Advanced Encryption Standard instruction set (AES-NI). Performance improvement between Bay Trail and Apollo Lake is also impressive here. You’d need 10 Raspberry Pi 3 to match MeLE PCG35 Apo in this particular test.
Intel is normally better with SIMD accelerated multimedia application, and FLAC audio encoding (single threaded) confirms that.
I was expecting a close fight between Rockchip RK3399 and Celeron J3455, but RK3399 only has two fast Cortex A72 cores against four x86 cores in the Intel Apollo Lake SoC.
Jean-Luc started CNX Software in 2010 as a part-time endeavor, before quitting his job as a software engineering manager, and starting to write daily news, and reviews full time later in 2011.
Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress
Two small remarks: 1) On Intel systems with intel_pstate driver running (Sandy Bridge or above and obviously a somewhat recent kernel) powersave and performance are expected to give similar results in synthetic benchmarks (since ramping up CPU clockspeeds fast enough). But on other platforms depending on kernel version powersave vs performance can result in huge performance differences (since powersave can lead to the CPU cores remaining on lowest allowed clockspeed all the time while performance chooses the maximum as on Intel) 2) OpenSSL performance and AES-NI:if we’re talking about AES here then ARMv8 SoCs with AES crypto extensions (like RK3399… Read more »
For now I’ve been quite disappointed by RK3399’s performance. I managed to make my build farm run on the H96max at 1.8 GHz (A72) + 1.4 GHz (A53), and at 1.8 GHz, the A72 shows slightly lower performance than the RK3288 on the MiQi (ie 16.2 sec instead of 16.0 for a build). And that’s the best I can take out of it, it’s in 32-bit (armv7t or armv8l). In 64-bit (armv8), it’s about 10-15% slower for the same task. It was said that the A72’s architecture was very close to the A17, the former with 3-decode and 5-issue, the… Read more »
@willy
Those settop boxes traditionally have notoriously bad thermal design, and a 28nm A72 can get really hot at near-2GHz — have you monitored the throttling?
Why not add a comparative about power consumption? It would be interesting…
Thanks.
Unless it is like for like hardware the results have no place in the real world.
Most TV boxes watch TV and comparing a £30 less board against a £130 + Intel J3455 Apollo Lake. Is like racing a child’s peddle bike against a F1 Racer. Funny but no value.
How about benchmarking a Xbox One against a Intel J3455 Apollo Lake? It would have just as much real world use.
@blu Yes absolutely, and it’s really at 1.8 GHz while running such tests, and doesn’t throttle. BTW the advertised 1.5 and 2.0 GHz frequencies are not even part of the frequencies list at all, that’s the beauty of “up to”, it probably means that it can go “up to 2.0 GHz if you hack your DTB”. Honestly the perf is not bad at all and it heats less than the 3288 at the same performance level, it just doesn’t have enough cores to compete with it. The RK3288 has 4 strong cores, but the 3399 has only two, and the… Read more »
ARM is not power efficient, it’s just cheap in some TV box :
50$ A53 x4 cpu burn = 5Watt
200$ intel I3 5005U cpu burn = 15-20W
Intel is 10x more powerfull in benchmark, so 3X more efficient benchmark pt / watt.
Conclusion expensive Arm is a no go
@willy
I don’t get it. Your workload scales horizontally? And 2 x A72 are as fast as 4 x A17?
@bob
The real fact is most Arm SoC spend most of the time not running at the claimed top speed, they can only run very sort bursts.
Even over designed cooler cannot help.
@bob
There are no quad-core A53 that a 5005U would be 10x as powerful — the average IPC difference between Broadwell and A53 is approx 2-2.5 in favor of the former. There might be isolated cases where AVX2-centric scenarios can produce 10x difference, but how common are they?
btw, 410c is a severely underclocked SBC — its normal ‘stock-cooling’ clock is ~600MHz IIRC.
@theguyuk
Right about idle/low use cpu need, for that i don’t understand strategy of arm don’t provide hw encoder/decoder driver for a good power efficiency…it’s a shame for their business
And maybe we miss a compilation souce code benchmarck, because i don’t think my little H3 compile 10X slower than my intel cpu…
What are the RAM speeds and configurations on these boxes? I suspect that may tell as much or more of the story, especially if the benchmark is blowing out the icache on the ARMs.
@theguyuk
As mentioned in the post RK3399 and Apollo Lake devices are in the same price range.
@tkaiser
OpenSSL benchmark is missing on RK3399 because the test failed. If I remember correctly, the download failed.
@Jonathan
The Phoronix test suite as well as the results on openbenchmarking.org should be considered useless garbage especially when trying to compare different platforms in the way it’s done here. Most of the ‘benchmarks’ use questionable or no optimization settings at build time so results vary a lot with the OS used and the platform it’s running on (compiler shipped and default settings).
I’ve seen some of these ‘benchmarks’ performing 3 times faster on the same hardware just by exchanging the OS and thereby default compiler and settings (switching from GCC 4.x to 6.x for example).
@theguyuk
when thinking about (relatively) low power server I actually do think whenever should I go with arm and x86 and such benchmark did made sense for me.
You can argue like other commenters the metric used was not the best, but the idea itself? no. Look at the scores, they are clearly comparable, especially if you think about the price involved for the performance.
Which kind of server task(s) could be represented by the collection of synthetic benchmarks above? Really curious 🙂
@bob
That may be true for these cheap implementations of A53 cores. Have a look at Nvidia Jetson TX2 benchmarks. Phoronix did some a good while back. Same is true for Apple’s cores.
My results for Hikey 960
https://pastebin.com/twarU4eC
@m][sko
Himeno Benchmark 3.0 -> 1033.08
That’s really a big gap compared to other ARM platforms (5x faster than RK3399), and even faster than the Apollo Lake processor.
I wonder what may have caused the jump beside just the faster cores.
@m][sko
Can you please provide output from
@Jean-Luc Aufranc (CNXSoft)
Those benchmarks are extremely sensitive to compiler version and flags. One wrong flag and/or compiler version can make all the difference. It’s essential when doing synthetic benchmarking to know very well what a given compiler produces from the sources (e.g. ‘Hey, it’s not generating AES instructions!’, ‘Hey, it produced this snafu in the inner-most loop!’, etc)
Arm boards for the general use case have lost their initial promise. Lack of support from Arm, lack of proper software support from vendors, and the throttling and performance has left the promise in tatters. And the cost of higher performing boards, even the 3399, takes one into low power lower priced Intel platforms at which point its game over as the software support on Intel just cannot be compared. But for boards like the Pi or Odroid with better software support, at least things like a low power HTPC makes sense. And if the SOC provider decides to provide… Read more »
@raul
I would say that there are vendors who don’t suffer the problems you mention. ODROUD from Hard Kernel for one provides boards without cooling problems. They support their boards with software.
blu : Those benchmarks are extremely sensitive to compiler version and flags. One wrong flag and/or compiler version can make all the difference. And that’s the main reason no one with a brain in his head should use this terrible Phoronix crap. I used Jean-Luc’s last results to rely on and tested on a ROCK64: https://openbenchmarking.org/result/1710279-TY-1710254TY63 According to the Phoronix results ROCK64 shows just 41% OpenSSL performance compared to the octa core Banana Pi M3 and less than 9 percent compared to the Apollo Lake box (obviously the latter making use of AES-NI here). So obviously the Phoronix ‘benchmark’ uses… Read more »
@tkaiser OpenSSL testes on MeLE PCG35 Apo: openssl speed rsa4096 -multi 4 OpenSSL 1.0.2g 1 Mar 2016 built on: reproducible build, date unspecified options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) blowfish(idx) compiler: cc -I. -I.. -I../include -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -m64 -DL_ENDIAN -g -O2 -fdebug-prefix-map=/build/openssl-WoFyGJ/openssl-1.0.2g=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wa,--noexecstack -Wall -DMD32_REG_T=int -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM sign verify sign/s verify/s rsa 4096 bits 0.006233s 0.000094s 160.4 10645.9 1234567 openssl speed rsa4096 -multi 4OpenSSL 1.0.2g 1 Mar 2016built on: reproducible build, date unspecifiedoptions:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) blowfish(idx) compiler: cc -I.… Read more »
Found out how (.phoronix directory). Openssl Phoronix: ./openssl speed rsa4096 -multi 4 WARNING: can't open config file: /usr/local/ssl/openssl.cnf OpenSSL 1.0.1g 7 Apr 2014 built on: Thu Oct 26 09:06:50 +07 2017 options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx) compiler: gcc -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -Wa,--noexecstack -m64 -DL_ENDIAN -DTERMIO -O3 -Wall -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM sign verify sign/s verify/s rsa 4096 bits 0.004955s 0.000077s 201.8 12934.5 12345678 ./openssl speed rsa4096 -multi 4WARNING: can't open config file: /usr/local/ssl/openssl.cnfOpenSSL 1.0.1g 7 Apr 2014built on: Thu Oct 26 09:06:50 +07 2017options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)… Read more »
@Jean-Luc Aufranc (CNXSoft) Thank you. So on x64 we’re talking about: openssl speed rsa4096 -multi 4: Phoronix: 201.8 sign/s and 12934.5 verify/s Ubuntu 17.10: 160.4 sign/s and 10645.9 verify/s 123 openssl speed rsa4096 -multi 4: Phoronix: 201.8 sign/s and 12934.5 verify/sUbuntu 17.10: 160.4 sign/s and 10645.9 verify/s openssl speed -elapsed -evp aes-128-cbc: Phoronix: 436592.26k (16 bytes) 598267.22k (8KB) Ubuntu 17.10: 415316.34k (16 bytes) 571697.83k (8KB) 123 openssl speed -elapsed -evp aes-128-cbc: Phoronix: 436592.26k (16 bytes) 598267.22k (8KB)Ubuntu 17.10: 415316.34k (16 bytes) 571697.83k (8KB) With both tests the PTS binary scores even better than Ubuntu’s (but on Intel totally different compiler… Read more »
@tkaiser
Nice catch. Yes, that’s part of what I was talking about — in this particular case what phoronix has produced is literally apples to shotshells.
I know lots will think I am nuts for suggesting it, but if you want TV boxes and ARM SBC to improve, people need to be encouraged to play more 3D games on them.
Just look how demanding 3D games have pushed Game Console and Gaming PC hardware. The same market can push ARM hardware.
Does the average office really need a i7, i5 to write letters etc, no.
Network gear is a different market same as financial data, Movie editing etc
@tkaiser
No Thomas, I meant 2*A72 are exactly as fast as 2*A17, or about half as fast as 4*A17. Yes the build workload scales reasonably well (provided the memory bandwidth is there and the number of files to build is high enough and these files are about the same size). So for this workload, the RK3288 is almost twice as fast as the RK3399 just because it has twice the number of really usable cores.
@bob Power efficiency doesn’t compare like this because it decreases with peak performance. You need to compare the power drawn by a device capable of achieving a given peak performance. For example, my RK3288, while less power efficient than my Atom 8350, is significantly faster (15-20%). These two are the only ones in this thermal envelope capable of delivering more or less comparable performance. If I want higher performance on x86, I have to seek a significantly larger design (a much faster one) which will eat more power than the RK3288. It might be a bit more power efficient but… Read more »
I always understood that the main trick Intel use, is to do more things in memory, and have as much of the data as you can, as close to the CPU cores as possible.
Bigger faster cache and faster memory?
@theguyuk That’s true for high end CPUs, when the cache algorithm are very advanced, allowing almost instant access to any memory location. But having compared memory performance between a few Atoms and the RK3288 in dual-channel configuration shows a significant difference in favor of the latter on in-cache and RAM patterns. One feature helping x86 CPUs is the trace cache replacing the instruction cache. The principle is to store decoded and fused instructions, saving a few pipeline stages for hot code paths. This also allows some old CISC instructions (eg: “rep movs” for memcpy()) to be expanded to very efficient… Read more »
RK3399 is more like Celeron N series， since Celeron J series usually have a higher power consumption.
@blu Well, almost all of those Phoronix ‘benchmarks’ are totally broken since how they’re built and executed is not with providing useful numbers in mind but only focusing on ‘portability/compatibility’ so that PTS users being tricked into believing they would do serious benchmarking get all those tests compiled on their hosts regardless of platform, OS and environment. Isn’t it really ‘funny’ when I’m interested in encryption performance of an inexpensive ARM thingie and compare with a way more expensive Intel box that Phoronix is telling me the ARM thing would be 11 times slower since the ‘test suite’ only produces… Read more »
@tkaiser
Phoronix Suite shows the CFLAGS / CXXFLAGS used for compilation. For example for OpenSSL:
I’d assume all the rest is default. That would mean in the case of some of the results above that AES instructions are enabled by default for x86, but disabled for ARM (maybe
-march=armv8-ais required to enable those?).@willy
Re x86 i-cache — Atoms, at least up to Silvermont, don’t have either uop cache or trace cache (latter was something found in Netburst; former appeared in SNB). The only i-cache optimisation employed by Atoms (read: Silvermonts) is op boundary marks in the i-cache, and thus Atoms are often decode-bound (one of the reasons the Cortexes fair so well against them).
@Jean-Luc Aufranc (CNXSoft) Yes, PTS is using $some defaults in ‘fire and forget’ mode. Sometimes build settings are hard encoded in the makefile, sometimes relying on what the environment defines (so if for example EXTRAOPTFLAGS=’-O3′ in /etc/profile some ‘benchmarks’ will show magnitudes better scores later, eg. the infamous Smallpt stuff) and sometimes just using what the operating system’s default compiler does for whatever reasons. Please think about again: what PTS reports as ‘OpenSSL’ performance is RK3328 being 11 times slower than the J3455. While with another benchmark and when using a sanely built openssl binary the ARM SoC easily outperforms… Read more »
If you are interested: This is how it looks on a ODROID XU4 ( Exynos5422 Cortex™-A15 2Ghz and Cortex™-A7 Octa core CPU ) on a Ubuntu 16.04.3 LTS – Kernel 4.13.0 # openssl speed -elapsed -evp aes-128-cbc You have chosen to measure elapsed time instead of user CPU time. Doing aes-128-cbc for 3s on 16 size blocks: 14966032 aes-128-cbc’s in 3.00s Doing aes-128-cbc for 3s on 64 size blocks: 4189907 aes-128-cbc’s in 3.00s Doing aes-128-cbc for 3s on 256 size blocks: 1104424 aes-128-cbc’s in 3.00s Doing aes-128-cbc for 3s on 1024 size blocks: 279763 aes-128-cbc’s in 3.00s Doing aes-128-cbc for… Read more »
I’m sorry but talking about efficiency comparing two different fabs is nonsense, each platform is designed for what they are, everything said.
Danand : This is how it looks on a ODROID XU4 Same numbers as in the referenced link in my first comment above (we tried to collect numbers from A7, A9, A15, A17, A53 — with and without ARMv8 crypto stuff — and A72. Currently A17 and A72 still missing). With RSA signing the Exynos doesn’t look that bad but when it comes to AES encryption the SoC is far behind compared to those SoCs with ARMv8 crypto extensions, both considering ‘raw performance’ and especially ‘performance per Watt’. An ODROID XU4/HC1 doing AES stuff on the CPU cores compared to… Read more »
@tkaiser
Let me know if and what A72 benchmarking you need – I have one macchiatobin idling here.
@blu
Would be great to get full output from
Compiler and version the binaries were built with are important of course. And if you can results from github.com/ssvb/tinymembench would also be great.
@tkaiser On H96-Max, I get this on the 2 [email protected] GHz : # openssl version -a OpenSSL 1.0.2k 26 Jan 2017 built on: reproducible build, date unspecified platform: linux-aarch64 options: bn(64,64) rc4(ptr,char) des(idx,cisc,16,int) idea(int) blowfish(ptr) compiler: aarch64-gcc47l_glibc218-linux-gnueabi-gcc -I. -I.. -I../include -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -O3 -Wall -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM OPENSSLDIR: “/usr/share/openssl” (built on linaro gcc-4.7.4, not rebuilt since). # taskset -c 4,5 openssl speed rsa4096 -multi 2 sign verify sign/s verify/s rsa 4096 bits 0.019186s 0.000257s 52.1 3887.3 And this on the 4 A53 cores at 1.4 GHz : # taskset -c 0-3 openssl speed rsa4096 -multi 4… Read more »
Model: MACCHIATOBin-8040
Clock: CPU 1300 [MHz]
DDR 800 [MHz]
FABRIC 800 [MHz]
MSS 200 [MHz]
macchiato:~$ ./gov.sh # columns: core governor clock
0 performance 1300000
1 performance 1300000
2 performance 1300000
3 performance 1300000
tinymembench: https://pastebin.com/bg1Wha41
openssl: https://pastebin.com/WRnVa07L
7z: https://pastebin.com/Z6RFZLkh
bench mark we want to see is huawei fastest versus apple latest
@blu Thank you! This is DDR4, right? As a quick reference Jean-Luc’s tinymembench numbers for RK3399 from few weeks ago. And the 7-zip numbers are the only ‘generic CPU’ benchmark resuilts I need for my use cases (server stuff). A72 with ARMv8 crypto extensions performs really impressive, this time also with very small chunks of data (the A53 implementation suffers here somewhat). @willy I put your AES encryption numbers and the Celeron results together: type 16 bytes 64 bytes 256 bytes 1024 bytes AES-NI 436592.26k 549185.24k 554032.38k 588775.08k A72 465434.04k 938648.81k 1221223.85k 1300021.93k A53 170812.96k 464463.87k 783981.14k 982121.81k 1234 type 16… Read more »
@tkaiser
Yep, DDR4 at the lowest rate (1600) — board and DIMM are capable of 2400, but I keep mine at the lowest setting for minimal active cooling (side-mounted 80mm silent fan).
On an unrelated note, Teres-A64 acquired! ; ]
@tkaiser
Maybe file a bug against PTS instead of just complaining on a random article on another site?
PTS is mostly used/developed on x86_64 so they probably aren’t even aware it doesn’t work properly (much slower than distro defaults) on arm.
@Anonymous
He’s commented over there on a number of articles when this topic has come up. Nothing has come from it. Michael shows no interest in fixing PTS.
So, instead @tkaiser is addressing it where it’s used so that people there can be aware of what junk it is.
And it really helps to think about Phoronix Media’s business model to understand why this will never change. 🙂
PTS is great in a lot of areas (especially presenting/visualizing and comparing numbers) but unfortunately the first step to collect useful data and not just garbage is totally flawed. Which is understandable having PTS target audience in mind (people preferring data over information and happy looking at nice graphs representing numbers without meaning).
IMO most important lesson to learn: Switch from passive to active benchmarking!
run the benchmarks on rock960, some improvements over the other rk3399.
https://www.96rocks.com/blog/2017/11/02/rock960-linux-benchmarks/
https://openbenchmarking.org/result/1711019-TY-1709271TY72
@vamrs
For whatever reason, John the ripper seems to be very sensitive to CPU frequency.
Were the Cortex A72 cores clocked at 1.8 GHz or higher in your boards?
With regards to AES on OpenSSL, the important flags appear to be -DAES_ASM -DVPAES_ASM -DBSAES_ASM for hardware crypto, or SIMD accelerations
The implementations are shown @ https://github.com/openssl/openssl/tree/master/crypto/aes/asm
Even 32-bit ARM (ARMv7) would be accelerated, as while there’s no AES instructions, it would still be accelerated with NEON if the SoC supports it.
ARMv8 does not mean AES crypto instructions are enabled, as at least one Cortex A53 based SoC does not have those.
cnxsoft : @vamrs For whatever reason, John the ripper seems to be very sensitive to CPU frequency. Nope, the problem is that for whatever reasons people try to use a tool that graphs numbers without meaning (PTS) as performance ‘benchmark’. These are the settings your binaries have been built with --build=arm-linux-gnueabihf --disable-browser-plugin --disable-libitm --disable-libquadmath --disable-sjlj-exceptions --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-nls --enable-objc-gc=auto --enable-plugin --enable-shared --enable-threads=posix --host=arm-linux-gnueabihf --program-prefix=arm-linux-gnueabihf- --target=arm-linux-gnueabihf --with-arch-directory=arm --with-arch=armv7-a --with-default-libstdcxx-abi=new --with-float=hard --with-fpu=vfpv3-d16 --with-mode=thumb --with-target-system-zlib -v 12345678910111213141516171819202122232425262728293031 --build=arm-linux-gnueabihf--disable-browser-plugin--disable-libitm--disable-libquadmath--disable-sjlj-exceptions--enable-checking=release--enable-clocale=gnu--enable-default-pie--enable-gnu-unique-object--enable-gtk-cairo--enable-java-awt=gtk--enable-java-home--enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++--enable-libstdcxx-debug--enable-libstdcxx-time=yes--enable-multiarch--enable-nls--enable-objc-gc=auto--enable-plugin--enable-shared--enable-threads=posix--host=arm-linux-gnueabihf--program-prefix=arm-linux-gnueabihf---target=arm-linux-gnueabihf--with-arch-directory=arm--with-arch=armv7-a--with-default-libstdcxx-abi=new--with-float=hard--with-fpu=vfpv3-d16--with-mode=thumb--with-target-system-zlib -v And these are @vamrs settings: --build=aarch64-linux-gnu --disable-browser-plugin --disable-libquadmath --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-fix-cortex-a53-843419 --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk… Read more »
It seems Amlogic are the greatest ‘offender’ here, as even Allwinner’s A53 have the crypto extensions.
Do we know exactly which SoCs by Amlogic lack the crypto extensions, besides the S905?
@blu
Don’t know, I just saw that in the code. They did not name names, just:
Maybe S905 (and variants?) SoC is the only one.
@Jean-Luc Aufranc (CNXSoft)
Now I can see S905 has it own crypto module according to: https://forum.odroid.com/viewtopic.php?t=22236#p151854
Yes, the crypto stuff is optional with A53 (BTW: it’s not only AES). Samsung/Nexell S5P6818 (A53) comes without, same with BCM2837 on RPi 3 and Amlogic S905 (while S905X is said to have crypto extensions enabled/licensed).
cnxsoft : @Jean-Luc Aufranc (CNXSoft) Now I can see S905 has it own crypto module The problem with those vendor proprietary crypto modules is how to get them working and how do they perform (especially with small chunk sizes that might be typical for encryption/decryption of VPN traffic). If you check Banana Pi R2 AES performance in the referenced link in my first comment above you see that MediaTek’s own crypto module performs very poorly with small chunk sizes. Also it’s PITA to get this stuff working since you would need to rebuild the kernel to make use of cryptodev,… Read more »
AES performance and A53 without crypto extensions (Raspberry Pi 3 / BCM 2837): Two times tested, first time with RPi foundation software (32-bit kernel ‘4.9.59-v7+ #1047 SMP Sun Oct 29’) and Raspbian Stretch userland (build ‘optimization’ for ARMv6!), then with a true 64-bit OS (64-bit kernel ‘4.11.12-pi64+ #1 SMP PREEMPT Sun Jul 30’ and original Debian Stretch arm64 userland). OpenSSL distro package (OpenSSL 1.1.0f 25 May 2017) used with ‘openssl speed -elapsed -evp aes-128-cbc’ built for 16 bytes 64 bytes 256 bytes 1024 bytes ARMv6 30556.36k 41850.11k 46540.89k 47880.87k ARMv8 31186.04k 47189.70k 52744.87k 54331.73k 123 built for 16 bytes 64 bytes 256… Read more »
@Jean-Luc Aufranc (CNXSoft)
The S905 in the ODROID C2 is one such AARCH64 processor without AES. If you need numbers from one, I would be glad to provide them if you don’t have one.
@willmore I use your C2 OpenSSL numbers below you posted to Armbian forum a while back. All results done with ‘openssl speed -elapsed -evp aes-128-cbc’ using arm64 Binaries on 64-bit platforms (since otherwise numbers are lower, see RPi 3 example above): Board MHz 16 bytes 64 bytes 256 bytes 1024 bytes BPi R2 1040 22082.67k 25522.92k 26626.22k 26912.77k BPi M3 1800 36122.38k 43447.94k 45895.34k 46459.56k Clearfog 1600 47352.87k 54746.43k 57855.57k 58686.12k XU4 2000 78690.05k 89287.85k 94056.79k 95104.34k MiQi 2000 87295.72k 94739.03k 98363.39k 99325.95k RPi 3 1200 31186.04k 47189.70k 52744.87k 54331.73k NPi M3 1400 44264.22k 54627.49k 58849.88k 59756.35k ODROID C2 1752… Read more »
blu : Do we know exactly which SoCs by Amlogic lack the crypto extensions, besides the S905? S905 lacks them while S905X and S912 have them enabled. It’s easy to check using the results searchable at browser.geekbench.com of another flawed benchmark collecting also mostly numbers without meaning: Geekbench. Forget about the ‘total scores’ since total BS but check detailed results and there AES numbers. AML-S905X-CC AKA ‘Le Potato’ gets a multi-core AES score of 1648 points (1.24 GB/sec), a Beelink GT1 2739 points (2.06 GB/sec) while a Raspberry Pi 3 for example scores just 79 points (61.0 MB/sec). The difference… Read more »
I still can’t figure out what thought process goes in the head of vendors who decide that:
1. Their SoC will target crypto applications, yet
2. They won’t license the proven and easilly accessible aarch64 crypto extensions, instead they’ll provide some proprietary engine.
@blu Well, at least Amlogic corrected this later and both S905X and S912 can be considered being the fastest AES/SHA crypto performers with A53 cores (since 1.5GHz clockspeed). Some new results (I partially fail to understand): https://forum.armbian.com/topic/5570-some-basic-benchmarks-for-le-potato/?tab=comments#comment-42725 Wrt proprietary crypto modules it depends. Marvell’s CESA for example is able to increase the above AES performance numbers for Clearfog (Cortex-A9) by 3 to 4 times. It’s also said that more recent ARMv8 Marvell SoCs like Armada 37xx (EspressoBin) contain a new ‘Marvell 5Gbps Security Engine’ (do a web search for) outperforming both CESA and the already licensed ARMv8 crypto extensions with… Read more »
@blu
They probably added that crypto module back in ARMv5 or so and there’s that *one* customer who uses it, so they just carry it along and that customer is happy.
Crypto customers are either silly middle managers who see ‘crypto’ in the list of features and check a box saying “yep, has crypto” or the legacy people mentioned above.