Rockchip RK3399 powered NanoPi R4S router SBC launched at the beginning of the month, and FriendlyELEC kindly sent a review sample to CNX Software. I intended to test thermally performance, Ethernet, and USB like I did for NanoPi R2S and NanoPi NEO3, but Armbian is not available right now, so I could not use some of the tools I normally used right now.
So instead, I tested the board/gateway with the image from FriendlyELEC. First FriendlyCore based on Ubuntu Core 20.04, but there some issues which we’ll detail in this preview, so I then switch to FriendlyWrt built upon OpenWrt 19.07 which works better, but I still encountered some problems. That’s just to say it might be better to wait a little longer until Armbian images are released, or until FriendlyELEC fixes some of the shortcomings.
NanoPi R4S gateway unboxing
Before testing the software, let’s see what I’ve received.
NanoPi R4S SBC inside its metal enclosure together with a 16GB class A1 microSD card that, as I found later, comes preloaded with FriendlyWrt.
The rear panel includes a USB-C port for power, WAN and LAN Gigabit Ethernet ports, and a reset button.
Yes, it is! I could mount the gateway on a camera’s tripod. This should allow for some innovative and inexpensive mounting options…
Before I teardown the device, let’s have a little family reunion with from left to right: NanoPi NEO3, NanoPi R2S, and NanoPi R4S. The latter is quite bigger than the other two.
NanoPi R4S “teardown”
Let’s open the enclosure. After taking out the four rubber pads and loosening the four screws on the bottom of the enclosure we can access the board.
After removing two more screws, we can completely take out the board and see the processor is in contact with the metal case through a thermal pad, as it should be.
FriendlyCore 20.04 (Ubuntu Core)
I initially thought the microSD card was blank so I went to the Wiki and downloaded the latest version of FriendlyCore namely rk3399-sd-friendlycore-focal-4.19-arm64-20201027.img and flash it to the MicroSD card using USBImager. I then inserted the card into NanoPi R4S, connected Ethernet cables to the WAN and LAN ports, and powered it using MINIX NEO P2 USB-C power adapter.
I can see the power LED (red) is on, and the status LED (green) is blinking, but the LAN and WAN LEDs are off. I go to my router’s web interface to check for new devices and find out the IP and nothing. Maybe the first boot is quite long, but five minutes later still no IPs. Finally, I decided to power cycle the board, it got an IP address, and I could finally access it via SSH using the default pi/pi credentials.
Let’s check out the details with inxi:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 |
sudo inxi -Fc0 System: Host: NanoPi-R4S Kernel: 4.19.111 aarch64 bits: 64 Console: tty 0 Distro: Ubuntu 20.04.1 LTS (Focal Fossa) Machine: Type: ARM Device System: FriendlyElec NanoPi R4S details: N/A serial: f748058df14cbaed Battery: ID-1: test_battery charge: 100% condition: N/A CPU: Topology: 6-Core (2-Die) model: N/A variant-1: cortex-a53 variant-2: cortex-a72 bits: 64 type: MCP MCM Speed: 1200 MHz min/max: 408/1416:1800 MHz Core speeds (MHz): 1: 408 2: 408 3: 408 4: 408 5: 1608 6: 1608 Graphics: Device-1: display-subsystem driver: rockchip_drm v: N/A Device-2: rk3399-dw-hdmi driver: dwhdmi_rockchip v: N/A Device-3: malit860 driver: mali v: N/A Display: server: X.org 1.20.8 driver: mali tty: 79x23 Message: Advanced graphics data unavailable in console for root. Audio: Device-1: rk3399-dw-hdmi driver: dwhdmi_rockchip Device-2: simple-audio-card driver: asoc_simple_card Sound Server: ALSA v: k4.19.111 Network: Device-1: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet driver: r8169 IF: eth1 state: down mac: d2:ed:28:cf:85:22 Device-2: rk3399-gmac driver: rk_gmac_dwmac IF-ID-1: eth0 state: up speed: 1000 Mbps duplex: full mac: 80:1f:12:fc:2f:96 Drives: Local Storage: total: 14.84 GiB used: 100.4 MiB (0.7%) ID-1: /dev/mmcblk1 model: SC16G size: 14.84 GiB Partition: ID-1: / size: 11.11 GiB used: 100.4 MiB (0.9%) fs: overlay source: ERR-102 Sensors: System Temperatures: cpu: 26.0 C mobo: N/A Fan Speeds (RPM): N/A Info: Processes: 147 Uptime: 1h 01m Memory: 3.75 GiB used: 206.1 MiB (5.4%) Init: systemd runlevel: 5 Shell: bash inxi: 3.0.38 |
Both Ethernet devices are detected but eth1 down. However even if we turn it on, there’s no link detected:
|
1 2 3 |
pi@NanoPi-R4S:~$ sudo ifconfig eth1 up pi@NanoPi-R4S:~$ sudo mii-tool eth1 eth1: no link |
I thought maybe it could be configured with npi-config that’s supposed to be pre-installed in FriendlyCore, but:
|
1 2 |
pi@NanoPi-R4S:~$ sudo npi-config sudo: npi-config: command not found |
We’ll also notice the temperature is odd at just 26°C, but the same value is reported by sbc-bench.sh:
|
1 2 3 4 5 |
sudo ./sbc-bench.sh -m [sudo] password for pi: Time big.LITTLE load %cpu %sys %usr %nice %io %irq Temp 09:14:27: 1608/ 600MHz 1.86 1% 0% 0% 0% 0% 0% 26.0°C 09:14:32: 1800/1416MHz 1.79 18% 8% 2% 0% 5% 1% 26.0°C |
The temperature measured with an IR thermometer is around 39°C.
That’s truly magic! Now if I look at sysfs, we can see three temperature zones:
|
1 2 3 4 5 6 |
pi@NanoPi-R4S:~$ cat /sys/devices/virtual/thermal/thermal_zone0/temp 34444 pi@NanoPi-R4S:~$ cat /sys/devices/virtual/thermal/thermal_zone1/temp 34444 pi@NanoPi-R4S:~$ cat /sys/devices/virtual/thermal/thermal_zone2/temp 26000 |
zone2 appears to be what is used now, but it’s clearly hard-coded to 26°C. However sbc-bench script checks for another file before going into /sys/devices/ directory:
|
1 2 |
cat /sys/class/hwmon/hwmon0/temp1_input 26000 |
That’s also hard-coded to 26C, so I’ll comment the part related to this file in the script, making it use thermal_zone0 instead.
|
1 2 3 |
sudo ./sbc-bench.sh -m Time big.LITTLE load %cpu %sys %usr %nice %io %irq Temp 04:57:34: 1416/ 408MHz 0.00 0% 0% 0% 0% 0% 0% 35.6°C |
It looks better. Let’s run the benchmark:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
sudo ./sbc-bench.sh -c sbc-bench v0.7.5 Installing needed tools. This may take some time... Done. Checking cpufreq OPP... Done. Executing tinymembench. This will take a long time... Done. Executing OpenSSL benchmark. This will take 3 minutes... Done. Executing 7-zip benchmark. This will take a long time... Done. Executing cpuminer. This will take 5 minutes... Done. Checking cpufreq OPP... Done. Memory performance (big.LITTLE cores measured individually): memcpy: 1625.4 MB/s memset: 8431.7 MB/s (1.0%) memcpy: 3589.5 MB/s (1.2%) memset: 8545.4 MB/s (2.3%) Cpuminer total scores (5 minutes execution): 10.39,10.38,10.37,10.36,10.35,10.34,10.33 kH/s 7-zip total scores (3 consecutive runs): 5690,5844,5787 OpenSSL results (big.LITTLE cores measured individually): type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes aes-128-cbc 117574.08k 354357.06k 696504.83k 946934.10k 1056814.42k 1063796.74k aes-128-cbc 345815.47k 794550.68k 1154592.77k 1280458.07k 1344244.39k 1352466.43k aes-192-cbc 112159.06k 315819.05k 570631.51k 730835.63k 796188.67k 799899.65k aes-192-cbc 330109.00k 726448.43k 982510.25k 1135289.00k 1181499.39k 1186491.05k aes-256-cbc 109038.23k 291820.14k 495495.59k 611941.38k 657126.74k 657926.83k aes-256-cbc 319159.24k 663780.86k 902803.03k 978935.81k 1016075.61k 1018975.57k Full results uploaded to http://ix.io/2HEU. Please check the log for anomalies (e.g. swapping or throttling happenend) and otherwise share this URL. |
No throttling was detected, and for instance, 7-zip scores (~5800) are roughly the same as for RockPi 4C with the Rockchip RK3399 clocked a 1.8/1.4GHz for both SBC.
Since we can’t install armbianmonitor, there’s no pretty chart, but we can still check out the log above and temperature for 7-zip multi-core and cpuminer never exceeded 60C:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
System health while running 7-zip multi core benchmark: Time big.LITTLE load %cpu %sys %usr %nice %io %irq Temp 05:47:07: 1800/1416MHz 5.16 0% 0% 0% 0% 0% 0% 45.6°C 05:47:28: 1800/1416MHz 5.41 88% 1% 87% 0% 0% 0% 50.6°C 05:47:48: 1800/1416MHz 5.70 83% 1% 82% 0% 0% 0% 50.6°C 05:48:11: 1800/1416MHz 5.54 81% 1% 79% 0% 0% 0% 50.6°C 05:48:31: 1800/1416MHz 5.45 76% 1% 74% 0% 0% 0% 48.1°C 05:48:55: 1800/1416MHz 5.47 90% 1% 88% 0% 0% 0% 51.1°C 05:49:16: 1800/1416MHz 5.71 85% 1% 84% 0% 0% 0% 52.2°C 05:49:38: 1800/1416MHz 5.60 83% 0% 82% 0% 0% 0% 51.7°C 05:50:01: 1800/1416MHz 5.86 82% 1% 80% 0% 0% 0% 52.2°C 05:50:22: 1800/1416MHz 5.85 74% 1% 72% 0% 0% 0% 50.0°C 05:50:42: 1800/1416MHz 5.82 96% 1% 94% 0% 0% 0% 52.2°C 05:51:04: 1800/1416MHz 5.69 85% 1% 84% 0% 0% 0% 52.2°C 05:51:25: 1800/1416MHz 5.58 84% 1% 83% 0% 0% 0% 53.3°C 05:51:47: 1800/1416MHz 5.43 81% 1% 79% 0% 0% 0% 53.3°C 05:52:08: 1800/1416MHz 5.35 76% 1% 74% 0% 0% 0% 50.6°C 05:52:31: 1800/1416MHz 5.68 91% 2% 88% 0% 0% 0% 52.8°C System health while running cpuminer: Time big.LITTLE load %cpu %sys %usr %nice %io %irq Temp 05:52:37: 1800/1416MHz 5.48 0% 0% 0% 0% 0% 0% 50.0°C 05:52:59: 1800/1416MHz 5.63 99% 0% 99% 0% 0% 0% 56.1°C 05:53:22: 1800/1416MHz 5.73 100% 0% 99% 0% 0% 0% 56.1°C 05:53:44: 1800/1416MHz 5.88 100% 0% 99% 0% 0% 0% 56.1°C 05:54:07: 1800/1416MHz 5.92 100% 0% 99% 0% 0% 0% 56.7°C 05:54:30: 1800/1416MHz 6.16 100% 0% 99% 0% 0% 0% 57.2°C 05:54:53: 1800/1416MHz 6.12 100% 0% 99% 0% 0% 0% 57.2°C 05:55:16: 1800/1416MHz 6.13 100% 0% 99% 0% 0% 0% 57.2°C 05:55:39: 1800/1416MHz 6.14 100% 0% 99% 0% 0% 0% 58.3°C 05:56:02: 1800/1416MHz 6.10 100% 0% 99% 0% 0% 0% 58.3°C 05:56:25: 1800/1416MHz 6.12 100% 0% 99% 0% 0% 0% 58.3°C 05:56:47: 1800/1416MHz 6.09 100% 0% 99% 0% 0% 0% 58.3°C 05:57:10: 1800/1416MHz 6.11 100% 0% 99% 0% 0% 0% 58.3°C 05:57:33: 1800/1416MHz 6.08 100% 0% 99% 0% 0% 0% 59.4°C |
That means cooling is working very well, and I suppose running the CPU at 2.0 GHz might also be fine. By comparison, NanoPi R2S reached 85°C and throttled a little during the same test. Note this is winter here, so while the ambient temperature was 30°C for R2S test against around 24°C now.
Let’s check the Gigabit Ethernet port that works (eth0) with iperf using a full-duplex transfer:
|
1 2 3 4 5 6 7 |
Client connecting to 192.168.1.2, TCP port 5001 TCP window size: 298 KByte (default) ------------------------------------------------------------ [ 6] local 192.168.1.4 port 51780 connected with 192.168.1.2 port 5001 [ ID] Interval Transfer Bandwidth [ 4] 0.0-60.0 sec 3.07 GBytes 439 Mbits/sec [ 6] 0.0-60.0 sec 6.14 GBytes 878 Mbits/sec |
- Upload only:
|
1 2 3 4 5 6 |
Client connecting to 192.168.1.4, TCP port 5001 TCP window size: 391 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.1.2 port 38860 connected with 192.168.1.4 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 6.44 GBytes 921 Mbits/sec |
- Download only:
|
1 2 3 4 5 6 |
Client connecting to 192.168.1.2, TCP port 5001 TCP window size: 340 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.1.4 port 51820 connected with 192.168.1.2 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 6.57 GBytes 941 Mbits/sec |
FriendlyWrt (OpenWrt)
FriendlyCore has some issues, and limited information in NanoPi R2S Wiki, so instead let’s try FriendlyWrt, and I flashed rk3399-sd-friendlywrt-5.4-20201111.img.zip to the MicroSD card. I did not get any problems to get an IP address this time, and the WAN and LED are working as expected (green ON) on the board. I could login as root without any password using SSH:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
ssh root@192.168.1.2 The authenticity of host '192.168.1.2 (192.168.1.2)' can't be established. RSA key fingerprint is SHA256:BPBd3bkZXAHzM6S7s8oE883esQlNccHuuOuJzlqVADM. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added '192.168.1.2' (RSA) to the list of known hosts. BusyBox v1.30.1 () built-in shell (ash) ___ _ _ _ __ _____ _____ | __| _(_)___ _ _ __| | |_ \ \ / / _ \_ _| | _| '_| / -_) ' \/ _` | | || \ \/\/ /| / | | |_||_| |_\___|_||_\__,_|_|\_, |\_/\_/ |_|_\ |_| |__/ ----------------------------------------------------- FriendlyWRT 19.07.4, r11208-ce6496d796 ----------------------------------------------------- === WARNING! ===================================== There is no root password defined on this device! Use the "passwd" command to set up a new password in order to prevent unauthorized SSH logins. -------------------------------------------------- |
But instead of carrying on with the command, I went on my laptop web browser to access LuCi web interface instead.
We can also see how the LAN and WAN ports are configured. The WAN port is a DHCP and DHCPv6 client…
But for testing purpose, I deleted the bridge, and set eth1 LAN port as DHCP client so I could get an IP on the same network:
|
1 2 3 4 5 6 7 |
Client connecting to 192.168.1.2, TCP port 5001 TCP window size: 484 KByte (default) ------------------------------------------------------------ [ 6] local 192.168.1.4 port 54200 connected with 192.168.1.2 port 5001 [ ID] Interval Transfer Bandwidth [ 4] 0.0-60.0 sec 5.56 GBytes 796 Mbits/sec [ 6] 0.0-60.0 sec 5.90 GBytes 845 Mbits/sec |
and repeated the same with the LAN port:
|
1 2 3 4 5 6 7 |
Client connecting to 192.168.1.10, TCP port 5001 TCP window size: 246 KByte (default) ------------------------------------------------------------ [ 6] local 192.168.1.4 port 42058 connected with 192.168.1.10 port 5001 [ ID] Interval Transfer Bandwidth [ 4] 0.0-60.0 sec 6.01 GBytes 860 Mbits/sec [ 6] 0.0-60.0 sec 5.83 GBytes 835 Mbits/sec |
All good. I’ve also tested the USB 3.0 port by connecting a USB 3.0 hard drive on the port next to the MicroSD card, but I got some errors:
|
1 2 3 4 5 6 7 8 9 |
[ 4016.615349] usb usb8-port1: Cannot enable. Maybe the USB cable is bad? [ 4017.582838] usb usb8-port1: Cannot enable. Maybe the USB cable is bad? [ 4018.534752] usb usb8-port1: Cannot enable. Maybe the USB cable is bad? [ 4019.486422] usb usb8-port1: Cannot enable. Maybe the USB cable is bad? [ 4020.438725] usb usb8-port1: Cannot enable. Maybe the USB cable is bad? [ 4021.390783] usb usb8-port1: Cannot enable. Maybe the USB cable is bad? [ 4022.342405] usb usb8-port1: Cannot enable. Maybe the USB cable is bad? [ 4023.294942] usb usb8-port1: Cannot enable. Maybe the USB cable is bad? [ 4024.247236] usb usb8-port1: Cannot enable. Maybe the USB cable is bad? |
So I switched to the other USB 3.0 port, and it worked just fine:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
[ 4501.341310] usb 6-1: new SuperSpeed Gen 1 USB device number 2 using xhci-hcd [ 4501.367210] usb 6-1: New USB device found, idVendor=0bc2, idProduct=2312, bcdDevice= 6.36 [ 4501.368004] usb 6-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3 [ 4501.368674] usb 6-1: Product: Expansion [ 4501.369152] usb 6-1: Manufacturer: Seagate [ 4501.369561] usb 6-1: SerialNumber: NA495NNC [ 4501.390736] scsi host0: uas [ 4501.394727] scsi 0:0:0:0: Direct-Access Seagate Expansion 0636 PQ: 0 ANSI: 6 [ 4501.401310] sd 0:0:0:0: [sda] Spinning up disk... [ 4504.889125] .ready [ 4504.891192] sd 0:0:0:0: [sda] 1953525167 512-byte logical blocks: (1.00 TB/932 GiB) [ 4504.892476] sd 0:0:0:0: [sda] Write Protect is off [ 4504.893052] sd 0:0:0:0: [sda] Mode Sense: 2b 00 10 08 [ 4504.894397] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 4504.948967] sda: sda1 sda2 sda3 sda4 [ 4504.957469] sd 0:0:0:0: [sda] Attached SCSI disk |
I then configured a SAMBA share in LuCi and the command line, to transfer files from my laptop to the USB drive connected to NanoPi R4S.
It works, but at around 16MB/s it’s really slow especially for a USB 3.0 drive. Other Arm-based platforms with Gigabit Ethernet and USB 3.0 running OpenWrt could achieve close to 50MB/s in the same test (and same USB hard drive). Maybe it’s just a question of software optimization.
Conclusion
NanoPi R4S has been better designed than the previous NanoPi R2S notably in terms of thermal design, as the board, while fitted in the metal case, never exceeded 60°C under load. Networking performance seems to be fine too. However, it looks like FriendlyELEC focused on OpenWrt image (FriendlyWrt) more than on Ubuntu Core for now which has a few issues. I also had one issue with one of the USB 3.0 ports but it may be transient. If you plan on using OpenWrt with the router/gateway, then it may be good, but for Debian based Linux distributions waiting for Armbian images may be advised both for stability and improved performance.
I’d like to thank FriendlyELEC for sending NanoPi R4S for review, and if interested you can get the model with 4GB RAM and metal enclosure as reviewed here for $69 plus shipping on FriendlyELEC store or Aliexpress.
Jean-Luc started CNX Software in 2010 as a part-time endeavor, before quitting his job as a software engineering manager, and starting to write daily news, and reviews full time later in 2011.
Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress
Armbian images for NaniPi M4 V2 might work out of the box (most probably same LPDDR DRAM initialization). And if you do the test I would choose mainline kernel (FriendlyELEC obviously uses Rockchip’s ‘new’ 4.19 BSP kernel branch and maybe that one is not fully up to the task yet).
Real support might take a while. Orders are still in the mail….
WIP: https://github.com/armbian/build/pull/2415
Might be worth a try to use m4 images indeed.
http://www.armbian.com/nanopi-m4-v2/ Armbian Focal does not boot (tested on 4GB version).
armbian.com/rockpro64/ on the other hand does boot.
debian linux 11.3 arm64: nanopi-r4s native build
https://github.com/inindev/nanopi-r4s/releases/tag/v11.3
Ethernet performance is very poor for something that’s being called a router.
Samba performance is terrible and could easily be improved by better software tuning. Been involved in tuning a router in the past and it went from 25MB/s to 100MB/s.
Poor Samba performance is most probably due to combination of UAS with Seagate USB3 disks and missing IRQ affinity settings (both Ethernet and XHCI interrupts being processed on the little cpu0 core where maybe also the smbd task is scheduled to). Can be checked easily with htop and by looking at /proc/interrupts after such a benchmark attempt.
USB-C power port act as OTG or not?
R2S does. However R4S?
I’ve just connected MINIX USB-C hub with SSD, and the SSD is not recognized, so no OTG support I’m afraid.
I wonder what can be a real use case for such a form factor?
For a NAS, it misses SATA connectivity like odroid HC4 (which is multimedia with HDMI).
For a router, 1 input, 1 output: I call it a cable and it is cheaper.
An ethernet repeater with firewall functionality: the topology is limited with only 1 output port and why use such a powerful multimedia chip for this?
I must be missing something.
“For a router, 1 input, 1 output: I call it a cable and it is cheaper.”
This device can do routing stuff: NAT, firewall, IPv6 tunneling.
NAS is feasible as it has USB3. No fantastic speeds, but maybe enough for medium usage. I have a very small Intel box, with a 5TB 2.5 USB disk connected, and it works as a NAS for me.
This is placed between your VDSL router and your WiFi router and GbE switch. You disable everything on your router and setup your network on the Nanopi-R4S, configure your VPNs, firewall, DHCP and network rules using OpenWRT. Now your network configuration is completely independent from your router. You can switch your provider and router at any moment and don’t care if your router supports the services you need.
I bought it to replace my 1Gb fiber french ISP box consuming 20watts at idle. Ten times this r4s ! To remove the ISP box, I need WAN side VLAN tagged packets, kind of DHCP specific config, 6to4 tunnelling for ipv4 traffic, etc, the r4s will handle that on wan side. Also I have a manageable switch in which I want to setup separated vlans + firewall rules (separated guest wifi, children controlled wan accès + specific dns, home automation critical subnet, dmz….) Last but not least, I want a remote access using wireguard. And if possible vpn also for… Read more »
Hi,
Very interresting, thanks for that.
Would you have any info about its power consumption (at idle and under heavy load) ?
From my measurements it’s 0.44A at 5V idle and 1.0A at 5V at load.
2.2W idle is high. And what does ‘at load’ mean?
I’ve only tested the device as a router, so the load means both interfaces working at max throughput for 2 mins. You can have a look here if you’re interested.
I believe the increased consumption is due to the fact that the docker daemon runs in the background (OpenWRT) and uses 20-30% of the CPU at idle.
Thank you for the article. I wonder how IRQ affinity settings in the FriendlyWRT image look like (checking /proc/interrupts after a while).
I’ve tried first to leave it idle for 10 mins, then only test the WAN and then the bridged network (WAN+LAN 1Gbps). It seems that everything goes only through CPU0. This is the /proc/interrupts after all tests.
https://pastebin.com/TNS8H3p6
> It seems that everything goes only through CPU0 As expected. This explains also a lot wrt poor Samba performance (where UAS blacklisting of the Seagate disk is missing too). Once you send eth0 and eth1 IRQs to cpu2/cpu3 (with appropriate tunables for RPS – Receive Packet Steering – as well) performance with smaller packet sizes might improve. Maybe with small packet sizes it’s necessary to move the stuff to the two big cores which are almost twice as fast as the A53 cores with such tasks. At least avoiding the artificial bottleneck on cpu0 would already be a great… Read more »
Thanks for the info regarding RPS, this seems interesting. I will try to test this. Tbh, this sounds more interesting to test on the Nanopi-R2S, because the R4S is buffed enough as it is to handle the bridged interface even like that. I may try to test on both for comparison.
OK, cool. I’ve actually seen significant difference by moving the IRQs and the RPS to CPU4 for eth0 and CPU5 for eth, which are the 1.8GHz cores. This is the script I’ve used: # CPU4 and CPU5 are the 1.8GHz cores # Set CPU4 to handle eth0 IRQs echo 10 > /proc/irq/27/smp_affinity echo 10 > /sys/class/net/eth0/queues/rx-0/rps_cpus # Set CPU5 to handle eth1 IRQs echo 20 > /proc/irq/234/smp_affinity echo 20 > /sys/class/net/eth1/queues/rx-0/rps_cpus 12345678 # CPU4 and CPU5 are the 1.8GHz cores# Set CPU4 to handle eth0 IRQsecho 10 > /proc/irq/27/smp_affinityecho 10 > /sys/class/net/eth0/queues/rx-0/rps_cpus # Set CPU5 to handle eth1 IRQsecho 20 >… Read more »
Today I’ve done more thorough tests and it seems that there’s no real benefit in a benchmark environment by using those network optimizations. I’ve wrote more details here. I think that my yesterday enthusiasm was based on a number that probably is wrong, since it seems that the UDP packets are actually lost when the MTU is set to 512 and using those optimizations. I didn’t had more time to investigate this, though.
> it seems that there’s no real benefit in a benchmark environment
Those IRQ affinity optimizations are not for a benchmark environment but for real world scenarios where ‘everything on cpu0’ pretty often creates an artificial bottleneck. Another topic is SMP affinity (using taskset/cgroups since as soon as the process scheduler also prefers cpu0 already busy with interrupts performance suffers for sure)
That’s my thoughts, too; and I’ve also come to this conclusion, that it’s good to have them anyways for real usage.
It has a graphics driver/chip, but no graphics / HDMI output, right?
Yes, correct. There’s no video output. It might still be possible to use the GPU for other things like GPGPU for example using OpenCL.
In an ideal world… Wouldn’t some future chipset carry video over USB C alt mode, so that ‘headless’ boxes didn’t need to waste space on a display output?
[Noting your earlier comment that this sadly doesn’t even OTG]
Would it be possible to install GNOME and VNC, for example in order to access from a PC and to have a typical Ubuntu GUI?
@Jean-Luc Aufranc (CNXSoft), will it be possible to run some routing tests with 512 byte packet size?
Yes. But it’s more likely to happen if you tell me how to test it.
Hi @Jean-Luc Aufranc (CNXSoft), I really appreciate the time you put into this! You can control packet size either with iperf’s ‘-M’ option (to set MSS), or by setting MTU at interface level for iperf client and server (https://www.cyberciti.biz/faq/how-can-i-setup-the-mtu-for-my-network-interface/). You need to set MSS to 472 or MTU to 512 for 512B packet size. Before testing, make sure you have two hosts (referred forward as ‘iperf client’ and ‘iperf server’) that can handle 900Mbps in direct connect with ~512B packet size (it should be enough, I would be surprised if R4S can route at that speed out fo the box).… Read more »
At this point in time such tests are pretty much useless since as shown above the vendor’s software offering lacks the needed optimizations (stuff like this for example)
Without PCIe and Ethernet interrupts sent to cpu4 and cpu5 (the A72 cores clocking in at 1.8 GHz) routing performance will be a joke anyway. Maybe Active-state power management (ASPM) also needs some love…
Thomas, there seems to be lots of bugs in this script. First it seems to assume that smp_affinity uses decimal notation while it’s using hexadecimal, so values written as 16 or 32 should be written as 10 or 20. Second, some settings look completely wrong just as if the person writing them thought they were CPU numbers. I think some entries mixed smp_afffinity_list and smp_affinity, though I can’t say for sure since it takes time to review it.
> there seems to be lots of bugs in this script
Possible. Nobody at Armbian is interested in such stuff any more and it’s more or less an unmaintained mess. I only mentioned it to outline that certain tunables exist and need to be set. And both SMP and IRQ affinity on big.LITTLE designs (where cpu0 is always a little core) are key to performance with such use cases we’re talking here about.
Yep. However the little cores are not necessarily the worst ones to handle network I/O. Very often the user-land work associated with networking is much more expensive than the driver counter-part, and it’s not uncommon to see better performance by binding the application to the big cores and the network layer to thelittle ones. Most of the driver’s job is just accessing memory, and that’s not worse in little cores than big ones. For the cases where crypto is involved, similarly the crypto performance will be roughly the same.
In addition, for CPUs like the S922X or A311D, mixing A73 and A55, I’d bet better memory performance and latency on the A55 core than on the A73, and it could actually make the network run faster on the little core.
I’m totally aware the software is not optimized for the actual purpose of the board (or for anything else) at this point. I’m not even expecting it to be. I just want to see some figures at this state of software as, with the right setting, those can be improved by 30-40% (from my experience with routing on NEO4 with various USB to Ethernet adapters). I’m just hopping the Ethernet over PCIe will generate less IRQ load than Ethernet over USB.
Well, see Dim’s comments above.
Still, I don’t think any tests or even benchmarking makes any sense unless at least reasonable SMP/IRQ affinity settings and RPS is in place. The artificial cpu0 bottleneck is a joke.
For doing lots of network testing, I second this! I even refuse to waste my time reading network tests reports if the report suggests that nobody knows how affinity was done nor which modules were loaded. Just load ip_conntrack and lose 20%. Leave your IRQ on cpu0 (or forget to kill irqbalance) and randomly lose 80%. Misconfigure RPS and lose an extra 30%. Such numbers which are extremely sensitive to latency and per-packet processing time mean nothing with no precise context around.
@Jean-Luc Aufranc (CNXSoft)
I see you have the 4GB LPDDR4 version.
Any chance you could connect a UART console and capture the DRAM init messages?
Would like to know if they got the DRAM routing on the board “in-spec” this time.
Possible. Except I’m not home right now. I should be back around the 8th of January.
Finally, there are nightly (21.02.0-trunk) builds from Armbian 🙂
https://www.armbian.com/nanopi-r4s/
Works like a charm…
I built my own image last night and ran sbc-bench:
http://ix.io/2KKP
No big surprises.
Regarding the temperature: I have the version without the case (unfortunately) and use a small 30mm fan connected to the GPIO.
Works… until I can buy a metal case (and one for the Neo3, too) 🙂
Thanks for the review.
Helped me at lot
I see no one has posted a user review, on the Friendlyelec product page yet. Why so quiet?
Anybody have luck bringing up a serial console over UART with OpenWRT? I manage to get some garbage in minicom or cutecom after connecting, but can’t seem to get a login prompt. This is the hardware module I am using I am using (found on Amazon): IZOKEE CP2102 Module USB to TTL 5PIN Serial Converter Adapter Module Downloader for UART STC 3.3V and 5V with Jumper Wires Basically connected GND to GND, RX to TX, TX to RX. Not using the 3.3V or 5V wires. Poked around in dmesg and saw this: [ 0.000000] Kernel command line: console=ttyS2,1500000 earlycon=uart8250,mmio32,0xff1a0000 root=PARTUUID=5452574f-02 rw… Read more »
Usually, garbage output on the serial console is due to one of the following:
You seem to do everything right, so I do not have an explanation…
The settings in minicom should be something like 1500000 8N1.
Alternatively try bootterm: https://github.com/wtarreau/bootterm
It’s easier to use, and the program I’m now using as well: https://www.cnx-software.com/2020/12/14/bootterm-a-developer-friendly-serial-terminal-program/
Thank you, I’ll try that and report back.
FYI I did stick to 8-bit, and some cursory googling implies this is correct. If the lines are inverted, it’s because something is mislabeled. I’ll explore that possibility, but seems far-fetched.
Do you use any particular TTL to USB adapter you can recommend? I’m not sure that should matter, but I hear reports….
One other thing I forgot to mention, I turned on this setting:
uci set system.@system[0].ttylogin=’1′
But it didn’t make a difference so far.
CP2102 should work fine at 1.5 Mbps. Some adapters will not however (e.g. CH340 loses many characters and is unusable). Otherwise it’s often useful to buy at least one adapter made of a real FTDI chip, they’re extremely performant and adapt to about everything. Using that to confirm a port works helps quite a bit! Also it’s worth re-checking the pins and if you’re certain to be connecting to the correct UART. I don’t know the addresses, but it could be possible that by accident you’re using the wrong one. My M4 doesn’t respond right now, but connecting to a… Read more »
Thanks to Willy and Jean-Luc for the feedback, and help. Happy to report a lot of success! So I grabbed a DSD TECH SH-U09C5 USB to TTL UART Converter Cable with FTDI Chip Support 5V 3.3V 2.5V 1.8V TTL I still had problems. So I changed USB ports and now it works with software flow control. Maybe I have a bad USB port. I was unable to to have success with bootcom, but worked with minicom and cutecom. I had no luck figuring out the bootcom flow control settings . This is the command I used minicom -D /dev/ttyUSB1 -b… Read more »
Cool! Regarding dead USB ports, it happened to me a few times. I know that the DC lines on some of my machines’ ports are weak (such as the netbook I’m currently typing on), it’s probable that I damaged the power transistors used in the current regulators by trying to boot some SBCs directly off them.