ntttcp (Windows NT Test TCP) is a network benchmarking utility similar to iperf3 that works in both Windows and Linux written and recommended by Microsoft over iperf3, so we’ll test the alternative in this mini review.
iperf3 is a utility of choice for our reviews of single board computers and mini PCs running either Windows or Linux, but we’ve noticed that while Ethernet (up to 2.5GbE) usually performs just as well in Windows and Linux, WiFi is generally much faster in Ubuntu 22.04 than in Windows 11. So when XDA developers noticed a post by Microsoft saying iperf3 should not be used on Windows 11, it caught my attention.
Microsoft explains iperf3 should not be used in Windows for three main reasons:
- The maintainer of iperf – ESnet (Energy Sciences Network) – says “iperf3 is not officially supported on Windows, but iperf2 is. We recommend you use iperf2. Some people are using Cygwin to run iperf3 in Windows, but not all options will work”
- iPerf3 is Emulated on Windows – iPerf3 does not make Windows native API calls as it only knows how to make Linux/POSIX calls, and this may impact performance.
- I usually download iperf3 3.1.3 for Windows released in 2016, and Microsoft notes the one offered by ESnet (version 3.16) is more recent but still 15 versions behind, so users are not running the latest version of the utilities.
So what’s the alternative for iperf3? Microsoft maintains two:
- ntttcp (Windows NT Test TCP) open-source utility for Windows and Linux with a command line similar to iperf3 according to Microsoft in the sense it aims to isolate network stack throughput.
- ctsTraffic for Windows-to-Windows testing only, also open-source and maintained on Github. ctsTraffic focuses on end-to-end goodput scenarios.
We can disqualify ctsTraffic immediately here at CNX Software since our tests typically involve a mix of Linux and Windows machines. Microsoft compared iperf3 to ntttcp utilities with high-speed network interfaces (10 GbE+), and the latter reports much higher performance. I only have hardware with 2.5GbE and WiFi 6, but I still wanted to test it, especially to check WiFi. So I decided to give ntttcp a try following the instructions on Microsoft Learn. It ended up being a challenge as those would not work on my system, and it took me a while to find the right parameters…
My testbed is comprised of four main components:
- Khadas Mind Premium mini PC and Khadas Dock with 2.5GbE and a WiFi 6 running Windows 11 Pro
- UP Xtreme i11 Edge mini PC with 2.5GbE acting as a server running Ubuntu 20.04
- A 2.5GbE TP-Link switch
- Xiaomi Mi AX6000 WiFi 6 router (not shown in the photo above)
The Khadas Mind Premium was selected as it’s the only spare Windows system with 2.5GbE and WiFi 6 that I own and have already reviewed. You’ll notice the network tests in Windows 11 and Ubuntu 22.04 using iperf3 reveal much lower performance in the Microsoft OS as summarized in the table below.
| WiFi 6 Tx | WiFi 6 Rx | 2.5GbE Tx | 2.5GbE Rx | |
|---|---|---|---|---|
| WIndows 11 Home | 712 Mbps | 590 Mbps | 700 Mbps | 2.30 Gbps |
| Ubuntu 22.04 | 1.40 Gbps | 991 Mbps | 2.35 Gbps | 2.35 Gbps |
The Khadas mini PC is an outlier when it comes to 2.5GbE upload performance, but WiFi is faster in Linux in all mini PC reviews we’ve done. Since the Mind Premium review was done a while ago (August 2023), I updated Windows to the latest version and drivers and tested networking performance again with iperf3 and ntttcp in Windows 11 Home using the same command line as in the Microsoft blog post.
The first step was to install ntttcp (Linux) on UP Xtreme i11 mini PC running Ubuntu 20.04:
|
1 2 3 4 |
git clone https://github.com/microsoft/ntttcp-for-linux cd ntttcp-for-linux/src make sudo make install |
We can run the receiver command as follows.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
devkit@UPX-i11:~/ntttcp-for-linux$ ntttcp -r -m 1,*,192.168.31.12 -t 60 -V NTTTCP for Linux 1.4.0 --------------------------------------------------------- *** receiver role ports: 1 cpu affinity: * server address: 192.168.31.12 domain: IPv4 protocol: TCP server port starting at: 5001 receiver socket buffer (bytes): 65536 test warm-up (sec): no test duration (sec): 60 test cool-down (sec): no show system tcp retransmit: no quiet mode: disabled verbose mode: enabled --------------------------------------------------------- 12:25:30 DBG : user limits for maximum number of open files: soft: 1024; hard: 1048576 12:25:30 DBG : Interface:[lo] Address: 127.0.0.1 12:25:30 DBG : Interface:[enp44s0] Address: 192.168.31.12 12:25:30 DBG : Interface:[docker0] Address: 172.17.0.1 12:25:30 DBG : Interface:[flannel.1] Address: 10.42.0.0 12:25:30 DBG : Interface:[cni0] Address: 10.42.0.1 12:25:30 INFO: 2 threads created 12:25:30 DBG : ntttcp server is listening on 192.168.31.12:5001 12:25:30 DBG : ntttcp server is listening on 192.168.31.12:5000 12:25:47 DBG : New connection: 192.168.31.69:50666 --> local:10000 [socket 6] socket read error: 104 12:26:11 DBG : socket closed: 6 12:26:32 DBG : New connection: 192.168.31.69:50675 --> local:10000 [socket 6] socket read error: 104 12:26:37 DBG : socket closed: 6 12:26:49 DBG : New connection: 192.168.31.69:50683 --> local:10000 [socket 6] |
Only one core is used to emulate iperf3 and the V (verbose) options help a lot to troubleshoot issues. After downloading the ntttcp.exe binary to Windows we can run it immediately as a sender in a command prompt:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 |
C:\Users\jaufr\Downloads>ntttcp.exe -s -m 1,*,192.168.31.12 -l 128K -t 60 -V Copyright Version 5.39 buffers_length: 131072 num_buffers_to_send: 9223372036854775807 send_socket_buff: -1 recv_socket_buff: -1 port: 5001 sync_port: 0 no_sync: 0 wait_timeout_milliseconds: 600000 async_flag: 0 verbose_flag: 1 wsa_flag: 0 use_ipv6_flag: 0 send_flag: 1 udp_flag: 0 udp_unconnected_flag: 0 verify_data_flag: 0 wait_all_flag: 0 run_time: 60000 warmup_time: 0 cooldown_time: 0 dash_n_timeout: 10800000 bind_sender_flag: 0 sender_name: max_active_threads: 1 no_delay: 0 node_affinity: -1 udp_uso_size: 0 udp_receive_coalescing: 0 tp_flag: 0 use_hvsocket_flag: 0 no_stdio_buffer: 0 throughput_Bpms: 0 cpu_burn: 0 latency_measurement: 0 use_io_compl_ports: 0 cpu_from_idle_flag: 0 get_estats: 0 qos_flag: 0 packet_period: 0 jitter_measurement: 0 mapping[0]: 1 4/21/2024 10:10:45 proc_speed: 2611 MHz 4/21/2024 10:10:45 SetupThreads 4/21/2024 10:10:45 Threads: 1 Processor: -1 Host: 192.168.31.12 4/21/2024 10:10:45 created thread 0 port 5001 4/21/2024 10:10:45 StartSenderReceiver start thread 0 port 5001 4/21/2024 10:10:45 SetupNet port 5001 4/21/2024 10:10:45 connected to port 5001 4/21/2024 10:10:45 SetupNet complete on port 5001 4/21/2024 10:10:45 All threads ready! 4/21/2024 10:10:45 SetupNet port 6001 4/21/2024 10:10:47 ERROR: SetupNet failed: Connect attempt failed, GetLastError: 10061 - No connection could be made because the target machine actively refused it. 4/21/2024 10:10:47 PORT#: 6001 4/21/2024 10:10:49 ERROR: SetupNet failed: Connect attempt failed, GetLastError: 10061 - No connection could be made because the target machine actively refused it. |
But as you can see from the log above, it did not quite work as expected. It turns out that for Windows to Linux tests, we need to use the “ns” (No Sync) parameter. It’s mentioned in the Microsoft’s blog post
There is a known interoperability limitation when testing between Windows and Linux. Details can be found in this ntttcp for Linux wiki article on GitHub.
It took me a couple of hours to find out, but once I did that, I could complete the WiFi 6 Tx (upload) test:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
C:\Users\jaufr\Downloads>ntttcp.exe -s -m 1,*,192.168.31.12 -l 128K -t 60 -ns Copyright Version 5.39 Network activity progressing... Thread Time(s) Throughput(KB/s) Avg B / Compl ====== ======= ================ ============= 0 60.011 100476.513 131072.000 ##### Totals: ##### Bytes(MEG) realtime(s) Avg Frame Size Throughput(MB/s) ================ =========== ============== ================ 5888.375000 60.009 1459.307 98.125 Throughput(Buffers/s) Cycles/Byte Buffers ===================== =========== ============= 785.001 4.024 47107.000 DPCs(count/s) Pkts(num/DPC) Intr(count/s) Pkts(num/intr) ============= ============= =============== ============== 906.366 4.807 3218.159 1.354 Packets Sent Packets Received Retransmits Errors Avg. CPU % ============ ================ =========== ====== ========== 4231054 261456 5 0 0.991 |
98.125 MB/s or about 785 Mbps, slightly better than with iperf3, but still far from the performance in Linux.
I then connected an Ethernet cable to test 2.5GbE upload:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
C:\Users\jaufr\Downloads>ntttcp.exe -s -m 1,*,192.168.31.12 -l 128K -t 60 -ns Copyright Version 5.39 Network activity progressing... Thread Time(s) Throughput(KB/s) Avg B / Compl ====== ======= ================ ============= 0 60.015 90110.806 131072.000 ##### Totals: ##### Bytes(MEG) realtime(s) Avg Frame Size Throughput(MB/s) ================ =========== ============== ================ 5281.250000 60.014 1458.979 88.000 Throughput(Buffers/s) Cycles/Byte Buffers ===================== =========== ============= 703.998 3.927 42250.000 DPCs(count/s) Pkts(num/DPC) Intr(count/s) Pkts(num/intr) ============= ============= =============== ============== 11814.824 0.449 31926.446 0.166 Packets Sent Packets Received Retransmits Errors Avg. CPU % ============ ================ =========== ====== ========== 3795663 318708 1 0 0.867 |
88 MB/s or 704 Mbps, so it’s basically the same as with iperf3 even after having updated the drivers.
iperf3 has a reverse transfer option, but I could not see any such option on ntttcp. So I had to type the commands to run ntttcp.exe in receiver mode on Windows and ntttcp in sender mode on Linux. We’ll need to run CMD as administrator, open the firewall, and the networking benchmark tool with other parameters:
|
1 2 |
netsh advfirewall firewall add rule program=C:\Users\jaufr\Downloads\ntttcp.exe name="ntttcp" protocol=any dir=in action=allow enable=yes profile=ANY C:\Users\jaufr\Downloads>ntttcp.exe -r -m 1,*,192.168.31.141 -ns -t 60 -V |
The Linux command for the Ubuntu sender is quite different than the same command for the Windows sender as parameters are different:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
devkit@UPX-i11:~/ntttcp-for-linux$ ntttcp -s -m 1,*,192.168.31.141 -b 128K -N -t 60 -V NTTTCP for Linux 1.4.0 --------------------------------------------------------- *** sender role *** no sender/receiver synch connections: 1 X 1 X 1 cpu affinity: * server address: 192.168.31.141 domain: IPv4 protocol: TCP server port starting at: 5001 sender socket buffer (bytes): 131072 test warm-up (sec): no test duration (sec): 60 test cool-down (sec): no show system tcp retransmit: no quiet mode: disabled verbose mode: enabled --------------------------------------------------------- 14:45:55 DBG : user limits for maximum number of open files: soft: 4096; hard: 4096 14:45:55 INFO: Starting sender activity (no sync) ... 14:45:55 INFO: 1 threads created 14:45:55 DBG : New connection: local:42880 [socket:3] --> 192.168.31.141:5001 14:45:55 INFO: 1 connections created in 1708 microseconds 14:46:55 INFO: Test run completed. 14:46:55 INFO: Test cycle finished. 14:46:55 INFO: Thread Time(s) Throughput 14:46:55 INFO: ====== ======= ========== 14:46:55 INFO: 0 60.00 2.28Gbps 14:46:55 INFO: 1 connections tested 14:46:55 INFO: ##### Totals: ##### 14:46:55 INFO: test duration :60.00 seconds 14:46:55 INFO: total bytes :17068195840 14:46:55 INFO: throughput :2.28Gbps 14:46:55 INFO: retrans segs :54998 14:46:55 INFO: cpu cores :8 14:46:55 INFO: cpu speed :2000.462MHz 14:46:55 INFO: user :0.46% 14:46:55 INFO: system :0.78% 14:46:55 INFO: idle :97.94% 14:46:55 INFO: iowait :0.00% 14:46:55 INFO: softirq :0.82% 14:46:55 INFO: cycles/byte :1.16 14:46:55 INFO: cpu busy (all) :5.16% 14:46:55 INFO: tcpi rtt :397 us |
But I still managed it. Needless to say, I did have an overly positive view of ntttcp utility so far. The only benefit I see is that we have some extra data such as CPU usage during the transfer. 2.28 Gbps is about what we would expect for a 2.5GbE connection.
I usually disconnect the Ethernet cable to test WiFi 6 with iperf3 and run the same command. But here we also need to change the IP address on both the server and client side to test it again:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
devkit@UPX-i11:~/ntttcp-for-linux$ ntttcp -s -m 1,*,192.168.31.69 -b 128K -N -t 60 NTTTCP for Linux 1.4.0 --------------------------------------------------------- 14:54:58 INFO: Starting sender activity (no sync) ... 14:54:58 INFO: 1 threads created 14:54:58 INFO: 1 connections created in 6716 microseconds 14:55:58 INFO: Test run completed. 14:55:58 INFO: Test cycle finished. 14:55:58 INFO: 1 connections tested 14:55:58 INFO: ##### Totals: ##### 14:55:58 INFO: test duration :60.00 seconds 14:55:58 INFO: total bytes :4530110464 14:55:58 INFO: throughput :604.01Mbps 14:55:58 INFO: retrans segs :0 14:55:58 INFO: cpu cores :8 14:55:58 INFO: cpu speed :2800.000MHz 14:55:58 INFO: user :0.53% 14:55:58 INFO: system :0.41% 14:55:58 INFO: idle :99.01% 14:55:58 INFO: iowait :0.00% 14:55:58 INFO: softirq :0.04% 14:55:58 INFO: cycles/byte :2.93 14:55:58 INFO: cpu busy (all) :1.98% |
604 Mbps so there’s no improvement here.
| WiFi 6 Tx | WiFi 6 Rx | 2.5GbE Tx | 2.5GbE Rx | |
|---|---|---|---|---|
| iperf3 | 551 Mbps | 608 Mbps | 736 Mbps | 2.30 Gbps |
| ntttcp | 785 Mbps | 604 Mbps | 704 Mbps | 2.28 Gbps |
The table above summarizes the results after I ran iperf3 again. While ntttcp is faster for WiFi 6 upload, the results are not conclusive as the other results are more or less unchanged. I suppose it only matters for high-speed networking with 10GbE or greater connections.
The tests above were done to compare ntttcp to iperf3 with similar parameters, but Microsoft says multithread and larger buffer sizes should be used to test bandwidth. Let’s try again with WiFi 6 download using 8 threads and 1024KB buffer size:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
devkit@UPX-i11:~/ntttcp-for-linux$ ntttcp -s -m 8,*,192.168.31.69 -b 1024K -N -t 60 NTTTCP for Linux 1.4.0 --------------------------------------------------------- 15:00:06 INFO: Starting sender activity (no sync) ... 15:00:06 INFO: 8 threads created 15:00:06 INFO: 8 connections created in 5242 microseconds 15:01:06 INFO: Test run completed. 15:01:06 INFO: Test cycle finished. 15:01:06 INFO: 8 connections tested 15:01:06 INFO: ##### Totals: ##### 15:01:06 INFO: test duration :60.00 seconds 15:01:06 INFO: total bytes :4234149888 15:01:06 INFO: throughput :564.55Mbps 15:01:06 INFO: retrans segs :573 15:01:06 INFO: cpu cores :8 15:01:06 INFO: cpu speed :2800.000MHz 15:01:06 INFO: user :0.52% 15:01:06 INFO: system :0.54% 15:01:06 INFO: idle :98.62% 15:01:06 INFO: iowait :0.00% 15:01:06 INFO: softirq :0.32% 15:01:06 INFO: cycles/byte :4.39 15:01:06 INFO: cpu busy (all) :2.90% --------------------------------------------------------- |
564 Mbps is slower than with only one thread and a 128KB buffer, although I reckon WiFi results can be fairly volatile.
Based on the tests done above there’s very little difference between iperf3 and ntttcp results, ntttcp Linux has not been updated for over three years, so I’m not convinced, and we’ll keep using iperf3 for the networking tests in reviews of Windows mini PCs…
Jean-Luc started CNX Software in 2010 as a part-time endeavor, before quitting his job as a software engineering manager, and starting to write daily news, and reviews full time later in 2011.
Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress
After checking with Linixium, his own WiFi 6 tests show that ntttcp is faster than iperf 3.1.3 (downloaded from free.fr that typically shows up first in a web search), but iperf 3.16 (downloaded from https://files.budman.pw/) has the same performance as ntttcp.
This could explain the difference in WiFi 6 throughput between Windows (iperf 3.1.3) and Linux (iperf 3.9 in Ubuntu 22.04).
What matters in any case is not to get *the* most accurate tool but a collection of different tools so that discrepancies can be spotted and investigated. Among the regular issues not tested by such tools by default is small packets (which happen with visioconfs for example), where it’s visible that all NICs are not equal at all.
Have you tried using iPerf with -w 4M or alternatively -P xx ?
It allows you to change the window size or run parallel streams once you have more bandwidth available than the default settings are suitable for.
I haven’t tried, but considering there’s no issue with 2.5GbE (on most platforms), I’m not sure how increasing the buffer size may help with the slower WiFi on Windows vs Linux.
I find at home that my iperf3 test between windows clients gets ~ 2.3Gbps, I was surprised to see your results lower. However what I do see is that in general Windows does not raise it’s receive window high enough for latency even as low as 12ms. Linux will in that case and deliver the full 1Gpbs to one connection over the Internet. This is increasingly important as VPNs usually are single connections, and usually UDP. Which using a parallel tests can get windows to go the whole way. Too bad I cannot tell VPN clients to bond several connections.… Read more »
I also get about 2.30 to 2.35 Gbps on most devices with iperf3 using 2.5GbE. It’s unclear why WiFi is slower in Windows compared to Linux, but as noted in the article, even using a parallel test with 8 threads does not improve the results
Forgive me, I should have been more specific with that comment. I mean the 2.5GbE TX numbers. Though my test devices are Server 2022 and Windows 10 pro, the numbers are usually very similar between the two systems. I do have a 2.5GbE USB adapter but I’m not sure if my NUC has USB 3 ports. If not, one day I’ll live boot my laptop and test. I’ve only seen parallel test help when the connections leave the local network and the problems are probably in the Internet connection’s router. My guess is that a lot of it has to… Read more »
Results won’t differ with 2.5GbE between various tools.
For the purpose of this test, we should ignore the low 2.5GbE Tx transfer rate on the Khadas Mind Premium because it’s a device-specific issue. I wanted to use another mini PC with fully working 2.5GbE in both directions, but the Mind Premium was the only device with Windows and WiFi 6 available to me at the time.
Embrace Extend Extinquish (EEE)
What about using iperf2?