Microsoft recently announced that they brought Ubuntu userspace to Windows, and that this features will be officially released in Windows 10’s Anniversary Update and called Windows Subsystem for Linux. But people part of the company’s insider program can already try the beta version of “Bash on Windows”, and Phoronix ran some benchmarks in bash in Windows 10, and repeated the tests in Ubuntu 16.04, Ubuntu 14.04, and Clear Linux. The test machine was based on an Intel Xeon E3-1280 v5 Skylake CPUwith 16GB of RAM and 120GB Samsung 850 EVO SSD.
Many of the results show Windows Subsystem for Linux (I’ll just call it Windows 10 in the rest of the post) just performing a little slower than on the Linux distributions, but there are also some outliers, which I’m going to cover here.
The most surprising results is when Windows 10 clearly outperforms Linux at its own game which should not be happening.
That’s the case for Stream 1.2 triad and add benchmarks. Stream is supposed to benchmark the system memory (RAM) performance. The copy operation from the same benchmark is still faster in Linux however, except in Ubuntu 14.04.
The table below summarize the operation for the 4 stream tests:
|
1 2 3 4 5 6 7 8 |
------------------------------------------------------------------ name kernel bytes/iter FLOPS/iter ------------------------------------------------------------------ COPY: a(i) = b(i) 16 0 SCALE: a(i) = q*b(i) 16 1 SUM/ADD: a(i) = b(i) + c(i) 24 1 TRIAD: a(i) = b(i) + q*c(i) 24 2 ------------------------------------------------------------------ |
I don’t have any explanation for the issue, but maybe some people can provide some clues in the comments.
There were also benchmarks where bash on Windows 10 is much slower, likely due to the use of NTFS instead of EXT-4.
The Compile bench “tries to age a filesystem by simulating some of the disk IO common in creating, compiling, patching, stating and reading kernel trees. It indirectly measures how well file systems can maintain directory locality as the disk fills up and directories age. This current test is setup to use the makej mode with 10 initial directories”. Since Ubuntu bash on Windows is designed for developers this may actually matter. The poor performance is confirmed with Timed PHP compilation benchmark.
So it may pay off to try some other file systems if possible in Windows 10.
SciMarks v2.0 Fast Fourier Transform is another benchmark that’s quite faster in bash in Windows 10.
That one is also odd, so there must be some operations that the Windows kernel does faster than the Linux kernel, even after the overhead of converting Linux calls to Windows calls.
Windows 10 got back to struggling with Redis open-source data structure server benchmark that’s likely reliant on storage I/Os.
The other benchmark results where more or less in line with expectations, although there were some regressions between Ubuntu 16.04 and Ubuntu 14.04.
Jean-Luc started CNX Software in 2010 as a part-time endeavor, before quitting his job as a software engineering manager, and starting to write daily news, and reviews full time later in 2011.
Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress
“ubuntu userland” i.e. GNU? GNU/M$, who’d have thunk that’d be the next thing they’d want to EEE is GNU Bash.
All tests would depend on the compiler somewhat which might explain the 14.04 differences in the memory tests. Since CPU bound things should only depend on the hardware, the main OS contributors would be the page table setup or context switching rate, or power throttling/burst modes.
Those NTFS times mirror my experience with it (native) – it’s total junk.
I feel Ubuntu Bash is a misnomer. Bash is a GNU project not something Ubuntu created.
@FransM
You’ve right and I guess it should be “GNU bash with Ubuntu userspace” but that’s a big too long for a title, especially I had to use other words too…
Anyway that Bash on Ubuntu on Windows thing makes it very hard to use the right names.
So he messed up storage configuration in Windows, tested different compilers and obviously syscalls (according to the screenshots the Linux subsystem reports a kernel 3.4.0). And the most interesting thing (huuuuuge standard deviation, see the Stream values above) is not even commented. Another round of useless passive benchmarking as usual 🙂
It might be interesting how results differ if PTS would start to use optimised settings and not nonsense ‘defaults’ and demanding tests like cpuminer.
BTW: I don’t get why bash is mentioned at all since it’s about the performance of this “Windows Subsystem for Linux” and neither bash nor Ubuntu (I would assume you can replace the whole Ubuntu userland with something else)
@tkaiser
In Windows, you type “Bash” to get to the terminal, and the tools are exactly the same as in Ubuntu (same binaries). I think they use Ubuntu 14.04 binaries now.
@cnxsoft
You start C:\Windows\System32\bash.exe (could also be called C:\Windows\System32\enter_hell.exe 😉 ) and then you get a root shell that is bash by default (I would suspect it could also be zsh or sh later), the userland will be downloaded the first time you start WSL and is a stripped down Ubuntu 14.04 LTS (x86-64) and most importantly there is no interconnection to the Windows world apart from sockets and filesystem access (/mnt/c/ — OMG). It’s a nice feature for developers that want to quickly try out things or are more familiar with Unix tools but there is no real interaction with Windows applications possible.
Therefore the whole benchmarking approach is even more questionable than as usual with the Phoronix stuff. And when you’re talking about performance both bash/Ubuntu aren’t relevant but WSL’s syscall implementation. And even in this early stage it already seems quite performant and we might even see Linux stuff being executed faster using the Windows kernel 😉
I had some hope there would be more integration between Linux and Windows possible with WSL. Like on OS X for example where I have full access to application objects from BSD using osascript and the like or am able to interact between Cocoa apps and BSD stuff through Automator.
The storage benchmarks being bad probably reflects that the abstraction layer needs to do a lot of work for every filesystem access, emulating what the Linux expects from its filesystems, directory structure, acess rights. Benchmarks that are only bottlenecked by RAM usage or CPU power should have it easier.
The differences in math performance might be due to different compilers used for making the kernels of Windows10 and Ubuntu, using different features of the CPU. Just guessin’… Otherwise it makes absolutely no sense that such simple calls yield such different results.