Getting the modification time of a file–GetFileAttributesEx() on Windows and stat() on non-Windows platforms–seems to be about 100 times slower on Windows than it is on Linux.
It cannot be overstated how much slower File I/O on Windows is compared to Linux. Yes, orders of magnitude, I’ve measured it myself. This started to become really bad about 5-10 years ago. Raw throughput on one single large file performs okay, but especially operations on many small files are painfully slow, even on the fastest Desktop SSDs. Sometimes you literally wait for seconds for operations that should complete in at most a few milliseconds (and actually do so on ext4/Linux). I don’t know what’s going on, but I wouldn’t be surprised if there were some accidentally-quadratic bugs in there. There is simply no reasonable explanation for this kind of overhead. It really hurts development on Windows. The time between writing and running code is substantially longer, this adds up due to the iterative nature of software development. If Microsoft cannot put a lid on the escalating file system slowness and reverse the trend, we will see more and more developers pushing for Linux and Mac OS machines in the future (or they do everything in WSL2).
There are hooks for “antivirus” software all over Windows, which definitely has to be a decent part of it. Forget files, their mmap equivalent does this for executable pages, which was in the news recently for affecting Firefox hard.
Ninja is one of the tools that makes me happy that not all open source developers have forgotten the (good bits of the) UNIX philosophy. It does less than make, and it does it far better, and it composes well with other tools.
I’ve never used Ninja, but I absolutely love samurai, which is a tiny, ninja-compatible build system written in C99. What you say is spot on tho: I really do enjoy how small Ninja build files are, how easy they are to compose, and that it’s quite a bit simpler than make proper, whilst not being too far away from make either. Honestly, the most common thing I use it for is my document pipelines: generating PDFs from docs I run often, slides from asciidoc/markdown to HTML, &c. Really lovely to use, all told
I’ve never felt the need for a C99 version rather than C++ (C++ is better supported by MSVC than C99 and equally well supported by GCC and Clang), so I haven’t had a use case for Samauri. I do wish packagers would stop pretending Ninja depends on Python - the only thing it uses Python for is showing GraphVis files visualising the build, which in over a decade of routinely using Ninja, I have never wanted to do.
The author of Ninja is currently working on a new build tool written in Rust that has some interesting ideas.
This article is linking to this online event on Tuesday, with some interesting looking topics! A shame the site doesn’t link to the actual lightning talk titles (stuff like “ How to create the nastiest test inputs ever.”), but it sounds right up the alley of lots of people here.
It cannot be overstated how much slower File I/O on Windows is compared to Linux. Yes, orders of magnitude, I’ve measured it myself. This started to become really bad about 5-10 years ago. Raw throughput on one single large file performs okay, but especially operations on many small files are painfully slow, even on the fastest Desktop SSDs. Sometimes you literally wait for seconds for operations that should complete in at most a few milliseconds (and actually do so on ext4/Linux). I don’t know what’s going on, but I wouldn’t be surprised if there were some accidentally-quadratic bugs in there. There is simply no reasonable explanation for this kind of overhead. It really hurts development on Windows. The time between writing and running code is substantially longer, this adds up due to the iterative nature of software development. If Microsoft cannot put a lid on the escalating file system slowness and reverse the trend, we will see more and more developers pushing for Linux and Mac OS machines in the future (or they do everything in WSL2).
There are hooks for “antivirus” software all over Windows, which definitely has to be a decent part of it. Forget files, their
mmapequivalent does this for executable pages, which was in the news recently for affecting Firefox hard.Ninja is one of the tools that makes me happy that not all open source developers have forgotten the (good bits of the) UNIX philosophy. It does less than make, and it does it far better, and it composes well with other tools.
I’ve never used Ninja, but I absolutely love samurai, which is a tiny, ninja-compatible build system written in C99. What you say is spot on tho: I really do enjoy how small Ninja build files are, how easy they are to compose, and that it’s quite a bit simpler than
makeproper, whilst not being too far away frommakeeither. Honestly, the most common thing I use it for is my document pipelines: generating PDFs from docs I run often, slides from asciidoc/markdown to HTML, &c. Really lovely to use, all toldI’ve never felt the need for a C99 version rather than C++ (C++ is better supported by MSVC than C99 and equally well supported by GCC and Clang), so I haven’t had a use case for Samauri. I do wish packagers would stop pretending Ninja depends on Python - the only thing it uses Python for is showing GraphVis files visualising the build, which in over a decade of routinely using Ninja, I have never wanted to do.
The author of Ninja is currently working on a new build tool written in Rust that has some interesting ideas.
This article is linking to this online event on Tuesday, with some interesting looking topics! A shame the site doesn’t link to the actual lightning talk titles (stuff like “ How to create the nastiest test inputs ever.”), but it sounds right up the alley of lots of people here.
Plus the name of the event, “It’ll Never Work In Theory”, cracks me up.