1.  

    Another shitty license, better use MIT/BSD.

    1. 2

      isn’t this like with c/c++/java/javascript previously ?

      1. 25

        No, it’s different. I know C++ for a long time and mainly write C++ in my full-time job for years, but with Rust, I’m about 5 times as productive as with C++. Since I learned Rust, I churned out various programs that I would’ve never been able to develop in C++, simply due to time and complexity. I can now write software that I previously couldn’t have developed as a single person. Say about the language what you will, to me personally, Rust is pure empowerment that manifests in real software that is used by real people daily and saves time, money and nerves. And trust me, in the years before I learned Rust, nothing annoyed me more than the Rust evangelists whose posts kept popping up in every forum.

        1. 4

          Is there anything you could share that you feel is a particularly good example of something you’ve been able to develop in Rust on your own that you couldn’t have otherwise?

          I’m asking because I am learning Rust for fun right now, and I’m hitting the wall a little bit. (By that I mean “the wall” in the same sense that people who run endurance races use the term. You hit a certain point in the race where you suspect you won’t be able to continue. If you can push past that, you can have a good long race. But it’s a challenge.) I can get things done and there’s a lot that I like about it, but I don’t feel very productive yet.

          I’ve mainly written C++ and python over the past 20 years, so if it’s something you could show, seeing some of the things you’ve churned out with Rust that you couldn’t previously would be really interesting to me as I’m deciding whether I can afford the energy/aggravation to “push through”.

          1. 8

            Is there anything you could share that you feel is a particularly good example of something you’ve been able to develop in Rust on your own that you couldn’t have otherwise?

            Sure, these are some of my spare time projects:

            Of course all the things that can be done with Rust can also be done with C++… It might just take a decade longer to implement and fix until it’s stable ;)

            The projects above are of a size that pushes the boundaries of what I as a single individual can create and maintain in my spare time. I can confidently say that Sherlog would have never happened with C++.

            1. 3

              Thanks! That’s exactly the kind of thing I wanted to look at.

              I’ve learned new languages/libraries/frameworks enough over the years that this part isn’t surprising. I’m at that miserable point where every time I try to do something new, I spot a way to maybe do it, then spend a while looking at documentation to get it right, and in some cases spend time casting around search engines, forums, chat, etc. to find out what’s best/idiomatic.

              Some of this feels a little harder with rust than other things, but probably isn’t really. This is just the slow part of learning something.

              So it’s helpful to see those projects that you consider the “payoff”. Plus Sherlog looks neat. Thanks for sharing it.

              1. 1

                Focused, hard work is the real key to success. Keep your eyes on the goal, and just keep taking the next step towards completing it. If you aren’t sure which way to do something, do it both ways and see which works better.

                John Carmack

                Also, the Rust Discord is frequented by super nice and helpful people. They helped me more than once when I got stuck. Same thing with the gnome IRC, they also have a dedicated Rust channel.

            2. 4

              Personally its: If it compiles, it’s nearly done. I don’t have to think about += vs append - which in python might just change a lot of what is happening (rust won’t just nest two lists without me realizing, because types..). Or that I don’t have to worry about using c dependencies and their quirks that much (if at all with some very good bindings). Other than that: Yes it takes some time to get really productive. Till you acquired your base of crates and “framework” to re-use for most things (in which you can look up some of your specific solutions).

              1. 2

                100% agree here. Not a major rust user, but when I do it is both annoying and really nice how the compiler checks what I would usually catch in manual testing and debugging.

          2. 7

            Well, it’s a wave, they come and go :)

            1. 1

              Hahaha nice one, I’ll chill on a sandy beach of C/Fortran/Python and mine some coins =)

          1. -1

            Isn’t planning for the bonus just greed?

            1. 4

              That’s certainly what the company would like you to believe!

              1. 0

                And what do you believe?

                1. 1

                  I think it’s not. My last company had bonus included in our annual salary. It said something like: here is your base, there is your bonus, based on projections. It adds up to this total amount. It means that company was informing that is what you get… but don’t count in it.

            1. 2

              Is writing/publishing your main source of income?

              1. 3

                Unfortunately no, but I’ve treated it as a hobby up until this year. You know how people keep saying: “I want to be a writer” but never take it seriously? Well, that was me. I did write six books, and participated in two fiction anthologies, and it was the most fun experience for me. It is what I actually enjoy doing. Still, I’ve kept being a software developer for the past twenty years. I’m still working with software development as my main source of income, but I’m slowly ramping my book publishing with the goal to eventually becoming just a writer. There is a long road ahead of me.

              1. 1

                GCC and target architectures.

                1. 21

                  This post is a good illustration of why I keep an eye on the evolution of Zig. Rust’s core values are performance, safety, and ergonomics. Simplicity is not a core value of Rust and it’s becoming an increasingly complex language. (That’s not to say that Rust doesn’t care about simplicity: only that when simplicity and another core value are in conflict, simplicity will always lose.) If I were still in my twenties, in might not bother me as much, but as I’m inching to 40, I find it hard to keep up with the evolution of the language. I worry that there will come a point where I’m locked out of the ecosystem, because I haven’t been able to keep up. I think that as Rust releases go by, my Rust style will become less and less “modern”. In a sense, Zig is a possible escape hatch.

                  And it’s a shame, because I really like the first two core values of Rust, performance and safety. Zig has the same focus on performance as Rust, but since Rust’s memory safety often comes at the expense of simplicity, which is a Zig core value, I expect that Zig will not be able to match Rust’s guarantees against use-after-free, double-free, iterator invalidation, data races, etc.

                  1. 12

                    My concern with Rust is that not knowing it in 2029 will be like not knowing C or C++ today; it’s going to be ubiquitous in systems programming and a lack of knowledge will be a really career-limiting state.

                    My other concern with Rust is that it just collapses under its weight, complexity, feature creep, and speed of change (and this is said by someone who is growing to really like Rust, at least in its synchronous form).

                    1. 11

                      My other concern with Rust is that it just collapses under its weight, complexity, feature creep, and speed of change

                      C++ has enormous complexity, both intentional and accidental, yet it remains ubiquitous. Language complexity is sometimes justifiable.

                      Besides, I’d much rather learn numerous obscure details of how Rust keeps me safe, instead of numerous obscure details of how C++ is screwing me over. And I say this as someone who genuinely likes C++.

                      1. 5

                        This is part of what’s pulling me to brush up on Ada again.

                        Mostly because it is still used to build complex systems but it’d stayed relatively stable. Granted it’s user base is still pretty small but it’s there and enough to sustain a company to back some development of tooling.

                        1. 3

                          My concern with Rust is that not knowing it in 2029 will be like not knowing C or C++ today; it’s going to be ubiquitous in systems programming and a lack of knowledge will be a really career-limiting state.

                          Good point. But can you imagine many devs knowing all this crazy syntax, I doubt!

                          1. 7

                            It’s no more crazy than C++, its competition.

                        2. 9

                          I share your feelings and have also been curious about Zig. I’m not excited about going back to the land of memory leaks, use-after-free, and so on, though… I’d be curious to hear about others’ experience here; maybe zig’s additional creature comforts mitigate those issues somewhat.

                          1. 7

                            I have been using Zig in earnest for about 8 months now. At that time my default choice for new projects was Rust, so I picked up the biggest Rust project I had (comrak, perhaps the 2nd most popular CommonMark/GFM formatter in the ecosystem) and converted it to Zig. It was quite refreshing.

                            Since then I’ve built a number of tools with Zig, and I have honestly not had to deal with memory leaks, use-after-free, double-free, etc. almost ever. The general purpose allocator does what you would otherwise have to reach for Valgrind for, but better. The careful design of defer, errdefer and control flow around error handling make it quite a joy to use.

                            If you already grok manual memory management as a concept (say, you’ve used C quite a bit before), then Zig feels like it gives you all the tools you need to do it sanely without much overhead, like what you always wanted. A lot of people are repelled by the idea of doing it ‘manually’, but my experience is that it is the poor affordances C gives that generates that repulsion. I do recommend it.

                          2. 9

                            Yes, people who want simplicity to win in conflict at least once, should look elsewhere than Rust. This has been the case even before Rust 1.0. It’s just not what Rust is for. Simplicity is the lowest priority item in Rust.

                            1. 7

                              And it’s a shame, because I really like the first two core values of Rust, performance and safety. Zig has the same focus on performance as Rust, but since Rust’s memory safety often comes at the expense of simplicity, which is a Zig core value, I expect that Zig will not be able to match Rust’s guarantees against use-after-free, double-free, iterator invalidation, data races, etc.

                              …which is kind of the point where I fall of the bandwagon with Zig. Ok, so it’s simple, and in return for that simplicity it will merrily let you shoot entire limbs off in familiar ways that have had familiarly terrible consequences for users for as long as I’ve been alive.

                              What’s the value proposition here again? C but not better, just different?

                              1. 14

                                What’s the value proposition here again? C but not better, just different?

                                No macros and no void pointers thanks to comptime, slices (fat pointers) by default and a type system that can encode when a pointer-to-many expects a sentinel value at the end, optionals so no null ptr derefs, runtime checks for array boundaries, sentinels, non-tagged unions, and arithmetic programming errors, error unions to make it harder to mistakenly ignore an error, defer and errdefer for ergonomic cleanup of resources, etc.

                                You might want to take a look at the language reference and report your impressions on how better or not Zig is, compared to C.

                                1. 1

                                  merrily

                                  Not quite… Zig purposefully makes the ergonomics of major footguns poor and the resulting code ugly. For example, converting an array of u32 to u8 is 3 lines of ugly code that screams out “please CR this”

                                  Also zig async is amazing. It’s modestly hard, but the way it’s structured forces you to think about what the hardware is actually doing instead of abstracting away. Once you get it, it’s very easy, and there is no colored async, and you will probably have a correct mental model of what your code is doing

                              1. 6

                                Wow, good god! Okay, I hope I don’t need to use Rust for… at least couple of years. It’s funny how many people say that “rust is not changing”, but lib devs require the most fresh rust…

                                1. 22

                                  It’s funny how many people say that “rust is not changing”, but lib devs require the most fresh rust…

                                  I’m a “lib dev” (and a member of the library team), and several of my libraries still compile with Rust 1.28, which was released almost 2.5 years ago. That includes the regex crate. If I pushed its MSRV up to the most stable release, not a lot would change. I’d get some new niceties from an updated std and maybe a small convenience in some syntax. But that’s it. And there is absolutely nothing released in the last 2.5 years that has compelled me to upgrade its MSRV. (Nothing in particular is making me stay on Rust 1.28 either. I’ll eventually upgrade, but I do it at my leisure.) The last major feature (platform specific vector functions and runtime CPU feature detection) relevant to the regex crate was released in Rust 1.27.

                                  What’s more, aside from my role as a member of the library team, I’ve never needed to worry, care or use Pin. Likely because I don’t use async Rust because I don’t have a need to.

                                  There is a really simple explanation for this: different parts of the Rust ecosystem move at different speeds. Async Rust is still pretty young and there’s a lot of rough edges that need to be smoothed out.

                                  1. 11

                                    Just say no to async (I do) and you will be fine. Rust is a stable language that is joy to use (at least for me). Just avoid async.

                                    1. 7

                                      Yeah that’s basically my strategy with Python. Similar thoughts from a prominent Python developer:

                                      https://lucumr.pocoo.org/2016/10/30/i-dont-understand-asyncio/

                                      The number of problems that require async/await is very small. You generally need it for huge scalable cloud services, which is a problem most people don’t have. And even then most of the code can dispatch to threads; in fact it’s almost required to dispatch to threads in such settings for utiilization (to use all your cores).

                                      The existing cloud services are already written with non-Rust technologies (C++, Go, Erlang, etc.).

                                      Or maybe if you’re writing a BitTorrent client. You can do that in a bunch of different languages or with a manual event loop.

                                      Honestly there have been so many BitTorrent clients written that I wonder if any of the authors actually thinks async/await is an improvement for that problem (I have not written one). My guess is that 90%+ of them are written without any such language features, given their age.

                                    2. 9

                                      I mean, you don’t have to use async at all.

                                      I actually done use it in most of my code bases due to the poor patterns/ergonomics around async/await.

                                      I’ve worked on async/await patterns in other languages and Rust’s definitely has the leakiest abstraction. That may or may not be a good thing depending on how much you want that stuff to be transparent.

                                      As I noted in another comment, I find channels and threads easier to reason about.

                                      1. 4

                                        This is the really weird thing to me; it seems like there was this big push in the last decade towards async that was largely driven by the rise of a runtime (Node and browsers) in which async was literally unavoidable. On a runtime that has access to real threads, there are a handful of cases where the overhead of threads causes bottlenecks, but in my experience these are exceedingly rare and easy to avoid. On top of that, they tend to be domains in which the BEAM runtime dominates so conclusively that it’s difficult to justify building these systems without OTP.

                                        How is it happening that “the async question” is such a dominant factor in the discussions around Rust? Is it just due to people coming from other runtimes and assuming that “you can’t scale without async” or is there more to it?

                                        1. 3

                                          I’m not entirely sure. I spent most of my career was spent in C/C++ and so I got comfortable with multithreading in those languages early on. I wouldn’t argue that this is the best way, but it’s a way that has familiar patterns. In this model, Rust actually shines due to blocking a lot of the bad behaviors - sharing objects between threads and not properly locking, etc.

                                          However, async/await in Rust has felt awkward from the start to me. I don’t care enough about the async/await pattern to be too broken up about it though. If the community manages to iron it out and make it less rough around the edges, then I’ll invest more of my time into it.

                                          1. 2

                                            I believe early on two big use-cases without threads influenced the need for async. That’s not a first-hand knowledge, so it might be wrong. First, Fuchsia uses async Rust for its network stack, including the user-space drivers. Fuchsia devs were a major force shaping async. Second, you need async for wasm to interop with JS promises, and wasm was early or recognized as an important thing for Rust.

                                            I don’t know what explains “today”s discussions, I suspect it’s a combination of many less-technical factors. Async vs threads is important for web, and a lot of people do web. Async vs threads is controversial so you can argue a lot which is better than. Async in rust specifically is young and complicated, so there’s a lot to explain.

                                            1. 1

                                              That makes sense - I’ve definitely encountered the wasm scenario. You need async there simply because you’ve only got one thread, period. Instead of managing the cooperative multithreading aspect on a single thread, it’s easier to just use async.

                                              I think these are good uses of async.

                                            2. 1

                                              I still maintain that async/await is probably one of the best “bang-for-your-buck” concurrency patterns out there, up there with Actor models (popularized by BEAM and Akka), balancing developer simplicity with performance. I think the question of whether async is more effective than modern threads (with smaller thread stack sizes, and memory that’s only virtually allocated by the kernel until used) at solving C10K is different altogether. For the average developer, being able to do await func() to have a function run “asynchronously” is a lot simpler than thinking about thread pool sizes, sharding, and other things dealing with threads. I do think for the average concurrent application (so I’m talking about low-to-medium scale), threads are just as effective as async and perhaps even more so, but the developer experience of working with async is compelling enough that users are interested in async.

                                              Julia also has spent time baking in async into the language. It’s increasingly a higher level pattern that I think many developers enjoy using. I know here a lot of folks tend to prefer threads, but I think that’s not very applicable to the average application developer looking to use concurrency in their application without much work. Libraries like Rust’s rayon do really offer a compelling way to use threads without thinking too hard about the underlying complexities of spawning and retiring threads, but rayon style threading is only applicable to certain types of workloads.

                                        1. 6

                                          Can I ssh -X yet?

                                          1. 7

                                            You can

                                            waypipe ssh user@theserver weston-terminal!

                                          1. 1

                                            I’m wondering how many languages like Chapel, Fortran has survived. In 10 years there will be still fortran and openmpi, not sure about Chapel =)

                                            1. 0

                                              Good god the dependencies: https://github.com/uutils/coreutils/blob/master/Cargo.lock. Hopefully something like big stdlib (as in Python) or stl+boost (in C++) appears in rust. I bet coreutils can be written in Python without any dependencies, and in C++ with stl and boost.

                                              1. 22

                                                You’re looking at a computer generated file, which is easily over half self dependencies, and over a quarter about things like bench-marking tools, and the rest of it is mostly completely reasonable, and duplicated by dependencies organizing themselves into multiple crates (note that that file includes every transient dependency).

                                                If you want to see the actual lists of dependencies, cd into src and run rg '\[dependencies\]' -A20, and look for things under [dependencies] that don’t have path set (self dependencies).

                                                I’ll copy in some for you as an example, and annotate the first use of every dependency (note this is just the list of things that rg happened to spit out at the bottom for me)

                                                uu/id/Cargo.toml
                                                [dependencies]
                                                clap = "2.33"  # Command line parsing 
                                                
                                                uu/date/Cargo.toml
                                                [dependencies]
                                                chrono = "0.4.4" # Date time library
                                                clap = "2.33"
                                                
                                                uu/seq/Cargo.toml
                                                [dependencies]
                                                clap = "2.33"
                                                
                                                uu/chmod/Cargo.toml
                                                [dependencies]
                                                libc = "0.2.42"  # Literally just libc bindings (maintained by core team)
                                                walkdir = "2.2" # Does what it says on the tin
                                                
                                                uu/tsort/Cargo.toml
                                                [dependencies]
                                                getopts = "0.2.18"   # Command line parsing
                                                
                                                uu/tac/Cargo.toml
                                                [dependencies]
                                                getopts = "0.2.18"
                                                
                                                uu/nice/Cargo.toml
                                                [dependencies]
                                                getopts = "0.2.18"
                                                libc = "0.2.42"
                                                
                                                uucore_procs/Cargo.toml
                                                [dependencies]
                                                proc-macro2 = "1.0" # Rust maintained crate for writing procedural macros
                                                quote = "1.0" # Rust maintained crate for writing procedural macros
                                                syn = { version="1.0" } # Rust maintained crate for writing procedural macros
                                                
                                                uu/install/Cargo.toml
                                                [dependencies]
                                                clap = "2.33"
                                                libc = ">= 0.2"
                                                
                                                
                                                1. 3

                                                  Thanks for the reply. Can you explain, say I want to compile coreutils locally without any dev dependencies. What crates I should download (besides rust)?

                                                  1. 7

                                                    When you say locally, I assume you mean without internet (since cargo build downloads all dependencies and builds them locally by default), and without offline local caches that may or may not exist (which you can restrict cargo to using with --offline).

                                                    What you should do is be lazy and run cargo vendor, which will happen to download dev dependencies too. Unfortunately there doesn’t seem to be a flag to exclude dev dependencies, probably if you added one they would accept it.

                                                    If you’re just trying to get an idea what the dependency tree looks like, I recommend just running cargo tree -e normal.

                                                    If for some reason you’re insistent on finding every recursive dependency by hand though, the following command should work. It lists all the dependencies in a tree, filters out self dependencies, selects just the dependency name + version, and then sorts + de-duplicates them. This is a pretty bad way to get an idea what the dependency tree looks like though.

                                                    cargo tree -e normal | grep -v coreutils | egrep -o ' [a-z_-]+ v[^ ]+' | sort | uniq
                                                    
                                                    1. 1

                                                      I assume you mean without internet Yes.

                                                      Thanks!

                                                2. 4

                                                  Yeah, if you want someone to pile on, one could say some of the code is really dirty too. :) But I think it kinda makes sense to concentrate on compatibility with GNU coreutils first and only then root out unnecessary dependencies or increase performance or what not. All of those are much better suited to tackle after you have an existing test suite that just passes.

                                                  1. 11

                                                    FWIW, I believe the coreutils project started before Rust 1.0. If you thought Rust changed a lot now, you should have seen it before 1.0, especially in the months leading up to 1.0. Any code I wrote that survived that transformation became a complete and total mess.

                                                    1. 1

                                                      Yup, that’s also a very fair point. My comment above was in context of a conversation I had with Sylvestre earlier today and the diff at https://github.com/uutils/coreutils/commit/0a02aeda71c5faa1eb726deed69e7141bc20919d#diff-f27fff3d4fe45a009932c1fffaba0e58e6bb9fc085e22514e03c68d3d950dd4aR99, which I should have linked to. Oops :)

                                                    2. 1

                                                      Yeah, if you want someone to pile on, one could say some of the code is really dirty too. :)

                                                      It really is a replacement for coreutils, then.

                                                  1. 2

                                                    Again and again, languages that compile to C win. So at this point, it doesn’t matter how C is generated, “compiled” from another language or written directly, undef behavior is the main issue.

                                                    1. 2

                                                      Compilers can guarantee that the code they generate is free of undefined behaviour. Or at least of certain classes of undefined behaviour. Same way Rust compilers can guarantee the x86-64 instruction they generate are safe, even though using x86-64 assembly directly definitely isn’t.

                                                    1. 1

                                                      So basically one can write (is it possible?) a binary scanner that scans versions of the statically linked libs in a binary application (or library) and see if those have any CVSs and exploit.

                                                      1. 2

                                                        Project Zero has looked at this a bit. As I understand it it gets tricky because once you’re statically linking something, the compiler starts mashing up your code with the library code, so it’s not trivial to find library code. They had a 40% hit rate with the algo written up here: https://googleprojectzero.blogspot.com/2018/12/searching-statically-linked-vulnerable.html

                                                      1. 2

                                                        Rust people like to say Rust releases every six weeks, comparing to C++ how hard it is to use even C++17, but going forward, the days when you could use new Rust features within a year will be gone and won’t return. It is sad but a natural process.

                                                        1. 2

                                                          I think this will be more of a concern for packagers than developers.

                                                          I don’t believe a lot of people that actually develop in any of the newer language like Rust or Go, or even older languages like Node.js, Python or Ruby, care about how their application is packaged. You either use the language ecosystem, provide binaries, or provide another standalone installation method.

                                                          Packaging only becomes a concern once a packager shows interest, and then it’s too late for large changes to the build & release process.

                                                          1. 4

                                                            It may be true that developers don’t care, but users definitely do. So developers should care, once they acquired enough users.

                                                          2. 2

                                                            I’m still targeting the latest stable only, and get away with it. At Cloudflare we update Rust the day it’s released, and it’s in production within a week.

                                                            1. 3

                                                              I used C++17 at work in 2018, but no, that’s not representative. I think it will be the same for Rust.

                                                              1. 1

                                                                At Cloudflare we update Rust the day it’s released, and it’s in production within a week.

                                                                How many engineers work with Rust there? (compared to a small startup).

                                                                1. 2

                                                                  At least 3 startups worth of engineers ;) It’s now powering many critical components and user-facing services.

                                                                  Since Cloudbleed things are moving to Rust where possible. Rust is regularly picked for new projects (Golang and some other memory-safe languages are used too).

                                                                  So far compiler upgrades haven’t caused any major problems. The biggest issue was regression in compilation speed in 1.46.

                                                              1. 6

                                                                Downvoted as off-topic. This is business news, not technical content.

                                                                1. 6

                                                                  This is the kind of news that has a gigantic impact on future technical content.

                                                                  1. 2

                                                                    Is this a new thing here to explain your downvote?

                                                                    1. 5

                                                                      No, some people want to be transparent.

                                                                  1. 1

                                                                    Was the “web” interface written from scratch? Can cgit be used for this (https://git.zx2c4.com/cgit/about/)?

                                                                    1. 3

                                                                      Not trivially: Cgit does not run on 9front. This is hosted with a completely custom git implementation and it’s a lot easier to cat a few files than to port cgit. Also, a lot more satisfying to be fully self hosted.

                                                                      1. 2

                                                                        Thanks for the reply.

                                                                        Cgit does not run on 9front.

                                                                        You mean it can’t be compiled on plan9? (I have no idea what compilers plan9 has).

                                                                        1. 2

                                                                          There are many APIs that it uses – like mmap – which plan 9 does not have or want. Plan 9 is not a posix environment.

                                                                    1. 1

                                                                      Yeah!! Really nice post! Really nice pull requests, really nice examples of open source and closed source software working together!

                                                                      1. 1

                                                                        open source and closed source software working together!

                                                                        What do you mean by “closed source”? Qt 6 is indeed open source (GPL/LGPL), unless I’m very much mistaken.

                                                                          1. 1

                                                                            That says very directly that Qt is available under the LGPL/GPL. Both of those are OSI Open Source and FSF free software. What’s the “closed source software” you’re referring to when you call this “really nice examples of open source and closed source software working together”?

                                                                      1. -3

                                                                        It seems to be a common theme of prog-lang-started-by-child-prodigy projects that they adopt features where I simply can’t fathom how they are going to maintain and develop them in the mid-to-long-term.

                                                                        Perhaps I’m the only one who is concerned by the complexity these party-trick features seem to involve?

                                                                        (The other option is that this stuff is really that easy and all the hundreds of full-time C/C++ compiler engineers are just idiots for not doing it.)

                                                                        1. 33

                                                                          There are more details on why and how this works here: zig cc: a Powerful Drop-In Replacement for GCC/Clang

                                                                          The other full time C/C++ compiler engineers are not idiots; they just have different goals since they work for companies trying to turn a profit by exploiting open source software rather than working for a non-profit just trying to make things nice for everyone.

                                                                          1. 6

                                                                            The other full time C/C++ compiler engineers are not idiots; they just have different goals since they work for companies trying to turn a profit by exploiting open source software rather than working for a non-profit just trying to make things nice for everyone.

                                                                            This feels like a big statement, and that’s fine, but would you mind elaborating? Which companies do you mean? What goals do they have that are incompatible with something like zig cc?

                                                                            1. 5

                                                                              I think the point there was just that e.g. Apple has no particular interest in making using clang to cross-compile Windows binaries easy. They wouldn’t necessarily be against it, but it’s not something that aligns with their business interests whatsoever, so they’re very unlikely to spend any money on it. (Microsoft actually does value cross-compilation very highly, and has been doing some stuff in that area with clang, and so is almost a counterexample. But even there, they focus on cross-compilation in the context of Visual Studio, in which case, improving the CLI UI of clang again does not actually do anything for them.)

                                                                          2. 40

                                                                            Am I the only one who is concerned by the complexity these party-trick features seem to involve?

                                                                            (The other option is that this stuff is really that easy and all the hundreds of full-time C/C++ compiler engineers are just idiots for not doing it.)

                                                                            This mindset is one of the major causes why modern software sucks so much. The amount of tools that can be improved is humongous and this learned helplessness is why we keep having +N layer solutions to problems that would require re-thinking the existing toolchains.

                                                                            I encourage you to read the Handmade Manifesto and to dive deeper into how Zig works. Maybe you’re right, maybe this is a party trick, but the reality is that you don’t know (otherwise you would take issue with specific approaches Zig employs) and you’re just choosing the safe approach of reinforcing your understanding of the status quo.

                                                                            Yes, there are a lot of snake oil sellers out there, but software is not a solved problem and blanket statements like this one are frankly not helping anybody.

                                                                            1. 1

                                                                              I think you are wrong and the exact opposite is the case:

                                                                              We can’t have nice things because people don’t learn from their predecessors.

                                                                              Instead they go out to reinvent flashy new stuff and make grandiose claims until it turns out they ignored the inconvenient last 20% of work that would make their new code reliable and complete – oh, and their stuff takes 200% more resources for no good reason.

                                                                              So yeah, if people don’t want to get suspected of selling snake oil, then they need to be straight-forward and transparent, instead of having these self-congratulatory blog articles.

                                                                              Build trust by telling me what doesn’t work, and what will never work.

                                                                              1. 17

                                                                                Here’s what doesn’t work https://github.com/ziglang/zig/labels/zig%20cc.

                                                                            2. 7

                                                                              Clang could provide the same trivial cross compilation if it were a priority. Zig is mostly just using existing clang/llvm features and packaging them up in a way that is easier for the end user.

                                                                              1. 21

                                                                                “just”

                                                                                1. 4

                                                                                  Perhaps not obvious, but I meant the “just” to be restricted to “mostly just using existing clang/llvm features”. I’m in no way denegrating Andrew’s efforts or the value of good UX.

                                                                              2. 5

                                                                                Another option is that it’s easy if you build it in at the start and much more difficult to add it later. It’s like the python 2 to 3 migration. It wasn’t worth it for some projects, but creating a new python 3 project is easy. Path dependence is a thing.

                                                                                1. 2

                                                                                  I think the hard part is adding these kinds of features after the fact. But assuming it’s already in place, I feel like this is actually not a very hard thing to maintain?

                                                                                  I think a lot of complexity with existing tools is around “oh we’re going to have this be global/implicit” and that permeating everywhere, so then when you want to parametrize it you have to play a bunch of tricks or rewrite everything in the stack to get it to work.

                                                                                  But if you get it right out of the door, so to speak, or do the legwork with some of the dependencies… then it might just become a parameter passed around at the top level (and the lower levels already had logic to handle this, so they don’t actually change that much).

                                                                                  case in point: if you have some translation framework relying on a global, your low-level will read that value and do a lookup, and the high level will not handle it. If you parameterize it, now your high-level stuff has to pass around a bunch of translation state, but the low-level (I guess the hard part, so to speak?) will stay basically the same. At least in theory

                                                                                  I do kinda share your skepticism with the whole “let’s rewrite LLVM” thing… but cross compilation? Having a build system that is “just zig code” instead of some separate config lang? These seem good and almost simpler to maintain. I don’t think C compiler engineers are idiots for not doing X, just like… less incentivised to do that, since CMake isn’t a problem for someone who has spent years doing it.

                                                                                  1. 2

                                                                                    I agree with you. This doesn’t make any sense for Zig to take on. Andrew shared it with me as he was working on it and I thought the same thing then: what? Why does a compiler for one language go to this much trouble to integrate a toolchain for another language? Besides being severely out of scope, the problem space is fraught with pitfalls, for example with managing sysroots and dependencies, maintaining patched forks of libcs, etc. What a huge time sink for a group who should ostensibly have their hands full with, you know, inventing an entire new programming language.

                                                                                    The idea of making cross-compilation easier in C and C++ is quite meritous. See Plan 9 for how this was done well back in the naughts. The idea that it should live in the zig build tool, however, is preposterous, and speaks rather ill of the language and its maintainers priorities. To invoke big corporate compiler engineers killing open source as the motivation is… what the fuck?

                                                                                    Sorry Andrew. We don’t always see eye to eye, but this one is particularly egregious.

                                                                                    1. 7

                                                                                      No, this makes a lot of sense. Going back to the article, Go’s toolchain (like Plan 9’s) is good at cross-compilation, but “I recommend, if you need cgo, to compile natively”. This sort-of works for Go because cgo use is low. But Zig wants to encourage C interoperability. Then, Zig’s toolchain being good at cross-compilation is useless without solving C’s cross-compilation, because most of Zig will fail to cross-compile because of C dependency somewhere. By the way, most of Rust fails to cross-compile because of C dependency somewhere. This is a real problem.

                                                                                      Once you solved the problem, it is just a good etiquette to expose it as CLI, aka zig cc, so that others can use it. The article gives an example of Go using it, and mentions Rust using it in passing.

                                                                                      I mean, yes, zig cc should be a separate project collaboratively maintained by Go, Rust, and Zig developers. Humanity is bad at coordination. Big companies are part of that problem. Do you disagree?

                                                                                      1. 2

                                                                                        The best way, in my opinion, to achieve good C interop is by leveraging the tools of the C ecosystem correctly. Use the system linker, identify dependencies with pkg-config, link to system libraries, and so on. Be prepared to use sysroots for cross-compiling, and unafraid to meet the system where it’s at to do so. Pulling the concerns of the system into zig - libc, the C toolchain, statically building and linking to dependencies - is pulling a lot of scope into zig which really has no right to be there. Is the state of the art for cross-compiling C programs any good? Well, no, not really. But that doesn’t mean that those problems can jump domains into Zig’s scope.

                                                                                        I am a believer that your dependency’s problems are your problems. But that definitely doesn’t mean that the solution should be implemented in your domain. If you don’t like the C ecosystem’s approach to cross compiling, and you want to interoperate with the C ecosystem, the correct solution involves going to the C ecosystem and improve it there, not to pull the responsibilities of the C ecosystem into your domain.

                                                                                        Yes, other languages - Go, Rust, etc - should also be interested in this effort, and should work together. And yes, humanity is bad at cooperation, and yes, companies are part of that problem - but applying it here doesn’t make sense. It’s as if I were talking about poaching as contributing to mass extinction, and climate change for also contributing to mass extinction, and large corporations for contributing to climate change, and then conclude that large corporations are responsible for poaching.

                                                                                        1. 3

                                                                                          There’s another path to achieving C interop, which is by using whatever feels more convenient but staying true to whatever ABI boundaries. In terms of Zig, this is achieved in a few ways: It uses its own linker (currently LLD) which is useful when you don’t have a local system linker (pure linux/windows install) and still works with existing C code out there. It uses paths for dependencies, leaving it up to the user to specify how they’re found (e.g. pkg-config). It links to system libraries only if told explicitly but still works without them - this is also useful when building statically linked binaries which still work with existing C code.

                                                                                          For cross-compiling, sysroot is a GCC concept. This doesn’t apply to other environments like clang (the C compiler Zig uses), or the defaults of Mac/Windows. Zig instead uses LLVM to emit any supported machine code (something which requires having multiple compilers for in GCC), bundled the build environment needed (lib files on windows, static libc on linux if specified, nothing if dynamically linking), and finally links them together to the appropriate output using LLD’s cross-linking ability.

                                                                                          Having this all work seamlessly from whatever supported system is what makes it appealing. For example, andrew (creator of Zig) has showcased in the past cross-compiling the compiler on an x86 machine to aarch64, then using qemu to cross-compile the compiler again from the aarch64 vm back to x86, and it works. This applies also to other operating systems, which is a feature that isn’t present in current cross compiling tools, even clang.

                                                                                          For the issue of problem domains, this is not something you could address by trying to fix existing C tools. Those already have a defined structure as andrew noted above given they have different goals and are unlikely to change it. This could be why Zig takes upon solving these problems locally, and pulls the responsibility of what it wishes to provide, not the entire C ecosystem. I believe its partially of similar sub-reasons why Go has its own build system but also claims to compile to different environments.

                                                                                          I also agree that different ecosystems could pitch in for what seems to be a universally helpful tool, but as its been going on today, maybe they have different design goals. Where another path such as using the existing C ecosystem (for various situational reasons) makes more sense than the idealistic one Zig has chose to burden.

                                                                                          1. 1

                                                                                            It links to system libraries only if told explicitly but still works without them - this is also useful when building statically linked binaries which still work with existing C code.

                                                                                            System libraries can also be static libraries, and there’s lots of reasons to link to them instead. We do build statically linked programs without the Zig tools, you know!

                                                                                            For cross-compiling, sysroot is a GCC concept. This doesn’t apply to other environments like clang

                                                                                            Clang definitely uses sysroots. Where does it find the static libs you were referring to? Or their headers? The answer is in a sysroot. Zig may manage the sysroot, but it’s a sysroot all the same.

                                                                                            There’s more to take apart here, but on the whole this is a pretty bad take which seems to come from a lack of understanding about how Linux distributions (and other Unicies, save for macOS perhaps) work. That ignorance also, I think, drove the design of this tool in the first place, and imbued it with frustrating limitations which are nigh-unsolvable as a consequence of its design.

                                                                                            1. 3

                                                                                              The explicitly provided system libraries is not about dynamic vs static linking, its about linking them at all. Even if you have the option to statically link libc, you may not want to given you can do its job sometimes better for your use case on platforms that don’t require it (e.g. linux). The closest alternative for C land seems to be -ffreestanding (correct me if i’m wrong)? This is also an option in zig, but it also gives the option to compile for platforms without having to link to any normal platform libraries.

                                                                                              Clang has the option to use sysroots, but it doesn’t seem to be required. In zig’s case, it uses whatever static libs you need by you explicitly linking to them rather than assuming they exist upon a given folder structure in the same directory. Zig does at least provide some methods of finding where they are on the system if you don’t know there they reside given the different configurations out there. I’d say this differs from a sysroot as its more modular than “system library directory”.

                                                                                              Without a proper explanation, the idea that this approach “stems from lack of understanding” or has “frustrating limitations which are nigh-unsolvable” don’t seem make such sense. As we’re both guilty of prejudice here, i’d relate your response to one of willfully ignorant to unknown systems and gate-keeping.

                                                                                              1. 1

                                                                                                Clang has the option to use sysroots, but it doesn’t seem to be required.

                                                                                                Your link does not support your statement. I don’t think you understand how cross-compiling or sysroots actually work.

                                                                                                Again, it’s the same with the rest of your comments. There are basic errors throughout. You have a deep ignorance or misunderstanding of how the C toolchain, linking, and Unix distributions work in practice.

                                                                                                1. 4

                                                                                                  Given you haven’t actually rebutted any of my claims yet, nor looked into how clang supports using sysroots, we probably won’t be getting anywhere with this. Hope you’re able to do more than troll in the future.

                                                                                                  1. 1

                                                                                                    Clang totally uses sysroot, see here. (Long time ago, I wrote the beginning of Clang’s driver code.) I don’t know where to begin, but in fact, ddevault is correct about all technical points and you really are demonstrating your ignorance. Please think about it.

                                                                                                    1. 3

                                                                                                      Please re-read by post above which literally says “clang supports using sysroots”, a claim that agrees with yours. My original point a few messages back was about how clang doesn’t need sysroot in order to cross-compile, which still stands to be disproved, as its just short for a bunch of includes.

                                                                                                      Once again, just as ddevault, you enjoy making claims about others without specifying why in an attempt to prove some point or boost your ego. Either ways, if this is your mindset, there’s no point in further discussion with you as well.

                                                                                    2. 1

                                                                                      What features in this case?

                                                                                      1. 1
                                                                                    1. 23

                                                                                      This is fucking crazy good! Congrats on that deep integration with C/C++. I see the future here.

                                                                                      1. -2

                                                                                        It’s funny that it seems that engineers are so smart, but so naive sometimes.