Threads for leighmcculloch

  1. 1

    My experience programming in Zig is mixed hard and easy too.

    Writing application logic is relatively easy, as the primitives in the language are a short easy learning curve.

    However, I also found learning how to correctly setup a build.zig file though, isn’t intuitive. Even after doing it a few times there’s no way I could do it from memory, and I end up copy-pasting a lot of the time there.

    Comptime had a smaller learning curve for me than Rust’s macro does.

    However, using comptime to write multiple types that satisfy an interface are non-trivial, and so this is a case where I frequently forget how to do it, and end up copy-pasting from a past project.

    And lastly memory. The article talks about the difficulty in choosing an allocator, but I found that to be the easy bit. The APIs of most types make it pretty easy to know when you need an allocator, and you just pass it in.

    However, remembering who “owns” some heap memory and is responsible for deallocating it, is just as hard as it is in C.

    1. 10

      This post has a curious absence of even mentioning distrust of Google with yet more of our personal data, which was a major theme of discussion in all venues I saw this issue mentioned in, and certainly seemed to play a major part in distrust of an opt-out configuration.

      1. 15

        Because they’ve been very clear: it doesn’t collect personal data. It’s designed to determine how and when people use various language features, anonymously.

        1. 3

          Yes, because Google is not a company that would like to collect more and more data overtime, because it is so trustworthy that a ton of people rejected this.

          1. 4

            The post claims to summarise the discussion. It glaringly does not summarise the discussion.

            Your comment doesn’t actually address what I said, it addresses something else entirely. Do you have an answer to what I said?

            1. 7

              I’m not understanding what you’re angry about. I agree with mperham that it’s quite a stretch to consider which language features are being used by a random golang toolchain installation as “personal” information.

              I also don’t see any claim to summarize the discussion in the original blog post.

              Seems like you just want to be angry at Google and yell at anyone who dares defend them, even when (in this case) Google seems to be trying to do the “right” thing.

              1. 2

                I couldn’t find where it claims to summarize the discussion. Can you quote the passage that leads you to believe that?

                1. 3

                  The second paragraph. You’re being disingenuous.

                  1. 2

                    I notice you didn’t provide a quote and instead accused me of bad faith. Cool.

                    1. 3

                      See my top-level comment for an explicit examination of the second paragraph. I would characterize it as manufacturing consent or astroturfing; a top-down plan is being described as community-approved.

            2. 1

              I assumed other topics fell under the statement in the second paragraph that said:

              In the GitHub discussion, there were some unconstructive trolls with no connection to Go who showed up for a while

            1. 15

              The main selling point of gofmt, for me, is not that it formats my code perfectly (I don’t care. I really really don’t care. I’ve worked with code bases that had half the functions completely squished onto line, from declaration to closing brace. I work with code bases that have multiple indentation levels in the same file. Nothing about formatting has ever prevented me from doing my job. I don’t get why people get so worked up over it).

              It’s that it makes sure time and energy isn’t wasted discussing meaningless subjective differences in opinion about formatting. It’s the joy of knowing that if I’ve run gofmt, whoever reviews it can’t be lazy and just point out a few formatting mistakes; they have to come up with something meatier.

              gofumpt might be made with the best possible intentions, it does re-open the can of worms that gofmt closed.

              1. 6

                I’ve liked the formatting gofumpt has enforced, but this has been my experience too. It creates diffs when I’m working across projects because not all are using it and I feel like I’ve gone back in time on this issue.

                1. 2

                  Setting this key in your repo’s .vscode/settings.json file will get all the VS Code people using it.

                  "go.formatTool": "gofumpt"
                  
              1. 6

                Typescript for now is the worst. It doesn’t show the line or the values but a cryptic, technically correct, error message. […]

                typescript/Error2.ts(4,17):
                

                That tells me the error is at line 4 column 17. What am I missing?

                1. 6

                  The author wants the line repeated and the error underlined with carets.

                  For my part, I use Go and I’m pretty happy with the short error messages because the LSP does error highlighting. It would be sort of weird for me to be surprised by a compilation error.

                  1. 4

                    Go’s short but high quality error messages make understanding what’s wrong a quick low effort glance. Extra lines would make it less convenient for me.

                    However, I get why the longer error messages are needed in Rust. I’m much more likely to be puzzled by a Rust compiler error, and when I am I need all the help I can get.

                    I think the type of error messages needed really depends on how complex the compiler is to understand.

                  2. 4

                    Typescript supports a pretty multi line output format, and claims that it’s enabled by default. Perhaps the author wasn’t using a TTY for stdout? Using a non-TTY switches to single-like machine readable output.

                    https://www.typescriptlang.org/tsconfig/#pretty

                  1. 5

                    I find this a bit silly. In which way is vim editing more keyboard centric than VSCode’s own shortcuts? To be honest I think current modern cua mode editors with their multi cursor, mark next occurrence, select inside, etc. Are kind of crossing keyboard usage effectiveness when compared to the two old editors. I am only missing a way to toggle between vertical and horizontal terminal split. Other than that, I don’t use the mouse at all.

                    Why do people equate keyboard-centric to vim movement?

                    1. 6

                      A big component is arrow keys. Using literal arrow keys sucks, as they are far from home row, but account for a big chunk of keystrokes. This can be solved by:

                      1. 2

                        Is that really that big of a deal compared to long keybond sequences that need to be typed repeatedly? What about escape? It is also outside the home row.

                        Feels like a meme without really ever asking these questions.

                        1. 11

                          In my experience, yes it is. It is much faster and much lower mental effort to type something without moving hands. Reaching for arrow keys it not that much worse better than reaching for mouse. Again for me, even godawful Emacs keybindngs were a marked improvement over arrows.

                          Escape is on CapsLock of course!

                          1. 1

                            It is not worse than reaching the mouse, it is leaps and bounds less disruptive for typing than reaching the mouse. The arrow keys are always in the same relative position, and closer, to the letters than the mouse. And place your hand with the keys home, end,pgup, pgdn, insert, which are central to cursos movement.

                            Why is moving the right hand (most people most agile hand) to the arrows such a deal breaker to com users, but moving the left hand to escape is fine? What exactly is the difference?

                            1. 2

                              I don’t need to move my hand to reach escape key – just extend the hand. The wrist stays where it is. To move to the arrow keys, I need to rellocate the whole hand.

                              I totally can see how having to reach for arrow keys is not a big deal for some folks. However, the bit about “What exactly is the difference?” I don’t get. Obviously there’s a huge difference between just extending the finger versus moving the whole hand?

                              And, to clarify, on the layout I use, escape is capslock, arrows are ijkl, home end pgup/down are uo<>.

                          2. 2

                            What about escape? It is also outside the home row.

                            I, like many vim users, has a map for escape for this reason. I use jj from normal mode.

                            1. 1

                              How do you escape insert mode?

                              1. 1

                                I mistyped. Should have read “jj from insert mode”.

                          3. 1

                            Using literal arrow keys sucks

                            Yes, but only if you’re arrow keys are in a sucky location. There’s little reason to use hjkl if arrow keys are in an ideal location.

                            I realize alternative keyboards arent as affordable or accessible though.

                        1. 3

                          I think there is so many dimensions to productivity it’s difficult for me personally to make a call on memory safety alone, but the author did a phenomenal job of discussing the options and they largely align with my own experiences.

                          For me my order of selection is: GC by default for anything I can use them on and I’d prefer that over the safety something like a borrow checker adds, then RC/borrow checker/MMM if I absolutely need it, but the selection of exactly which type specifically and which technology of that last group is more likely to be driven by other factors.

                          On the topic of the Rust’s borrow checker specifically, it has helped me write some applications that just worked first time in ways that I haven’t experienced in other languages, and anecdotally where less bugs have shown up. On the flip side I’ve struggled to TDD in Rust. I find that when I program in Rust I need to have a clear mental model of the types and interfaces ahead of time. Also, and maybe I’ve been unlucky with this one, but in my experience it’s much more likely that changing code over here could results in side-effect changes to code over there. Because of the difficulty to TDD I have less confidence in code I’ve written in Rust because I haven’t written failing tests before the functionality.

                          Other dimensions that I feel like play a large role in development velocity are API stability, solid standard libraries, fast and comprehensive tooling, and active communities on Stack Overflow, Discord, and elsewhere who are legitimately welcoming. These all go a long way to providing the primitives I need, helping me get to where I need to go whilst getting out of the way, getting me unstuck when I hit a wall, and make me feel like I’m part of a community where we’re all growing and not going it alone. The GC languages that have all these things are launch pads to developer productivity.

                          Maybe in the future we’ll get some languages with the other memory models that turn that around though.

                          1. 3

                            I’m looking at https://www.roc-lang.org/ and its notion of “platform”, allowing to chose/mix different memory models.

                          1. 7

                            Zig says no to almost every feature… it puts a high value on high leverage features that accomplish a lot, with a little…

                            +1 Similar philosophy, although different decisions, to Go. And it’s again really refreshing.

                            1. 3

                              Would you choose UUID v6/v7 if they were more stable and prevalent do you think? Or does the base32 encoding of ULIDs play a siginifics role in preferring them?

                              1. 1

                                Yeah, maybe, although I do think you still need a better encoding for URLs and such.

                                1. 1

                                  Well, canonical hex encoding isn’t enforced in any way. You can use whatever encoding format you want.

                              1. 2

                                This is great. A similar tool for generating Go and Rust structs from JSON documents is here: https://transform.tools/json-to-go.

                                1. 2

                                  json-to-go seems to only cope with a single JSON document. To handle multiple documents, see https://github.com/twpayne/go-jsonstruct

                                1. 7

                                  s := []string{“a”, “b”, “c”}

                                  a := [3]string(s)

                                  The ability to copy a slice to an array type of fixed size looks really convenient. Not that this isn’t impossible today with a few lines of code, but this feels natural and there are a few places I would have reached for this if it had been there. 👏

                                  1. 4

                                    For anyone curious, the current way, since go1.17, to do this conversion is

                                    a := *(*[3]string)(s) // a is [3]string
                                    

                                    The newer form makes this conversion more straightforward to write.

                                    1. 4

                                      and this sure as heck beat copying from the slice to the array before that:

                                      copy(array[:], slice[:len(array)])

                                    2. 3

                                      Yeah, I think for anyone working with fixed size blocks, like hashes, this is a real QoL improvement.

                                      1. 1

                                        Another usage site I’ve seen is binary encoding/decoding. binary.Write requires the provided data to be fixed size values, and converting a slice to an array makes it fixed size to meet this requirement.

                                    1. 3

                                      This would largely fix one of my great bugbears in the language. It still has a bunch of issues, though: it doesn’t look to cover enumerations, and ideally there’d be a match statement of some kind to allow for pattern matching, which this proposal lacks. I’ve seen a few other proposals for sum types, and hopefully some of the ideas from those might make it into a revised version of this.

                                      1. 3

                                        I think this proposal strikes the balance between uprooting the type system in Go and doing nothing. The fact that sum types would be interfaces means that most Go code would be unaffected and would seamlessly be able to support the use of sum types with existing semantics, but with the benefit that type authors can define types more accurately.

                                        I think the biggest issue is that this won’t bring about an Option<T>-like revolution to Go, since all sum-types could be nil. But, I think it might be a worthy trade-off.

                                      1. 7

                                        It’s probably worth noting that this is a problem only for large projects. Yes, a CI build of LLVM for us takes 20 minutes (16 core Azure VM), but LLVM is one of the largest C++ codebases that I work on and even then incremental builds are often a few seconds (linking debug builds takes longer because of the vast quantity of debug info).

                                        There’s always going to be a trade off between build speed and performance: moving work from run time to compile time is always going to slow down compile speed. The big problem with C++ builds is the amount of duplicated work. Templates are instantiated at use point and that instantiations is not shared between compilation units. I wouldn’t be surprised if around 20% of compilation units in LLVM have duplicate copies of the small vector class instantiated on an LLVM value. Most of these are inlined and then all except one of the remainder is thrown away in the final link. The compilation database approach and modules can reduce this by caching the instantiations and sharing them across compilation units. This is something that a language with a more overt module system can definitely improve because dependency tracking is simpler. If a generic collection and a type over which it’s instantiated are in separate files then it’s trivial to tell (with a useful over approximation) when the instantiations needs to be regenerated: when either of the files has changed. With C++, you need to check that no other tokens in the source file have altered the shape of the AST. C++ modules help a lot here, by providing includable units that generate a consistent AST independent of their use order. Rust has had a similar abstraction from the start (as have most non-C languages).

                                        1. 9

                                          I think that’s true for C++ but I use Rust for small projects and I definitely find compile times to still be a problem. Rust has completely replaced my usage of C++ but is strictly worse in terms of compilation time for my projects. Even incremental builds can be really slow because if you depend on a lot of crates (which is incredibly easy to do transitively), linking time will dominate. mold helps a lot here.

                                          I think this is always going to be a problem regardless of the compiler or optimizations because it’s just too tempting to use advanced macros / type level logic or pull in too many dependencies. In C++ I would just avoid templates and other slow to compile patterns, but in Rust these utilities are too powerful and convenient to ignore. Most of the Rust versions of a given library will try to do something cool with types and macros that’s usually great but you end up paying for it in compile times. I hope the Rust community starts to see compile times as a more pressing concern and designing crates with that in mind.

                                          1. 3

                                            I use Rust for small projects and I definitely find compile times to still be a problem.

                                            This is my experience. Unless we’re talking about ultra tiny workspaces, any small to medium sized project hurts, especially compared to build times with other tool chains like the Go compiler.

                                        1. 14

                                          Are compilation times a problem with Rust? Yes.

                                          From personal experience I was expecting the answer to be yes, but really hoping reading this blog that it was the opposite and there was a magical switch to flip to make it all better.

                                          The only thing I’ve found to make a material difference is running sccache locally and in CI, and using it to cache build artifacts. I can’t imagine building without sccache as it makes a noticeable difference to rebuilds, but it’s not enough. On GitHub this is the action I use to cache: https://github.com/stellar/actions/blob/main/rust-cache/action.yml.

                                          It’s not just build time, but also build resources. On GitHub Actions I’ve lost hours debugging not enough disk space errors, trying to keep a workspace build under the disk space limitations.

                                          1. 2

                                            I frequently run out of disk space, just building some TUI programs locally, and ending up with many gigabytes of output.

                                            I now have a script that removes <any-dir-with-Cargo.toml>/target.

                                          1. 6

                                            This looks awesome. What are you using to get syntax highlighting on the .up files so that when you use the ^ it changes colors up to the end of the code?

                                            1. 5

                                              That is thanks to the vim syntax support developed by @dvogel:

                                              https://github.com/adhocteam/pushup/tree/main/vim-pushup

                                              1. 2

                                                This is a very minor thing, but you can simplify the installation for vim-pushup if you make it a separate repo. That way, people can clone it directly into {neo,}vim’s package path (or use a package manager to do that). Copying individual files (from a larger git repository) directly into ~/.vim/ seems harder to manage than a repo that can easily be cloned and checked for updates.

                                                1. 1

                                                  Very nice. Thanks! Is there already a VSCode plugin?

                                                  1. 1

                                                    No, but LSP server is on the roadmap https://github.com/adhocteam/pushup/issues/15

                                              1. 2

                                                I’m using this in one of my projects and it works well so I guess there’s no urgency but I should look for alternatives. Does anyone have any recommendations?

                                                1. 5

                                                  My go-to’s for the past few projects have been Chi and, more recently, Echo.

                                                  Mostly Echo because it was the default for oapi-codegen whereas Chi is what I use when I’m not generating code. Both have worked well and seem actively maintained. Echo’s backing company, Labstack, does seem to have a dead homepage so… be wary.

                                                  1. 6

                                                    I use go-chi also, and I really enjoy it. I think it’s been able to learn a lot of lessons from earlier routers and middleware providers.

                                                    1. 6

                                                      I’m using chi as well. It provides just enough for API servers and is sufficiently unopionated.

                                                1. 2

                                                  Company: Stellar Development Foundation (non-profit)

                                                  Company site: http://stellar.org/foundation

                                                  Position(s): Senior, Staff, Backend, and Fullstack Software Engineers

                                                  Location: Offices in New York and San Francisco, or remote in US or Canada.

                                                  Description: Building an open financial network to support use cases like aid assistance in Ukraine, the anchor network (example), and other things that provide access to the global financial system for all. Software we build or maintain include p2p servers, APIs, full stack web applications with frontends, mobile applications, SDKs, wasm-based runtime.

                                                  Tech stack: Go, Rust, JavaScript (React), Python, C++

                                                  Compensation: See link below for salary ranges of each role.

                                                  Contact: Job Board

                                                  1. 7

                                                    I like Go. It’s a simple language, every project has a similar code style so it’s universally easy to read, it’s very easy to build production applications which have almost no dependencies, the list of positives goes on and on…

                                                    Every language has issues. I also like Ruby and that runtime has big issues too. Languages are tools and tool usage is always context specific.

                                                    But I have a commercial product written in Go and I’ve never had a large scale project like it that requires so little maintenance. It just continues to compile and work, year after year, without any effort on my part. This is the exact opposite of something like JS apps which need constant updating to giant dependency trees to keep them modern and working,

                                                    1. 7

                                                      A lot of people seem to focus on the pros and cons of the Go language itself, which unquestionably does have some polarizing design choices. But to me the productivity people get with Go seems more to do with the system around the language: the standard library, module system, single-binary deployment, tooling, and most importantly stability as a core value.

                                                      In use it feels like a system built by hardened cynical engineers who want the tooling to “just work” and get out of the way of their code. The goal is not to present “simple” or “magical” facilities, but to build everything with understandable mechanisms so you always know what’s going on under the hood.

                                                      The oft-quoted comment by Russ Cox about how it’s to enable novice programmers makes no sense to me, as the assumption everywhere in Go seems to be that you will constantly be aware of the ways you can screw things up, and read every word of the documentation before using a library.

                                                      1. 2

                                                        But to me the productivity people get with Go seems more to do with the system around the language: the standard library, module system, single-binary deployment, tooling, and most importantly stability as a core value.

                                                        This is my experience. While the language might not have every feature that I’ve dreamt for, everything I’ve built in it continues to work,easy to setup CI for, easy to debug, profile, and now fuzz too. The tooling is top notch.

                                                        There’s a lot of problems to solve in software engineering. The Go team have chosen a subset that are pragmatic.

                                                        1. 2

                                                          Then what’s the point of creating a new language instead of creating good tooling for an existing language?

                                                          1. 2

                                                            They aren’t independent. If you look at the design of Go, there are decisions in the language that enable the tooling, and vice versa. As an obvious example, the namespace for modules is literally version control URL prefixes. As a counterexample thought exercise, imagine how you’d make “good tooling” in this sense for C++ without being able to modify the language.

                                                      1. 5
                                                        1. 0

                                                          Realtalk: The only reason I have not and will not ever try it is the embarassing “brony”-name, and I say that as someone working with Coq.

                                                          1. 8

                                                            This makes me deliberately want to create and promote programming language whose name is an explicit My Little Pony reference. “Fluttershy” or something like that.

                                                            1. 6

                                                              wut? nothing on pony-lang’s website suggests anything at all to do with my little pony or “bronies”

                                                              https://www.ponylang.io/blog/2017/05/an-early-history-of-pony/

                                                              1. 0

                                                                Indeed, but the association can easily be falsely made.

                                                                1. 16

                                                                  I think you’re projecting pretty hard here.

                                                                  That article clearly gives the origins of the name.

                                                                  Back in the flight sim days, when I would make my friends groan by telling them yet again about all the things I was going to do when I wrote a programming language, one of the people I would tell was Nathan Mehl. And one time, when I gave him yet another laundry list, he said: “yeah, and I want a pony”.

                                                                  1. 2

                                                                    That’s also the origin of the “pony” mascot for Django.

                                                          1. 19

                                                            I think zig does offer more safety than the author lets on. It definitely offers flexibility, a good bootstrap story, and will work to make memory mistakes visible to the programmer.

                                                            Rust is better for critical stuff than C. Zig seems nicer to use, to me, than both.

                                                            1. 34

                                                              I think Zig’s approach to memory safety is not reasonable for any new system language. Temporal memory errors is about the single biggest problem large code bases, and Zig is the only new language that does not make any real attempt to fix this. We’ve got decades of evidence demonstrating that manual memory management is fundamentally unsafe as it requires no mistakes by a developer, but is also infeasible to analyze without language level support for tracking object life time. There are numerous ways a language can do this, from zero runtime overhaad (rust w/borrow check) through to a full tracing GC - though I believe the latter simply is not feasible in kernel level code so seems impractical for a general “systems language”.

                                                              I know whenever I bring up temporal safety someone chimes in with references to fuzzing and debug allocators, but we know that those are not sufficient: C/C++ projects have access to fuzzers and debug allocators, and the big/security sensitive ones aggressively fuzz, and use aggressive debug allocators while doing so, and those do not magically remove all security bugs. In fact a bunch of things the “debug” allocators do in Zig are already the default behaviour of the macOS system allocator for example, and the various webkit and blink allocators are similarly aggressive in release builds.

                                                              1. 5

                                                                I think it depends on your priorities and values and what you’re trying to do. Rust does not guarantee memory safety, only safe Rust does. If what you’re doing requires you to use a lot of unsafe Rust, that’s not necessarily better than something like Zig or even plain C. In fact, writing unsafe Rust is more difficult than writing C or Zig, and the ergonomics can be terrible. See for why wlroots-rs was abandoned for example.

                                                                From my experience, I’ve found Rust to do great when no unsafe code or complex lifetimes are involved, such as writing a web server. It also works great when you can design the memory model with lifetimes and the borrow checker in mind. When you can’t, or you have to deal with an existing memory model (such as when interacting with C), it can fare very poorly in terms of ergonomics and be more difficult to maintain.

                                                                1. 12

                                                                  Rust does not guarantee memory safety, only safe Rust does.

                                                                  And that’s great, because people are writing very small amounts of code in unsafe Rust.

                                                                  I’ve also implemented a programming language VM in Rust and found that you can usually define good abstractions around small amounts of unsafe code. It isn’t always easy (and wlroots legitimately seems like a hard case, but I don’t have enough domain expertise to confirm) but given how rarely you need to do it it seems manageable.

                                                                  But I think focusing on a few hard cases is missing the forest for the trees. If you can write the vast majority of code in a pleasant, safe and convenient language it very quickly makes the small part that you do need to be unsafe or fight lifetimes worth it.

                                                                  I would love to see a language that does even better and handles more cases. But until that comes along I would strongly prefer to use Rust which helps me write correct code than the alternatives just because in a few sections it doesn’t help me.

                                                                  1. 3

                                                                    But I think focusing on a few hard cases is missing the forest for the trees. If you can write the vast majority of code in a pleasant, safe and convenient language it very quickly makes the small part that you do need to be unsafe or fight lifetimes worth it.

                                                                    That’s fine, but we don’t need to have only one be all end all programming language. We can have different options for different situations, and that’s what Zig and other languages provide in contrast to Rust.

                                                              2. 15

                                                                I really enjoy using Zig more than using Rust. I have to work with existing C codebases, and Rust seems to hold C code at arms length, where Zig will let me integrate more fluidly. Both Rust and Zig seem like great replacements for C, but Zig seems to play more nicely with existing software.

                                                                1. 5

                                                                  I enjoy using Zig a lot more than Rust as well. While Rust has many fancy features, I am frequently fighting with them. And I’m not talking about the borrow checker, but rather limitations in the type system, tooling, or features being incomplete.

                                                                  But all that aside I think where I would use Zig in a commercial setting is limited, only because it is too easy to introduce memory use issues, and I don’t think the language is doing enough, at least yet, to prevent them.

                                                                2. 9

                                                                  This. I’ve been using Rust way more than Zig, and even used Rust professionally, for about 3 years, and I have to say Zig really gets a lot right. Zig’s memory safety system may not be on-par with Rust but that’s because it opts to make unsafe code easier to write without bugs. Traversing the AST (which is available via std.zig.Ast), you can get the same borrow checker system in place with Zig 🙂 Now that Zig is maturing with the bootstrapping issue figured out, I bet we’ll see this sooner than later, as it’s a real need for some.

                                                                  Zig is builder friendly. You ever try C interop with Rust? Rust is enterprise friendly.

                                                                  1. 17

                                                                    But lifetimes in rust are based on the control-flow graph not on the AST. Lexical lifetimes got replaced by non-lexical lifetimes back in 2018. Also there are lifetimes annotations, how would you get those?

                                                                    1. 1

                                                                      I know neither zig nor rust, and do not plan to, but: usually, a control-flow graph (along with more complicated structures besides) is built from an ast; is there some reason you can not do this for zig?

                                                                      1. 8

                                                                        It’s like the “draw two circles and then just draw the rest of the owl” meme.

                                                                        You can make CFG from AST easily, but then liveness and escape analysis is the hard part that hasn’t been solved for half a century, and even Rust needs the crutch of restrictive and manual lifetime annotations.

                                                                    2. 7

                                                                      Traversing the AST, you can get the same borrow checker system in place with Zig

                                                                      Do you have more reading on this? Because my understanding is that Rust’s borrow checker isn’t possible to infer all lifetimes from the AST alone. Hence it’s infamous lifetime annotations ('a). While Rust inference is generally pretty good, there are definitely plenty of scenarios I’ve run into where the inferred lifetimes are wrong, or ambiguous in which both cases require the additional syntax.

                                                                      1. 1

                                                                        Because my understanding is that Rust’s borrow checker isn’t possible to infer all lifetimes from the AST alone.

                                                                        It would absolutely be a learning experience for it to be explained to me why :) I don’t see why not: the AST describes the full program.

                                                                        1. 13

                                                                          If you could prove the safety (or lack thereof) of mutable aliasing entirely through syntactic analysis, the issues we have with C would be a lot less glaring. Having a parser in the standard library is incredibly cool, as is comptime, but you still have to build a static verifier for borrow checking your language/runtime of choice.

                                                                          I do think there’s a good opportunity for conventions and libraries to emerge in the Zig ecosystem that make writing code that’s “safe by default” a lot easier. You could absolutely force allocations and lookups in your code to use something like a slotmap to prevent race-y mutation from happening behind the scenes.

                                                                          Unfortunately that confidence doesn’t extend to 3rd-party libraries. Trying to transparently interface with arbitrary C code (which may in turn drag in embedded assembler, Fortran, or who knows what) means bringing all of the memory safety issues of C directly into your code base. See: literal decades of issues in image format decoding libraries, HTML parsers, etc. that get reused in nominally “memory-safe” languages.

                                                                          All of which is precisely why Rust does keep C/C++ at arm’s length, no matter how appealing trivial interop would be. Mozilla, MS, and Google/Android have custodianship over hundreds of millions of lines of C++ code, and yet they also understand and want the safety benefits Rust provides enough to deal with the more complicated and toilsome integration story vs. something like Zig or D.

                                                                          1. 1

                                                                            Right right, I made a big mistake to say “same”. Others mention annotations - those could be created in comments or in novel ways Im sure ! We’re starting to see things like CheckedC so yea, my thoughts still stand…

                                                                          2. 10

                                                                            But there isn’t enough information in the AST without annotations. Else you wouldn’t need mut and lifetime annotations in rust. The reason for those annotations is that the compiler couldn’t proof that the safe rust code doesn’t introduce new memory bugs (safe code can still trigger memory bugs introduced by unsafe code) without those annotations. The AST describes the whole program, that doesn’t mean that every property that is true for your code can be automatically inferred by an general algorithm. A well known example would be if your program computes in a finite amount of time.

                                                                        2. 3

                                                                          Doesn’t zig still require manually freeing memory at the “right time”?

                                                                      1. 14

                                                                        The issues with library (“crate”) organization are already apparent, and unless something is done about it relatively soon I think we’ll see a fracturing of the Rust ecosystem within 5 years. IMO the fundamental problem is that crates.io is a flat namespace (similar to Hackage or PyPI).

                                                                        For example, the other day I needed to create+manipulate CPIO files from within a Rust tool. The library at https://crates.io/crates/cpio has no documentation and limited support for various CPIO flavors, but it still gets the top spot on crates.io just due to the name. There’s also https://crates.io/crates/cpio-archive, which is slightly better (some docs, supports the “portable ASCII” flavor) but it’s more difficult to find and the longer name makes it seem less “official”.

                                                                        If I wanted to write my own CPIO library for Rust, it wouldn’t be possible to publish it on crates.io as cpio. I would face the difficult choice between (1) giving it an obscure and opaque codename (like hound the WAV codec, or bunt the ANSI color-capable text formatter) or (2) publishing it the C/C++ way as downloadable tarball[0] on my website or GitHub or whatever.

                                                                        Go has a much better story here, because libraries are identified by URL(-ish). I couldn’t publish the library cpio, it would be john-millikin.com/go/cpio or github.com/jmillikin/go-cpio/cpio or something like that. The tooling allows dependencies to be identified in a namespace controlled by the publisher. Maven has something similar (with Java-style package names anchored by a DNS domain). Even NPM provides limited namespacing via the @org/ syntax.

                                                                        [0] By the way, from what I can tell Cargo doesn’t support downloading tarballs from specified URLs at all. It allows dependencies to come from a crates.io-style package registry, or from Git, but you can’t say “fetch https://my-lib.dev/archive/my-lib-1.0.0.tar.gz”. So using this option limits the userbase to less common build tools such as Bazel.

                                                                        1. 3

                                                                          If I wanted to write my own CPIO library for Rust, it wouldn’t be possible to publish it on crates.io as cpio

                                                                          library name doesn’t have to match package name. You can publish jmillikin_cpio which people could use as use cpio.

                                                                          1. 8

                                                                            Yes but I think the point was that if I, as someone who doesn’t know anything about rust ecosystem, was looking for a cpio package, i probably would not go beyond the “official” cpio.

                                                                            1. 7

                                                                              If you had to choose between jcreekmore/cpio and indygreg/cpio, you still wouldn’t know which is the better one.

                                                                              1. 12

                                                                                That’s the point though I think: it makes it more obvious that you actually need to answer that question, because both look equally (not) “official”/“authoritative”.

                                                                                1. 4

                                                                                  I think you’re really stretching this example, because short of at_and_t_bell_labs/cpio there can’t be any official/authoritative cpio package. There may be a popular one, or ideally you should get a quality one. So to me this boils down to just search and ranking. crates-io has merely text-based search without any quality metrics, so it brings up keyword spam instead of good packages.

                                                                                  The voting/vouching for packages that @mtset suggests would be better implemented without publishing forks under other namespaces as votes. It could be an upvote/star button.

                                                                                  1. 4

                                                                                    If crates.io had organization namespaces, then an “official” CPIO library might have the package name @rust/cpio.

                                                                                    This would indicate a CPIO package published by the Rust developers, which would be as close to “official” as putting it into the standard library.

                                                                                    1. 3

                                                                                      That would be good for officialness, but I think it’s nether realistic nor useful.

                                                                                      We are approaching 100K crates. Rust-lang org already has more work than it can handle, and can’t be expected to maintain more than a drop in a bucket of the ecosystem. See what’s available on GitHub under rust-lang-nursery and rust-lang-deprecated.

                                                                                      And official doesn’t mean it’s a good choice. You’d have rust-lang/rustc-serialize inferior to dtolnay/serde. And rust-lang/mpsc that is slower and less flexible than taiki-e/crossbeam-channel. And rust-lang/tempdir instead of stebalien/tempfile, and rust-lang/lazy_static instead of matklad/once_cell.

                                                                                      1. 3

                                                                                        We are approaching 100K crates. Rust-lang org already has more work than it can handle, and can’t be expected to maintain more than a drop in a bucket of the ecosystem.

                                                                                        Yep! That’s true! In a healthy ecosystem, the number of official packages is extremely small as a percentage. Look at C++, C#, Java, Go – there might be a few dozen (at most) packages maintained by the developers of the language, compared to hundreds of thousands of third-party packages.

                                                                                        And official doesn’t mean it’s a good choice. You’d have rust-lang/rustc-serialize inferior to dtolnay/serde. And rust-lang/mpsc that is slower and less flexible than taiki-e/crossbeam-channel. And rust-lang/tempdir instead of stebalien/tempfile, and rust-lang/lazy_static instead of matklad/once_cell.

                                                                                        Also yep! And also (IMO) totally normal and healthy. The definition of “good choice” will vary between users. Just because a package is maintained by the language team doesn’t mean it will be appropriate for all use cases. That’s why Go’s flags package can co-exist with third-party libraries like github.com/jessevdk/go-flags or github.com/spf13/pflag.

                                                                                      2. 1

                                                                                        I wish you had put that at the top of your original post. :)

                                                                                        1. 2

                                                                                          I think it’s a minor point, to be honest. Even if crates.io never gets organizational namespaces, just being able to upload to a per-user namespace would be a sea-change improvement over current state.

                                                                                  2. 1

                                                                                    Personally, I think this is a great use case for a social-web system; we’ve already seen this with metalibraries like stdx and stdcli, though none have stood the test of time. I think a namespacing system with organizational reexports could really shine; I’d publish cpio (sticking with the same example) as mtset/cpio, and then it could be included in collections as stdy/cpio or embedded/cpio or whatever. Reviews and graph data would help in decisionmaking, too.

                                                                                2. 6

                                                                                  There’s some issues with that approach.

                                                                                  First, I do want the package name to match the library name, or at least be ${namespace}${library_name} where ${namespace} is something clearly namespace-ish. If I did not have this requirement then I would name crates.io packages with a UUID. And to be honest, I don’t think anyone would do that remapping – people would type use jmillikin_cpio::whatever and grumble about the arrogance of someone who uses their own name in library identifiers.

                                                                                  Second, a namespace provides access control. I’m the only person who can create Go libraries under the namespaces john-millikin.com/ or github.com/jmillikin/, but anyone in the world can create crates.io packages starting with jmillikin_. It’s just a prefix, it has no semantics other than implying something (ownership) to human viewers that may or not be true.

                                                                                  1. 4

                                                                                    And to be honest, I don’t think anyone would do that remapping – people would type use jmillikin_cpio::whatever and grumble about the arrogance of someone who uses their own name in library identifiers.

                                                                                    To clarify, it’s the author of the library who sets its default name. You can have the following in Cargo.toml of your cpio library:

                                                                                    [package]
                                                                                    name = “jmillikin_cpio”
                                                                                    
                                                                                    [lib]
                                                                                    name = “cpio”
                                                                                    

                                                                                    Users would then use jmillikin_cpio in their Cargo.tomls, but in the code the name would be just cpio.

                                                                                    This doesn’t solve the problem of access control, but it does solve the problem of names running out.

                                                                                    1. 3

                                                                                      Yes, per my post I’m aware that’s possible, I just think it would be bad. What you propose would be semantically equivalent to using a UUID, since the package name and library name would no longer have any meaningful relationship.

                                                                                      In other words, I think your code example is semantically the same as this:

                                                                                      [package]
                                                                                      name = "c3f0eea3-72ab-4e79-a487-8b162153cfd1"
                                                                                        
                                                                                      [lib]
                                                                                      name = "cpio"
                                                                                      

                                                                                      Which I dislike, because I think that it should be possible to compute the library name from a package name mechanically (as is the idiom in Go).

                                                                                  2. 4

                                                                                    Using a prefix doesn’t provide access control which is an important feature of namespacing. If there’s no access control, you don’t really have a namespace.

                                                                                    For example, I might publish all my packages as leigh_<package> to avoid collisions with other people, but there’s nothing stopping someone else from publishing a package with a leigh_ prefix.

                                                                                    This is a real problem, especially with the prevalent squatting going on on crates.io.

                                                                                    For example, recently I was using a prefix for a series of crates at work, and a squatter published one of the crates just before I did. So now I have a series of crates with a prefix that attempts to act as a namespace, and yet one of the crates is spam.

                                                                                    Most other ecosystems have proven that namespacing is an effective tool.