Threads for l1x

  1. 11

    This is entirely subjective but I have looked at zig a few times and it does feel enormously complicated to the point of being unapproachable. This is coming from someone with a lot of experience using systems programming languages. Other people seem to really enjoy using it, though… to each their own.

    1. 8

      Huh, that’s interesting to hear. Out of curiosity what where the features you found the most complicated?

      I’ve had the exact opposite experience actually. I’m comparing against Rust, since it’s the last systems language I tried learning. Someone described rust as having “fractaling complexity” in its design, which is true in my experience. I’ve had a hard time learning enough of the language to get a whole lot done (even though I actually think rust does the correct thing in every case I’ve seen to support its huge feature set).

      Zig, on the other hand, took me an afternoon to figure out before I was able to start building stuff once I figured out option types and error sets. (@cImport is the killer feature for me. I hate writing FFI bindings.) It’s a much smaller core language than rust, and much safer than C, so I’ve quite enjoyed it. Although, the docs/stdlib are still a bit rough, so I regularly read the source code to figure out how to use certain language features…

      1. 17
        • An absurd proponderance of keywords and syntaxes. threadlocal? orselse? The try operator, a ++ b, error set merging?
        • An overabundance of nonstandard operators that overload common symbols. When I see | and % I don’t think “saturating and wrapping”, even though it makes sense if you think about it a lot. Mentioned earlier, error set merging uses || which really throws me off.
        • sentinel terminated arrays using D, Python, and Go’s slicing syntax is just cruel.
        • why are there so many built-in functions? Why can’t they just be normal function calls?
        • there seem to be a lot of features that are useful for exactly one oddly shaped usecase. I’m thinking of anyopaque, all the alignment stuff… do “non-exhaustive enums” really need to exist over integer constants? Something about zig just suggests it was not designed as a set of orthogonal features that can be composed in predictable ways. That isn’t true for a lot of the language but there are enough weird edges that put me off entirely.
        • Read this section from the documentation on the switch keyword and pretend you have only ever used algol family languages before:
        Item.c => |*item| blk: {
                    item.*.x += 1;
                    break :blk 6;
                }
        

        It’s sigil soup. You cannot leverage old knowledge at all to read this. It is fundamentally newcomer hostile.

        • what does errdefer add to the language and why does e.g. golang not need it?
        • the async facility is actually quite unique to zig and just adds to the list of things you have to learn

        Any of these things in isolation are quite simple to pick up and learn, but altogether it’s unnecessarily complex.

        Someone said something about common lisp that I think is true about Rust as well: the language manages to be big but the complexity is opt-in. You can write programs perfectly well with a minimal set of concepts. The rest of the features can be discovered at your own pace. That points to good language design.

        1. 13

          I’m thinking of anyopaque, all the alignment stuff…

          Maybe your systems programming doesn’t need that stuff, but an awful lot of mine does.

          what does errdefer add to the language and why does e.g. golang not need it?

          A lot! Deferring only on error return lets you keep your deallocation calls together with allocation calls, while still transferring ownership to the caller on success. Go being garbage-collected kinda removes half of the need, and the other half is just handled awkwardly.

          I didn’t quite see the point for a while when I was starting out with Zig, but I pretty firmly feel errorsets and errdefer are (alongside comptime) some of Zig’s biggest wins for making a C competitor not suck in the same fundamental ways that C does. Happy to elaborate.

          Someone said something about common lisp that I think is true about Rust as well: the language manages to be big but the complexity is opt-in. You can write programs perfectly well with a minimal set of concepts.

          Maybe, if you Box absolutely everything, but I feel that stretches “perfectly well”. I don’t think this is generally true of Rust.

          I think Zig’s a pretty small language once you actually familiarise yourself with the concepts; maybe the assortment of new ones on what looks at first blush to be familiar is a bit arresting. orelse is super good (and it’s not like it comes from nowhere; Erlang had it first). threadlocal isn’t different to C++’s thread_local keyword.

          I get that it might seem more unapproachable than some, but complexity really isn’t what’s here; maybe just a fair bit of unfamiliarity and rough edges still in pre-1.0. It’s enormously simplified my systems programming experience, and continues to develop into what I once hoped Rust was going to be.

          1. 8

            Being unfamiliar, nonstandard and having features for what you consider ‘oddly shaped usecase’ may be exactly what makes it a worthwhile attempt at something ‘different’ that may actually solve some problems of other languages, instead of being just another slight variant that doesn’t address the core problems?

            I personally think it’s unlikely another derivative of existing languages is likely to improve matters much. Something different is exactly what is needed.

            1. 2

              The question is how different we need. Everything must be different or just some aspects?

            2. 3

              what does errdefer add to the language and why does e.g. golang not need it?

              This is a joke, right? Please tell me this is a joke.

              Go does not need errdefer because it (used to?) has if err != nil and all of the problems that came with that.

        1. 2

          DNS over TLS was a great idea

          1. 3

            If you’re trying to attack DoH by that you’re not making a good argument. If you resolve that DNS without https you still might want to verify certificates. Or you’re even at the point that you resolved your IP address. Fine and now what ? Talk https to the service and fail to verify their certificates ? Or pray that timesyncd was running in between so it may randomly work.

            1. 3

              Generally speaking I do not like CA based encrypted communication. It breaks down easily and there are many ways to break it. You should be able to get the current time without much hassle especially without DoH or DoT getting involved which can be broken without proper time.

              1. 1

                Fully agreed that the CA system has many (well-documented) flaws. But it seems to me that the only realistic alternative at this point is plaintext, which seems worse?

          1. 7

            I wish there was a programming language that had a stable version that only changed for security reasons.

            1. 6

              If you avoid undocumented APIs, Java code has amazing longevity. It’s not quite what you’re asking for (the language and the runtime are both evolving) and sometimes “avoid undocumented APIs” is harder than it should be because innocent-looking dependencies might be doing funny business under the covers. But vanilla Java code compiled 20 years ago still runs perfectly well today.

              Libraries are where it starts to get tricky, though. “Only changes for security reasons” can look pretty similar to, “Only works on obsolete OS versions and hardware that’s no longer being manufactured” for some kinds of libraries.

              1. 6

                Common Lisp. Largely because the standards committee came up with a quite decent base specification, then packed up and turned the lights off.

                1. 2

                  I like programming in common lisp because I know that if I find a snippet of code from 20 years ago, there’s very little chance it won’t work today. And more importantly, that the code I wrote 10 years ago will still work in 10 years (unless it interacts with something outside (like a C library) that’s now out of date).

                  1. 3

                    Better yet, the code I write today is guaranteed to work if I can time travel back 30 years!

                2. 4

                  C and FORTRAN still have good support for ancient codebases.

                  1. 3

                    So maybe an LTS? Nobody is going to support a project indefinitely without financial support, so the best you can get is extension of the lifecycle.

                    1. 3

                      And companies like Redhat will happily sell you that and use your money to pay developers to backport and test fixes. The system works!

                    2. 1

                      JavaScript does this. Node.js doesn’t, and browsers don’t (though they’re reasonably close), and the ecosystem definitely doesn’t. But the core language from TC39 does

                    1. 1

                      ARM would be nice. New macs and rpi are popular.

                      1. 2

                        Fuchsia runs on ARM64, I just don’t think we’re building a darwin/arm64 host toolchain, yet.

                      1. 5

                        Fast or elegant. Writing Java in Clojure for higher performance is meh. This is what we get when trying to use a too high level of abstraction. It is really good that we have a simple way to observe performance and fix the bottlenecks though.

                        1. 7

                          The reality is that idiomatic Clojure without any optimizations is fast enough for vast majority of situations. Clojure is already significantly faster than a lot of popular languages like Ruby or Python out of the box. So, the amount of times you actually need to do these kind optimizations is pretty rare in most domains Clojure is used. However, having the ability to tweak performance to this level is really handy when you do end up in a scenario where that’s needed. I’m very much a fan of writing code with maintainability in mind as the primary concern, and optimizing things on case by case basis as the need arises. Clojure is hands down one of the best languages I’ve used in this regards.

                          1. 1

                            Clojure is already significantly faster than a lot of popular languages like Ruby or Python out of the box.

                            Any sources, if you don’t mind my asking?

                            1. 4

                              Totally anecdotal, but my rewrite of the Riemann monitoring system from Ruby to Clojure improved throughput by something like two orders of magnitude right off the bat, and that was the first thing I ever wrote in Clojure. With a little work, we went from ~700 events/sec (in ruby) to ~10 million events/sec per node. Not an apples-to-apples comparison there–that’s years of optimization, batching and network tricks, and different hardware, but like… coming from MRI, the JVM was a game changer.

                              1. 2

                                Here’s a post from AppsFlyer discussing a significant performance boost when we moving from Python to Clojure.

                            2. 1

                              I completely agree with you. My only problem is that is very hard to sell Clojure to people. Both clients and co-workers. Clojure and F# are the two default languages I use a lot for writing code for myself. Unfortunately team mates and clients force me to use Python, TS, C#, etc.

                              1. 3

                                My team’s been using Clojure for close to a decade now, and what I observed over that time is that developers experienced in imperative style often have a hard time getting into FP. Yet, when my team hires students who have little to no programming experience, they’re able to pick up Clojure very quickly and tend to love the language.

                                I think the big problem is with mismatched expectations. People build up a set of transferable skills that allow them to quickly move from one language to another as long as those languages are within the same family. If you learn a language like Ruby then you can quickly pick up Python, Js, or Java. You’ll have to learn a bit of syntax and libraries, but how you approach code structure and solve problems remains largely the same.

                                However, using a language like Clojure or F# requires a whole new set of skills that you wouldn’t pick up working with imperative languages. People tend to confuse this with FP being inherently more difficult as opposed to just being different.

                          1. 1

                            Does anyone use Prolog ? Because I only know of this being taught in school, nothing in production. For Ada we know of at least one industry using it.

                            1. 5

                              There are few projects. For example: https://github.com/terminusdb/terminusdb

                              1. 2

                                Prolog is used in a log of linguistics and processing. I know of a couple of companies using it.

                                Fun fact: my only paper credit ever is a piece of Prolog code ;).

                                1. 2

                                  Erlang was originally implemented in Prolog. The paper Use of Prolog for developing a new programming language (Armstrong, Virding, and Williams, 1992) is a short and fun read.

                                  1. 1

                                    Gerrit Code Review uses Prolog rules for allowing to customize rules (e.g. “What kind of Code-Review/Verified/… flags are needed, and by to whom, to allow a change to be submitted).

                                    The verifier of pre-2014 SPARK (that Ada variant) was written in Prolog.

                                    1. 1

                                      I just found out about ProbLog, and this is supposedly built in ProbLog: http://bioinformatics.intec.ugent.be/phenetic/index.html#/index

                                      1. 1

                                        Tau prolog is used by the Yarn package manager.

                                        1. 1

                                          I think it’ll become more and more important going forward. Mostly this is a safe bet since type systems are getting more important, but there’s a lot of other places where these ideas haven’t reached yet .. personally; I’d like to replace file systems with a logic programming engine.. and you also end up with a variant of logic programming if you only want to describe coordination free programs (which are a super important class of programs in the cloud world)

                                          1. 1

                                            Can you elaborate on what “coordination of free programs” means and how logic programming improves the situation there?

                                            1. 1

                                              The class of coordination-free distributed programs.. the programs that do not require coordination. Something like conflict free replicated datatypes for example (CRDTs).

                                              Edit: Logic programming enters the picture when you start talking about propagator networks; you can look at edward kmett talk about it or the keynote from popl this year.

                                          2. 1

                                            I use SWI Prolog for hobby stuff & a few little things in prod. There are a few companies that I know of which use it, to the extent that they’ve sponsored development of SWI and reached out for Prolog devs; in most cases though, the fact that they use Prolog is not advertised.

                                            1. 1

                                              More links to practical applications via here: https://lobste.rs/s/wdftoh/practical_application_prolog_1993

                                              My favourite is AusPig, an expert system for pig farming.

                                            1. 1

                                              Java is much older than Rust and carries backward compatibility weights. I am surprised it is not much slower.

                                              1. 2

                                                C++ is much older than java and I’d be surprised if it wasn’t in the same performance ballpark than Rust for optional. Old age is not an excuse for bad design.

                                                1. 5

                                                  Are you sure it is bad design? Perhaps Java’s design just makes different trade-offs for performance (like consistency or lower mental model overhead or… I’m sure there could be many).

                                                  1. 1

                                                    I can confidently say that it isn’t for consistency or mental overhead when compared to c++‘s generic implementation. Else this mess wouldn’t exist: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/function/package-summary.html and requiring fall back on external libs if you want more arguments, like https://projectreactor.io/docs/extra/3.2.0.M3/api/reactor/function/Consumer8.html

                                                    1. 1

                                                      I can confidently disagree, having seen the C++ standard library implementations of gcc, llvm, msvc, and a few others.

                                                      1. 1

                                                        why would the stdlib implementation matters ? what matters is that as a C++ dev I can just do optional<int>, optional<float>, optional<std::string>, optional<whatever_type> while in Java I have to remember that I must use OptionalInt, OptionalLong, OptionalDouble, and Optional<T> for everything else.

                                                        Anecdotally, I find libc++‘s implementation (https://github.com/llvm/llvm-project/blob/main/libcxx/include/optional) fairly readable, outside of the necessary __uglification of headers (and when one takes into account all the various optimizations that it does that Java’s does not seem to do - if I read https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/util/Optional.java correctly, it means that Java’s Optional does not work with value types since it relies on value being nullable ? how is that remotely serious)

                                                        1. 1

                                                          What matters is that in rust I can do Option<NonZeroU32> and have a type with sizeof 4. C++ can’t do that - how is that remotely serious.

                                                          I hope this will help you see your replies in this thread in a different way :)

                                                          1. 1

                                                            ? Of course it can, boost::optional did this for e.g. storing optional<T&> in a pointer for years. Sure, it’s a bit of work to specialize std::optional for your NonZero type but there’s nothing technically impossible

                                                            1. 1

                                                              Not really. Sure, you could maybe handcode specializations for a few selected types? Maybe? It’s not a trivial template. But even then this only handles some explicit/short list of types. In rust this is automatic for any type with ‘spare’ bits and works recursively (https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=004c4172a53ba7ab00fc7afb5e3adceb).

                                                              Also, since two posts before you complained about java needing external libs then don’t use boost as an excuse for c++ standard libs ;)

                                                          2. 1

                                                            If you were implementing your own generic libraries instead of just consuming them, then your view might be different.

                                                            For the language designers, both viewpoints are important.

                                                            1. 1

                                                              Having had to do this in Java, C# and admittedly mostly) C++, I really prefer the C++ way, especially nowadays with concepts.

                                                              1. 1

                                                                Concepts help, but they are quite recent. Also, C++ has objectively quite a few more footguns than the other languages, despite individual preferences.

                                                    2. 3

                                                      Java has a huge boat-anchor, the JVM. It’s stuck with a bytecode instruction set, binary format, and runtime model designed in the mid-1990s before the language had even seen serious use. There have been minor changes, like clarifying the memory concurrency model, but AFAIK no serious breaking changes.

                                                      This makes some improvements, like stack-based objects, either impossible, or achievable only with heroic effort by the JIT and only in limited circumstances. (See also: the performance limitations of Java generics due to the requirement of type-erasure.)

                                                  1. 4

                                                    and also were not suited to modern data science algorithms and workflows

                                                    The line when I closed the article. If you cannot make Java work for “modern” data science algorithms and workflows than you are holding it wrong.

                                                    1. 1

                                                      Can you elaborate?

                                                      1. 1

                                                        Java and specifically the JVM is one of the most optimized platforms out there. Several companies implemented many products using these especially for big data use cases. If you cannot make Java + JVM (or Graalvm) work for data science algos + workflows what makes you think that you can make Rust do these things where you have to implement roughly 10x code because there are no libraries that you can build on.

                                                        Google, Amazon, Netflix all publish tons of libraries that are designed for high performance computing and large scale data systems that millions of users use every single day. Can you tell the same thing about Rust?

                                                        I love the ideas of Rust but I miss roughly 90% of the libraries that I would need to be productive in it. There is also a higher barrier of entry with the concept of lifetimes and the additional complexity these introduce.

                                                    1. 1

                                                      But why?

                                                      What makes companies trying to solve really hard problems. Isn’t it obvious that they need a different data solution?

                                                      1. 4

                                                        In many cases, it’s lower risk (or at least, seen that way) to use something that’s well understood and do something convoluted in the application, than switch to something that _might) solve the problem, but nobody at the company understands.

                                                        Not to mention, that they’d left the migration way too late, and were already seeing complaints of poor performance.

                                                      1. 10

                                                        I think the SQLite recursive queries are cool, but constructing them is very challenging for my brain. We use a couple of them for loading graphs of data on the client at $WORK and they’re all bamboozling to me. I’d rather write a Datalog variant - I wish I had differential-datalog for every data store I work with.

                                                        1. 10

                                                          I explain a recursive CTE line by line here, perhaps you find it useful.

                                                          1. 2

                                                            Yes I think this (graph in SQL) is borderline misuse of technology but it works! :)

                                                          1. 13

                                                            I wonder why the kernel community seems to have structural issues when it comes to filesystem - btrfs is a bit of a superfund site, ext4 is the best most people have, and ReiserFS’s trajectory was cut short for uh, Reasons. Everything else people would want to use (i.e. ZFS, but also XFS, JFS, AdvFS, etc.) are hand-me-downs from commercial Unix vendors.

                                                            1. 13

                                                              On all of the servers I deploy, I use whatever the OS defaults to for a root filesystem (generally ext4) but if I need a data partition, I reach for XFS and have yet to be disappointed with it.

                                                              Ext4 is pretty darned stable now and no longer has some of the limitations that pushed to me XFS for large volumes. But XFS is hard to beat. It’s not some cast-away at all, it’s extremely well designed, perhaps as well or better than the rest. It continues to evolve and is usually one of the first filesystems to support newer features like reflinks.

                                                              I don’t see why XFS couldn’t replace ext4 as a default filesystem in general-purpose Linux distributions, my best guess as to why it hasn’t is some blend of “not-invented-here” and the fact that ext4 is good enough in 99% of cases.

                                                              1. 3

                                                                It would be great if the recent uplift of xfs also added data+metadata checksums. It would be perfect for a lot of situations where people want zfs/btrfs currently.

                                                                It’s a great replacement for ext4, but not other situations really.

                                                                1. 1

                                                                  Yes, I would love to see some of ZFS’ data integrity features in XFS.

                                                                  I’d love to tinker with ZFS more but I work in an environment where buying a big expensive box of SAN is preferable to spending time building our own storage arrays.

                                                                  1. 1

                                                                    I’m not sure if it’s what’s you meant, but XFS now has support for checksums for at-rest protection against bitrot. https://www.kernel.org/doc/html/latest/filesystems/xfs-self-describing-metadata.html

                                                                    1. 2

                                                                      This only applies to the metadata though, not to the actual data stored. (Unless I missed some newer changes?)

                                                                      1. 1

                                                                        No, you’re right. I can’t find it but I know I read somewhere in the past six months that XFS was getting this. The problem is that XFS doesn’t do block device management which means at best it can detect bitrot but it can’t do anything about it on its own because (necessarily) the RAIDing would take place in another, independent layer.

                                                                  2. 3

                                                                    I don’t see why XFS couldn’t replace ext4 as a default filesystem in general-purpose Linux distributions

                                                                    It is the default in RHEL 8 for what it’s worth
                                                                    https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/managing_file_systems/assembly_getting-started-with-xfs-managing-file-systems

                                                                  3. 2

                                                                    Yep. I use xfs for 20 years now when I need a single drive FS and I use zfs when I need multiple drive FS. The ext4 and brtfs issues did not increase my confidence.

                                                                  1. 2

                                                                    I still use fish. It does the autocomplete part very well which is 90% what I need from a shell. I do not have time to go through gazillion lines of shell configuration to get the same in zsh. I do not want to maintain this much configuration. My fish config is 40 lines. Most of it is git related.

                                                                    1. 2

                                                                      I really like fish(1) interactive mode and its really a decent shell. I would say that currently zsh(1) and fish(1) are the best currently available interactive shells.

                                                                      I tried to use it for a while (not forcing myself - just trying it) and I REALLY miss that I can not use these one liners/typical loops or pipes that I have been using for more then a decade in the fish(1) shell - like those:

                                                                      # for I in *.mp4; do ff.mp3.320.sh "${I}"; done
                                                                      # find . -type f -iname \*.jpg | while read I; do echo -n "${I} "; identify -format %Q; echo; done
                                                                      

                                                                      I really can not understand why the fish(1) developers would NOT implement POSIX /bin/sh syntax and invent their own one … its really a NO GO for me.

                                                                    1. 1

                                                                      There are many things wrong with AWS permissions. The article example is kind of weird. Generally speaking the new services does not play too well with the older services (config service with iam in this scenario).

                                                                      1. 16

                                                                        The most interesting part is this tool:

                                                                        https://github.com/GoogleContainerTools/distroless

                                                                        Pretty amazing small images.

                                                                        1. 20

                                                                          The devops people are slowly rediscovering static linking

                                                                          1. 10

                                                                            The main advantage of distroless seems to be dynamically linking glibc properly in a minimal container.

                                                                            1. 16

                                                                              what if the container was a file, and the file was an executable. that’d be pretty minimal. they’ll get there some day…

                                                                              1. 14

                                                                                And then the cycle will begin again. “Hey I have this file but it doesn’t run because it needs to be in the presence of this other file”.

                                                                            2. 4

                                                                              I am not sure what you mean. Static linking was always an option at different levels. Single JAR file is a statically linked package for Java apps for example. You can do the same thing with AWS Lambda packages (single zip).

                                                                              1. 11

                                                                                At least last I checked, a JAR file is a self-contained package but not actually statically linked. All the class path lookup stuff is still done, dynamically, when the program is run.

                                                                            1. 4

                                                                              Funny, I was expecting to read about all the subject he listed instead of password storage.

                                                                              1. 6

                                                                                Debian has become my favorite Linux distro recently. It is surprisingly good.

                                                                                1. 10

                                                                                  Why surprisingly? It’s the foundation of a lot lot lot of distros

                                                                                  1. 5

                                                                                    That’s interesting, because I can’t stand it. Things are changed at random so programs are different, and package management seems to like to break at random. What makes it your favourite?

                                                                                    1. 4

                                                                                      I’ve been using stable since 2008. I can remember one instance where pulling in a package update broke a program: it was when Chromium switched to requiring a GPU and then had a horrid CVE that no one could backport to stable; suddenly I couldn’t use it over ssh -X any more. Hard to place much blame on Debian for that though.

                                                                                      I’d imagine you’d see more breakage running unstable but like … that’s literally what it says on the tin.

                                                                                      1. 2

                                                                                        What in the world? There is hardly a less “things break at random” system than Debian.

                                                                                        1. 2

                                                                                          Apache’s configs being entirely different for no reason has bitten me more than once. Still not sure why that is, it’s pretty horribly annoying.

                                                                                          1. 1

                                                                                            Why would you think it’s for no reason? Actually that’s an example of exceptional care being taken to avoid things breaking at random. It means that you can have packages that provide Apache modules, and packages that provide entire websites (say, Wordpress or netdata), and expressible dependencies between them, and they don’t step on each other, local config is never blown away by upgrading packages, and generally speaking the sysadmin doesn’t get unpleasant surprises.

                                                                                            Admittedly, it doesn’t really feel “different” to me, because I got used to it 20 years ago and have only rarely had to deal with Apache on systems that didn’t do it that way, but what I will say is that it works, and that it’s not arbitrary — basically all of the packaging guidelines (which amounts to a book worth of material) come from a place of “how to be predictable, reproducible, and avoid breakage”. But it is assumed that admins know their system; being different from some other distro or upstream defaults isn’t breakage in and of itself, when those differences are justified and documented (which they certainly are in the case of Apache).

                                                                                            1. 2

                                                                                              I’d rather have the config files match with examples online (and the official docs!) than have any of those features, though. What do other distros do with Apache modules? I’ve never had breakage in them and they don’t do anything like debian.

                                                                                          2. 1

                                                                                            This might have been around 2004, when I encountered a legacy Debian Stable that had missed out on a stable release. So it was like oldoldstable.

                                                                                            Upgraded directly to the contemporary stable and everything went perfectly, despite skipping over a major release altogether.

                                                                                            It was beautiful.

                                                                                        2. 3

                                                                                          It’s been mine since around 1996. I try running other operating systems but I’m so used to the availability of software, the stability of updates and the general attention to detail that I have a hard time using anything else.