1. 23

    Kinda late on UNIX bashing bandwagon :)

    Also, Windows owes more of it’s legacy to VMS.

    1. 10

      It does, but users don’t use any of the VMSy goodness in Windows: To them it’s just another shitty UNIX clone, with everything being a file or a program (which is also a file). I think that’s the point.

      Programmers rarely even use the VMSy goodness, especially if they also want their stuff to work on Mac. They treat Windows as a kind of retarded UNIX cousin (which is a shame because the API is better; IOCP et al)

      Sysadmins often struggle with Windows because of all the things underneath that aren’t files.

      Message/Object operating systems are interesting, but for the most part (OS/2, BeOS, QNX) they, for the most part, degraded into this “everything is a file” nonsense…

      Until they got rid of the shared filesystem: iOS finally required messaging for applications to communicate on their own, and while it’s been rocky, it’s starting to paint a picture to the next generation who will finally make an operating system without files.

      1. 10

        If we talk user experiences, it’s more a CP/M clone than anything. Generations later, Windows still smells COMMAND.COM.

        1. 6

          yes, the bowels are VMS, the visible stuff going out is CP/M

          1. 4

            Bowels is a good metaphor. There’s good stuff in Windows, but you’ve got to put on a shoulder length glove and grab a vat of crisco before you can find any of it.

        2. 10

          I think you’re being a little bit harsh. End-users definitely don’t grok the VMSy goodness; I agree. And maybe the majority of developers don’t, either (though I doubt the majority of Linux devs grok journald v. syslogs, really understand how to use /proc, grok Linux namespaces, etc.). But I’ve worked with enough Windows shops to promise you that a reasonable number of Windows developers do get the difference.

          That said, I have a half-finished book from a couple years ago, tentatively called Windows Is Not Linux, which dove into a lot of the, “okay, I know you want to do $x because that’s how you did it on Linux, and doing $x on Windows stinks, so you think Windows stinks, but let me walk you through $y and explain to you why it’s at least as good as the Linux way even though it’s different,” specifically because I got fed up with devs saying Windows was awful when they didn’t get how to use it. Things in that bucket included not remoting in to do syswork (use WMI/WinRM), not doing raw text munging unless you actually have to (COM from VBScript/PowerShell are your friends), adapting to the UAC model v. the sudo model, etc. The Windows way can actually be very nice, but untraining habits is indeed hard.

          1. 6

            I don’t disagree with any of that (except maybe that I’m being harsh), but if you parse what I’m saying as “Windows is awful” then it’s because my indelicate tone has been read into instead of my words.

            The point of the article is that those differences are superficial, and mean so very little to the mental model of use and implementation as to make no difference: IOCP is just threads and epoll, and epoll is just IOCP and fifos. Yes, IOCP is better, but I desperately want to see something new in how I use an operating system.

            I’ve been doing things roughly the same way for nearly four decades, despite the fact that I’ve done Microsoft/IBM for a decade, Linux since Slackware 1.1 (Unix since tapes of SCO), Common Lisp (of all things) for a decade, and OSX for nearly that long. They’re all the same, and that point is painfully clear to anyone who has actually used these things at a high level: I edit files, I copy files, I run programs. Huzzah.

            But: It’s also obvious to me who has gone into the bowels of these systems as well: I wrote winback which was for a long time the only tools for doing online Windows backups of standalone exchange servers and domain controllers; I’m the author of (perhaps) the fastest Linux webserver; I wrote ml a Linux emulator for OSX; I worked on ECL adding principally CL exceptions to streams and the Slime implementation. And so on.

            So: I understand what you mean when you say Windows is not Linux, but I also understand what the author means when he says they’re the same.

            1. 2

              That actually makes a ton of sense. Can I ask what would qualify as meaningfully different for you? Oberon, maybe? Or a version of Windows where WinRT was front-and-center from the kernel level upwards?

              1. 2

                I didn’t use the term “meaningfully different”, so I might be interpreting your question you too broadly.

                When I used VMS, I never “made a backup” before I changed a file. That’s really quite powerful.

                The Canon Cat had “pages” you would scroll through. Like other forth environments, if you named any of your blocks/documents it was so you could search [leap] for them, not because you had hierarchy.

                I also think containers are very interesting. The encapsulation of the application seems to massively change the way we use them. Like the iOS example, they don’t seem to need “files” since the files live inside the container/app. This poses some risk for data portability. There are other problems.

                I never used Oberon or WinRT enough to feel as comfortable commenting about them as I do about some of these other examples.

            2. 2

              If it’s any motivation I would love to read this book.

              Do you know of any books or posts I could read in the meantime? I’m very open to the idea that Windows is nice if you know which tools and mental models to use, but kind of by definition I’m not sure what to Google to find them :)

              1. 4

                I’ve just been hesitant because I worked in management for two years after I started the book (meaning my information atrophied), and now I don’t work with Windows very much. So, unfortunately, I don’t immediately have a great suggestion for you. Yeah, you could read Windows Internals 6, which is what I did when I was working on the book, but that’s 2000+ pages, and most of it honestly isn’t relevant for a normal developer.

                That said, if you’ve got specific questions, I’d love to hear them. Maybe there’s a tl;dr blog post hiding in them, where I could salvage some of my work without completing the entire book.

            3. 7

              but users don’t use any of the VMSy goodness in Windows: To them it’s just another shitty UNIX clone, with everything being a file or a program (which is also a file). I think that’s the point.

              Most users don’t know anything about UNIX and can’t use it. On the UI side, pre-NT Windows was a Mac knockoff mixed with MSDOS which was based on a DOS they got from a third party. Microsoft even developed software for Apple in that time. Microsoft’s own users had previously learned MSDOS menu and some commands. Then, they had a nifty UI like Apple’s running on MSDOS. Then, Microsoft worked with IBM to make a new OS/2 with its philosophy. Then, Microsoft acquired OpenVMS team, made new kernel, and a new GUI w/ wizard-based configuration of services vs command line, text, and pipes like in UNIX.

              So, historically, internally, layperson-facing, and administration, Windows is a totally different thing than UNIX. Hence, the difficulty moving Windows users to UNIX when it’s a terminal OS with X Windows vs some Windows-style stuff like Gnome or KDE.

              You’re also overstating the everything is a file by conflating OS’s that store programs or something in files vs those like UNIX or Plan 9 that use file metaphor for about everything. It’s a false equivalence: from what I remember, you don’t get your running processes in Windows by reading the filesystem since they don’t use that metaphor or API. It’s object based with API calls specific to different categories. Different philosophy.

              1. 3

                Bitsavers has some internal emails from DEC at the time of David Cutler’s departure.

                I have linked to a few of them.

                David Cutler’s team at DECwest was working on Mica (an operating system) for PRISM (a RISC CPU architecture). PRISM was canceled in June of 1988. Cutler resigned in August of 1988 and 8 other DECwest alumni followed him at Microsoft.

            4. 5

              I have my paper copy of The Unix Hater’s Handbook always close at hand (although I’m missing the barf bag, sad to say).

              1. 5

                I always wanted to ask the author of The Unix Hater’s Handbook if he’s using Mac OS X

                8~)

                1. 5

                  It was edited by Simson Garfinkel, who co-wrote Building Cocoa Applications: a step-by-step guide. Which was sort of a “port” of Nextstep Programming Step One: object-oriented applications

                  Or, in other words, “yes” :)

                  1. 2

                    Add me to the list curious about what they ended up using. The hoaxers behind UNIX admitted they’ve been coding in Pascal on Macs. Maybe it’s what the rest were using if not Common LISP on Macs.

                2. 7

                  Beat me to it. Author is full of it right when saying Windows is built on UNIX. Microsoft stealing, cloning, and improving OpenVMS into Windows NT is described here. This makes the Linux zealots’ parodies about a VMS desktop funnier given one destroyed Linux in desktop market. So, we have VMS and UNIX family trees going in parallel with the UNIX tree having more branches.

                  1. 4

                    The author doesn’t say Windows is built on Unix.

                    1. 5

                      “we are forced to choose from: Windows, Apple, Other (which I shall refer to as “Linux” despite it technically being more specific). All of these are built around the same foundational concepts, those of Unix.”

                      Says it’s built on the foundational concepts of UNIX. It’s built on a combo of DOS, OS/2, OpenVMS, and Microsoft concepts they called the NT kernel. The only thing UNIX-like was the networking stack they got from Spider Systems. They’ve since rewritten their networking stack from what I heard.

                      1. 4

                        Says it’s built on the foundational concepts of UNIX.

                        I don’t see any reason to disagree with that.

                        The only thing UNIX-like …

                        I don’t think that’s a helpful definition of “unix-like”.

                        It’s got files. Everything is a file. Windows might even be a better UNIX than Linux (since UNC)

                        Cutler might not have liked UNIX very much, but Windows NT ended up UNIX anyway because none of that VMS-goodness (Versions, types, streams, clusters) ended up in the hands of Users.

                        1. 10

                          It’s got files. Everything is a file.

                          Windows is object-based. It does have files which are another object. The files come from MULTICS which UNIX also copied in some ways. Even the name was a play on it: UNICS. I think Titan invented the access permissions. The internal model with its subsystems were more like microkernel designs running OS emulators as processes. They did their own thing for most of the rest with the Win32 API and registry. Again, not quite how a UNIX programming guide teaches you to do things. They got clustering later, too, with them and Oracle using the distributed, lock approach from OpenVMS.

                          Windows and UNIX are very different in approach to architecture. They’re different in how developer is expected to build individual apps and compose them. It wasn’t even developed on UNIX: they used OS/2 workstations for that. There’s no reason to say Windows is ground in the UNIX philosophy. It’s a lie.

                          “Windows NT ended up UNIX anyway because none of that VMS-goodness (Versions, types, streams, clusters) ended up in the hands of Users.”

                          I don’t know what you’re saying here. Neither VMS nor Windows teams intended to do anything for UNIX users. They took their own path except for networking for obvious reasons. UNIX users actively resisted Microsoft tech, too. Especially BSD and Linux users that often hated them. They’d reflexively do the opposite of Microsoft except when making knockoffs of their key products like Office to get desktop users.

                          1. 3

                            Windows is object-based.

                            Consider what methods of that “object” a program like Microsoft Word must be calling besides “ReadFile” and “WriteFile”.

                            That the kernel supports more methods is completely pointless. Users don’t interact with it. Programmers avoid it. Sysadmins don’t understand it and get it wrong.

                            I don’t know what you’re saying here.

                            That is clear, and yet you’re insisting I’m wrong.

                            1. 3

                              Except, that’s completely wrong.

                              I just started Word and dumped a summary of its open handles by object type:

                              C:\WINDOWS\system32>handle -s -p WinWord.exe
                              
                              Nthandle v4.11 - Handle viewer
                              Copyright (C) 1997-2017 Mark Russinovich
                              Sysinternals - www.sysinternals.com
                              
                              Handle type summary:
                                ALPC Port       : 33
                                Desktop         : 1
                                Directory       : 3
                                DxgkSharedResource: 2
                                DxgkSharedSyncObject: 1
                                EtwRegistration : 324
                                Event           : 431
                                File            : 75
                                IoCompletion    : 66
                                IoCompletionReserve: 1
                                IRTimer         : 8
                                Key             : 171
                                KeyedEvent      : 24
                                Mutant          : 32
                                Process         : 2
                                Section         : 67
                                Semaphore       : 108
                                Thread          : 138
                                Timer           : 7
                                Token           : 3
                                TpWorkerFactory : 4
                                WaitCompletionPacket: 36
                                WindowStation   : 2
                              Total handles: 1539
                              

                              Each of these types is a distinct kernel object with its own characteristics and semantics. And yes, you do create and interact with them from user-space. Some of those will be abstracted by lower-level APIs, but many are directly created and managed by the application. You’ll note the number of open “files” is a very small minority of the total number of open handles.

                              Simple examples of non-file object types commonly manipulated from user-land include Mutants (CreateMutex) and Semaphores (CreateSemaphore). Perhaps the most prominent example is manipulating the Windows Registry; this entails opening “Key” objects, which per above are entirely distinct from regular files. See the MSDN Registry Functions reference.

                              1. 0

                                None of these objects can exist on a disk; they cannot persist beyond shutdown, and do not have any representation beyond their instantaneous in-memory instance. When someone wants an “EtwRegistration” they’re creating it again and again.

                                Did you even read the article? Or are you trolling?

                                1. 3

                                  None of these objects can exist on a disk; they cannot persist beyond shutdown, and do not have any representation beyond their instantaneous in-memory instance. When someone wants an “EtwRegistration” they’re creating it again and again.

                                  Key objects do typically exist on disk. Albeit, the underlying datastore for the Registry is a series of files, but you never directly manipulate those files. In the same sense you may ask for C:\whatever.txt, you may ask for HKLM:\whatever. We need to somehow isolate the different persisted data streams, and that isolation mechanism is a file. That doesn’t mean you have to directly manipulate those files if the operating system provides higher-level abstractions. What exactly are you after?

                                  From the article:

                                  But in Unix land, this is a taboo. Binary files are opaque, say the Unix ideologues. They are hard to read and write. Instead, we use Text Files, for it is surely the path of true righteousness we have taken.

                                  The Windows Registry, which is a core part of the operating system, is completely counter to this. It’s a bunch of large binary files, precisely because Microsoft recognised storing all that configuration data in plain text files would be completely impractical. So you don’t open a text file and write to it, you open a Registry key, and store data in it using one of many predefined data types (REG_DWORD, etc…).

                                  Did you even read the article? Or are you trolling?

                                  It sounds like you’re not interested in a constructive and respectful dialogue. If you are, you should work on your approach.

                                  1. -3

                                    What exactly are you after?

                                    Just go read the article.

                                    It’s about whether basing our entire interactions with a computer on a specific reduction of verbs (read and write) is really exploring what the operating system can do for us.

                                    That is a very interesting subject to me.

                                    Some idiot took party to the idea that Windows basically “built on Unix” then back-pedalled it to be about whether it was based on the same “foundational” concepts, then chooses to narrowly and uniquely interpret “foundational” in a very different way than the article.

                                    Yes, windows has domains and registries and lots of directory services, but they all have the exact same “file” semantics.

                                    But now you’re responding to this strange interpretation of “foundational” because you didn’t read the article either. Or you’re a troll. I’m not sure which yet.

                                    Read the article. It’s not well written but it’s a very interesting idea.

                                    Each of these types is a distinct kernel object with its own characteristics and semantics

                                    Why do you bring this up in response to whether Windows is basically the same as Unix? Unix has lots of different kernel “types” all backed by “handles”. Some operations and semantics are shared by handles of different types, but some are distinct.

                                    I don’t understand why you think this is important at all.

                                    It sounds like you’re not interested in a constructive and respectful dialogue. If you are, you should work on your approach.

                                    Do you often jump into the middle of a conversation with “Except, that’s completely wrong?”

                                    Or are you only an asshole on the Internet?

                                    1. 4

                                      Or are you only an asshole on the Internet?

                                      I’m not in the habit of calling people “asshole” anywhere, Internet or otherwise. You’d honestly be more persuasive if you just made your points without the nasty attacks. I’ll leave it at that.

                            2. 2

                              networking for obvious reasons

                              Them being what? Is the BSD socket API really the ultimate networking abstraction?

                              1. 7

                                The TCP/IP protocols were part of a UNIX. AT&T gave UNIX away for free. They spread together with early applications being built on UNIX. Anyone reusing the protocols or code will inherit some of what UNIX folks were doing. They were also the most mature networking stacks for that reason. It’s why re-using BSD stacks was popular among proprietary vendors. On top of the licensing.

                                Edit: Tried to Google you a source talking about this. I found one that mentions it.

                  1. 8

                    I would be very interested to see how this compares with say Carp. I’m not knowledgeable in this space though so it might be universes apart.

                    1. 3

                      Despite having posted this myself, I’m interested in such a comparison, too. Though, I hadn’t heard of Carp until today. (That’s the right link, right? The author recommends RabbitVM instead.)

                      So, I’d like to see a comparison to PicoLisp.

                      1. 7
                        1. 5

                          PicoLisp has a single data type internally, and as a result doesn’t do anything fancy in terms of garbage collection (there’s no real avenue for heap fragmentation). As such it’s just a mark-sweep collector.

                          newlisp does do something more fancy, Automatic Memory Management in newLISP, which is similar in nature to what Interim does, and what Rust does as well, albeit in a completely different way.

                          newLISP follows a one reference only (ORO) rule. Every memory object not referenced by a symbol is obsolete once newLISP reaches a higher evaluation level during expression evaluation. Objects in newLISP (excluding symbols and contexts) are passed by value copy to other user-defined functions. As a result, each newLISP object only requires one reference.

                          1. 1

                            newLISP memory management in practice felt like Tcl to me, and not in a good way. The details are different, but you end up doing similar kluges, where you pass a handle to data that must be manually GC’d if you need to pass large stuff around/mutate data. That’s radically different from what Rust does, both in broad strokes and in the details.

                            [Edit: accidentally wrote PicoLisp the first time I drafted this comment.]

                            1. 1

                              Yes, Rust is undoubtedly better in practice, though far more complicated to learn, and implement (from a compiler standpoint). A couple of hacks here and there and you get “arena-like” allocation properties, and it’s easy to implement? Yeah, seems like a good trade off for small, embeddable languages, to me.

                          2. 3

                            Oops! There’s some naming conflict there. I’m the author of the Carp and Rabbit you mentioned, both of which are small learning projects. https://github.com/carp-lang/Carp is probably the Lisp you are looking for.

                        1. 10

                          Quite a few of the things in this post are genuinely preferences (e.g. whether the type is before or after the argument, whether open classes are a good or a bad idea, whether data classes should…I guess not exist because they don’t work the way the author wants? I’m not clear what the point is), but some of them seem odd to me.

                          First, in 2014/2015, when Kotlin was much less mature and well-supported than it is today, and when documentation was much scarcer, I oversaw a team convert several medium-sized Java applications to Kotlin. We did this at a company that had tried and failed to adopt Scala several times due to its complexity, and several people on my team had survived those attempts. But we were up and moving in Kotlin in a couple of weeks, and successfully converted everything within a couple of months—with quantitative improvements in crashes and performance (largely due to a pile of NPEs the compiler caught in the process and resource leaks plugged by Kotlin’s .use semantics). That would seem to explicitly challenge his claim that it’s as complicated as Scala, and that the interop with Java is a problem in practice.

                          I also found their notes on companion/static objects odd. For example, “[o]ld good public static void main() is still the only way to launch a Java app,” which is true at the bytecode level, but in Kotlin, you’d normally just write top-level functions for that purpose. In fact, you can indeed do that for main itself: most Kotlin programs start with a simple top-level fun main(args: Array<String>). In fact, that’s literally the first block of code on the Kotlin home page. The reason for the companion objects is because statics in Java have piles of odd behaviors due to the fact that they are more global functions than genuine class methods, which is the exact reason companion objects exist in the first place: you can use them wherever an object would work, you can reasonably override methods, etc., and it’ll all work the same as it does for class instances. I.e., it’s simpler. Unless I’m misremembering, in our entire codebase, I’m not sure we had any statics, and we used the companion objects very rarely. Namespaced top-level functions and variables almost always took their place.

                          Similarly, the note about the lack of a Maybe monad is weird. First, the particular example is nonstandard to the point it’s hard to use it as a discussion point; (number ?: "0").toInt + 1 or number?.let { it.toInt + 1 } ?: 0 are both shorter than either the existing Kotlin or Java versions, and easier to read as well. But Kotlin can also just use the Java monad, and if you want a real monad, people have written the Maybe monad in Kotlin. It might be nice if it were in Kotlin’s stdlib, but it’s not hard to get.

                          There are a lot of reasons I can think of not to use Kotlin: you’re effectively tied to IntelliJ, compilation times are slower, Kotlin doesn’t really do SAMs, Kotlin makes it very difficult to figure out if you’re working with a primitive or not when you’re trying to write performance-sensitive code, etc. But most of the stuff in this article seems really does ring odd to me.

                          1. 4

                            This has a lot of overlap with ideas I’ve been working on independently (like small computing, composable GUIs, implicit over-the-network message passing on cluster computers, and stack based languages with visible state). Presumably Don & I have both been paying a lot of attention to Alan Kay :). Glad there are fellow-travelers around – for a long time it was easy to believe I was the only Xanadu/Smalltalk/ZUI guy around!

                            1. 5

                              I really like the “tentative guidelines for composable uis” post. Going to save and reflect on that a bit.

                              I’ve been working on a live-codeable interactive environment inspired by Self/Smalltalk etc., but using a Lua prototype-inherited scopes system (where code is “eval’d” in reified environments that can inherit from each other). After some iteration on such things I’ve realized it helps to have a concrete use case (allows testing, motivation, and empathizable communication with other people), but at the same time the choice of good use cases is important: if I choose a “make boring CRUD apps” use case, it basically involves porting existing libs + concepts and limits experimentation – so I went with a ‘generative art tool’ use case. This seems to let folks that look at it challenge their current ways of thinking about “software development.” It can evolve from static to animated art, then simulations (thus allowing games) and eventually hoping for network collaboration etc. Here are some videos:

                              https://www.youtube.com/watch?v=zDGzEUJscYE (making an art sketch)

                              https://www.youtube.com/watch?v=5-mxbhHBFOw (making another sketch)

                              https://www.youtube.com/watch?v=rRMeOGc1JLQ (slightly older, using it on the phone, you can see the browsing / inheritance here)

                              As you may have noticed, here too there is a concept of ‘sending messages’ to the scopes – all a message is is some code to eval at the scope, and that’s how you program objects to begin with anyways.

                              Some of the ‘composable UI’ stuff I’ve explored here is that like the .__tostring(...) metatable function Lua datatypes can support, the ‘console’ window will call a .__toui(...) metatable function on values you try to print in it (if defined), so that objects can provide their own renderers, you can set custom renderers for slots that you add to scopes (going to explore this soon for color picker widgets), …

                              I’m trying to use terminology like ‘perform’ etc. and more other art/human oriented words to move this tool away from “software engineering as a career choice” style orientation, as I think some of your blog posts also touch on. ‘Mindstorms’ by Seymour Papert along with some other readings are fun to explore here…

                              1. 2

                                This is fantastic!

                                Some of my earlier experiments in the composable-UI vein used Lua, but I found that it was easy to run up against both coroutine problems (lack of preemption) & limitations in the maximum number of identifiers in the global namespace. Your work here looks a lot more advanced than mine ever got.

                                (My current prototype is in Io, but I discovered that I would essentially need to rewrite Io to get a working system, because it stopped being maintained years ago & has problems with its speculative execution based thread planning.)

                                1. 3

                                  Yeah def. understand the global identifiers thing – in my prototype above globals are by default written to the ‘current object’ and scopes can inherit, so it sort of works like process environments in UNIX. You can really bend Lua to your will a lot – I do it by setting the metatable of the environment that code is eval’d in.

                                  Thanks for the nice words. :) I really like your writings so will be digging in there more. Definitely feels like you’ve thought about this stuff a lot and there’s good overlap. Will update you as I make more progress on this. Let me know if / when you have any more sources for me to grok!

                                  1. 2

                                    There’s a group of people interested in the subject of composable UIs, hypertext, and utopian attempts to fix what’s broken about computing as a whole, over on mastodon. Most of my discussion happens there, & a lot of what I write on Medium is a refined version of discussions I have there. You might find that stimulating – I don’t represent the views of the whole community, which overlaps with the generative art scene.

                                  2. 1

                                    (My current prototype is in Io, but I discovered that I would essentially need to rewrite Io to get a working system, because it stopped being maintained years ago & has problems with its speculative execution based thread planning.)

                                    I didn’t realize Io wasn’t maintained anymore! That makes me sad. By happenstance, I’d looked recently and found the repo itself to be quite active, but it does mostly look like keep-it-going maintenance, not heavy work. Ah well. I recall it having its own pretty cool UI toolkit back in the day, too.

                                    That said, would you mind explaining a bit about the speculative thread planning? I was just an undergrad last I used it, and I’d thought Io had a pretty normal cooperative threading system; this makes it sound like I really misunderstood something pretty cool, but I can’t find much (any?) info about this on the language site. (All I could find was the note, “The Scheduler object is responsible for resuming coroutines that are yielding. The current scheduling system uses a simple first-in-first-out policy with no priorities.”)

                                    1. 1

                                      There’s some kind of complicated heuristic for determining whether or not coroutines have already exited, when determining whether or not to transfer control to them. I ran into false positives with regard to that behavior, which were not entirely reproducible. I asked about the behavior in the irc channel, & was told that this was a known bug with the scheduler system, and one of the reasons active development was halted – the author didn’t think he could get the behavior right in C, if I understand the history correctly.

                                      I started implementing a new version in Go (a language I don’t know, but one that has support for channels and real multithreading built-in). This should allow me to more easily make it support a smalltalk-style image-based format with a history & support for transactions & rolling back execution, too, so it’s a general win. (Plus, since I don’t need to keep full compatibility with Io, I can break that compatibility if it makes it easier to make my composable UI system – the important bits are message passing, multi-threading, a prototype-based object system, and a simple syntax with few keywords to memorize, all of which can be preserved.)

                                2. 2

                                  Hell yes on composable components! Great articles, thanks!

                                  Here’s the money shot from an HN article I just posted about that:

                                  Valerie Landau interviewed by Martin Wasserman

                                  Q: Do you have any last minute comments or observations about him to finish up. Or a good anecdote?

                                  A: I think – I wanted to say one thing that Doug told me many years ago. And this is really for the software developers out there. Once, this was in the 90’s. And I said, Doug, Doug, I’m just started to get involved with software development, and we have this really cool tool we’re working on. Do you have any advice, about … for a young software developer. He looked at me and said:

                                  “Yes. Make sure that whatever you do is very modular. Make everything as module as possible. Because you know that some of your ideas are going to endure, and some are not. The problem is you don’t know which one will, and which one won’t. So you want to be able to separate the pieces so that those pieces can carry on and move forward.”

                                  https://news.ycombinator.com/item?id=17121629

                                1. 10

                                  I have not read the entire paper. Just the first 5 pages and the conclusion.

                                  I’d like to point out that there are significantly different ways for creating computer programs and this analysis seem to implicitly assume that the “static way” is the only way.

                                  In the “static way”,

                                  1. the programmers write large chunks of code in an editor (sometimes with some tests written beforehand),
                                  2. the code is sent to a compiler which help point out errors,
                                  3. if it compiles, the programs runs and the programmer observes the output
                                  4. the programmers makes updates as needed. Eventually adding other large chunks of code.

                                  (There are of course many variantions on this. Please correct me for a more accurate description.)

                                  In the more “dynamic way” of programming, a programmer is not only created in small pieces but potential new lines are tried out in the interpreter before deciding to add them or not. It is typically characterized by a short feedback loop.

                                  1. the programmers write a statement or line,
                                  2. this is sent to the interpreter,
                                  3. the programmer observes the result,
                                  4. make modifications and go back to 1 and/or add/keep some that line for the final program.

                                  Usually, there are lot of (language and IDE) features to make the 1-3 loop as fast as possible.

                                  In the Samlltalk IDEs, its possible to even run programs with mostly missing methods and Classes and fill them in at runtime (without ever restarting). To make such workflows possible, “superfluous” dynamic features are absolutely needed but will never be seen in individual codebases in the final product.

                                  The main reason I think this article assumes a strong static view is that they never discuss this aspect of dynamic languages (in the pages I’ve read).

                                  We started our analysis by looking at all 1850 software projects stored in Squeaksource in a snapshot taken in early 2010.

                                  They don’t look at dynamic feature use during the program creation phase at all, just the final product.

                                  One issue is that its always possible to program in a more static way in a dynamic language and IDE. And the see the lack of help from the compiler as a hinderance. In one never uses the dynamic features then it is easy to conclude they are a hinderance. But these features aren’t there solely for the purpose of being included in the final program.

                                  They found that the Python program in their corpus made a heavier usage of dynamic features during their startup phase, but that many of them also used some of these features during their entire lifetime.

                                  Of course, to perform any kind of study, some kind of assumptions need to be made. But they don’t say that this aspect is ignored to simplify the study.

                                  There are also things that can be done which can improve a language on one of the static/dynamic axis without regression on the other, especially at the beginning.

                                  As a final note, I’d like to mention one last use of dynamic features: to learn about a new module. I’ll usually read the high level readme of a new module and then immediately start playing with its objects in the interpreter without reading the documentation for individual classes and functions.

                                  Reflexive methods allow me to quickly find out what’s available and try things purely based on their names. I’ll send mock inputs to functions to try and figure out what their inputs and outputs are, observe the results and update my hypothesis about what they do.

                                  I find this gives me a much better and certain understanding of foreign modules and its a lot of fun to just poke around instead of just reading!

                                  Now I’m wondering if I should make a screencast of this.

                                  1. 9

                                    I completely agree with your point here, and I had the same reaction to the paper. I don’t program in Smalltalk anymore, but when I did, one of my favorite demos was what I called “debugger-driven development.” I’d start by just writing some code that would be what I wanted an API to look like–e.g., for a fortune program, maybe Fortune loadAll randomFortune–then execute DoIt, bringing up the debugger (since that class doesn’t even exist)–and then I’d add recursively add the missing methods until the original line completed. This leverages a whole swath of metaobject and reflective capabilities that the final code wouldn’t need, but without which, the entire Smalltalk development experience would be neutered beyond recognition.

                                    My other common demo was pretty different, but also did a great job using metaprogramming and reflective capabilities during development not needed in prod: I’d find a web API that was pretty cleanly designed and relevant to the audience I was speaking to. Next, I’d show how to write a kind of generic catch-all implementation via #doesNotUnderstand:, like what you might do in Ruby or Python. But then I’d modify it to generate the missing methods on-the-fly, and then (using a deliberately-made mistake in my first implementation) show how things like method modification timestamps and reflection could find the improperly-generated methods and regenerate them. (Sometimes, I’d even demo regenerating them as-needed with #perform: if I felt like it.) Finally, when we were done, I’d delete the #doesNotUndestand: implementation, customize a route or two, and be done. The final product didn’t use reflection, metaclasses, #doesNotUnderstand, or any of that kind of stuff, but the entire development process did. And unsurprisingly, a lot of the code for development did use things like self basicNew, foo perform: (a, b) asSymbol, and so on, even though the final wouldn’t.

                                  1. 6

                                    The fact that Guix is written in Scheme is a big appeal for me as opposed to Nix’s custom language. I preferred Nix as a way to support a standard environment (it has more packages), but this new feature makes the distribution of fat binaries a lot simpler than the other solutions. Less is more!

                                    1. 1

                                      FWIW, I tried to dissuade Gentoo from using Bash and Nix from creating their own language, both at basically around the 0.0.1 timeframe. I guess I am not terribly persuasive. Guix and Nix should merge. The separation is kinda ridiculous.

                                      1. 3

                                        Guix and Nix should merge.

                                        Seems like a great idea until you consider Guix’s commitment to freedom, and as a result blobless experience. Unless NixOS adopted that stance as well, the philosophical incompatibility would doom it. Nix adopting guile is more likely, I’d say, especially since guile did have a lua like front end that might make it a bit easier to slowly migrate everything…

                                        1. 2

                                          It is similar to vegetarian and non-vegetarian, one can have a blobless, freedom filled diet and then occasionally should they choose, sprinkle some bin01bits on top.

                                          1. 1

                                            I upvoted, but as a vegan, I kind of take offense to vegetarians (in a half hearted way, of course), who only “half” commit. But, I recognize that not everyone does it for the animals (even vegans).

                                            But, why would you go out of your way to run a completely free system, only to sprinkle some blobbits on it? That completely invalidates the point! That blob, is where the nasty things that disrespect your freedoms are.

                                            1. 1

                                              you wouldn’t run it for the freeness, but supposedly guix has some other strengths as well

                                          2. 1

                                            I didn’t realize Guix forbade blobs (though I’m not surprised, given its origin). Is there a with-blob version of Guix? I didn’t see one, but that doesn’t necessarily mean no…

                                            1. 1

                                              Obviously, you can acquire and install the blobs yourself, and I’m sure there are blog posts around in support of that. But, yeah, it’s like Trisquel, gNewsense, and the others that have similar governance for totally-libre.

                                              1. 1

                                                I haven’t used it in a long time, but I thought that you could point Guix at the package store from Nix, similar to how you can point Debian at apt repos from other sources. You would have to be really careful with this; I remember early on getting burned because I asked Nix to install Firefox and it gave me Firefox-with-adobe-flash which was pretty gross.

                                            2. 3

                                              Ha! Well, there must be an alternate universe where you managed to convince them ;) I think they do borrow some ideas and even some code (I remember a FOSDEM talk from Ludovic last year mentioning that). Implementation wise, I would suspect Guix has the upper hand, but the restriction to GNU packages is problematic not you need specific packages.

                                          1. 6

                                            Sometimes I wonder why recent languages don’t support multiple [return] values. We are even using destructuring assignment as a poor man’s multiple values. It solves the problem described in the article w/o cluttering the functions code with conditionals

                                            1. 2

                                              Most recent languages honestly do (Python, Elixir, Go, Rust, Kotlin, and Swift (I think) to name a few), and they’re getting backported in some others (C# 7 and C++17, for example) by improving tuples.

                                              I think the bigger issue with most of these languages is they don’t have anything like ML/Rust/Erlang-style fail-if-the-return-looks-like-this mode that lets you effectively use them as if they were exceptions when you want—e.g., the frequent pattern in Erlang/Elixir of Foo, ok = some_fun(), or Rust’s let v = bar()?;. Without that, you either do what Ruby’s doing here (throw an exception if exception: true is passed), or you have to explicitly check yourself (where Go is the extreme example of if foo, err := bar(); err != nil { ... } being a constant refrain).

                                              1. 1

                                                I don’t now about Elixir, Go or Kotlin, but neither Python or Go support multiple-values. In Go you have to accept to assign all the values the function returns, which kinda defeats the whole purpose of having multiple return values in the first place! (I checked in a REPL just to be sure). I haven’t kept up with recent developments in Python3 but AFAIK Pythonistas use lists or dictionaries to mimic multiple-values. Similarly in ES6 people use dictionaries + destructuring assignment in ES6. For Example

                                                const foo = () => ({a: 1, b: 2})
                                                
                                                const {a} = foo()
                                                
                                                console.log(a)
                                                

                                                Prints 1. The ES6 approach is better than Golangs and Python solution but it still forces the caller to reuse the name the callee decided on[0] plus all the return values are allocated on the stack.

                                                AFAICT Rust again doesn’t support multiple value but uses pattern matching (which destructuring assignment is one example of) to mimic them.

                                                Multiple values are useful for more than exception handling btw, one example is the division operator

                                                [0]: Yeah I know one can rebind the name, but the syntax is cumbersome to use and one still have to remember the name of the value as opposed to its purpose, which is easier to remember).

                                                1. 2

                                                  Well, now I’m confused. On the one hand, you say,

                                                  …neither Python [n]or Go support multiple-values

                                                  …and then immediately link to a REPL that starts with “Go supports multiple return values,” and even shows an example of ignoring some of the returned values. Likewise, while I agree with you that Python uses sequences for multiple returns behind-the-scenes, the use in practice looks like

                                                  def foo(a, b): return b, a
                                                  c, d = foo(1, 2)
                                                  

                                                  which is indistinguishable in practice from what I do in an ML. (And in Python 3.5 and later, you can even do things like a, b, *rest = foo().) Python even has multiple-return division operations (see math.fmod from as far back as Python 2.7).

                                                  Could you give an example of a language that does what you want, and how it differs from the previous? I feel as if we’re using the same words, but for radically different things.

                                                  1. 2

                                                    I’m sorry I wasn’t more clear in my previous reply.

                                                    Could you give an example of a language that does what you want, and how it differs from the previous?

                                                    Common Lisp.

                                                    and how it differs from the previous?

                                                    • The caller doesn’t have to be aware that the callee supports multiple values unless they want to use it.
                                                    • The compiler can determine at the call site how many values are going to be used and allocate appropriately.

                                                    which is indistinguishable in practice from what I do in an ML.

                                                    Yes, and to the best of my knowledge MLs don’t support multiple values. Pattern matching enables one to mimic multiple values, which is what most recent language are doing and it provides at least 90% of the value of having multiple values.

                                                    The Go REPL example shows that the caller must be aware of the amount of values the callee is returning, which leads to clumsy UX. I’m guessing you wanted to link to modf but I’m not sure what you were getting at. The CL example of / shows how we can re use the same function instead of having mod and div. Similarly be taken advantage retrieving the value in a map where nil indicates absence without affecting the UX of the ‘happy path’ (ie. nil is not possible a value in the map).

                                                    –––––––

                                                    It is kinda ironic that my comment was motivated out of the idea of less is more in language design, in light of all the feature creep I’m seeing recently in JS and Ruby, but it appears that the approach of recent languages is better example of less is more ¯_(ツ)_/¯

                                                    1. 1

                                                      to the best of my knowledge MLs don’t support multiple values

                                                      Returning multiple values (like in Go) is just a special case of tuples, which most MLs support. So you don’t need an extra language feature for this. But I wouldn’t use this for errors like Go does - it makes more sense to use a variant/sum type for this.

                                            1. 1

                                              The Laundry Files appears to be leaking.

                                              1. 26

                                                The circular shape of the letters hints at the eyes of the Go gopher

                                                They’re really stretching with that line. Two circles could be a lot of things and the first things that come to mind don’t have anything to do with the Go gopher. I don’t like this. It’s generic corporate, and I’ll miss the ode to Plan 9.

                                                1. 29

                                                  they needed an excuse to update the website and make it not work without javascript.

                                                  1. 16

                                                    i kind of respect the cynicism that went into this reply

                                                    1. 2

                                                      But, they didn’t update the website. The branding is entirely absent from the website. It’s boggling.

                                                      1. 3

                                                        oh, they will

                                                        1. 1

                                                          Over two weeks later and still no redesign of the website.

                                                    2. 15

                                                      It’s generic corporate

                                                      but… Go doesnt have generics! \s

                                                      Maybe it just supports the corporate interface.

                                                      1. 1

                                                        It’s generic corporate

                                                        Actually, it makes sense.

                                                        I’ll miss the ode to Plan 9

                                                        IMHO, Plan 9 had the technical potential to disrupt the centralized web. Years before it became a problem.
                                                        Now it’s a niche OS, developed by weird hackers that don’t buy mainstream buzzwords.

                                                        So, from certain points of view, Plan 9 is dangerous.
                                                        Something that any good programming minion should forget…

                                                        1. 4

                                                          Wait, does the COC rotate every time you click it? If so, are they being serious? The COCs generally looked really laudable, but now I can’t tell if they’re just posting them ironically.

                                                          1. 1

                                                            Wait, does the COC rotate every time you click it?

                                                            Yes.

                                                            If so, are they being serious?

                                                            Yes.

                                                            The COCs generally looked really laudable, but now I can’t tell if they’re just posting them ironically.

                                                            As my doctor say, irony is the worst side effect of intelligence.
                                                            A pretty serious side effect, that most people cannot tollerate.

                                                            (but since my doctor is my wife, she could be ironic about that… :-D)

                                                      1. 1

                                                        If you just want this kind of thing for dev tooling, the asdf extensible version manager (which has nothing to do with Common Lisp’s asdf-install) is probably a bit closer to solving your needs than a snap.

                                                        1. 1

                                                          Despite all Node.js-related content in this article, is anybody here using Snap or Flatpak already?

                                                          1. 3

                                                            I used FlatPak to install one or two Desktop apps but I wasn’t impressed. That could be the packagers’ fault or the system’s.

                                                            All real problems aside t’s also a bit annoying as you seem to have to run the applications with a long command line that I kept forgetting, maybe just providing a directory with shims/wrapper scripts with predictable names would’ve gone a long way (I mean, /usr/local/bin might be debatably ok as well)

                                                            My solution for non-GUI-heavy things so dar has been nixpkgs - so I can for example run a brand new git or tmux on Ubuntu 16.04.

                                                            1. 2

                                                              You might also be interested in checking out Exodus for quickly getting access to newer versions of tools like that. It automatically packages local versions of binaries with their dependencies, so it’s great for relocating tools onto a server or into a container. You can just run

                                                              exodus git tmux | ssh my-ubuntu-server.com
                                                              

                                                              and those tools are made available in ~/.exodus/bin. There’s no need for installing anything special on the server first, like there is with Snap, Flatpak, and Nix.

                                                              1. 1

                                                                Thanks, I’ve heard about exodus but I think it’s a bit of a hack (a nice one though) and first I’d need to have those new versions installed somewhere, which I usually don’t :)

                                                                I’m actually a big fan of package managers and community effort - just sometimes I’m on the wrong OS and would have certain tools in a “very fresh” state - so far nixpkgs is perfect for me for this.

                                                            2. 2

                                                              I use snap for a few things, and have even made a classic snap or two of some silly personal stuff. They seem to work fine, but ultimately feel out of place due to things like not following XDG config paths. They also get me very little over an apt repo, or even an old-school .deb, since most of the issues (e.g. you must be root) remain. Generally speaking, given that Linux distros already have package managers, I’m more interested in things like AppImage, which bring genuine non-package but trivial to install binaries to Linux.

                                                              (What I really want is to live in a universe where 0install took off, but I think that universe is gone,)

                                                              1. 2

                                                                Yes, quite a few popular projects: Spotify, Skype, Firefox, Slack, VLC, Heroku, etc.

                                                              1. 10

                                                                Pattern matching is one of the only features I regularly feel is lacking in Ruby.
                                                                There are other things I dislike or wish weren’t present, but the only missing feature is this.
                                                                Combining conditionals and assignment just feels 10x better than having them separate.

                                                                I even built a pattern matching library with some gross fun hacks on top of a specific namespace, thread-local singleton instance variables and caller_locations to allow for kinda-mostly pattern matching. I’m going to see if I can dig up the code, because aside from a first/rest pairing, I managed to get nested matching across Arrays and Hashes, and an Any object to match anything.
                                                                Then I bumped into a Ruby Tapas video on the topic and stole the hubcaps to implement Erlang-like guard clauses.

                                                                1. 6

                                                                  Have you looked at Qo at all? If so, any strong opinions about where it’s lacking? (Admittedly, it’s a giant hack, but it’s a pretty good giant hack.)

                                                                  1. 12

                                                                    Hello! Author of Qo here. Yeah, you’re right, it is a bit of a giant hack XD

                                                                    Anyways, I’d love to hear any ideas or suggestions on it. Typically I use the issue tracker to keep tabs on what I want to do with it, so feel free to jump in there as well.

                                                                    Admittedly I’m also the one who wrote a small-scale novel on pattern matching in the description to try and cover bases as I really really really want this feature, as evidenced by writing Qo in the first place.

                                                                    1. 3

                                                                      What do you think about the %p() or %m() syntaxes? I think they’re really ugly personally. Is it possible to make a spin on case which parses the when clauses as patterns without any special extra delimiters? You somewhat hit on that when talking about Scala style I think. Something like this maybe?

                                                                      match val 
                                                                      when [:constant, "constant", [1], variable_binding]
                                                                        variable_binding + 1
                                                                      else
                                                                        0
                                                                      end
                                                                      

                                                                      If you already covered that, apologies, I read your comments quickly last night and might have missed it.

                                                                      1. 4

                                                                        EDIT Codefied my idea here - https://bugs.ruby-lang.org/issues/14709#note-6

                                                                        match(value) do |m|
                                                                          m.when(/name/, 42) { |name, age| Person.new(name, age) }
                                                                          m.else { |object| raise "Can't convert!" }
                                                                        end
                                                                        

                                                                        An example practical usage:

                                                                        def get_url(url)
                                                                          Net::HTTP.get_response(URI(url)).then(&match do |m|
                                                                            m.when(Net::HTTPSuccess) { |response| response.body.size }
                                                                            m.else { |response| raise response.message }
                                                                          ))
                                                                        end
                                                                        

                                                                        Original:

                                                                        Not a problem, there are a lot of them (may Matz and Koichi forgive me)

                                                                        I hadn’t quite covered it yet. That’s always been the trick about this: what should it look like?

                                                                        Truthfully I don’t know quite yet, but I’m working on ideas. In regards to the %p and %m I would agree with some Reddit comments that they can tend slightly Perl-ish. I’d like a more fluent syntax if possible that reads intuitively, and I don’t think that quite does it.

                                                                        I had initially proposed this:

                                                                        new_value = match value
                                                                          when %m(:_, 20..99) { |_, age| age + 1 }
                                                                          else { |object| ... }
                                                                        end
                                                                        

                                                                        …which is quite similar to your suggestion. I’d almost consider switching the syntax a bit to something more like this:

                                                                        new_value = match value
                                                                          when matches(:_, 20..99) { |_, age| age + 1 }
                                                                          else matches(:_) { |object| ... }
                                                                        end
                                                                        

                                                                        When possible I would prefer very clear words as syntax over the percent shorthands. With this you could almost amend case to look for matches

                                                                        1. 1

                                                                          Ahhh, I hadn’t seen how binding the match results as block arguments would be valuable, but your example of using === overloads like Class.=== and Regexp.=== have convinced me. I learned pattern matching in Erlang so I was thinking about the Erlang style mostly, and I didn’t think of how pattern matching would be most useful in Ruby. Blocks are a good way to reuse the case style matching that’s already well understood.

                                                                        2. 1

                                                                          OOH I like this! You should counter-propose this syntax.

                                                                      2. 2

                                                                        I only just saw it via this proposal. Hope to find time to play with it today after work.

                                                                    1. 7

                                                                      One of the most popular libraries for doing such things is gobject. Compared with C++ it also offers much better interop with other languages. For example, gobject-based GTK has decent bindings for lots of languages, but C++-based Qt has only for few (for other languages bindings are incomplete, outdated or QML-only). Also there is Vala for this system, which is more like object syntax preprocessor for C than separate language (AFAIK, Objective C used similar approach in earlier versions).

                                                                      1. 3

                                                                        Indeed. I have extremely ambivalent feelings about GObject, but it’s largely COM done a bit more sanely. The plethora of language bindings testifies to that. My main complaint is that they didn’t literally just build off GNU’s already existing ObjC runtime.

                                                                      1. 4

                                                                        Thanks for posting this.

                                                                        A general problem I have with mercurial (I started using it for pet projects I work on at home, never at work), that a lot of material you can google is fairly old and lots of it outdated. Whenever someone refers to a hg extension, one needs to further investigate if this extension is still alive, and still the prefered way of doing things.

                                                                        1. 1

                                                                          The feature that this article describes is in core.

                                                                          1. 9

                                                                            Just to elaborate, because this is the third or fourth Mercurial discussion coming up in as many days, and I’m getting tired of the same discussion happening ad nauseam:

                                                                            1. Out-of-the-box, with no configuration done, Mercurial doesn’t allow editing history–but ships with all the functionality required to do that. You just have to turn it on. Turning it on takes up to three lines in a config file and requires no third-party tools whatsoever.
                                                                            2. Out-of-the-box, Mercurial does come with phases (discussed here) and the associated phase command that allows explicitly altering them. You don’t actually use the phase command that much; phases are actually more for consumption by history editing commands.
                                                                            3. If you enable any of the history editing extensions–again, which ship with Mercurial–including rebase, which is probably all you need, and histedit, if you do really need the equivalent of git rebase -i, you will find they are phase-aware. In particular, they will allow you to alter changesets that are secret or draft, but not public. Because changesets will become public on push by default, this is by itself awesome, as it can trivially help you avoid accidentally rebasing something someone else might’ve pulled. Having this would’ve eliminated quite a few Git horror stories.

                                                                            All of the above ships in core. You need to add at most three lines to your .hgrc or equivalent to get all of it. Which is fine, because you also need at least two lines just to set your email and name, much like you’d have to at least do git config --global user.email and git config --global user.name. A couple extra lines isn’t a big deal.

                                                                            The only thing interesting in this space that doesn’t yet ship in Mercurial, and which I’m really excited about, is something called changeset evolution, which will allow cleanly and easily collaborating on in-progress, frequently-rebased/collapsed/edited branches. But that’s not baked yet, and Git doesn’t have anything equivalent to it yet anyway.

                                                                            1. 5

                                                                              The problem is making it clear to new users or users coming from git how to enable those extensions. There’s also the problem that the new tweakdefaults option is trying to solve: that hg’s backward compatibility guarantees mean that new features (e.g. new since hg 1.0) don’t get surfaced in the hg UI unless you’ve customized your setup (or had it customized for you as in a corporate setup).

                                                                              git’s out-of-the box experience enables a lot of power user features. This certainly isn’t great for safety but it is great for discovery - thus these perennial discussions on forums like lobsters and HN.

                                                                              I’m hoping with evolve getting upstreamed we might see more projects using mercurial. On the other hand, for open source projects the only real place to host them is either to use hgweb and roll a custom hosting and development workflow (basically what mercurial itself does) or use bitbucket, which is run by people who don’t prioritize or think much about open source fork-based workflows. It would be amazing if there were more competition in this space. Kallithea doesn’t support pull requests. Rhodecode has a questionable past participating in the free software community. I’m not aware of much else in this space.

                                                                              What would really change things is if one of the big players like github or gitlab decided to add support for other VCS tools although I’m not exactly holding my breath for that to happen.

                                                                              1. 4

                                                                                Unfortunately, I agree. I have noodled with basically rewriting Kiln (only not called that because I’m not at Fog Creek) based on changeset evolution and with an explicit fork-based workflow focus, but I’ve been waiting to see if evolution really does get fully into core, and then what the story is with hg absorb, since properly supporting Mercurial and evolution looks really different in an hg absorb-based world than one without it.

                                                                                In particular, my idea is that anyone can push to a repository, but it’ll automatically go into a new pull request in draft phase. At that point, if hg absorb stays a thing, and a normal user can be expected to just run hg absorb repeatedly as they address issues, then I can rely on obsmarkers. Otherwise, the story empirically gets a lot more interesting; I’ve had trouble on prototypes not requiring the user to know they’ve got a custom branch, basically doing the same kluge as Gerrit, albeit with a different mechanism.

                                                                                Edit: just to be clear, I’ve prototyped bits of this a few times, but nothing I want to release—doubly so since it’s all in Smalltalk anyway. But it’s been helpful to try to think through what a proper approach to this would work like.

                                                                                1. 2

                                                                                  AFAIK the only blocker on absorb getting upstreamed is the need to rewrite the linelog interface in C.

                                                                                2. 2

                                                                                  I’d like to add that RhodeCode is actively supporting usage of Evolve/Phase with changeset evolution. Based on feedback from our users we started to ship evolve extension enabled and within the CE and EE editions.

                                                                                  This works with Pull requests, can be enabled globally or per repository.

                                                                                  You might question the past, but we since almost 2 years provide a open-source free, no limitation version of CE edition of RhodeCode (similar to Gitlab CE/EE). You can use evolve there and it works :) I said it many times, and I’ll said it again. We did mistakes with certain past releases, but currently, our business model is based on a free GPL open-source version. This will be here to stay, we always try to promote Mercurial, and it’s great workflows using mentioned extensions.

                                                                                  I doubt Gitlab/Github will ever support mercurial. They openly said they won’t for many reasons.

                                                                                  We currently work on a simple app for Digitalocean, we hope it’ll make usage of Mercurial hosting much easier for people that don’t want to host it themselves.

                                                                                  1. 1

                                                                                    Kallithea doesn’t support pull requests.

                                                                                    Looks like they now do.

                                                                                    I’m not aware of much else in this space.

                                                                                    Phabricator, Redmine and Trac also support Mercurial as well.

                                                                                    However none of them are as convenient as the hosted and “free for open source” offerings of Bitbucket, GitHub and GitLab.

                                                                                  2. 2

                                                                                    I feel the need to fork hg2k5, which will be mercurial with only the original features. :)

                                                                                    1. 1

                                                                                      You’d have to start with forking Python 2.4 so that you could run it.

                                                                              1. 3

                                                                                Oreo came out awhile ago, right? Is this a new feature that’s coming out everywhere due to the HAL, or is this an old feature that just happened to get a blog post? I’m genuinely asking here, I don’t know.

                                                                                1. 1

                                                                                  Sounds like it may be the latter.

                                                                                1. 1

                                                                                  Can I ask a potentially ignorant question? Why would someone who’s not already using Subversion choose to run it at this point? What are some of its advantages over Git or Fossil or Mercurial?

                                                                                  1. 6

                                                                                    For software version control? Probably very little (especially as you included mercurial in the alternatives)

                                                                                    I think however, that SVN could be the basis of quite a good self-hostable blob/file storage system. WebDAV is a defined standard and accessible over HTTP and you get (auto-)versioning of assets for ‘free’.

                                                                                    1. 1

                                                                                      Why would Mercurial in particular stand out on this list? Are you extrapolating from your own experience? I don’t think there are complete and reliable global usage statistic about any of these systems, are there?

                                                                                      1. 2

                                                                                        On top of what stephenr says, Mercurial has an increasingly solid story for large assets from things like remotefilelog and other similar work from Facebook. That means I’d feel comfy using it for e.g. game asset tracking, at least to a point. Git is getting there too (specifically the work from Microsoft), but it’s a bit less mature at the moment.

                                                                                        1. 0

                                                                                          Git is not the easiest thing in the world to learn/use.

                                                                                          If you just day “why use svn when git exists” it’s easy: because svn is easier to learn and understand.

                                                                                          Mercurial muddies that because you get the benefits of dvcs with usability that’s pretty close to svn.

                                                                                          I’ve worked in the last few years with entire teams that used no vcs.

                                                                                          1. 1

                                                                                            Yeah, very much agreed that hg hits a rather nice middle ground. Their UI design is great.

                                                                                            Still, I don’t think we could infer anything from this about the actual number of users across the various vcs. Not sure though if I simply misunderstood what you meant.

                                                                                            1. 1

                                                                                              Oh I’m not at all claiming to have stats on actual usage.

                                                                                              It was a hypothetical: if hg wasn’t an option, some developers will be more productive with svn than git.

                                                                                            2. 1

                                                                                              why use svn when git exists

                                                                                              I think this sums it up well: https://sqlite.org/whynotgit.html

                                                                                              Not about subversion in particular though, just a bash at git.

                                                                                          2. 1

                                                                                            Are you referring to mod_dav_svn? The last time I tried it it was pretty unreliable. It often truncated files silently. That’s probably not Subversion’s fault. Apache HTTPd’s WebDAV support doesn’t seem to be in a great state.

                                                                                            1. 1

                                                                                              That’s the only subversion http server that I’m aware of.

                                                                                              I suspect that post is about mod_dav - WebDAV into a regular file system directory.

                                                                                              Mod_dav_svn provides WebDAV + Svn into a repo backend.

                                                                                          3. 4

                                                                                            I know some game studios still run subversion, because of large art assets along side code, and the ability to check out specific subdirectories.

                                                                                            1. 3

                                                                                              SVN is still heavily used by companies that are not pure software dev shops yet still produce their own software, e.g. in the financial sector and manufacturing industry.

                                                                                              I don’t think many people on lobsters still encounter SVN at their place of work, but that is due to lobster’s demographic rather than everyone on the planet migrating away from SVN. That is not the case. (Some people still use tools like ClearCase at work!)

                                                                                              1. 2

                                                                                                For something closer to my heart, LLVM’s official repository is still SVN.

                                                                                            1. 6

                                                                                              Can someone explain a reason you’d want to see the descendants of a commit?

                                                                                              1. 7

                                                                                                Following history. Code archeology.

                                                                                                Many people use the VCS history as a form of documentation for the project.

                                                                                                1. 4

                                                                                                  But history is the past… you can always see the past in git…

                                                                                                  1. 16

                                                                                                    Suppose I’ve isolated an issue to this bug fix commit. In what version of gRPC did that commit release?

                                                                                                    Github tells you it’s on the v1.8.x branch, so if you head over to the, v1.8.x branch, you can see it landed after v1.8.5, so it must have released in v1.8.6. Easy enough right?

                                                                                                    Well that’s not the whole story. That commit was also cherry-picked over to the v1.9.x branch here, because v1.9.x was branched before the bug was fixed.

                                                                                                    Besides, that was silly to begin with. Why did you go to the v1.8.x branch and then manually search for it. Why couldn’t it just tell you when it got merged? That would have been nice.

                                                                                                    Many projects maintain many release branches. Some just backport bug fixes to older releases, some have more significant changes. Sometimes a bug fix only applies to a range of older releases. Do you want to track all that with no notion of descendants? It’s not fun.

                                                                                                    Even just looking at pull requests, it would be nice to see whether a pull request eventually got merged in or not, what release it got merged into, and so on. That’s all history too.

                                                                                                    So no, you can’t always see the past in git. You can only see the direct lineage of your current branch.

                                                                                                    1. 3

                                                                                                      I used to find this hella handy at Fog Creek, especially for quickly answering which bug fixes were in which custom branch for some particular client. We actually made a little GUI out of it, it was so helpful.

                                                                                                      (Interestingly, while Kiln supports that in Git too, it at least used to do so by cheating: it looked up the Mercurial SHAs in the Harmony conversion table, asked Mercurial for the descendants, and then converted those commits back to their Git equivalents. Because Harmony is now turned off, I assume either they’ve changed how this works, or no longer ship the Electric DAG, but it was cool at the time.)

                                                                                                      1. 2

                                                                                                        Why couldn’t it just tell you when it got merged?

                                                                                                        I don’t know why GitHub doesn’t, but Git can:

                                                                                                        $ git tag --contains b15024d6a1537c69fc446601559a89dc8b84cf6f
                                                                                                        v1.8.6
                                                                                                        

                                                                                                        That doesn’t address the cherry-picking case though. I’m not aware of any built-in tooling for that. Generally Git avoids relying on metadata for things that can be inferred from the data (with file renames being the poster child of the principle), so I’m not surprised that cherry-picks like this aren’t tracked directly. Theoretically they could be inferred (i.e. it’s “just” a matter of someone building the tooling), but I’m not sure that’s doable with a practical amount of computation. (There are other operations Git elects not to try to be fast at (the poster child being blame), but many of them still end up not being impractically slow to use.)

                                                                                                        1. 1

                                                                                                          Does Fossil track cherry-picks like this though? So that they’d show up as descendants? In git the cherry-picked commit technically has nothing to do with the original, but maybe Fossil does this better. (It’s always bothered me that git doesn’t track stuff like this - Mercurial has Changeset Evolution which has always looked suuuuper nice to me.)

                                                                                                          1. 4

                                                                                                            According to the fossil merge docs, cherry pick is just a flag on merge, so I imagine it does. I was just highlighting the utility of viewing commit descendants.

                                                                                                            1. 7

                                                                                                              Mercurial also tracks grafts (what git calls a cherry-pick) in the commit metadata.

                                                                                                              1. 5

                                                                                                                This is actually illustrative of the main reason I dislike mercurial. In git there are a gajillion low level commands to manipulate commits. But that’s all it is, commits. Give me a desired end state, and I can get there one way or another. But with mercurial there’s all this different stuff, and you need python plugins and config files for python plugins in order to do what you need to do. I feel like git rewards me for understanding the system and mercurial rewards me for understanding the plugin ecosystem.

                                                                                                                Maybe I’m off base, but “you need this plugin” has always turned me away from tools. To me it sounds like “this tool isn’t flexible enough to do what you want to do.”

                                                                                                                1. 7

                                                                                                                  Huh? What did I say that needs a plugin? The graft metadata is part of the commit, in the so-called “extras” field.

                                                                                                                  I can find all the commits that are origins for grafts in the repo with the following command:

                                                                                                                  $ hg log -r "origin()"
                                                                                                                  

                                                                                                                  And all the commits that are destinations for grafts:

                                                                                                                  $ hg log -r "destination()"
                                                                                                                  

                                                                                                                  This uses a core mercurial feature called “revsets” to expose this normally hidden metadata to the user.

                                                                                                                  1. 2

                                                                                                                    Right but how much manipulation of grafts can you do without a plugin? I assume you can do all the basic things like create them, list them, but what if I wanted to restructure them in some way? Can you do arbitrary restructuring without plugins?

                                                                                                                    Like this “extras” field, how much stuff goes in that? And how much of it do I have to know about if I want to restructure my repository without breaking it? Is it enough that I need a plugin to make sure I don’t break anything?

                                                                                                                    In fairness, I haven’t looked at mercurial much since 2015. Back then the answer was either “we don’t rewrite history” or “you can do that with this plugin.”

                                                                                                                    But I want to rewrite history. I want to mix and blend stuff I have in my local repo however I want before I ultimately squash away the mess I’ve created into the commit I’ll actually push. That’s crazy useful to me. Apparently you can do it with mercurial—with an extension called queues.

                                                                                                                    I’m okay with limited behavior on the upstream server, that’s fine. I just want to treat my working copy as my working copy and not a perfect clone of the central authority. For example, I don’t mind using svn at all, because with git-svn I can do all the stuff I would normally do and push it up to svn when I’m done. No problem.

                                                                                                                    And I admit that I’m not exactly the common case. Which is why I doubt mercurial will ever support me: mercurial is a version control system, not a repository editor.

                                                                                                                    1. 12

                                                                                                                      For the past several years, as well as in the current release, you still have to enable an extension (or up to two) to edit history. To get the equivalent of Git, you would need the following two lines in ~/.hgrc or %APPDATA%\Mercurial.ini:

                                                                                                                      [extensions]
                                                                                                                      rebase=
                                                                                                                      histedit=
                                                                                                                      

                                                                                                                      These correspond to turning on rebase and rebase -i, respectively. But that’s it; nothing to install, just two features to enable. I believe this was the same back in 2015, but I’d have to double-check; certainly these two extensions are all you’ve wanted for a long time, and have shipped with Hg for a long time.

                                                                                                                      That said, that’s genuinely, truly it. Grafts aren’t something different from other commits; they’re just commits with some data. Git actually does the same thing, IIRC, and also stores them in the extra fields of a commit. I’m not near a computer, but git show —raw <commit sha> should show a field called something like Cherry-Pick for a cherry-picked commit, for example, and will also explicitly expose and show you the author versus committer in its raw form. That’s the same thing going on here in Mercurial.

                                                                                                                      And having taught people Git since 2008, oh boy am I glad those two extra settings are required. I have as recently as two months ago had to ask everyone to please let me sit in silence while I tried to undo the result of someone new to Git doing a rebase that picked up some commits twice and others that shouldn’t have gone out, and then pushing to production. In Mercurial, the default commands do not allow you to shoot your foot off; that situation couldn’t have happened. And for experienced users, who I’ve noticed tend to already have elaborate .gitconfigs anyway, asking you to add two lines to a config file before using the danger tools really oughtn’t be that onerous. (And I know you’re up for that, because you mention using git-svn later in this thread, which is definitely not something that Just Works in two seconds with your average Subversion repository.)

                                                                                                                      It’s fine if you want to rewrite history. Mercurial does and has let you do that for a very long time. It does not let you do so without adding up to three lines to one configuration file one time. You and I can disagree on whether it should require you to do that, but the idea that these three lines are somehow The Reason Not to Use Mercurial has always struck me as genuinely bizarre.

                                                                                                                      1. 5

                                                                                                                        Right but how much manipulation of grafts can you do without a plugin?

                                                                                                                        A graft isn’t a separate type of object in Mercurial. It’s a built-in command (not a extension or plugin), which creates a regular commit annotated with some meta-data recording whence it came from. After the commit was created it can be dealt with like any other commit.

                                                                                                                        And how much of it do I have to know about if I want to restructure my repository without breaking it?

                                                                                                                        Nothing. Mercurial isn’t Git. You don’t need to know the implementation inside-out before you’re able to use it effectively. Should you need to accomplish low level tasks you can use Mercurial’s API, which like in most properly designed software hides implementation details.

                                                                                                                        But I want to rewrite history. (…) Apparently you can do it with mercurial—with an extension called queues.

                                                                                                                        The Mercurial Queues extension is for managing patches on top a repository. For history editing you should use the histedit and rebase extensions instead.

                                                                                                                        I just want to treat my working copy as my working copy and not a perfect clone of the central authority.

                                                                                                                        Mercurial is a DVCS. It lets you do exactly that. Have you run into any issues where Mercurial prevented you from doing things to your local copy?

                                                                                                                        For example, I don’t mind using svn at all, because with git-svn I can do all the stuff I would normally do and push it up to svn when I’m done.

                                                                                                                        Mercurial also has several ways to interact with Subversion repositories.

                                                                                                                        mercurial is a version control system, not a repository editor.

                                                                                                                        Indeed it is. And the former is what most users (maybe not you) actually want. Not the latter.

                                                                                                                        1. 2

                                                                                                                          Mercurial’s “Phases” and “Changeset Evolution” may be of interest to you, then.

                                                                                                                          1. 6

                                                                                                                            It’s also worth noting that mercurial’s extension system is there for advanced, built-in features like history editing. Out of the box, git exposes rebase, which is fine, but that does expose a huge potential footgun to an inexperienced user.

                                                                                                                            The Mercurial developers decided to make advanced features like history editing opt-in. However, these features are still part of core mercurial and are developed and tested as such. This includes commands like “hg rebase” and “hg histedit” (which is similar to git’s “rebase -i”).

                                                                                                                            The expectation is that you will want to customize mercurial a bit for your needs and desires. And as a tool that manages text files, it expects you to be ok with managing text files for configuration and customization. You might think that needing to customize a tool you use every day to get the most out of it to be onerous, but the reward mercurial gets with this approach is that new and inexperienced users avoid confusion and breakage from possibly dangerous operations like history editing.

                                                                                                                            Some experimental features (like changeset evolution, narrow clones and sparse clones) are only available as externally developed extensions. Some, like changset evolution, are pretty commonly used, however I think the mercurial devs have done a good job recently of trying to upstream as much useful stuff that’s out there in the ecosystem into core mercurial itself. Changset evolution is being integrated right now and will be a built-in feature in a few releases (hopefully).

                                                                                                      1. 8

                                                                                                        What sort of new problems doesn’t increasing Ruby’s performance 3x open up for Ruby? As a non-Rubyist, Ruby is so slow a 3x improvement is still too slow, to me.

                                                                                                        1. 16

                                                                                                          That’s fair, but Ruby’s performance is on par with Python’s (or better in at least some benchmarks, actually), which is plenty good enough for many, many people. So for those using Ruby, that’d still be an incredibly welcome improvement.

                                                                                                          That said, I’m curious when you last tried Ruby. It’s definitely still an order of magnitude slower than most compiled languages, no question, but the new GC that landed around 2.2, and the bytecode based VM that landed around 1.9 or so, have made a huge, very noticable difference in places I’ve used Ruby. It’s been enough that I’ve kept reaching for it over e.g. Go in quite a few instances.

                                                                                                          1. 3

                                                                                                            Ok, but that’s not really answering my question. Is this performance push just to get some benchmark numbers higher or is it opening new opportunities for Ruby? Certainly people aren’t going to be replacing their big data jobs with Ruby with a meager 3x improvement. People aren’t going to be writing video games in pure Ruby or doing high frequency trading.

                                                                                                            1. 9

                                                                                                              Is this performance push just to get some benchmark numbers higher or is it opening new opportunities for Ruby?

                                                                                                              Performance concerns aren’t even at the top of my list of “why I wouldn’t write new software in Ruby”; I’m more concerned with long-term maintainability due to the dynamicity making static analysis more or less a lost cause.

                                                                                                              In the interview he talks a bit about the possibility of gathering type data from running code, but without breaking existing code I’m having a hard time imagining a solution that would be reliable.

                                                                                                              1. 5

                                                                                                                I’m also skeptical. Ruby users, for a while, seemed to revel in meta-programming and run-time code generation, which greatly hinders static analysis.

                                                                                                              2. 6

                                                                                                                Well, I think that a 3x improvement would at least help push the bar for “Ruby is too slow for me, I’ll rewrite it in X-compile-language” further away.

                                                                                                                1. 5

                                                                                                                  That’s what I was trying to convey also. Ruby’s strength is that it’s a very expressive language, but its speed at one point made it an effective no-go for classes of work. (Hell, I remember writing a site in Django instead of Rails for performance reasons.) The speed improvements so far have really just kept it relevant, to be honest, by getting it back to and somewhat exceeding Python; a further 3x would mean I could start looking at it over things like C# for what I might call mildly performance demanding stuff (e.g. maybe daemons with nontrivial but not excessive workloads).

                                                                                                            2. 8

                                                                                                              The upcoming support for guilds is much more exciting in terms of addressing new problems than single-threaded execution improvements, IMHO.

                                                                                                              A less humble opinion of mine is that guilds are too little, too late, and still too primitive to have a transformative effect on Ruby and its ecosystem. I think too many devs (like me) are now accustomed to more advanced/higher-level concurrency abstractions, and guilds feel like a slightly safer way of revisiting the mid-90’s.

                                                                                                              1. 2

                                                                                                                It’s used a whole lot in web dev so 3x faster will make a lot of websites faster.

                                                                                                                1. 1

                                                                                                                  I think it just helps people who have already invested in Ruby justify their continued investment. I wouldn’t expect a ton of new users at this point unless they develop another “rails”.

                                                                                                                  1. 1

                                                                                                                    What are your favorite languages?

                                                                                                                    1. 2

                                                                                                                      I try to spend as much time in Ocaml as I can, currently.

                                                                                                                      1. 1

                                                                                                                        Wow…I’ve never heard of anyone using that. :)

                                                                                                                        Taking a look now - it does look pretty cool.

                                                                                                                  1. 7

                                                                                                                    Another language that’s worth taking a look at for CLI apps is OCaml,this blog series has a good rundown of of porting a Python app.

                                                                                                                    1. 11

                                                                                                                      I really want to agree, because I love MLs and love how simple OCaml’s runtime is, but I find OCaml so frustrating in practice. There are, what, three standard libraries, which are incompatible with one another, and then also several incompatible build systems. I’m sure it’s possible to work around this and make them interpolate, but it drove me completely nuts last time I tried that route.

                                                                                                                      1. 1

                                                                                                                        Is the Reason flavour any better in those areas?

                                                                                                                        1. 1

                                                                                                                          I haven’t tried it for the dual reason that I don’t trust Facebook and it’s more a language that can target OCaml than genuinely OCaml. That may be unfair, and some old co-workers of mine love it, but I just haven’t tried it enough to form an opinion.

                                                                                                                      2. 2

                                                                                                                        I really never felt at home in MLs. I like haskell (although I am wary of its laziness, I usually prefer strictness as a default), I like Rust (although I would be happy to just have a GC instead of borrow-checking, but I can see what rust is aiming at, its just not my use case).

                                                                                                                        OCaml in my experience (1-2 years ago) lacked good tooling and the ecosystem I find in Rust or Haskell. Also I miss typeclasses or traits.

                                                                                                                        1. 2

                                                                                                                          The situation is still not perfect in OCaml world, but the new dune (former jbuilder) build system is becoming the de-facto standard and tooling in the last couple of years has been improving a lot. The example of cargo, among other things, has been driving a lot of changes

                                                                                                                      1. 9

                                                                                                                        The thing that really irritates me is that the Flatpak developrs not only knowingly violates the XDG base directory spec, but also tries to obscure the fact that they are doing it.

                                                                                                                        It took half a decade to get to a state where one can reasonably assume that applications follow the standards of the operating system they run on. I have zero appetite for new applications regressing on that.

                                                                                                                        1. 1

                                                                                                                          Snaps (as far as I can tell from my limited usage) don’t necessarily do the same. “Classic” snaps (which are almost entirely developer-centric) honor XDG, which blissfully covers everything I personally need except Slack; everything else is more like Flakpak. This does irritate me, but I’m also not clear how they’d do it with the sandbox model they chose. (Not saying they can’t, mind; just it’s not obvious to me how.)

                                                                                                                          1. 2

                                                                                                                            I think they could still sandbox stuff, just do it in the established .config, .cache, .local directories.