Threads for mattrose

  1. 5

    Why does nobody complain about how OpenSSL doesn’t follow the UNIX philosophy of “Do one thing well”?

    1. 33

      Probably because there’s already so many other things to complain about with openssl that it doesn’t make the top 5 cut.

      1. 17

        Because the “Unix philosophy” is incredibly vague and ex-post-facto rationalization. That, and I suspect cryptography operations would be hard to do properly like that.

        1. 3

          Does UNIX follow the UNIX philosophy?

          I mean, ls has has 11 options and 4 of them deal with sorting. According to the UNIX philosophy sort should’ve been used for sorting. So “Do one thing well” doesn’t hold here. Likewise, other tenets are not followed too closely. For example, most of these sorting options were added later (“build afresh rather than complicate old programs” much?).

          The first UNIX, actually, didn’t have sort so it can be understood why an option might’ve been added (only t at the time) and why it might’ve stayed (backwards compatibility). Addition of sort kinda follows the UNIX philosophy but addition of more sorting options to ls after sort was added goes completely contrary to it.

          1. 3

            Theoretically, yes: it seems that Bell Labs’ UNIX followed the UNIX philosophy, but BSD broke it.

            Reference: http://harmful.cat-v.org/cat-v/

          2. 3

            Everyone’s still wondering if the right way to phrase it is that “it does too many things” or “it doesn’t do any of them well” ¯\_(ツ)_/¯

            1. 2

              Maybe because it’s not really a tool you’re expected to use beyond a crypto swiss army knife. I mean, it became a defacto certificate request generator, because people have it installed by default, but there are better tools for that. As a debug tool it is a “one thing well” tool. The one thing is “poke around encryption content / functions”.

              Otherwise, what would be the point of extracting things like ans1parse, pkey, or others if they would be backed by the same library anymore. Would it change anything if you called openssl-asn1parse as a separate tool instead of openssl asn1parse?

              1. 1

                For the same reason no one complains about curl either?

                1. 1

                  related, here’s a wget gui that looks similarly complex https://www.jensroesner.com/wgetgui/#screen

              1. 1

                Nearly impossible to detect on the running server. However, if you have something like a pihole looking for dns exfiltration attempts, this becomes much easier to detect. It does require multiple layers of protection though, I’ll give it that.

                1. 2

                  Since I haven’t seen any mention of it tampering with the kernel or hooking actual syscalls (as opposed to userspace syscall wrappers), it sounds like its concealment mechanisms should be pretty simple to bypass using statically-linked executables? (A static busybox build, say.)

                  1. 1

                    This was my take. LD_PRELOAD wouldn’t work in the statically linked context

                  2. 1

                    Or if you’re running in AWS there’s also their guardduty alert which I hope would pick it up: https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_finding-types-ec2.html#backdoor-ec2-ccactivitybdns

                    1. 1

                      The grsecurity patchset includes a feature called Trusted Path Execution (TPE). It can integrate with the RTLD to completely mitigate LD_PRELOAD abuses. I’m working on implementing something similar in HardenedBSD this weekend. :-)

                    1. 1

                      To get connected to an ISP back in the windows 3.1 days you had to configure your serial terminal program to connect to your ISPs modem pool, log in, find Trumpet Winsock (the 3rd party TCP stack mentioned in the OP), figure out a way to transfer the file to your computer, and then set up and configure Trumpet Winsock. It was a very long and painful procedure to connect windows to the internet.

                      1. 1

                        (Author of the big post here)

                        It’s true. I must admit, back then, I only used CIX, which was direct dial-up - no need for PPP or anything. Plain text only, but it let me use email and batched FTP (you got emailed the files you downloaded) and Usenet.

                        My email address from 1991 is still live and still works. 🙂 It’s 31 now. And I often answer emails on a keyboard that’s from the same year.

                        1. 1

                          *blog post

                        2. 1

                          figure out a way to transfer the file to your computer

                          well, presumably the same way you got all the rest of your software onto your computer. Physical media (if your ISP was nice they’d probably give you a floppy), or a BBS, or maybe your LAN if you were in an office (yes, there were LANs that didn’t provide a default route to the internet). It only seems terrible from the perspective of already expecting everything to be on the internet.

                          I had an ISP that, long past Windows 3.1 days, required a chat script to log in, because they were Unix-friendly so their modem pool had a plaintext login prompt and then put you into a shell, from which you could run ppp or slip or lynx or pine. So to make Windows happy you had to turn off CHAP and such and configure it to answer the username/password prompts and then run ppp.

                          1. 1

                            I was thinking more of setting up a zmodem transfer. This was before the days when ISPs were big corporations that would blindly mail out floppies or CDs. The only standard way was to just dial in and your provider would usually have a rudimentary menu that would let you initiate a file transfer of software that would connect via SLIP or PPP, and usually use chat scripts. We didn’t support PAP until we got our fancy new CISCO AS5200s and RADIUS. In those days a shell server available over the modem, or through telnet, was also a must have for any ISP.

                            1. 1

                              Oops, I didn’t even notice that line!

                              Yes, it was fiddly, but no more so than any other late 1980s/beginning of the 1990s OS. Actually it was easier on Windows, because the software was free and widely-available.

                              I never even tried with OS/2.

                              In Europe and the UK, all phone calls cost money, including local calls. Free local calls are an American peculiarity. So dial up internet accesses was very expensive and billed by the minute. So most of us tried to avoid doing it. It was much cheaper to use work or educational access instead.

                              I used a provider where you got shell access, as you say. You could negotiate PPP later on, but I didn’t. I told it to Zip all my emails, downloaded the zip, then hung up. This was automated so it was as quick as possible.

                              My client unpacked the zip, imported all my emails into folders, I read and replied offline, then connected. It zipped then up, uploaded the zip, and hung up.

                              This process was called blinking. I might do it multiple times a day and only pay for a few minutes of call time.

                          1. 5

                            FreeBSD is finally getting rid of csh as the default sh? I remember changing that to sh back in the FreeBSD 4 days in the late 90s.

                            1. 3

                              I generally have three shells installed on any FreeBSD system:

                              • /usr/local/bin/{bash,zsh,whatever} from packages that I use as my login shell.
                              • /bin/sh for running system shell scripts.
                              • /bin/csh sitting there for no reason.

                              From the mailing list threads, that’s a pretty common arrangement and so there’s been a gradual move to remove csh. It will still be in packages for people that actually choose to use it.

                              The one feature from csh that I liked was the context-dependent up-arrow behaviour. It did a backwards search based on what you’d typed already. That’s been added to /bin/sh, so I’m very happy to see csh go away.

                              The recommendation for at least the last 20 years has been to use a POSIX- (or Bourne-)style shell for scripting and csh only for interactive use. It really doesn’t make sense to have csh in the base system if nothing needs it and users can easily install another shell.

                              It is a good idea to leave root’s shell set to something in /bin because then you can still log in as root if you can’t mount /usr/local. The toor user exists for this though: you can set toor‘s shell to your favourite interactive shell and use that for root login when the system is healthy, root for when it isn’t.

                              1. 1

                                For years, I’ve ignored the advice and set root’s shell to zsh from ports regardless. I can’t remember ever having it break but it wouldn’t be hard to fix anyway. Especially for a jail or VM where you can use the host system.

                                1. 3

                                  For a jail, there’s no reason not to do this.

                                  I was bitten by this when I had the root shell set to bash and (pre-pkg) some shared libraries in my system got out of sync and bash didn’t work with the readline (I think) that I had installed. I couldn’t log in, root couldn’t log in. With ZFS, I typically have /usr/local on a separate dataset, but if the kernel can mount the root boot environment then it can usually mount this one but pre-ZFS I had a system where the root and /usr/local filesystems were on different disks - and the second disk failed (root any my home directory were on a mirrored pair and /usr/local/etc was symlinked from the other disk - not a setup I’d probably repeat). I was glad of root’s shell being on the root FS then.

                              2. 2

                                It does seem long overdue. But there were a few dissenters to the proposal on the mailing list. Using pw does already default to sh for new users so the csh default only really applied to root. May still have had the effect of encouraging some BSD newbies to think learning tcsh would be a good idea.

                              1. 13

                                The rsync thing has screwed me up in the past. I generally have to write a sync script and then just never manually use rsync as a workaround to not being able to remember how it works.

                                1. 5

                                  I should probably do this more often, I just habitually use the -n option to do a dry-run and make sure it’s doing what I actually expect.

                                  1. 4

                                    For me, rsync is the intuitive. If I want to backup my work, I sync those directories:

                                    rsync -a work/ /backup-work/
                                    

                                    Try that with any other command:

                                    cp -a work/ /backup-work/
                                    

                                    Oops, this only does the right thing the first time. The second time, when backup-work exists, it means something else!

                                    Ref idempotent commands, I think this is even an objective argument for rsync’s trailing slash semantics. Except that it wouldn’t/shouldn’t have to be about the trailing slash.

                                    1. 3

                                      Yeah, with rsync you basically always want a trailing slash on both source and destination. The exception is when you’re syncing a single file… in which case of course you don’t want a trailing slash. :-)

                                      1. 2

                                        Yeah, I hit this peculiarity of rsync a couple of weeks ago and I had to write down the exact rsync command I wanted.

                                      1. 2

                                        I tried the beta of this, and … I guess it would be fine, if I expected my terminal emulator to work like an IDE, but I don’t. I expect it to behave like a terminal emulator. I found warp just got In my way every time it tried to “help” me.

                                        1. 1

                                          I still have a hard time to getting used to URLs being clickable in my terminal, but this made me even less interested in Warp. Aside from all the other red flags, that is.

                                        1. 1

                                          Just chiming in here to say that “Broken Windows Policing” which was all the rage in the 90s, has been extensively questioned and criticized in the 40 years since it’s introduction, and the data we have collected to prove or disprove the methods is … inconclusive at best. Social Sciences are messy at best, but I would consider this theory highly questionable.

                                          1. 2

                                            Has anyone tried this? I’m thinking of installing it on my M1 MacBook Pro.

                                            1. 3

                                              I tried it on an M1 Air for 20 minutes and it was very smooth. Linux itself seems pretty good, but it’s Arch and KDE which I’m not too familiar with, so I switched back pretty quickly to macOS to get some work done. I’ll have another bash soon with a bit more time and patience.

                                            1. 11

                                              when you resolve a DNS name in a Python program, it checks /etc/hosts, but when you use dig, it doesn’t.

                                              This one seems surprising at first but when you think about it, it makes sense: The python program (or any other program, including ping) is asked to resolve a name into an IP address, so it goes through the whole /etc/nsswitch.conf routine, usually starting with checking /etc/hosts.

                                              dig and it’s older companions like host and nslookup are not actually designed to resolve a name into an IP, they are designed to query DNS services. What you’ve put on the local server is irrelevant to them.

                                              Later edit after completing the article. nscd used to be far more popular on *BSD and Solaris 20 years ago. They may still be the solution there. For some reason Linux never incorporated it by default, so it was eventually re-invented on linux

                                              1. 15

                                                trying to keep myself and my family safe In the middle of an occupation of the capital of a G7 country. They protested at my son’s school yesterday.

                                                Maybe trying to set up the gaming laptop into a full-fledged setup with dual monitors and an actual keyboard and mouse when I’m not doing that.

                                                1. 3

                                                  How is the situation down there conflict wise? The article shows kids playing soccer on the streets. I hope it’s safe!

                                                  1. 1

                                                    Everyone is safe, but the situation is very uneasy. Many of the occupiers have brought their kids, so I’m not surprised they’re playing soccer, but I wouldn’t want any of my family in the middle like that.

                                                  2. 3

                                                    Yikes, try and stay safe. I’m over in Orleans myself but this stuff is frankly terrifying.

                                                    1. 3

                                                      Aw geez. And just a couple of months ago I was thinking of Canada as being a potential refuge place if things go even more crazy in Europe. Best of luck and endurance to you all.

                                                      Mars it is then, I guess.

                                                      1. 2

                                                        If it makes you feel better, there’s likely nowhere on the planet that is truly a refuge if things go super sour.

                                                        1. 1

                                                          I’d be fine with a small unknown island. But then somebody asks me where my low-latency internet and power come from.

                                                      2. 2

                                                        I’m really sorry to hear. This is a super scary time and having the threat of violence looming so close has got to be nervous making.

                                                        1. 1

                                                          Hey, me too! I’m over in Gatineau in the ’burbs but I generally spend a lot of time in Ottawa proper.

                                                          1. 0

                                                            My sympathies; I’m flying into rat-licker central (Alberta :/ ) for my weekend; I expect I’ll see some of the dumbasses myself along the way, although not as bad as your part of the country. Good luck!

                                                            1. 1

                                                              Hey now, we are rat free!

                                                          1. 8

                                                            So, as a weird aside, Kevin Mitnick is, to this day, widely hated on the WeLL, a BBS that is still around that he hacked in 1995

                                                            1. 6

                                                              I dreamt of being on the WeLL, in the 90’s. I was too poor and too far away to have an account there, but I read about it in Wired and thought of how cool it would be.

                                                              I ended up joining years later, just for fun, but ended up canceling a couple of months later as it ended up being too much money per month for what was essentially a mid-life crisis purchase. :)

                                                              1. 5

                                                                I think we read the same article. Did it lead with a WeLL member describing meeting his birth mother?

                                                                Back then (this was when “The Web” was cool, kids) it was presented as “the text-only meeting place for the people who designed the Web”. IIRC Bruce Sterling was active there. Maybe still is.

                                                                Even then, to me, it had the faint whiff of men in older middle-age in loose jean and comfy sneakers. Maybe I was unfair.

                                                                1. 2

                                                                  Sterling is indeed still active on the WeLL, he regularly sits in on the yearly State of the World discussions.

                                                                2. 3

                                                                  I joined in the late 90s and still host the Linux conference there. It was far more vital back in the 90s and I can understand not being terribly impressed with it if you joined later. It is like a second home to me now though, and if you stick around the same people online for 25 years they eventually grow on you :)

                                                                  1. 1

                                                                    Heh. I may have to join again. Nostalgia is creeping up on me.

                                                              1. 5

                                                                This page is full of good advice (as opposed to a lot of Bash/Shell instructions online). It’s referred to a lot for example in the #bash channel on Libera.Chat, and in the Exercism Bash track.

                                                                1. 2

                                                                  Thank you for answering “Why should I pay attention to this, the 1 billionth “bash guide” posted to lobste.rs in the past year.” question. It actually has a lot of useful info.

                                                                1. 35

                                                                  Why did GitHub remove his account/projects?

                                                                  1. 44

                                                                    That’s the part that bothers me.

                                                                    I understand it wasn’t a nice thing to do, and that people are upset, but it’s his own code in his own repos. He even announced ahead of time he would do something like this, so “buyer” beware.

                                                                    I realize GitHub TOS covers them to remove accounts and repos at their discretion, but it’s a little unsettling that they’ll actually do so arbitrarily without a clear TOS violation. It might be time I move everything to Sourcehut and treat GitHub as a mirror…

                                                                    1. 24

                                                                      It might be time I move everything to Sourcehut…

                                                                      The Sourcehut guy has always seemed a little unstable to me (didn’t he get banned from this site, in fact?) So, why would I trust him any more than I trust GitHub?

                                                                      1. 33

                                                                        I banned him and I would not call him unstable. Not just because that kind of insult is inappropriate here, but because it obviously doesn’t apply. He writes inflammatory hyperbole that’s not a good fit for this site, but he’s a skilled, accomplished professional who looks like he’s seeing a lot of success in his life.

                                                                        1. 11

                                                                          I didn’t mean to insult him. Maybe “erratic” would have been a better word without any mental health connotations (which I absolutely didn’t intend)? Poor word choice on my part, I’m sorry for that.

                                                                          …but he’s a skilled, accomplished professional who looks like he’s seeing a lot of success in his life.

                                                                          Sure, same goes for the GitHub guys. A person who can’t tone it down enough to keep a Lobsters account just isn’t someone I feel I can trust to host my code, particularly given that he’s in charge of the whole operation. Obviously everyone is free to decide who to trust and for what reasons.

                                                                          1. 9

                                                                            A person who can’t tone it down enough to keep a Lobsters account just isn’t someone I feel I can trust to host my code

                                                                            Bear in mind, Linus Torvalds would also probably have been banned from here multiple times in the past.

                                                                            I’d be perfectly happy to trust someone that volatile a lot (and I guess I do, running Linux since 1996 :) ). But I would be careful which groups and forums I invited them to :)

                                                                            1. 2

                                                                              …I guess I do, running Linux since 1996

                                                                              Very different, at least to me. If Linux was a service, control would have been taken away from Linus a long time ago (I mean, as it is they made him step back for awhile to work on his issues). But it’s not, it’s just code that other people then build into something, often applying patches in the process. If Linus had a meltdown there is already sufficient infrastructure in place that the vast majority of us wouldn’t even notice.

                                                                              I wouldn’t trust a code hosting service Linus ran by himself either.

                                                                              1. 1

                                                                                Nobody made Linus step back. He recognized that he had issues and took a sabbatical to deal with them himself. Are you saying you wouldn’t trust a service by a guy who has been diligently working on the same project for 30 years? Not to mention the guy who invented the base of all of the services discussed in this thread.

                                                                                Why do people assume that “Bigger is better” when it comes to web services? The two most reliable services I use are Pinboard, run by an insanely opinionated and outspoken developer, and NewsBlur, who was, and may still be, a one man shop that just quietly does his own thing. In the same time as those services have been faithfully up and running, Google has shut down more services than I can count, because “It didn’t fit with their corporate vision”

                                                                                It’s misguided, and harmful.

                                                                                1. 1

                                                                                  Nobody made Linus step back.

                                                                                  We’ll probably never know for sure, but the subtext (well, and the text) of his announcement email sure makes it sound like his hand was forced, at least to me.

                                                                                  Are you saying you wouldn’t trust a service by a guy who has been diligently working on the same project for 30 years?

                                                                                  No, I’m saying I wouldn’t trust a service run by a guy who randomly goes off on people in totally inappropriate ways (his admission). Or, as is the case here, a guy who can’t even behave himself well enough to keep a Lobsters account.

                                                                                  Not to mention the guy who invented the base of all of the services discussed in this thread.

                                                                                  That has literally nothing to do with anything. A person can be productive or brilliant and also have other, undesirable, qualities.

                                                                                  Why do people assume that “Bigger is better” when it comes to web services?

                                                                                  I don’t, so I can’t answer that.

                                                                                  Google has shut down more services than I can count…

                                                                                  Agree with you there! I don’t trust Google for anything but search (I don’t even use Gmail), because that’s the one thing I don’t think they’ll ever kill (or break). I don’t think GitHub is going anywhere either, the worst case scenario is that Microsoft sells it.

                                                                                  It’s misguided, and harmful.

                                                                                  If there was a person who had the views you seem to ascribe to me, then I might agree!

                                                                        2. 30

                                                                          That’s unfair to Drew. He’s passionate, and rude, and opinionated, and submissions here from his site generally stirred up giant flamewars. But I do believe he’s got what it takes to keep sourcehut running.

                                                                          1. 18

                                                                            GitHub will keep running, too. I’m not sure we’ve answered the question of

                                                                            why would I trust him any more than I trust GitHub?

                                                                            1. 8

                                                                              Not only is the sourcehut software available under the AGPL, the issue trackers and such give you export and import functions to pull your data into another instance easily. The software itself is not trivial to host, but it’s not prohibitively hard either. If I needed to eject because Drew became untrustworthy, I am very comfortable that I could do that.

                                                                              Even though that’s a non-zero amount of work, GitHub gives me no comparable ability. That’s a good reason to trust him more than I trust GitHub, in my opinion.

                                                                              1. 3

                                                                                GitHub gives me no comparable ability.

                                                                                The GitHub command line client provides this functionality, as does the API. Obviously, the data formats are specific to the way GH works, but there are ways to extract most if not all of the relevant data (I use this heavily with my team to script up our findings workflow, for example).

                                                                                1. 5

                                                                                  Interesting. Unless I’m missing something, you can’t stand up your own self-hosted instance of github, and import that, can you? The ability to stand up my own instance of the forge and import my data, to use on a self-hosted site, is what I meant by “comparable”. (That’s the angle I was coming from… if Drew won’t let me use his hosted service, I can just set up my own copy on any host I want since upstream is AGPL, then import my data from the sr.ht export since sr.ht exposes those functions.)

                                                                                  1. 2

                                                                                    GitLab supports importing to a self-hosted instance from GitHub [1], although I’m sure it’s not perfect, so it may or may not be useful. It also isn’t clear to me based on a 15 second review whether you can import from some kind of local data dump or raw GitHub API responses or if your GitHub account needs to be currently active.

                                                                                    [1] https://docs.gitlab.com/ee/user/project/import/github.html

                                                                                    1. 2

                                                                                      That looks much better than I thought, particularly if it turns out to work off saved data/responses. And it’s nice that Gitlab enable that for all their tiers.

                                                                                    2. 1

                                                                                      Unless I’m missing something, you can’t stand up your own self-hosted instance of github, and import that, can you?

                                                                                      GitHub Enterprise can be bought as a GitHub-hosted or self-hosted thing. These support (most of) the same APIs as the public GitHub, so you can run your own instance if you are willing to pay.

                                                                                      1. 2

                                                                                        It would be an interesting experiment to see whether they would sell an enterprise installation to someone whose account they forcibly closed. I was sort of assuming that if they won’t let you be a customer of their public service, they won’t sell you the private one, but that is uninformed speculation.

                                                                                2. 3

                                                                                  Because sourcehut is open source so nothing is lost when I leave. More than that chances are if sourcehut goes a bad route there would likely be others jumping in.

                                                                                3. 2

                                                                                  Not that you exactly claim otherwise, but Drew also makes some nice things and has created a business structure that enables at least one other developer to make some nice things.

                                                                                  Quite apart from that, though, and similarly quite apart from whether he has what it takes to keep sourcehut running, he’s given me an out so that I don’t, strictly speaking, need him to. He’s released the software that runs the forge under the AGPL, here. And it exposes ways for me to export the hosted stuff and import it into a self-hosted instance.

                                                                                  So regardless of whether I trust Drew personally, he’s made it so I don’t need to for this purpose.

                                                                                  If Drew got angry and decided I couldn’t be his customer anymore, I could stand up my own instance or pay someone to do that for me and import my data. My repos wouldn’t be down at all, my tickets, docs, etc. would be down for a day or so, and my mailing lists might see a bit more disruption than that. If github decided that I shouldn’t be their customer anymore, I’d have my repos. For the rest, I’d kind of be out of luck. (I think this last paragraph is more responsive to @glesica ‘s comment than the one I’m replying to, and I’m too lazy to split it to another reply.)

                                                                                4. 17

                                                                                  Because “more than I trust Microsoft” is a damn low bar.

                                                                                  1. 7

                                                                                    It’s like a little devil hovering over my right shoulder, and a slightly less predictable devil hovering over the other.

                                                                                5. 6

                                                                                  From other options there’s also fediverse approach with Gitea, and p2p approach will be available soon with Radicle.

                                                                                  1. 11

                                                                                    It might be time I move everything to Sourcehut and treat GitHub as a mirror…

                                                                                    That time was years ago, but hey, better late than never.

                                                                                    1. 5

                                                                                      Consider hosting your own, instead. I published a blog post with a list of defunct code hosting sites which I update occasionally. Maybe that list is a good reminder. Remember, it’s not just code that goes away with such sites, it’s also issue queues and in some cases, wikis and mailing lists too.

                                                                                      1. 4

                                                                                        Are you also start hosting a list of defunct private websites that used to host Git repos that are gone forever and where the disappearence came completely unexpected? I would trust Github more with staying online since that’s their job than a developer running a Gitweb on some VPS with some domain name that requires regular payment to stay online.

                                                                                        Kinda like I registered callcc.org after it lapsed to make sure the links to the CHICKEN website don’t break and it doesn’t get domain-squatted and I’m redirecting to the official website these days.

                                                                                        1. 1

                                                                                          Are you also start hosting a list of defunct private websites that used to host Git repos that are gone forever and where the disappearence came completely unexpected?

                                                                                          I can’t think of anything offhand where I’ve taken a dependency that’s done that. But when I do take a dependency on something, I generally mirror the SCM repo if there is one. And I am very reluctant to take dependencies on things I can’t have the source to. Since the things I depend on generally haven’t gone away, I haven’t bothered to publish my mirrors, but I would if the license permits it.

                                                                                          1. 3

                                                                                            But when I do take a dependency on something, I generally mirror the SCM repo if there is one.

                                                                                            I learned that the hard way when Rubyforge went down, a few employers ago. We weren’t that active in the Ruby community anymore, so we missed the notice. When the site went away and I had to do some small maintenance tasks on a legacy project, all the third party svn subtrees from Rubyforge were no longer working (and, more painfully, another project of ours was completely gone too). Note that Rubyforge was huge in the Ruby community back in the day.

                                                                                          2. 1

                                                                                            I would trust Github more with staying online since that’s their job than a developer running a Gitweb on some VPS with some domain name that requires regular payment to stay online.

                                                                                            Like I said, history has shown these hosting sites are not as trustworthy as people like to believe they are. The GitHub company can get sold to an untrustworthy partner (har har, like that’d ever happen… oh wait) or go out of business (what if MS decides to sell the company to, I dunno, Oracle or something because it’s not making a profit?), or there might be some other new VCS that comes out that completely blows git out of the water. I’m sure nobody saw coming what happened to Bitbucket - it started out as a Mercurial hosting site, then started offering git and finally dropped Mercurial after Atlassian took over. Its founders probably never would have let that happen if it were still in their power.

                                                                                            From my own perspective, I’ve personally ran into at least five hosting sites who were hosting projects I started or heavily contributed to that are no longer available now (Rubyforge, Dutch govt OSOSS’ uitwisselplatform, Berlios, Bitbucket and Google Code). And then there’s Sourceforge which at least still hosts some of my defunct projects, but had for a while been injecting malware into downloads. If I or my employers (as the case may be) had hosted our own projects from the start, this pain would’ve been completely avoided. These are projects in which I had a stake, and it was in my interest to not let them die.

                                                                                            Now, having said that, I agree that from a third party perspective (someone who is using the hosted code) that’s different. I understand your point saying you don’t want to rely on some random developer’s VPS being up, and neither would I. But then, people change repositories on code hosting sites all the time, too. They move to other hosting sites, or rename repositories etc. Links rot and die, which is unfortunate and something we all have to live with.

                                                                                            Case in point:

                                                                                            Kinda like I registered callcc.org after it lapsed to make sure the links to the CHICKEN website don’t break and it doesn’t get domain-squatted and I’m redirecting to the official website these days.

                                                                                            Thanks for doing that. AFAIK this domain was never communicated as being official, but I might be wrong.

                                                                                      2. 8

                                                                                        I don’t know what the GitHub rationale was, but the ‘limitation of liability’ bit in most open source licenses only goes so far. If I intentionally introduce malicious behaviour into one of my open source projects, knowing that it would damage downstream consumers, then I’d probably be liable under the Computer Misuse Act in the UK and similar legislation elsewhere. GitHub’s T&C’s don’t explicitly prohibit using their service for criminal purposes but that’s normally implicit: if GitHub didn’t act then they might end up being liable as an accessory (at least as an accessory after the fact). Their distribution channel (NPM) is being used by a malicious actor to attack other users.

                                                                                        It’s normally difficult to prove malicious intent in this kind of thing (incompetence and malice look similar) but it seems pretty clear here from the author’s own statements.

                                                                                        1. 12

                                                                                          I don’t know what the GitHub rationale was, but the ‘limitation of liability’ bit in most open source licenses only goes so far.

                                                                                          This is disturbing. Software is provided as is, with no liability whatsoever, but the author should still be liable for what happens when other people use it, because it broke things? What if the author decided to completely change the library’s API, or recycle it to just print squares of color, because they liked the name?

                                                                                          If find what the author did pretty stupid, but frankly, suggesting it falls into criminal behavior call for some stepping back and put things in perspective.

                                                                                          1. 8

                                                                                            There is a difference, and it’s not subtle at all, between making a possibly unwanted change in software that is provided without any warranty, and deliberately making a crippling change with the express intent of breaking other people’s applications.

                                                                                            To put it another way: if you accidentally commit an integer overflow bug that causes batteries to blow up, that is, presumably, just bad engineering. But if you deliberately commit a clever hack that causes people’s batteries to blow up, with the express intent of getting people injured, or at least destroying their phones, I think it makes a little sense to not put it under “well, it did say no warranty of any kind on the box, didn’t it?”.

                                                                                            Obviously, this didn’t kill anyone, so I’m obviously not thinking it ought to be treated as murder. But “no warranty” is not a license to do anything.

                                                                                            It’s not like software is being given special treatment here, it’s how warranties work everywhere. If you sell boats with two years’ servicing warranty and they break down after three years, that’s one thing, but if you fit them with bombs that go off after two years and one day, with the express intent of killing anyone on them, that still falls under “murder”, not “what happens after two years isn’t our problem, it says so on the purchase contract”.

                                                                                            (Edit: obviously IANAL and this is not legal advice, either, I’m only parroting second-hand, non-lawyer advice about how insurance works for some high-stakes software projects)

                                                                                            1. 5

                                                                                              I guess that makes sense, when you put it that way :)

                                                                                            2. 3

                                                                                              I am not a lawyer, this is not legal advice:

                                                                                              My understanding is that it comes down to intent. If I upload a buggy piece of crap to GitHub with an open source license, and you use it, then it sucks to be you. If I upload something to GitHub, wait for you to deploy it and then intentionally introduce a vulnerability or other malicious behaviour in it then legally dubious. Normally it’s very difficult to prove intent. If I submit a patch to the Linux kernel that introduces a vulnerability, if you wanted to prosecute me then you’d have to prove that I did so knowing that the bug was there and with the intent to cause harm. That’s very difficult to do in the general case (the NSA null-pointer dereference bugs are a great case in point here: people suspect that the NSA knew about that vulnerability class and introduced it deliberately, but no one can prove it and there’s enough reasonable doubt that it would never stick in court unless there was some corroborating evidence - it could easily have been accidental). If, before I submit the patch, I post publicly about how I am going to intentionally break things for the people using my code and then I push a new version out to public repositories then it’s likely to be easy to prove malicious intent. The author of these packages did exactly that: posted saying that he was going to break things for people if they didn’t pay him and then, when they didn’t pay him, broke things. That may (again, not a lawyer) count as blackmail, as well as computer misuse.

                                                                                              1. 3
                                                                                                1. Code license != Github TOS.
                                                                                                2. Liability could only be disclaimed to the extent permitted by law. You cannot put a sign “free food, no liability whatsoever” and then put poison inside and expect that disclaimer to save you from prison. E.g., GPL states “THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW.”
                                                                                            3. 7

                                                                                              I think until they make a statement about it, nobody knows but them. But my assumption is that this happened on a weekend, and whoever was on call figured that the easiest thing to do to minimize disruption till Monday was to suspend the account and hard revert the content until more people could be brought in. I’m also assuming suspending the account just automatically says that you violated the ToS.

                                                                                              1. 3

                                                                                                I could imagine that somebody identified this as a possible account hack and thus locked it.

                                                                                                1. 2

                                                                                                  They didn’t, they suspended his account so he can’t log in. You are still free to troll him on GitHub without any recourse whatsoever.

                                                                                                1. 6

                                                                                                  Lots of companies pay for software. There’s a whole giant industry selling commercial software… which leads to the question of why not making it proprietary.

                                                                                                  1. What sort of product is it?:
                                                                                                  2. Who would benefit from availability of source?
                                                                                                  3. Who would benefit from it being open source (you can give people source code under a proprietary license, so this is a different question than the previous one)?

                                                                                                  (I’m working on commercial product, with open source variant with slightly different use case as marketing, and … a bunch of people use the open source tool, and I’ve only gotten a single patch, ever. It’s not clear what being open source does for anyone in this particular example.)

                                                                                                  1. 1

                                                                                                    You have a good point, so let me answer your questions:

                                                                                                    1. It is a tool meant for developers: a build system.
                                                                                                    2. Everyone; it is actually crucial to the software supply chain that the source is available. If the build system is not Open Source (i.e., you can’t compile it yourself), you don’t know if it has been backdoored with a Trusting Trust attack, just like a compiler.
                                                                                                    3. End users. If it’s only source-available, then companies that distribute software that builds with it could conceivably make it really hard to build their software, even if that software is FOSS or source-available.

                                                                                                    But beyond the fact that it is actually crucial to be FOSS for security, there is another big reason: developers will not adopt a non-FOSS tool. If it is FOSS, it has a chance, and if it is not, then it has none.

                                                                                                    1. 4

                                                                                                      There are many build tools out there that are very successful and not open source. TeamCity is a good example.

                                                                                                      1. 3

                                                                                                        But beyond the fact that it is actually crucial to be FOSS for security, there is another big reason: developers will not adopt a non-FOSS tool. If it is FOSS, it has a chance, and if it is not, then it has none.

                                                                                                        Open source isn’t a requirement for commercially successful build tools; Incredibuild is a proprietary build system used by Adobe, Amazon, Boeing, Epic Megagames, Intel, Microsoft, and many other companies. Most of the market consists of pragmatists; they’ll adopt a new product if it addresses a major pain point.

                                                                                                        Is there a distributed build tool for Rust yet? That may be a market worth pursuing.

                                                                                                        1. 1

                                                                                                          I did not expect anyone to say that closed-source build systems were used, but you and a sibling named two.

                                                                                                          As far as making a distributed build tool for Rust, yeah, I can do that. Thank you.

                                                                                                        2. 1

                                                                                                          It is a tool meant for developers: a build system.

                                                                                                          I am curious how are you planning to legally structure dual-licensing of a build system. I believe most (all?) examples of dual-licensing where one license is free/open source involve a copyleft license (commonly GPL). In order to trigger copyleft’ness the user must produce a derivative work of your software (e.g., link to your library). I don’t see how using a build system to build a project results in derivative work. I suppose there are probably some dual-licensed projects based on AGPL but that doesn’t seem to fit the build system either.

                                                                                                          I also broadly agree with what others have said about your primary concern (that the companies will steal rather than pay): companies (at least in the western economies) are happy to pay provided prices are reasonable and metrics are sensible (e.g., many would be reluctant to jump though licensing server installation, etc). But companies, especially large ones, are also often conservative/dysfunctional so expect quite a bit of admin overhead (see @kornel comment). For the level of revenue you are looking at (say, ~$300K/year), I would say you will need to hire an admin person unless you are prepared to spend a substantial chunk of your own time doing that.

                                                                                                          This is based on my experience running a software company (codesynthesis.com ) with a bunch of dual-licensed products. Ironically, quite a bit of its revenue is currently used to fund the development of a build system (build2; permissively-licensed under MIT). If you are looking to build a general-purpose build system, plan for a many-year effort (again, talking from experience). Good luck!

                                                                                                          1. 1

                                                                                                            I am curious how are you planning to legally structure dual-licensing of a build system.

                                                                                                            It will also be a library.

                                                                                                            There are plenty of places in programming where it is necessary to be able to generate tasks, order those tasks to make sure all dependencies are fulfilled, and run those tasks (hopefully as fast as possible).

                                                                                                            One such example is a init/supervision system. There are services that need to be started after certain others.

                                                                                                            (Sidenote: I’m also working on an init/supervision system, so technically, companies don’t need to make their own with my library. It’s just an example.)

                                                                                                            I suppose there are probably some dual-licensed projects based on AGPL but that doesn’t seem to fit the build system either.

                                                                                                            This build system will be distributable, like Bazel, so yes, that does apply.

                                                                                                            I also broadly agree with what others have said about your primary concern (that the companies will steal rather than pay): companies (at least in the western economies) are happy to pay provided prices are reasonable and metrics are sensible (e.g., many would be reluctant to jump though licensing server installation, etc).

                                                                                                            What are reasonable prices, though?

                                                                                                            But companies, especially large ones, are also often conservative/dysfunctional so expect quite a bit of admin overhead (see @kornel comment). For the level of revenue you are looking at (say, ~$300K/year), I would say you will need to hire an admin person unless you are prepared to spend a substantial chunk of your own time doing that.

                                                                                                            I am going to do it, yes, but I’m also going to be helped by my wife.

                                                                                                            This is based on my experience running a software company (codesynthesis.com ) with a bunch of dual-licensed products. Ironically, quite a bit of its revenue is currently used to fund the development of a build system (build2; permissively-licensed under MIT). If you are looking to build a general-purpose build system, plan for a many-year effort (again, talking from experience). Good luck!

                                                                                                            Oh, I’m cutting features out of my build system, so I don’t expect it to take that long. Also, I’m not running a business like you are.

                                                                                                            Thank you.

                                                                                                            1. 2

                                                                                                              What are reasonable prices, though?

                                                                                                              The video Designing the Ideal Bootstrapped Business has some excellent advice on pricing; the author has sold at least 3 startups.

                                                                                                      1. 2

                                                                                                        This is one of my big beefs with awk. I love it, but passing shell variables to it is painful. This is a very good guide on the different ways of doing it, and the different pitfalls that inevitably come with each different method

                                                                                                        1. 14

                                                                                                          Wow, I can’t believe rasterman is still working on E. I ran it on my RedHat 5 desktop in the late 90s

                                                                                                          1. 6

                                                                                                            Well, he’s aiming for a 1.0 in 2052 ;) (https://0ver.org/)

                                                                                                          1. 3

                                                                                                            The only problem with monorepos that I’ve ever seen as a build engineer and developer and SysAdmin for 20 years is git. Don’t get me wrong I love git, but it just has this size limitation where it becomes unusable after a certain size. There are ways to get around this, but monorepos are a git problem, not a source code management problem. SCMs like Mercurial and commercial ones like Perforce are great in that you can pretty much throw anything in the repo and it will not affect checkout time after initial checkout, but git OTOH, becomes nigh unusable after a certain size.

                                                                                                            1. 1

                                                                                                              Point 1. There are many fields of software where the penalty for bugs is death. I’m thinking not only of programming of medical instruments like X-ray scanners, but even things like airplane automation (think of the 737 MAX bug) or even scarier stuff like Nuclear Reactor control routines, or Military applications. The culture of software developers in these fields is much more strict than your typical JS web developer. Saying that it’s somehow impossible to write better software is a surprisingly defeatist attitude, really.

                                                                                                              Point 2. There are interesting methods of conducting interviews, which at least try to erase biases, like scrubbing names and identifiers off of resumes, or Blind Auditions for Orchestras. Importantly, they don’t take them into account and try to adjust for biases, but they do attempt to give candidates a level playing field. Orchestras in particular have shown that this approach increases inclusion, but not to the point where orchestra’s ethnic and gender composition matches that of the surrounding community. The tech community needs look outside of itself and learn from other fields.

                                                                                                              1. 41

                                                                                                                Can’t help but think that this follows Betteridge’s Law of Headlines…

                                                                                                                1. 6

                                                                                                                  For the uninitiated:

                                                                                                                  Any headline that ends in a question mark can be answered by the word no.

                                                                                                                  https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines

                                                                                                                1. 1

                                                                                                                  If I’m reading this right, the oft-repeated nightmare that Quantum computers would render crypto ineffective was, possibly a little overstated?

                                                                                                                  1. 6

                                                                                                                    I’m not sure this was ever really a question.

                                                                                                                    It was always clear that Shor’s algorithm only applied to very specific problems. Unfortunately it turned out these were the exact problems that were used in pretty much all mainstream public key cryptography. But there always were alternatives.

                                                                                                                    One likely quantum safe cryptosystem is McEliece, which was developed in the 70s. It is not very practical due to very large keys, so it’s likely not gonna be the one that your future browser will use.

                                                                                                                    1. 2

                                                                                                                      Wikipedia has a good summary. As someone with very, very limited understanding of the mathematics of cryptography, I take the tldr; to be: current symmetric encryption and hash algorithms are probably fine, but will have to double their key size; current public-key algorithms are broken, but there are replacements waiting in the wings.