1. 9

    It is not as bad as the title suggests: if you specify a UserID in your unit file that starts with a digit, systemd will ignore the UserID option and run the process as managed by systemd as root.

    I do wonder if this works when using systemd as cron replacement, i.e. systemd.timer(5) from your user account. That would be a neat vulnerability.

    1. 23

      It is not as bad as the title suggests: if you specify a UserID in your unit file that starts with a digit, systemd will ignore the UserID option and run the process as managed by systemd as root.

      A fail-open approach to system security still sounds pretty terrible, honestly. Typos in service configuration shouldn’t end in root privileges.

      1. 2

        I mean, that’s how it usually works. At least, sysv init you’d have to manually change the user from root if you want that, otherwise your process runs as root.

        I think if you wanted to avoid this kind of thing you’d have to require that the user option be always specified or use some default very low permissions user that probably won’t work. Which is fine for me, but I assume the authors didn’t want that.

        1. 3

          True, it’s a potential gotcha in sysvinit scripts as well. Either you need to error out when your su fails rather than continuing to run the script, or use a construct like su -c that won’t run the command if it’s not able to change user.

          The systemd version is a more subtle gotcha though, because it looks more declarative. The su approach in a sysvinit script is obviously just an imperative command, so it’s clearer that you need to defensively program around it. It’s not as obvious that an invalid user in a systemd file is effectively the same as running an invalid su, ignoring the error, and then continuing to run the rest of the script.

          1. 2

            I mean, that’s how it usually works. At least, sysv init you’d have to manually change the user from root if you want that, otherwise your process runs as root.

            Well, yeah. Of course, repeating SysV’s security flaws when you should know better by now strikes me as a massive design mistake in a SysV successor.

            But this isn’t exactly the first wildly-questionable decision re: security we’ve seen from SystemD, so maybe I shouldn’t be surprised.

      1. 1

        Meh. It’s not hard to do better than Java. In Scala you could probably remove all the brackets (I’d have a go if the “code” was available in text rather than an image).

        1. 1

          It looks like the blog post he linked in a reply has code as text.

        1. 3

          Digestible summary without the sky is falling fearmongering?

          1. 6

            What I said repeatedly on HN since it was obvious. There’s a security flaw or backdoor in the Intel ME firmware that has network and DMA access. Intel ignored it for years no matter who they talked to. They seem to be getting around to patching at least that one.

            It’s why I’ve recommended against Intel since vPro and AMT were first planned under Trusted Computing Group. Those meetings were closed door with NSA participation. It was always about DRM and backdoors.

            1. 11

              Don’t assume people are on HN?

              1. 18

                The whole reason I’m here is to specifically not be on HN.

                1. 7

                  I didnt. I implied I said it to IT people and security pros on a tech site full of them with almost nobody buying the claim or caring. This was typical of forums without paranoids. One of most famous there even started debunking why the RNG wasnt a risk with nobody even pointing out the whole chip, incl RNG, was backdoored. Made it kinda moot point, eh?

                2. 3

                  Which tells me nothing I didn’t know (hadn’t read) years ago. The information delta of this article is zero.

                  1. 9

                    Greater than zero: Intel is now releasing patches for their backdoored firmware.

                    1. 2

                      Exactly. Only for 1 backdoor, though. There could be more. ;)

                      1. 2

                        Firmware change list: Fix remote security exploit in all 2008+ Intel platforms (CVE-2017-5689) Add new remote security exploit for NSA

                  2. 2

                    Are AMD any better, though? afaik they implemented similar kinda things.

                    It’s not realistic to avoid both companies, really.

                    1. 9

                      It is for security critical work. I was recommending POWER, PPC, and SPARC back then with their Open Firmware. The market overwhelmingly going x86 means you’ll pay more to avoid the backdoors. On embedded, I designed for VIA Artigos to get low-power x86 at 1GHz w/ crypto accelerator and TRNG.

                      Today, if still needing fast x86, I suggest buying old gear with 2.5-3.0GHz Xeons. Maybe even SMP boards. The better solution, which needs big $$$, is paying AMD’s Semi-Custom business to make one without the management stuff.

                      1. 5

                        Are AMD any better, though? afaik they implemented similar kinda things.

                        Correct, it just took them a bit longer.

                        Check out this timeline by Raptor Engineering.

                      2. 1

                        I’m not as well informed as I perhaps should be on this, but doesn’t AMD also have something similar?

                      3. 2

                        Intel hasn’t revealed the details of the flaw yet.

                        1. 1

                          That’s what I’m hoping for at the moment. I don’t know the first thing about CPUs so I have no idea whether to actually be worried.

                          1. 2

                            Only you can decide whether or not you should be be worried.

                            For those of us who are worried, there are some mitigations available.

                        1. 5

                          This article is rehashing a common misconception of immediate and future work. Teaching folks to assess and assign those accurately is important, especially for junior devs needing to make progress at the hands of the pedantry of peers and more experienced reviewers. Warning: suggestions are Github-workflow specific.

                          Arguing over formatting. Things like tabs vs spaces and brace placement same line or next line should be settled and enforced by the build.

                          Integrate a style checker with the project’s build system or CI system. ScalaStyle, scalafmt, Rubocop, CheckStyle, gofmt, whatever. Agree on a style and set it as law with automatic conformance. Argue over //noformat blocks or something if you really want to get into it (and you should not).

                          Reviewers competing for the most clever one liners or ever more layers of abstraction and indirection. Code should be maintainable and the appropriate level of abstraction. To me this means the minimum level of abstraction. This is obviously a contentious point. Talk it out!

                          There’s a difference between an immediate improvement and future improvement. Immediate improvements should block a merge. Reducing something to a one-liner when it doesn’t have a performance benefit is not an immediate improvement. It may be more idiomatic but it is a future improvement: submit another PR if you’re so adamant about idiomaticity but don’t force me to waste my time with it.

                          Arguing over variable, method, class, or package names. Decide on a methodology and stick with it.

                          Some style checkers handle this (no single- or double-letter variables, no keyboard mashing, etc.) but this could be a fight worth fighting sometimes. Naming is hard. Again, submit a PR to the branch if you’re so adamant. Show me the code!

                          Reviewers requiring weird things like the Assert.assertTrue example above. Ideally all standards would derive from industry standard norms, like SOLID principles. If a reviewer can’t express something in those terms then it shouldn’t be a coding standard.

                          See my sentiments about idiomaticity. Submit another PR if you’re so adamant about idiomaticity but don’t force me to waste my time with it.

                          Previously unstated requirements are presented as code review required changes. Missed requirements ought to be an enhancement for the next sprint.

                          Future improvements. New PRs later in the development cycle so that the unstated requirements can be captured as a separate unit of work.

                          Reviewers requiring changes in code that’s not part of the review. Code beautification and refactoring ought to be tasks in the next sprint as well.

                          WTF people do this? That’s future work: file another issue, cite the PR as the source (“discovered while reviewing #256”), and move on with life. Maybe submit a PR and reduce that mean-time-before-fix number.

                          1. 4

                            I think it is useful to make the original author do idiomatic code improvements. Otherwise the person will be more likely to continue to write non-idiomatic code. Especially if the code being reviewed was written by a more junior member of the team and/or someone new to the language. So my opinion is to do it during code review OR make them do the follow up PR.

                            Besides that I agree with you.

                            1. 1

                              Yeah, it’s not particularly useful to allow a junior to continue to “write PHP in Ruby” or whathaveyou indefinitely, and it should block them merging in my opinion. Otherwise the “temporary” non-idiomatic code will likely be as permanent as every other “temporary” hack.

                              “Merge everything but the most egregious problems and clean it up later in other PRs” is a recipe for rapidly eroding the quality of your codebase.

                              1. 2

                                By the time “PHP in Ruby” or the like makes it as far as a PR, it’s usually too late, in my experience, and a sign that some kind of necessary guidance is not being provided earlier in the process. I haven’t seen these kinds of problems fixed in the PR stage (unless they are very minor things).

                                What I’ve seen happen is that the reviewer is dismayed by the code and points out a few specific solecisms because it’s often infeasible to explain language idioms in depth in a PR comment. The junior developer is annoyed at what seems like nit-picking and will likely fix the specific points mentioned in a way that doesn’t really address the overall non-idiomatic-ness of the code (because they don’t see the larger patterns they are failing to follow, since if they did they would have written more idiomatic code in the first place). In the end, nobody is happy and the code is not much improved.

                          1. 19

                            Personally I will not ever write anything in C again if I can avoid it, i’ll choose rust instead (and I know C way better than rust right now).

                            If I was maintaining a C project, i’d probably start breaking it out into modules and writing new/changed modules in rust and exposing them via its C API features.

                            So, I drink the rust kool-aid.

                            However, I do think it is douchey to go around to random projects that you don’t contribute to and tell them they should be in rust. If you like rust, write your own projects in it/move the projects you work on to it.

                            1. 1

                              Since pretty much all comments seem to be critical, I’m just starting a new thread rather than replying to a certain comment.

                              First off: yes, I agree that for open-source, PR is the way to go. One can’t open the repository to complete strangers. I was talking about work, on a closed product, by a relatively small team; we all know each other.

                              About master being stable at all times, I maintain that this is a fallacy. Yes, of course master should build and the test suite should run flawlessly. But “tests can only prove the presence of bugs, not their absence” (Dijkstra). Inevitably, bugs will enter master and sometimes (quite often!) the fix is only a single line of code. You should not require a PR and code review for that.

                              1. 12

                                The vast majority of production outages I’ve seen have started with “this fix is only a single line of code”.

                                1. 2

                                  I think this meme is a cultural signaling mechanism. In my circles a developer would never even say the words “it’s just a single line of code” without using an ironic tone of voice, eye rolling, etc. :)

                                2. 9

                                  You should not require a PR and code review for that.

                                  You should actually. For one, if it is literally one line of code you probably didn’t write a test to make sure it doesn’t regress again which you should and the reviewer should make you do.

                                  Another thing is that one line of code changes quite often have unforeseen effects elsewhere. Review can help suss this out.

                                  Finally, having at least a PR (if not a review) is some automatic documentation that there was an issue and it was fixed.

                                1. 36

                                  From recent experience with 3 large teams that do pull-request-development, I reject the author’s premise that people don’t actually review code. I’m tasked with reviewing pull requests for security and I usually lose the race to point out flaws in code; the non-security reviewers beat me to the punch.

                                  Most PRs I see have multiple stylebook and efficiency critiques and almost all get iterated at least once and usually multiple times before being merged.

                                  I also reject the premise that the data being captured in Jira tickets is already present in the git log. No, it isn’t. The git log is “what”, but not “why”. Most Jira tickets have at least 2 people commenting in them, something no git commit comment has. Also: being able to demonstrate traceability of code back to tickets is super valuable if you have regulated security requirements.

                                  From what I can tell, in real teams, this process works really well.

                                  1. 9

                                    I agree with your points overall. We do PR development and we do good, real reviews.

                                    However, the git commit messages should be “why” not “what”. “what” is for the git diff, the message should tell you why it was changed.

                                    A good point that that is only one person’s point of view though, and something like jira will often have multiple people commenting.

                                    Edit: One thing I thought of, also the git summary line is typically “what” rather than “why”.

                                    1. 3

                                      Minor note: there is a case where people doing “code reviews” get bogged down in stylistic things or best practices and can gate and slow down deployment of new features.

                                      That is a problem with the people, and not with the idea–it works pretty well once that’s addressed.

                                      EDIT: Downvote incorrect? How, exactly, is this incorrect? I’ve seen this happen multiple times on teams that use code review. Fixing it is part of getting the team up to speed.

                                      1. 5

                                        A codebase should have a style guide. If the code is within the style guide, then there is no problem. If the code doesn’t fit the style guide then fix it. It’s not the reviewers fault that the author didn’t follow the style guide.

                                        Sometimes I have seen people try to dismiss review comments as stylistic when they are really more than that. Spaces around parameters is a stylistic issue. Validating input and checking error codes are not. Avoiding code duplication is not a stylistic issue. Documenting code is not stylistic issue.

                                        Often times the style comments come first because they jump out at the reader when they are reading the code to try to understand it. If the comments never progress beyond style then the reviewer hasn’t done their job.

                                        1. 3

                                          get bogged down in stylistic things or best practices

                                          If it is being described as “bogged down” then it might be too much, but in general a large part of code reviews should be making sure everyone is following the team coding standards and best practices. Tests can ensure the thing actually does what it is supposed to so reviews aren’t really about verifying functionality.

                                        2. 1

                                          If your team isn’t actually providing meaningful feedback during code reviews, that’s usually a problem with the team, not the process. My team is hit or miss, we have a few people that don’t read anything and hit approve, and some that actually provide meaningful feedback. And that’s a people problem that we’re actively working on fixing.

                                        1. 52

                                          “Domain registrars have issued 988 domains with "paypal” in them; all but 4 used for phishing"

                                          1. 12

                                            There could also be potentially valid cases for paypal in a domain, like “dont-use-paypal.com” or “paypal-is-awful.com” or something similar. Someone may even want one of those domains to have perfectly valid SSL too.

                                            1. 16

                                              OpenBSD developer uebayasi@ once had problems signing up to some web site which complained that he shall not use a username which contains ‘ebay’.

                                              1. 10

                                                I was unable to register bannedfromhackernews.com because it had the word hacker.

                                              2. 1

                                                Yes, but the idea of having trusted CAs is that there should be someone taking a look at these before approving the certificates.

                                                1. 8

                                                  That’s what EV certs are

                                                  Also, who’s to say that paypal.mycorp.com is abusive or legitimate?

                                                  1. 3

                                                    How could a CA tell? What’s to stop me from creating paypal.mysite.com, filling it with blog posts critiquing paypal’s business practices, and then once I get a cert for it, replacing it with a phishing site?

                                                    All a CA can do is verify that the person requesting a cert for a given domain name controls the domain. Constantly auditing the ‘worthiness’ of a cert-holder’s domain would be an enormously impractical burden.

                                                2. 1

                                                  You pose that as if they’re equivalent, but it’s not really the case, is it? Domain registrars have issued invalid certificates used for evil, yes. But is it quite so prevalent as with LE?

                                                  1. 16

                                                    The point jcs is trying to make, I believe, is that the domain registrars allowed someone to register the domains in the first place. For DV (domain validation) certificates, it is only verified that you own the machine at the IP address the DNS for that domain points to.

                                                    1. 4

                                                      Oh, yes, indeed. I got domain registrars confused w/certificate authorities.

                                                      Carry on. ;)

                                                1. 4

                                                  My and one other person’s comments over here from an earlier post https://lobste.rs/s/9uld7l/on_eme_html5

                                                  1. 6

                                                    I didn’t feel it was worth its own post, but I came across this Tim Berners-Lee blog post through https://defectivebydesign.org/blog/response_tim_bernerslees_defeatist_post_about_drm_web_standards.

                                                    Personal opinion: I am ideologically very hard line pro Free Software but like to think I have a practical bent. From my standpoint, I think having EME as a standard is better than the likely alternative. I’d prefer the companies to just accept/decide that it isn’t worth it and just stream their video regularly but I don’t think that’s likely.

                                                    I found it interesting that Tim says

                                                    People spend a lot of time on the web, they like to be able to embed Netflix content in their own web pages, they like to be able to link to it.

                                                    I don’t think it says anything either way about his overall argument, but AFAIK(and can see with a quick duckduckgo) you cannot embed Netflix videos in your site. I also don’t see a “Share link” button anywhere, though using the link in the URL while looking at a show might work? That’s just a minor nitpick with his argument in any case.

                                                    Edit: punctuation fix

                                                    1. 2

                                                      The page mentions git specifically as being vulnerable. While I’m sure that’s true, it seems highly impractical to attempt to move git away from SHA1. Am I wrong? Could you migrate away from SHA1?

                                                      1. 17

                                                        [Edit: I forgot to add, Google generated two different files with the same SHA-1, but that’s dramatically easier than a preimage attack, which is what you’d need to actually attack either Git or Mercurial. Everything I said below still applies, but you’ve got time.]

                                                        So, first: in the case of both Mercurial and Git, you can GPG-sign commits, and that will definitely not be vulnerable to this attack. That said, since I think we can all agree that GPG signing every commit will drive us all insane, there’s another route that could work tolerably in practice.

                                                        Git commits are effectively stored as short text files. The first few lines of these are fixed, and that’s where the SHA-1 shows up. So no, the SHA-1 isn’t going anywhere. But it’s quite easy to add extra data to the commit, and Git clients that don’t know what to do will preserve it (after all, it’s part of the SHA-1 hash), but simply ignore it. (This is how Kiln Harmony managed to have round-trippable Mercurial/Git conversions under-the-hood.) So one possibility would be to shove SHA-256 signatures into the commits as a new field. Perfect, right?

                                                        Well, there are some issues here, but I believe they’re solvable. First, we’ve got a downgrade vector: intercept the push, strip out the SHA-256, replace it with your nefarious content that has a matching SHA-1, and it won’t even be obvious to older tools anything happened. Oops.

                                                        On top of that, many Git repos I’ve seen in practice do force pushes to repos often enough that most users are desensitized to them, and will happily simply rebase their code on top of the new head. So even if someone does push a SHA-256-signed commit, you can always force-push something that’ll have the exact same SHA-1, but omit the problematic SHA-256.

                                                        The good news is that while the Git file format is “standardized,” the wire format still remains a bastion of insanity and general madness, so I don’t see any reason it couldn’t be extended to require that all commits include the new SHA-256 field. I’m sure this approach also has its share of excitement, but it seems like it’d get you most of the way there.

                                                        (The Mercurial fix is superficially identical and practically a lot easier to pull off, if for no other reason than because Git file format changes effectively require libgit2/JGit/Git/etc. to all make the same change, whereas Mercurial just has to change Mercurial and chg clients will just pick stuff up.)

                                                        1. 18

                                                          It’s also worth pointing out that in general, if your threat model includes a malicious engineer pushing a collision to your repo, you’re already hosed because they could have backdoored any other step between source and the binary you’re delivering to end-users. This is not a significant degradation of the git/hg storage layer.

                                                          (That said, I’ve spent a decent chunk of time today exploring blake2 as an option to move hg to, and it’s looking compelling.)

                                                          Edit: mpm just posted https://www.mercurial-scm.org/wiki/mpm/SHA1, which has more detail on this reasoning.

                                                          1. 1

                                                            Plenty of people download OSS code over HTTPS, compile it and run the result. Those connections are typically made using command line tools that allow ancient versions of TLS and don’t have key pinning. Being able to transparently replace one of the files they get as a result is reasonably significant.

                                                            1. 1

                                                              Right, but if your adversary is in a position that they could perform the object replacement as you’ve just described, you were already screwed. There were so many other (simpler!) ways they could own you it’s honestly not worth talking about a collision attack. That’s the entire point of both the linked wiki page and my comment.

                                                          2. 2

                                                            That said, since I think we can all agree that GPG signing every commit will drive us all insane, there’s another route that could work tolerably in practice.

                                                            It is definitely a big pain to get gpg signing of commits configured perfectly, but now that I have it setup I always use it and so all my commits are signed. The only thing I have to do now is enter my passphrase the first time in a coding session that I commit.

                                                            1. 4

                                                              Big pain? Add this to $HOME/.gitconfig and it works?

                                                              [commit]
                                                                  gpgsign = true
                                                              
                                                              1. 2

                                                                Getting gpg and gpg-agent configured properly and getting git to choose the right key in all cases even when sub keys are around were the hard parts.

                                                                1. 1

                                                                  That’s exactly what I did.

                                                                2. 3

                                                                  Sorry, to rephrase: mechanically signing commits isn’t a big deal (if we skip past all the excitement that comes with trying to get your GPG keys on any computer you need to make a commit on), but you now throw yourself into the web-of-trust issues that inevitably plague GPG. This is in turn the situation that Monotone, an effectively defunct DVCS that predates (and helped inspire) Git, tried to tackle, but it didn’t really succeed, in my opinion. It might be interesting to revisit this in the age of Keybase, though.

                                                                3. 2

                                                                  I thought GPG signing would alleviate security concerns around SHA1 collisions but after taking a look, it seems that Git only signs a commit object. This means that if you could make a collision of a tree object, then you could make it look like I signed that tree.

                                                                  Is there a form of GPG signing in Git which verifies more than just the commit headers and tree hash?

                                                                  1. 4

                                                                    You are now looking for a preimage collision, and the preimage collision has to be a fairly rigidly defined format, and has to somehow be sane enough that you don’t realize half the files all got altered. (Git trees, unlike commits, do not allow extra random data, so you can’t just jam a bunch of crap at the end of the tree to make the hash work out.) I’m not saying you can’t do this, but we’re now looking at SHA-1 attacks that are probably not happening for a very long time. I wouldn’t honestly worry too much about that right now.

                                                                    That said, you can technically sign literally whatever in Git, so sure, you could sign individual trees (though I don’t know any Git client that would do anything meaningful with that information at the moment). Honestly, Git’s largely a free-for-all graph database at the end of the day; in the official Git repo, for example, there is a tag that points at a blob that is a GPG key, which gave me one hell of a headache when trying to figure out how to round-trip that through Mercurial.

                                                                  2. 1

                                                                    Without gpg signing, you can get really bad repos in general. The old git horror story artile highlights these issues with really specific examples that are more tractable.

                                                                    Though, I don’t want to start a discussion on how much it sucks to maintain private keys, so sorry for the sidetrack.

                                                                    1. 1

                                                                      I don’t see why GPG-signed commits aren’t vulnerable. You can’t modify the commit body, but if you can get a collision on a file in the repo you can replace that file in-transit and nothing will notice.

                                                                      Transparently replacing a single source code file definitely counts as ‘compromised’ in my book (although for this attack the file to be replaced would have to have a special prelude - a big but not impossible ask).

                                                                    2. 4

                                                                      Here’s an early mailing list thread where this was brought up (in 2006). Linus’s opinion seemed to be:

                                                                      Yeah, I don’t think this is at all critical, especially since git really on a security level doesn’t depend on the hashes being cryptographically secure. As I explained early on (ie over a year ago, back when the whole design of git was being discussed), the security of git actually depends on not cryptographic hashes, but simply on everybody being able to secure their own private repository.

                                                                      1. 3

                                                                        the security of git actually depends on not cryptographic hashes, but simply on everybody being able to secure their own private repository.

                                                                        This is a major point that people keep ignoring. If you do one of the following:

                                                                        1. Clone a git repo from a thumb drive you found in a parking lot (or equivalent)
                                                                        2. Don’t review first what you merge into your repo.
                                                                        3. Don’t follow general security best practices

                                                                        then the argument that SHA3, or SHA256 should be used over SHA1 simply doesn’t matter.

                                                                        1. 2

                                                                          And here’s the new thread after today’s announcement

                                                                          (the first link in Joey Hess’s e-mail is broken, should be https://joeyh.name/blog/entry/sha-1/ )

                                                                      1. 40

                                                                        What an awful story! If true (and I rather suspect it is), hopefully it’ll add to the growing amount of evidence that Uber is a garbage company run by garbage people.

                                                                        There was one line though that might be worth pointing out:

                                                                        It turned out that keeping me on the team made my manager look good, and I overheard him boasting to the rest of the team that even though the rest of the teams were losing their women engineers left and right, he still had some on his team.

                                                                        This sort of thing is an example of bad behavior from perverse incentives–I am curious if similar issues have happened in other orgs that have some nominal commitment to diversity.

                                                                        The idea of some manager fucking over somebody’s transfer because they want to play Pokemon with the minorities and catch them all is pretty sickening.

                                                                        1. 13

                                                                          What an awful story! If true (and I rather suspect it is), hopefully it’ll add to the growing amount of evidence that Uber is a garbage company run by garbage people.

                                                                          Her story is not atypical as far as technology companies go. I’m 98 bid that it’s true and what I think distinguishes Uber is only that it’s open about its amorality: Kalanick revels in it, while other companies' executives are more cautious.

                                                                          What is different/unusual is that HR was honest about their inability to do anything about a politically favored but problematic manager, and the high likelihood that he would “Perf” her (that is, retaliate with a false negative review). Most companies these days are a bit fascist (“we run lean”, “be a team player”) but they usually don’t open the playbook like that except to executives.

                                                                          I’ll write a long-form reply on this after I feed my cats. Stay tuned.

                                                                          1. 1

                                                                            This sort of thing is an example of bad behavior from perverse incentives–I am curious if similar issues have happened in other orgs that have some nominal commitment to diversity.

                                                                            I don’t think there’s enough information in the article to support the position that the author’s inability to transfer was definitively due to broken pro-diversity incentives. There’s nothing indicating there was any tangible pro-diversity incentive in place, and the only supporting data from the article is an offhanded remark that sounds more like an attempt at one-upmanship than anything else. It’s equally plausible that there were any number of additional contributing factors to her manager’s decision to block her transfer, including keeping a talented engineer who makes his team look better, and that being able to brag about having a woman on the team was a secondary bonus.

                                                                            Certainly it’s possible to craft a broken system of incentives around any policy, but I think pointing the finger at unknown pro-diversity incentives here is a red herring at best. The amount of infighting and generally toxic behavior described in the article suggests a much broader set of poor incentives throughout the organization.

                                                                            *edited for clarity

                                                                            1. 4

                                                                              that being able to brag about having a woman on the team was a secondary bonus.

                                                                              This is awful even by itself, FWIW, and IMO shows that there is a diversity issue very clearly. Offhand comments often point to deeper issues.

                                                                              1. 2

                                                                                Hmm. I think we’re running into some linguistic ambiguity here. If by diversity issue, you mean that Uber built an internal culture that’s both incapable of supporting and is actively damaging to diversity, then yes, I agree completely, they have serious diversity issues, likely compounded and enabled by deeper issues in the power structure of the organization.

                                                                                If you mean as the top-level thread parent suggested that there’s some backfiring pro-diversity effort in play, I remain unconvinced.

                                                                          1. 37

                                                                            So it starts off saying “not meant to be insulting to anyone or anything” and then, a bit below, they decide to take “designed for stupid people” as an actual reason of why Go is not good? I can’t say I’m impressed.

                                                                            1. 21

                                                                              To be fair, the author is just listing what other people have said about go and their major points. The author themselves did not say that Go is “designed for stupid people”, they included someone who did say it.

                                                                              1. 20

                                                                                I refuse to accept that as valid; quoting someone is an action too. Author could have decided to leave that out, because let’s face it, it brings nothing to the table. Both of the articles cited for that point have a lot of other stuff that they are cited for, as well. To be fair, I’m probably nitpicking, but it still irritates the hell out of me when people pull together a list of reasonable concerns, and then plop in a “you’re stupid” trump card. I’ve worked with the “you’re stupid” trump card for a year and a half, now. I really dislike it. A lot.

                                                                                [Edit: when I say “work with”, I mean “had a coworker who kept a trump card unreasonable argument nearby at all times”]

                                                                                1. 3

                                                                                  Personally, I would feel i’d either have to leave the article out entirely (which may have been what I did) or include the “point” in the list. It would have felt incomplete/wrong to me otherwise.

                                                                                  1. 1

                                                                                    i got voted down as a troll for saying this in another subthread, but direct, deadpan quoting of someone who says something over-the-top is a definite form of humour. “psuedointellectual arrogance of Rob Pike and everything he stands for” was a big tell, but the overall “tone” of the page is subtly (and i’d bet intentionally) funny, regardless of whether it serves a useful purpose or not.

                                                                                2. 5

                                                                                  I doubt they’re falling Go users stupid; it’s probably a reference to Rob Pike’s (one of Go’s creators) comment “They’re not capable of understanding a brilliant language but we want to use them to build good software. So, the language that we give them has to be easy for them to understand and easy to adopt.”

                                                                                  1. 4

                                                                                    It was definitely a poorly expressed point by Rob Pike but making a language easy to use is absolutely the whole point of language design. And really, Go is very easy to learn and provides a very low barrier to getting real work done. So that’s a great thing.

                                                                                    1. 4

                                                                                      making a language easy to use is absolutely the whole point of language design

                                                                                      That is really not the case. “Ease of use” is only one dimension of a language, and one that’s commonly overoptimized for to the detriment of other properties like correctness, efficiency, readability, maintainability…

                                                                                      Google “rich hickey simple made easy”

                                                                                    2. 1

                                                                                      Okay, so a hit on pride, but how is that a shortcoming of the language? If it’s a driving point of design decisions, it would result in the language being weak-sauce, but in and of itself, it’s not a weakness of the language.

                                                                                      1. 2

                                                                                        Personally I’ve never understood the comment as developer mistakes seem like the greatest motivator for strong type systems and uh, not including null.

                                                                                    3. 1

                                                                                      “I’m sorry if you feel offended”.

                                                                                    1. 10

                                                                                      More importantly, it will he the last init system we will get considering how deeply and how widely it has itself integrated into various systems.

                                                                                      A new init system would have to emulate all of it before being able to innovate on anything. At this point it probably has already become a systemd clone.

                                                                                      1. 8

                                                                                        What are embedded devices like phones/tablets/car computers using? I’m pretty sure they’re not adopting systemd since they have their own parallel universe of distros. Are they using plain init and shell scripts?

                                                                                        With Android and almost all other phone OS’s except iOS, it seems like Linux is still getting bigger on devices. (Although it’s probably equally as big or bigger on the server.)

                                                                                        I think systemd is solving more of a desktop problem… where you are unplugging devices and where you have diverse hardware. On headless servers you might not need many of the features that systemd provides.

                                                                                        Ironically, desktop is where Linux is used the least, but it appears to be driving a lot of the design decisions. Upstart was also designed for the desktop as far as I can tell.

                                                                                        Although I guess both servers and devices want fast startup, and systemd apparently helps there.

                                                                                        My hope is that the embedded Linux architecture will take over the desktop and perhaps get rid of systemd along with it. There have been some experiments in using Android for desktops although I haven’t followed them closely.

                                                                                        1. 6

                                                                                          I think systemd is solving more of a desktop problem

                                                                                          I see this brought up a lot, but at least in my hobbyist server admin opinion systemd has the most advantages on the server. It is so so so much easier to write a service configuration that starts correctly, stops correctly, and properly waits for the right things to go up/down in both cases. No more 100s of lines of bash scripts, just a few lines of config and most services are going to “just work”.

                                                                                          I rarely have to write service definitions on the desktop.

                                                                                          1. 2

                                                                                            Yeah that’s true. init.d scripts are too often copied and pasted and buggy. There needs to be a better way to share this logic. Config files are one way, but I think a better shell language is another way, with different tradeoffs.

                                                                                          2. 2

                                                                                            SailfishOS (Jolla) is using Systemd IIRC.

                                                                                            1. 2

                                                                                              It does, yes.

                                                                                        1. 3

                                                                                          If we wanted to truly solve the problem of codebase-exhaustiveness, however, I posit that it is impossible to create a fully robust type checker for Ruby.

                                                                                          I don’t have any proof, but I don’t believe this to be true. The example given (a function with a block) could be type checked if blocks can have types specified, for instance. I think the problems of having type checked dynamic languages is hard for a number of reasons(that are more complicated than dealing with blocks) but I don’t think it is impossible.

                                                                                          1. 9

                                                                                            I don’t have any proof, but I don’t believe this to be true.

                                                                                            Oh, I do! http://erlang.org/doc/man/dialyzer.html

                                                                                            Anyway type checking aside you can definitely benefit from Maybe in a dynamic language. The requirement is actually not so much to have a static type system as it is to have pervasive pattern matching and buy-in from the standard library. Erlang is dynamic but often returns tuples of {ok, Value} or {error, Reason} from functions which could fail; the presence of the wrapper indicates you need to match against it to separate out the happy path from the failure. Obviously you can defy it, but you basically have to go out of your way to do this.

                                                                                            1. 2

                                                                                              Agreed on Maybe. I do not much care for nil/null.

                                                                                            2. 4

                                                                                              My observation is that people from dynamic programming backgrounds usually imagine things like String and Int when talking about types. Even though these are the kinds of types that appear the most often in your codebase, they’re not the things that determine the power of a type system. Whenever you want to reuse code and express patterns, you need abstractions, and that progressively demands more and more from the type system. So,

                                                                                              • you start with a String but then you need a function String -> Int
                                                                                              • then you want to work with such functions so you need (String -> Int) -> [String] -> [Int]
                                                                                              • then you want this to work for all types so you need forall a b. (a->b) -> [a] -> [b]
                                                                                              • then you want this to work for many kinds of data structures, so you need forall f a b. Functor f => (a->b) -> f a -> f b
                                                                                              • then you want to interleave side effects, so you need forall t f a b. (Traversible t, Applicative f) => (a -> f b) -> t a -> f (t b)

                                                                                              And this is really just the tip of the iceberg. So, please take a moment to think about a metaprogramming utility you use in Ruby, and try to express its type completely (not just “this takes boring stuff and returns cool stuff”), in any type system you can imagine. I think this exercise quickly demonstrates how easily types get intractable once you try to model the creative things people do in dynamic languages.

                                                                                              1. 2

                                                                                                First off, I find your “observation” and overall tone a bit condescending but it probably is true for many dynamic programmers. Myself, I’ve used typed languages enough to not have this simplistic view but I’m not well versed enough to be able to discuss it at a formal level. I plan to learn a lot more, but my list of learning projects is miles long so I don’t know when I’ll get to it.

                                                                                                Typing some forms of metaprogramming actually seems quite easy to me. If you are doing lisp macros for metaprogramming, a lisp macro takes a code form and returns a code form so you need a type representing a code form (which basically would be a list of literals and symbols) and then you can have a super generic type definition for macros. Since code forms are lists in lisp, each macro would be taking a specific “shape”(or shapes) of list and returning another specific “shape”(or shapes) of list so that should be able to be typed as well. That being said, depending on the implementation of the type system some things that we currently do in macros might not be possible simply due to being hard/impossible to express the type. I haven’t played with shen much yet, but I assume it must be doing something with types and macros.

                                                                                                For Ruby it is harder/possibly impossible to type metaprogramming since everything is basically side effect based from what I understand. For that matter, due to Ruby’s overall design of side effects everywhere ruby in specific may be not possible to build a robust type system for. There might be a way but I have no idea what it’d look like.

                                                                                                I don’t think that applies to “dynamic programming languages” in general though.

                                                                                                1. 6

                                                                                                  I’ll say that “Typing Ruby” is probably a pretty difficult endeavor. I come at this from a lot of strongly typed programming experience, significant experience with gradually-typed systems, and the experience of having written a few type systems. On top of that all, I used to use Ruby in anger quite a lot a long while ago. That said, I haven’t tried to actually write a type system for Ruby, so what follows is speculation.

                                                                                                  There are at least two technical challenges. Broadly, applying types to dynamic languages refines them and adds more information. The complexity curve of this new information can be quite high since it’ll be describing a system that was designed outside of the influence of the refinement.

                                                                                                  Firstly, this will make inference a really, really tricky proposition. Inference is very delicate in type systems and can go from complete-and-easy to completely inoperable in a flash. Annotation of methods and procs is a good first step—it sidesteps the need for inference by demanding that the programmer provides intent—but a practical type system will allow you to elide these types at least some of the time.

                                                                                                  Secondly, there will be a tradeoff between complexity of the type language and the set of features which can be given types both at all and in practice. Since rising complexity makes the type system harder to use and breaks inference even more this will probably instead be interpreted at the cost of loss of Ruby features.

                                                                                                  Both of these technical challenges apply pretty much only if your goal is to build a New Ruby With Types and are willing to lose the entire Ruby community and existing codebase. If that’s not your goal then you additionally need to handle code which is untyped alongside code which isn’t. There are roughly two approaches here that I know of:

                                                                                                  • Give all untyped code the any type and be forced into doing manual value-shape checks at the boundaries between typed and untyped code (a la Typescript)
                                                                                                  • Do the above semi-automatically by extending a contract system into the untyped fragment of the language (a la Typed Scheme)

                                                                                                  You’ll also run into the question of “Can we give types to existing code?” which again you can follow Typescript’s lead here, but note that this is extra burden on either the community or the original maintainer and it gets a little dicey.

                                                                                                  Finally, after all that we can scratch the surface a little bit of interesting Ruby features which would be difficult to typecheck. Assuming we don’t have a dependently typed system (which is a good assumption since those are still heavy research subjects) you’ve already struck the motherlode in naming Ruby’s runtime metaprogramming features. The easiest target is “eval”. This is pretty much right out. With dependent types we could add a new function “check” which, given a runtime type representation that reflects its compile-time type could link up a successful checking process with an option to evaluate the expression into that type… but let’s leave that for now.

                                                                                                  Idiomatic Ruby makes heavy use of method_missing. There are worse offenders used popularly, but this one is simple to analyze and crushingly difficult to type. Let’s just leave on the table its relationship to respond_to? which allows even more dynamic use of the functionality (and room for error when respond_to? and method_missing don’t align). Under the presence of method_missing the fragment

                                                                                                  x.method arg1, arg2, arg2
                                                                                                  

                                                                                                  can take essentially any type whatsoever. It also provides zero information about what the types of arg1, arg2, and arg3 are. We can special case this to first infer x’s type and then read out its known methods via its class hierarchy so that at least if a known method exists we can easily do the right thing statically, but the level of runtime shenanigans available to change the type of x.method when method_missing is in use is extreme.

                                                                                                  A sophisticated type language could constrain method_missing at least some of the time by constraining what sort of overloaded methods are allowed to exist, but this is (a) restricting its power to the point that it’s really almost completely defunct and (b) breaking most idiomatic use-cases of method_missing.

                                                                                                  You could make it so that method_missing just accepts any types and returns an any type, but again this breaks inference since you often will want method calls to put constraints on their arguments so that type information can “flow backwards” and, additionally, it will force manual runtime type guards in places which are normally elided. The common practice is to just “know” that method_missing calls have certain behaviors without any static description amenable to the language.


                                                                                                  So, where does this leave things? Really, there are a lot of great examples of type systems which are sufficiently rich and generic to make meaningful dents in dynamic languages today: Typed Scheme, Typescript, and Flow come immediately to mind. Additionally, Erlang’s Dialyzer should be studied although it tries to do a slightly different thing. All of them (except Dialyzer) face really enormous difficulties though in that they splinter the community. Javascript programmers are not immediately Typescript programmers, although Typescript programmers do sometimes get the opportunity to reuse some Javascript code.

                                                                                                  These examples are probably the best leaders for any Typed Ruby exploration. They also set the limits for what people have figured out how to do with respect to gradual type systems in practice.

                                                                                                  1. 2

                                                                                                    I’m sorry if that sounded condescending, it wasn’t my intention.

                                                                                                    1. 1

                                                                                                      Yeah, I wasn’t offended and I figured it wasn’t intentional. It still bugged me slightly though :).

                                                                                                2. 4

                                                                                                  I’m pretty sure you can do heinous tricks with class objects and VM to defeat even the most earnest attempt at type checking.

                                                                                                  1. 3

                                                                                                    Ugh. I meant to say eval instead of define_function. Does that change anything?

                                                                                                    1. 3

                                                                                                      I’d have to think about that one. My first instinct is that eval breaks type systems but not 100% sure.

                                                                                                      Edit: found that haskell has an eval. http://hackage.haskell.org/package/plugins-1.5.1.3/docs/System-Eval-Haskell.html. So they were able to do limited type checking it looks like but not “robust”.

                                                                                                      1. 5

                                                                                                        Notice the way that Haskell’s eval works

                                                                                                        • It constrains the return type with a Typeable constraint meaning that we have access to runtime type representatives
                                                                                                        • It can fail Maybe
                                                                                                        • The IO is incidental to Haskell’s pure nature—an impure language won’t suffer too much here

                                                                                                        Typeable and Maybe here are providing the structure for what’s known as a type safe cast. You can think of eval as working in stages:

                                                                                                        Here, Dynamic represents a value which can take any type whatsoever—the specific type information is not known at compile time, but it’s guaranteed that Dynamic values will be stored alongside runtime type representations which match their actual type. Stage 2 is the type-safe cast (and actually already has a definition as Data.Dynamic.fromDynamic).

                                                                                                        Stage 1 is just a hook into the Haskell compiler. It typechecks the code and if that check succeeds produces a runtime representation of the result type. It runs the code fragment and obtains the result and then packages that result up with its runtime type information into a Dynamic value.

                                                                                                        Stage 2 is a trick. Since a is left generic, it’s up to the caller of stage2 (and ultimately eval) to pick what a they want. In effect, it’s a guess by the user of eval as to what the return type of their fragment will be. We use Typeable to obtain a runtime type representation of the caller’s “guessed” type and then compare that representation with the type we inferred/checked in stage1. If they match then we coerce the result into the “guessed” type and things are OK. If they don’t match, we fail and return Nothing.

                                                                                                        Really, the stages probably ought to be mixed so that if the caller guesses the wrong type then perhaps the string fragment shouldn’t be run at all.

                                                                                                        Finally, as the docs note, there is a limit to how much this guess-and-check game can be played with real code in that the eval’d fragment more-or-less has to be monomorphic.

                                                                                                        Also worth noting is the unsafeEval variant there. As always in Haskell, unsafe means really, really unsafe and requires that the user is carefully checking invariants. It is equivalent to a call to unsafeCoerce and can be used to break type invariants in a completely arbitrary fashion. It also doesn’t provide much more power than normal eval.

                                                                                                        1. 2

                                                                                                          article updated. thanks for pointing it out!

                                                                                                    1. 2

                                                                                                      I work for a place that helps non-profits and other folks take donations and tips, so I’m perfectly happy with that. We work with some great orgs like the Salvation Army.

                                                                                                      1. 3

                                                                                                        I don’t support the salvation army due to the political stances they push using the money they get https://en.wikipedia.org/wiki/The_Salvation_Army#Controversy. If it was just political stances of the owner or something (Like Chik-fil-a) then I wouldn’t care, but the organization itself uses the money given to push their political agenda that I strongly disagree with.

                                                                                                      1. 7

                                                                                                        I experienced something similar to this in Rust maybe a year ago. I was trying to write a baby LISP interpreter as a way to get more comfortable in the language. It felt like I was fighting the compiler the whole way through, but it was because I was so used to programming in a C++ way. The thing that really killed the project for me was something so painfully simple that it had me scratching my head as to how this passed under peoples' noses: getting a substring from some base string.

                                                                                                        • There is no .substr() method, or any equivalent that’s built in.
                                                                                                        • You can’t take the slice version of the string, allocate a new slice, and copy index-to-index; slices are indexed by byte, and not by character. Furthermore, string sizes are done by character, and not by byte.

                                                                                                        So the only solution was to write my own substring method (accounting for Unicode/multibyte character weirdness described above), or bring in a new dependency to the project. It felt ridiculous that I wasn’t able to do this basic string manipulation that’s a pretty common operation in most languages. My solution was equally ridiculous; chop off N front characters, reverse the string, chop off M back characters, and reverse again.

                                                                                                        Maybe there’s a better way to do this, but it just felt like an extreme solution to something that the standard library should be able to solve. The first hit from googling “rust substring” gives this link detailing a few possible solutions, but nothing conclusive. It was a frustrating experience and it really turned me off to the language for a little while.

                                                                                                        1. 12

                                                                                                          If you can, please see my response to @azrazalea. Rust strings are UTF-8 encoded, which means we give up fast & easy substring indexing, but the rust devs have decided that the gains are worth the losses.

                                                                                                          1. 6

                                                                                                            Taking a substring in Rust by byte ranges is simple. If s is a String, then &s[i..j] is a &str substring of s from bytes i to j. If either i or j aren’t on a valid UTF-8 encoded codepoint boundary, then the expression causes a panic.

                                                                                                            Doing this by Unicode codepoint index is generally considered an anti-pattern and something you should try to avoid. Both Rust and Go leave this type of slicing/indexing out of their standard library, so Rust is not alone in this. But if you wanted to do it, the way I’d suggest would be to get the byte offset of the corresponding codepoint, and then slice it that way. You can get byte offsets by using the s.char_indices() iterator. For example, s.char_indices().nth(5).unwrap().0 will give you the byte offset of the beginning of the 5th codepoint.

                                                                                                            The reason why this type operation is not something you should generally do is because “Unicode codepoint” isn’t necessarily “one character” from the perspective of humans. A better approximation of “character” is a grapheme cluster, as defined by the Unicode standard.

                                                                                                            As I see it, there are two good reasons to make this operation very explicit:

                                                                                                            1. To give you appropriate pause that maybe it’s not what you want to do.
                                                                                                            2. To make the performance costs obvious. Slicing by bytes is O(1), but slicing by codepoint is O(n) given how Rust represents strings internally.
                                                                                                            1. 1

                                                                                                              it just felt like an extreme solution to something that the standard library should be able to solve

                                                                                                              I’m a beginner/hobbyist in rust, but from what I understand their philosophy is to have very very little in the standard library and for you to pull in a dependency for many things that other languages have in their standard library.

                                                                                                              In addition, strings are indeed just bytes in rust but from what I understand they went that way to avoid setting a “blessed” encoding for the language. Since they are just bytes, libraries can implement UTF-8, UTF-16, whatever support.

                                                                                                              As far as unicode support in standard library, the situation isn’t that great with programming languages in general. I don’t see rust as much of an outlier.

                                                                                                              1. 17

                                                                                                                This is incorrect. Strings in rust (meaning the String and &str types) are guaranteed to be UTF-8. In the stdlib, you can get the individual characters with .chars() and get their indices with .char_indices(). If you want to deal with graphemes, you need to use unicode-segmentation: https://unicode-rs.github.io/unicode-segmentation/unicode_segmentation/index.html . Manishearth had a great post on dealing with unicode in strings recently: http://manishearth.github.io/blog/2017/01/14/stop-ascribing-meaning-to-unicode-code-points/

                                                                                                                edit: wrong manishearth page! it is now corrected

                                                                                                                (also cc @intercal for all of the above)

                                                                                                                1. 1

                                                                                                                  Thanks for the correction! The need of a library for graphemes is what I was thinking of.