1. 23

  2. 22

    I think it’s generally worth following the recommended steps to get a project running to get an understanding of what it looks like in a known working setup before trying to configure the application in a different way to the developers.

    1. 7

      I am happy to talk to the fxa team to find out what they recommend and if there is more or better documentation available.

      1. 6

        This happens with a lot of bigger “open source” projects. Basically code dumps without any consideration for making it easy to deploy it by someone who’s not a developer and intimately familiar with the code base. I guess as good a definition of “devops” as any.

        The easiest to improve this is to get it properly packaged for the most popular Linux distributions. This is a great QA tool as well. If your software (and its dependencies) are easy to package it means you have your dependencies under control, and you can build the software from source at any point.

        Unfortunately nowadays you can be happy to get any of those projects to even work in a provider Docker container. Running them as a production service yourself is a completely different story, and practically impossible.

        1. 21

          To be fair, I don’t think every piece of open source is really meant to be run as-is by users so much as “here’s what we use to run this, you can build off it if you want.” It’s perfectly fair to release your internal tools for the purposes of knowledge sharing and reference without intending to support public consumption of them by non-developers.

          Further, it looks like the author made minimal effort to actually use what is distributed as it was meant to be:

          • The suggested route of using docker images wasn’t used. Wanting to understand something and be able to run it without a pre-built image is fine, but totally skipping the intended usage and trying to roll your own from the beginning is only likely to make the system harder to understand.
          • The projects appear to provide release packages, yet he pulled the source directly from git, and at whatever state master happened to be in at the time, rather than a branch or tag. At least one of them looks to be failing CI in its current state, so it’s not even clear that what he had was a correctly functioning version to start with.
          • He’s ignored the npm-shrinkwrap provided and automatically upgraded dependencies without any indication or testing to confirm that they will work. While it would be great to think that this wouldn’t be an issue, the state of npm is not such that this is a realistic expectation.
          1. 3

            Where is the purpose of knowledge sharing when you do not make things in order to be understandable? Knowledge sharing is not just take my stuff and understand. You have to make it understandable and be sure the person understand it well. That’s why you have wiki and documentation, to facilitate the understanding.

            Where do you find the suggested route is Docker ? You know, when you try to deploy something, you begin by somewhere. I read the install part of one the repository, Firefox Accounts Server, as I do for a lot of application I install and I followed the process. The written process is git clone and npm install. After some research, I discovered there was not an unique repository needed but there are several links together. Where is it written? How can I suppose to know?

            You can’t say I did minimal effort when I took so much time on it. I am used to deploy stuff and configure application. I configure by my own each microservice in order to make it works. The problem was after three days, I still have to guess things by my own, understand it, configure it properly and fix issues I got. I am sorry but it is too much. It is not my job to make the application easily understandable and easy to deploy. It is the job of maintainers and it is what I said in my blog post.

            When I compare other applications I deployed, and some of them are bigger than this, FXA has a lot of work to do. The master branch is actually a development branch, there is no master branch and the documentation told you to pull from it to deploy in production. :o

            Here we have just a big thing which is our stuff, deal with it and make it workable if you want. I made a try and failed. It is not suppose to be deploy by someone who is not working in this project full time. That’s all and it is questionable when it is a Mozilla Foundation and who is publicly saying that privacy matter.

            1. 5

              Knowledge sharing is not just take my stuff and understand. You have to make it understandable and be sure the person understand it well.

              Your opinion is not universal.

              Different cultures handle this problem differently. Some culture/language pairings are listener-responsible and some are writer-responsible.

              1. 2

                Clearly. I think the best way is to have writer and listener responsible but it is not the debate here I guess.

                1. 0

                  I’m in agreement. Can’t understand how people can defend bad practice of the art. Why would anyone who cares about good work defend anything like this? It’s like they are working against themselves, karmic-ally setting themselves up for a later fall through someone else’s failing, … for no sensible gain.

                2. 1

                  Shouldn’t they both be “responsible”? And please tell me which cultures? Are we talking professional vs unprofessional, or what? I’ve worked in hundreds of different cultures worldwide over many decades, and I’ve never seen a claim like this.

                  1. 2

                    Come to Asia, never cease being frustrated.

                    1. 1

                      Been there. Once I understood “hectoring”, learned pretty quick how to generate a larger, louder response.

            2. 4

              Here is some more concrete documentation that I found:

              Run your own Firefox Accounts Server https://mozilla-services.readthedocs.io/en/latest/howtos/run-fxa.html

              Self Hosting via Docker https://github.com/michielbdejong/fxa-self-hosting/blob/master/README.md

              Installing Firefox Accounts https://webplatform.github.io/docs/WPD/Infrastructure/procedures/Installing_and_theme_Firefox_accounts/

              The ‘code dump’ argument is a bit odd .. these projects are all super accessible and being developed in the open on Github. No project is perfect. If you think something specific, like self-hosting documentation, is missing, file a bug or make the investment and work on it together with the devs. Open source is not one way.

            3. 5

              These projects are way easier to deploy when you use Docker. It will hide 90% of the stupid stuff that is automated for you in the Dockerfile.

              1. 4

                Is this a bug or a feature? One could fail to explain the “stupid stuff”, and be tripped up by a part of it that really mattered.

                It’s not enough to be “open source”, it needs to be transparent and credible too, so that one can reasonably maintain it. Hiding things in a Dockerfile doesn’t pass this test.

                1. 8

                  I think it’s probably “enough” for a project to be whatever the maintainers want it to be. A Dockerfile is just an abstraction like a Makefile or even a shell script; the built artefact is effectively a tar file containing the assembled software, ready for deployment. I’m not a fan of the ergonomics of the docker CLI, but the idea that you’re “hiding” anything with Docker any more than you are with any other packaging and deployment infrastructure seems specious at best.

                  1. 0

                    Instead of focusing on a single word, try considering the other, opposing two - being credible and transparent. Clearly this isn’t.

                    For one thing, the reason you don’t do this is that it’s easy to be taken advantage of and place exploitative code in a big pile of things. For another it’s bad form to not communicate your work well, because maintainer’s struggling to deal with an issue don’t create more (and possibly even worse) versions they might claim “fix” something, and in the fog of code it might not be easy to tell which end is up.

                    I’m surprised you’d defend bad practice, since nearly everyone has had one of these waste a few hundred hours of their time. Your defense sounds even more specious than focusing on the wrong word and missing the point of the comment.

                    1. 2

                      I highlighted the word enough because your comment seems to have come from a place of entitlement and I was trying to call that out. The project doesn’t owe you anything.

                      Indeed, most of my comment was attempting to address your apparent suggestion that using a Dockerfile instead of some other packaging or deployment mechanism is somehow not transparent (or, I guess, credible?). I’m not really defending the use of Docker in any way – indeed, I don’t have any use for it myself – merely addressing what I think is specious criticism.

                      Regardless of what point you were trying to make, your comment comes across as an effectively baseless criticism of the project for not delivering their software in a way that meets your personal sense of aesthetics. Things are only hidden in a Dockerfile to the extent that they are conveniently arranged for consumption by consumers that do not necessarily need to understand exactly how they work. This isn’t any different to any other process of software assembly that abstracts some amount of the internal complexity of its operation from the consumer; e.g., I am not in the habit of reviewing the Linux kernel source that executes on my desktop.

                      If you want to know how the Dockerfile works, you could always look inside! It seems no more or less transparent or credible than a shell script or a markdown document that executes or describes a similar set of steps.

                      1. -1

                        I build them so I know whats inside. You’re looking for something to be outraged at, and find it in my words.

                        Perhaps you can defend those who write programs with meaningless variable names, and stale comments that no longer reflect the code they were next to.

                        Point your anger at somewhere else. Meanwhile who speaks up for something unintentionally vague or misleading. Or are you also going to defend syntax errors and typos next.

                        1. 1

                          I’m not angry – merely flabbergasted! How is a Dockerfile “vague or misleading”? By definition, it contains a concrete set of executable steps that set up the software in the built image.

                2. 1

                  I hate the docker setups that are just one piece of the setup and you are expected to spend a few days writing a docker compose file to piece together the whole thing

                  1. 1

                    Which problems do you encounter when writing docker-compose files? I’ve mostly had the experience that the upstream Dockerfile is horrible (for example Seafile is trying to pack everything into a single image - which causes feature-creep for the setup scripts) - but writing docker-compose.yaml always felt rather straight forward (besides Volumes, I’m still confused by Volume management on occasion).

                3. 3

                  I tried to install the Firefox Sync Server and failed at the same level…

                  1. 1

                    Firefox Sync Server is independent to Firefox Account Server. I created a tutorial to deploy it and I currently using an instance I host in my own server. You can find the link to the tutorial here

                    1. 1

                      Oh, I know, but I meant like it’s a bit undocumented and not an easy thing to deploy (at least I couldn’t). Thank you for your tutorial, I’ll take a look today and give it a try!

                  2. 3

                    I love blog posts about failures. It usually prevents the audience from repeating a mistake.

                    For this one, I’m not sure. Because I’m convinced there must be a solution to the problem. It’s (admittedly) difficult, but I don’t think giving up should be the end state

                    Did you file bugs or reach out to devs on IRC? My assumption is that developers will consider this aa documentation bug that is worth resolving.

                    1. 3

                      You’re right, there’s little detail. But it’s a short blog post, and there are advantages in it not being tl;dr. More of a warning than a diagnostic. Think that was what was meant.

                      I have noticed that sometimes when people do a long “tried this, didn’t work … tried this, didn’t work … tried this, didn’t work … “ that many pick the critique apart, even though playing “Twenty Questions” because nobody documented their work, thus the “guessing game”. You get criticized for not being a “good guesser”, or reading someones mind wrong.

                      You shouldn’t have to guess or “mind read”. Consider it a part of writing clear code, that what it attaches to is likewise spoken for. I wish it was the case that all software that I’ve read to discern details of operation allowed me to “compensate” for lack of documentation, but that’s the rare case. Many often fix the code and don’t update even the comments.

                      And I’ve had prior experience with Mozilla not unlike this, so not surprised.

                    2. 3

                      I had similar experience when I looked into using Firefox Persona many moons ago. I really wish projects like this were written in easily packagable and distributable languages like Nim, Go, Rust, Crystal, D…

                      On the bright side the fxa-auth-server’s API is well documented. This is a nice real-world project for someone to implement using one of the above languages.

                      1. 2

                        Can someone ELI5 why Firefox is not to be trusted anymore?

                        1. 4

                          They’ve done some questionable things. They did this weird tie-in with Mr. Robot or some TV show, where they auto-installed a plugin(but disabled thankfully) to like everyone as part of an update. It wasn’t enabled by default if I remember right, but it got installed everywhere.

                          Their income stream, according to wikipedia: is funded by donations and “search royalties”. But really their entire revenue stream comes directly from Google. Also in 2012 they failed an IRS audit having to pay 1.5 million dollars. Hopefully they learned their lesson, time will tell.

                          They bought pocket and said it would be open sourced, but it’s been over a year now, and so far only the FF plugin is OSS.

                          1. 4

                            Some of this isn’t true.

                            1. Mr. Robot was like a promotion, but not a paid thing, like an ad. Someone thought this was a good idea and managed tto bypass code review. This won’t happen again.
                            2. Money comes from a variety of search providers, depending on locale. Money ggoes directly into the people, the engineers, the product. There are no stakeholders we need to make happy. No corporations we got to talk to. Search providers come to us to get our users.
                            3. Pocket. Still not everything, but much more than the add-on: https://github.com/Pocket?tab=repositories
                            1. 3
                              1. OK, fair enough, but I never used the word “ad”. Glad it won’t happen again.

                              2. When like 80 or 90% of their funding is directly from Google… It at the very least raises questions. So I wouldn’t say not true, perhaps I over-simplified, and fair enough.

                              3. YAY! Good to know. I hadn’t checked in a while, happy to be wrong here. Hopefully this will continue.

                              But overall thank you for elaborating. I was trying to keep it simple, but I don’t disagree with anything you said here. Also, I still use FF as my default browser. It’s the best of the options.

                            2. 4

                              But really their entire revenue stream comes directly from Google.

                              To put this part another way: the majority of their income comes from auctioning off being the default search bar target. That happens to be worth somewhere in the 100s of $millions to Google, but Microsoft also bid (as did other search engines in other parts of the world. IIRC the choice is localised) - Google just bid higher. There’s a meta-level criticism where Mozilla can’t afford to challenge /all/ the possible corporate bidders for that search placement, but they aren’t directly beholden to Google in the way the previous poster suggests.

                              1. 1

                                Agreed. Except it’s well over half of their income, I think it’s up in the 80% or 90% range of how much of their funding comes from Google.

                                1. 2

                                  And if they diversify and, say, sell out tiles on the new tab screen? Or integrate read-it-later services? That also doesn’t fly as recent history has shown.

                                  People ask from Mozilla to not sell ads, not take money for search engine integration, not partner with media properties and still keep up their investment into development of the platform.

                                  People don’t leave any explanation of how they can do that while also rejecting all their means of making money.

                                  1. 2

                                    Agreed. I assume this wasn’t an attack on me personally, and just as a comment of the sad state of FF’s diversification woes. They definitely need diversification. I don’t have any awesome suggestions here, except I think they need to diversify. Having all your income controlled by one source is almost always a terrible idea long-term.

                                    I don’t have problems, personally, with their selling of search integration, I have problems with Google essentially being their only income stream. I think it’s great they are trying to diversify, and I like that they do search integration by region/area, so at least it’s not 100% Google. I hope they continue testing the waters and finding new ways to diversify. I’m sure some will be mistakes, but hopefully with time, they can get Google(or anyone else) down around the 40-50% range.

                                  2. 1

                                    That’s what “majority of their income” means. Or at least that’s what I intended it to mean!

                              2. 2

                                You also have the fact they are based in the USA, that means following American laws. Regarding personal datas, they are not very protective about them and even less if you are not an American citizen.

                                Moreover, they are testing in nightly to use Cloudfare DNS as DNS resolver even if the operating system configure an other. A DNS knows all domaine name resolution you did, that means it know which websiste you visit. You should be able to disable it in about:config but in making this way and not in the Firefox preferences menu, it is clear indication to make it not easily done.

                                You can also add the fact it is not easy to self host datas stored in your browser. I am not sure they are not sold when there first financial support is Google which have based is revenue from datas?

                                1. 3

                                  Mozilla does not have your personal data. Whatever they have for sync is encrypted in such a way that it cannot be tied to an account or decrypted.

                                  1. 1

                                    They have my sync data, sync data is personal data so they have my personal data. How do they encrypt it? Do you have any link about how they manage it? In which country is it stored? What is the law about it?

                                    1. 4

                                      Mozilla has your encrypted sync data. They do not have the key to decrypt that data. Your key never leaves your computer. All data is encrypted and decrypted locally in Firefox with a key that only you have.

                                      Your data is encrypted with very strong crypto and the encryption key is derived from your password with a very strong key derivation algorithm. All locally.

                                      The encrypted data is copied to and from Mozilla’s servers. The servers are dumb and do not actually know or do crypto. They just store blobs. The servers are in the USA and on AWS.

                                      The worst that can happen is that Mozilla has to hand over data to some three letter organization, which can then run their supercomputer for a 1000 years to brute force the decryption of your data. Firefox Sync is designed with this scenario in mind.

                                      This of course assuming that your password is not ‘hunter2’.

                                      It is starting to sound like you went through this effort because you don’t trust Mozilla with your data. That is totally fair, but I think that if you had understood the architecture a bit better, you may actually not have decided to self host. This is all put together really well, and with privacy and data breaches in mind. IMO there is very little reason to self host.

                                      1. 1

                                        “The worst that can happen is that Mozilla has to hand over data to some three letter organization, which can then run their supercomputer for a 1000 years to brute force the decryption of your data. Firefox Sync is designed with this scenario in mind.”

                                        That’s not the worst by far. The Core Secrets leak indicated they were compeling via FBI suppliers to put in backdoors. So, they’d either pay/force a developer to insert a weakness that looks accidental, push malware in during an update, or (most likely) just use a browser sploit on the target.

                                        1. 1

                                          In all of those cases, it’s game over for your browser data regardless of whether you use Firefox Sync, Mozilla-hosted or otherwise.

                                          1. 1

                                            That’s true! Unless they rewrite it all in Rust with overflow checking on. And in a form that an info-flow analyzer can check for leaks. ;)

                                        2. 1

                                          As you said, it’s totally fair to not trust Mozilla with data. As part of that, it should always be possible/supported to “self-host”, as a means to keep that as an option. Enough said to that point.

                                          As to “understanding the architecture”, it also comes with appreciating the business practices, ethics, and means to work to the privacy laws of a given jurisdiction. This isn’t being conveyed well by any of the major players, so with the minor ones having to cater to those “big guys”, it’s no surprise that mistrust would be present here.

                                        3. 2

                                          How do they encrypt it?

                                          On the client, of course. (Even Chrome does this the same way.) Firefox is open source, you can find out yourself how exactly everything is done. I found this keys module, if you really care, you can find where the encrypt operation is invoked and what data is there, etc.

                                          1. 2

                                            You don’t have to give it to them. Firefox sync is totally optional, I for one don’t use it.

                                            Country: almost certainly the USA. Encryption: looks like this is what they use: https://wiki.mozilla.org/Labs/Weave/Developer/Crypto

                                        4. 2

                                          The move to Clouflare as dns over https is annoying enough to make me consider other browsers.

                                          You can also add the fact it is not easy to self host datas stored in your browser. I am not sure they are not sold when there first financial support is Google which have based is revenue from datas?

                                          Please, no FUD. :)

                                          1. 3

                                            move to Clouflare

                                            It’s an experiment, not a permanent “move”. Right now you can manually set your own resolver and enable-disable DoH in about:config.

                                      2. 1

                                        Is it possible to re-implement the Sync APIs [1], and build your own monolitic based Account Server instead?

                                        [1] https://mozilla-services.readthedocs.io/en/latest/

                                        1. 1

                                          The answer to “is it possible” is almost certainly “yes”, but whether it’s feasible is a different story. If, for example, the API docs are vague or incorrect, then you may be signing up for a great deal of reverse-engineering work. If the API docs lag behind an implementation change required for a new browser feature, you may be stuck playing catch-up.

                                          Then you have to deal with the fact that it’s a pretty darn niche product with a fairly small pool of potential open-source contributors.

                                          All that said, I think it’s a worthwhile thing to do.