1. 7

    One wonders why not just, say, use JSON for this.

    1. 24

      I believe the prefs format predates JSON (or at least JSON’s popularity), and changing it now is a non-starter as it would break everyone’s user preferences. Even this changeset whose only backwards incompatible changes were fixing some glaringly obvious bugs caused some reports of failing CI around the web.

      We could try to migrate prefs files to a new format, but that would be a high risk/low reward operation.

      1. 5

        We could try to migrate prefs files to a new format, but that would be a high risk/low reward operation.

        I wish you folks would do that. :(

        1. 19

          Would you volunteer to respond to all the breakage reports it might cause?

          I may sound bitter now, but when a maintainer says something is too hard/risky, and a random user replies with “yeah, you should do it anyway” disregarding who is it that’s going to deal with problems, it’s just utterly disrespectful.

          1. 17

            Respect doesn’t enter into it–hell, I even agree with the assessment that the work is high-risk/low-reward…then again, I feel the proposed fix has some of the same issues. Like, the parser has worked well-enough that refactoring is maybe not a good use of time.

            If the decision is made “We must refactor it”, then it makes sense to go one step further and fix the underlying file format anyways. Then again, Mozilla has a history of derping on file formats.

            As for “all of the breakage reports it might cause”, given that the docs themselves discourage direct editing of files, it would seem that there probably isn’t a huge amount of breakage to be concerned about. Further, if the folks are clever enough to write a neat parser for the existing format, I’m quite sure they’re clever enough to write a tool that can correctly convert legacy config files into a new thing.

            (And again, it’s common advice that there are no user-servicable parts inside good chunks of it, because it’s a derpy file format.)

            Like, just to hammer this home, here is the format of a prefs.js file:

            # Mozilla User Preferences
            /* Do not edit this file.
             * If you make changes to this file while the application is running,
             * the changes will be overwritten when the application exits.
             * To make a manual change to preferences, you can visit the URL about:config
            user_pref("accessibility.typeaheadfind.flashBar", 0);
            user_pref("app.update.lastUpdateTime.addon-background-update-timer", 1520626265);
            user_pref("app.update.lastUpdateTime.blocklist-background-update-timer", 1520626385);
            user_pref("app.update.lastUpdateTime.browser-cleanup-thumbnails", 1520640065);
            user_pref("app.update.lastUpdateTime.experiments-update-timer", 1520626145);
            user_pref("app.update.lastUpdateTime.recipe-client-addon-run", 1520626025);
            user_pref("app.update.lastUpdateTime.search-engine-update-timer", 1520625785);
            user_pref("app.update.lastUpdateTime.telemetry_modules_ping", 1520625905);
            user_pref("app.update.lastUpdateTime.xpi-signature-verification", 1520626505);

            There is no reason that this shouldn’t be in a sane file format (read: JSON). This could be accomplished with a conversion tool, and gracefully deprecated.


            It even already contains JSON!

            user_pref("browser.onboarding.tour.onboarding-tour-performance.completed", true);
            user_pref("browser.pageActions.persistedActions", "{\"version\":1,\"ids\":[\"bookmark\",\"bookmarkSeparator\",\"copyURL\",\"emailLink\",\"sendToDevice\",\"pocket\",\"screenshots\"],\"idsInUrlbar\":[\"pocket\",\"bookmark\"]}");
            user_pref("browser.pagethumbnails.storage_version", 3);
            1. 7

              No disrespect taken :)

              For the record, I agree a standard format would be better. Also for the record I’ve never even looked at the prefs code before, so my statement was coming more from experience knowing how much the tiniest changes can blow up on the scale of the web.

              You never know, maybe we’ll support JSON and the legacy format at some point, but that smells like it might be unnecessary complexity to me.

              1. 2

                You said unnecessary complexity. Normally, I’d either say that’s a good thing or suggest a simple subset if it’s something like JSON. If Firefox already supports JSON, wouldn’t there already be a component included that could be called to handle it? Is that inaccessible? Or does it suck so much it’s worth rolling and including ones’ own parser that’s not a cleaned-up, subset of JSON? Just curious given Firefox is an older, big project.

                1. 5

                  The pref parser is a small isolated module, so I don’t think it would be technically difficult to implement (bear in mind I’m not familiar with it at all).

                  The complexity I’m referring to was more around ux, maintenance, and support, that come with providing two different ways of doing the same thing.

            2. 2

              ““yeah, you should do it anyway” disregarding who is it that’s going to deal with problems, it’s just utterly disrespectful.”

              Bringing up respect and morals when a FOSS project uses non-standard formats instead of standard ones that already existed with tooling people could’ve used? And that definitely would need extra work or fixes later? I doubt they were thinking of morality when they did it. More like “Let’s implement this feature the way I feel like doing it with my preferences and constraints right now.” Kind of a similar mindset to many people asking them for changes.

              A better question would be, “Is replacing non-standard stuff in the browser with well-supported, standardized stuff worth the effort to fix the breakage?” In this case, I’m not sure without knowing more specifics. The general answer for file formats is “Yes wherever possible for interoperability and ecosystem benefits.”

              1. 6

                non-standard formats instead of standard ones that already existed with tooling people could’ve used

                That’s untrue, the grandparent comment mentions this probably predates JSON’s popularity.

                Edit: Yeah, the bug itself is 17 years old, and the prefs format is probably older. Wikipedia says “Douglas Crockford originally specified the JSON format in the early 2000s;”, which means that at best the prefs format came around the same time Crockford first specified it, and at worst it probably came into being a couple eyears earlier.

                1. 1

                  Good thinking on the history. I did say “standard formats,” not JSON. Before JSON, the formats I used included LISP-style sexprs for easy parsing, Sun’s XDR, ASN.1, and XML. I also hoped simpler ones gaining popularity would lead to secure or verified implementations. That was effortless for LISP-based syntax with Galois doing a verified ASN.1 later. Most went with the overcomplicated formats or hand-rolled their own each with problems. For XML, I found I could just use a subset of it close to basic HTML tags that made it easier for someone to convert later with standard or customer tooling.

                  So, those were among alternative approaches back in those days that many projects were taking. Except LISP syntax which only LISPers were using. ;)

        2. 3

          Or Toml, since that’s Rust’s go-to data markup language.

          1. 4

            That’d be just a little too cute.

        1. 2

          The “Webmaster” at the bottom of the page is so wonderfully quaint. :)

          1. 7

            A E S T H E T I C

            Any chance you could get lynx looking at lobsters? :3

            1. 9

              Sure thing!

              https://u.teknik.io/bfG9p.jpg https://u.teknik.io/rAyqc.jpg

              Is it everything you were hoping for?

              1. 2

                Oh man, I was hoping for a screenshot of this thread!

              2. 2

                Casually shilling for a possibly relevant project

              1. 4

                Guy runs into two bugs involving stale data and concludes every app must have a refresh button? That seems a bit presumptuous.

                1. 18

                  That’s kind of reductive. The message I got from the post was “please provide users this simple and well-known way to work around synchronization bugs“. It’s usually easy to implement and has no drawbacks save for a tiny bit of UI clutter.

                  1. 1

                    Okay, I see your point. I’m partial to the pull-to-refresh affordance since it stays out of the way the 95% of time you don’t need it. In apps that aren’t list-based, however, I don’t know what you’d do (shake-to-refresh?).

                    I think I was irked by “every app that doesn’t have this is broken” and the assumption that the web-based paradigm of throwing everything out and getting a fresh copy is the best one for every app. In some apps, it’d be the equivalent of an elevator door close button, where hitting it doesn’t really do anything it isn’t already doing.

                    I also think the real issue is how phones react to (or don’t) changes in network connectivity.

                  2. 8

                    The author uses two examples to illustrate the fact that every application which synchronizes can get desynchronized (seriously, every single one) and providing users with a workaround is essential.

                    1. 5

                      Going to have to agree with OP. Sometimes a refresh button doubles as an invaluable debugging tool.

                      1. 4

                        Does it? It seems, at best, like a way to work around bugs by destroying the state you probably need to debug the problem.

                        1. 9

                          As a user, what does destroying debug data matter to me? I’m not trying to debug your app, I’m trying to call a taxi. Further, how does omitting the refresh trigger help debugging? Most people don’t file bug reports, they just get annoyed, and denying them the tools to fix their problem will just make them more annoyed.

                          If you want debug information, I’d argue a refresh button could be actively helpful: users press it when they think something’s stale, so by collecting telemetry on refresh presses you can get a clue as to when desynchronizations happen.

                          1. 2

                            I’m not asserting that you shouldn’t have one, I just don’t think it’s a debugging tool.

                          2. 1

                            Provide a refresh button in release, but not debug so that developers can’t use it to ignore real bugs :)

                            1. 1

                              Just because I have a hammer doesn’t mean I just go around hitting everything like it’s a nail…

                          3. 1

                            Even Gmail on Android allows the user to manually refresh despite it being timely, correct, and not error prone in 2017. I give some odds that the refresh action doesn’t do anything except address the user experience.

                            Not defending buggy apps, but there is a take-away here.

                          1. 2

                            Can it even be called a proper contest when there were only two participants and four submissions in total?

                            1. 22

                              Yes, it can. It was open and advertised for multiple months and this were the submissions we got.

                              Everything starts small, and the resulting submissions are interesting.

                              1. 2

                                Fair enough. The name of the contest made me expect more submissions for some reason.

                                1. 14

                                  Well, you have to pick the name before the submissions. :) We’d have preferred more, but there will definitely be a re-run, given that the submissions show an interesting gamut of things you have to watch out for Rust:

                                  • Soundness holes: One of them uses a (not yet fixed) soundness hole in borrow checking
                                  • API behaviour: Even if it’s not broken, it can be surprising. No one can keep all exact behaviours in their head all the time. One uses the behaviour of Path.join, which is similar to Pythons
                                  • Environment: One uses the fact that deploy environments may have a copy of the crates.io index.

                                  Still, we’re quite happy that no one found ways to hide something that are absolutely specific to Rust. Our running theory is that Rusts aversion of implicit behaviour helps a lot there.

                                  Now, this is in no way a proof of this theory: the number of people that know every corner of the language is smaller for Rust then for C, so maybe there’s stuff that just wasn’t found yet.

                                  1. 4

                                    This bug? It’s still open.


                                    1. 3

                                      Correct, fixed the description. I’m currently gearing up for running RustFest, so I wrote it down from memory.

                                      I mixed it with this one, which is fixed: https://github.com/rust-lang/rust/issues/41622

                                      Thanks for the critical reading :)

                                      1. 3

                                        Any time. :) wasn’t sure if I’d missed something.

                            1. 15

                              Why would you want to use uninitialized memory for entropy when arc4random exists? To me it seems entirely pointless to use undefined behavior for acquiring low-quality entropy when the system provides a much better source already.

                              1. 2

                                The implication is that on some BSD and open ssl implementations, they use unitialized memory in the arc4random code.

                                In any event, to me, the question is why would you want to break working code?

                                1. 10

                                  The standards committee would argue that the code has always been broken, it’s just “worked” because compilers haven’t been as aggressive as they could be.

                                  From a compiler perspective, it’s hard to distinguish between “this should obviously work” and undefined code that can be eliminated for a legitimate performance win. In this example, an expression optimizer that previously deleted all effects with indeterminate args now needs to prove that the expression doesn’t reduce to a constant. The alternative would be to remove the indeterminacy check entirely, which would be a non-starter for teams with a “no performance regressions” policy. Remember that this sort of thing doesn’t exist just to break people’s “obvious” code: you can save on code size by removing impossible branches of a function after inlining, for example.

                                  Also, if this were to be “fixed” in the standard, what would the fix be? You could say that an expression is determinate if, for all possible values for the indeterminate inputs, the output is the same, but this raises questions about value range (e.g. if int a = indeterminate & 1;, what can you say about a?) You could replace the concept of “indeterminate” with value ranges where an uninitialized value has the widest possible range, but then when can you eliminate formerly-indeterminate effects? Is having a range greater than a single value enough? And remember, both of these increase the burden on compiler writers considerably, and if this were introduced in a new standards version several compilers would need to disable a lot of code until it could be properly fixed.

                                  1. -2

                                    That would be a false argument. Note that the claim is that the contents of the variable are “implicitly” UB. The standard did not explicitly rule this read to be undefined - this is an expansion of UB. In fact it is an expansion of UB into territory explicitly exempted from UB; the exception for character data were intended to cushion the ill effects of previous UB edicts. More importantly, I’d really like to see an example of a non-trivial optimization - something that actually made working code run faster or take up less space using this approach. I can see how many micro-“optimizations” are possible but what are you optimizing when you claim the programmer intention is unknowable?

                                    “(e.g. if int a = indeterminate & 1;, what can you say about a?)” - that it is either 0 or 1. Indeterminate is an implicit I/O like operation. There is nothing you can do to eliminate indeterminate effects and there is no need to do so. Compiler writers are solving the drunkards lampost problem: it’s easier to do “optimizations” if you knew or could assume something about these values, so let’s assume that we do or can.

                                    C programmers would be much better served by compilers that did better static analysis than by compilers that produce unexpected program failure that is labeled “optimization”.’

                                    “ And remember, both of these increase the burden on compiler writers considerably, and if this were introduced in a new standards version several compilers would need to disable a lot of code until it could be properly fixed.”

                                    Far better than to impose silent failures on working code in e.g. widely used crypto applications.

                                    1. 15

                                      Far better than to impose silent failures on working code in e.g. widely used crypto applications.

                                      If you’re talking about seeding an RNG from stack “garbage” that approach was always crap. It never worked. The garbage on the stack is always the same.

                                  2. 9

                                    Because that code relies on explicitly undefined behavior.

                                1. 10

                                  So I guess this security researcher is gonna get hit by the mob, nice job journalists.

                                  1. 3

                                    To be fair it’s not as if it was the only security researcher the mob knows about. Why would the mob hit someone who just happened to shut down their malware by chance? There’s many researchers that can be easily found and that are known to actively work and track on malware families. There’s even some that are known to shut down botnet from conference…

                                    Don’t get me wrong, the press doxxing people is stupid, but saying the guy is now in danger because of this is rather naive.

                                    1. 8

                                      He may not be in immediate danger, but his doxx are irreversibly out there. Nothing says he won’t become more interesting to dangerous people in the future.

                                      1. 6

                                        Exactly, and the fact is we simply don’t know if he’s in danger right now. He really might be, and if he does get a nasty visit, then it really would be on the journalists, who outed him purely for their own ends. Totally irresponsible and selfish. Sure, if they found him then sufficiently motivated bad actors could too, but the point is, the journalists did it for them, for free.

                                        I saw a piece earlier which just said “he lives in Cornwall and works for a security company” and thought even that was irresponsible, given the extent of the potential nastiness of people who carry out organised crimes. But putting the guy’s name out there? Really? Out of control.

                                        1. 2

                                          The data was already irreversibly out there. Someone just brought it together. Do you believe that “the mob” doesn’t have the resources to do the same if it wants to? Only, probably in a way that would give him less warning.

                                          1. 3

                                            That’s addressed in the piece - the researcher says he always assumed it would be some criminal who tracked him down. But instead, it was a journalist who got there first. Probably because they’re specifically trained to do this kind of investigation, don’t you think?

                                      2. 2

                                        I don’t think it’s legitimate to blame the journalist for other people’s criminal actions.

                                        If the mob cares enough to harm this guy, then they were certainly looking for him already, and if the journalist could find him, then so could the mob.

                                        Second, the logical extreme of your position is that everybody should censor themselves because information may be used by criminals, which I don’t agree with.

                                      1. 12

                                        The problem of handling many keys for different hosts/security domains can be solved elegantly in your ~/.ssh/config:

                                        IdentitiesOnly yes
                                        Host aur.archlinux.org
                                        IdentityFile ~/.ssh/id_ed25519_aur
                                        User aur
                                        Host example.com
                                        IdentityFile ~/.ssh/id_ed25519_example.com
                                        User exampleuser

                                        In conjunction with IdentitiesOnly, IdentityFile will tell ssh to:

                                        […] only use the authentication identity and certificate files explicitly configured in the ssh_config files or passed on the ssh(1) command-line, even if ssh-agent(1) or a PKCS11Provider offers more identities.

                                        As seen in ssh_config(5).

                                        1. 0

                                          The PineA64 is a travesty of computing.

                                          1. 7

                                            Could you please elaborate? A blanket statement like that doesn’t contribute anything and doesn’t help people that are interested in the product to make an informed purchase.

                                            1. 1

                                              While I understand the appeal of having a fully-OSS design that you can contribute to and audit, you’d get more bang for your buck by just dropping $100 on eBay for a Thinkpad. But I agree that the parent comment is kinda useless.

                                              1. 3

                                                Apologies if my previous comment relied upon people knowing about the fiasco surrounding the board this is based on.

                                                Brian Benchoff’s un-review covers the pine64 experience in far more detail than I could here.

                                          1. 2

                                            Is there any advantage to using this over a simple reverse SSH tunnel?