1. 20

    1. 5
      1. 5

        This is a good observation of yourn:

        Allowing programs to read/write configuration files [in a comment-preserving manner] allows for automated cleanup/manipulation

        The vast majority (<- educated guess) of config files is currently off-limits for all but the simplest (find-and-replace/sed) automatic manipulation because they contain, or might contain, comments that should not be discarded. I’m only rephrasing your point, here, but I want to highlight it. Having comment-preserving parsing available for every config file format would be a huge win. It would also, I think, be necessary for [angersock’s dream](#c_pz2dj0) of one config format for all programs with simple config formats. Or, I guess, not necessary if the one format is to have no comments, like JSON. Heaven forfend.

        Also, one Nei commented on your blog about irssi’s tiny comment-preserving config rewriting library, where I saw this reminder of how recent both DVCS and Internet at home are. Timo Sirainen committed on Apr 14 2000: “Sorry for a big update - I still don’t have internet connection at home”

        1. 3

          I ran into this recently not with comments but with ordering. I was trying to read a YAML file, modify a key, then write the file back out. The file is typically edited manually by humans (and this is intentional). The YAML library would alphabetize the keys when it wrote the file so everything would end up in the wrong order and I would have ended up with huge, bizarre diffs in the commit log, as manual changes since the last automated change were reordered, if I’d continued down that path. Very annoying.

      2. 2

        One possible lesson is follow the git model of using the CLI to interact with the configuration file. Or do what Gerrit does and use git config to write to your config file!

        Sure, I can dive into .gitconfig if I need to (it looks TOML-ish, is it?), but I haven’t had the need since I had to workaround weirdness on Windows invoking a GUI diffing tool (many years ago).

        On a related note, I’m actually playing with textql as a front-end to a nested directory structure of config files (in JSON, ironically enough) so that I can batch update/query across the files.

        1. 1

          I can imagine more possible CLIs for a particular config format than I can imagine config formats. I don’t think having a CLI that abstracts as little as git config helps with any problem here.

        2. 0

          See, if we wanted to make the world a better place, everyone would pick a month to submit patch requests to all open source projects to replace their config files with JSON and to add a converter for legacy configs.

          And yes, I also want a pony. And world peace.

          1. 30

            Wait, making every configuration file with JSON would make the world a better place? I think it would turn it far worse! A lack of comments in my configuration file makes future me confused and frustrated at past me.

            1. 5

              When I’m stuck with JSON-for-config, I’ll sometimes duplicate keys in the file - putting the comment in the first one. Every JSON decoder I’ve used so far ignores the first value and takes the second one.

              I’m well aware of how insane that sounds, but I’d rather have comments than sensible files.

              1. 3

                I do something similar but make it an _{{key}}-comment (e.g. password has a sibling _password-comment before it). This works great in config files as it can be ignored by the application. It does fall apart in something like package dependencies though =(

                1. 3

                  I do something similar, except I duplicate a specific key, so order is not important:

                    "//": "Here's a comment",
                    "//": "Another comment"

                  But yeah, I’d rather just avoid JSON for such things.

                  1. 1

                    Some of these problems are addressed by JSON5: http://json5.org/ There is obviously the whole https://xkcd.com/927/ problem, but IMO it’s not as bad as it is with something dramatically different like TOML.

                    But in general I think it’s much better for a whole host of reasons to use Lua for configuring end-user applications.

                    1. 1


                      Kind of reminds me of UCL.

                  2. 2

                    One problem with this, and with @alva’s solution below, is that the comments won’t survive deserialization/serialization in a meaningful way. @twolfson’s solution resolves that, though there’s still no guarantee the comment key is serialized anywhere near the key it is meant to be commenting.

                  3. 1

                    Having all configuration files in the same format would make the world a better place.

                    JSON does have some technical issues though, such as inability to represent all floating point values.

                    1. -1

                      Is the world really betterr off having different formats for grub, xorg, Apache, nginx, rust, npm, and all the other myriad ways of storing the same flavor of data?

                      1. 7

                        No, but there are enough standards for this that don’t suck as much for configuration data as json.

                        1. 1

                          yeah, all three of the systems suggested in the OP are better than json. i believe yaml has some parsing issues, but the other two look very clean and usable.

                          1. -1

                            Other than the common “muh comments” complaint, why do you think JSON sucks for configuration data?

                            1. 12

                              Two reasons that come to mind for me are: you get a lot of syntactic ceremony that other formats don’t require (every file starting with ‘{’, string quoting, commas between list items, etc.) and the selection of types is odd. Your JSON parser is going to convert bare numbers to numbers but you have to quote your strings. What do you do if you have other types, like URLs or date/times or something particular to your application? You’ll have to quote them as strings and then do a second pass over your JSON configuration object to convert it to something else. What happens if your language has a more interesting suite of numeric types than JSON does? Do you have to tell people to quote their numbers so that you can parse them properly? The behavior is hidden from you by your JSON parser, so you aren’t likely to be able to detect when something like this has gone wrong.

                              I like my code to require a certain amount of ceremony to catch problems before running, but I like my configuration files to be fairly lenient; if I can recover something from them, I can alert the user that the parse failed on these items or whatever and proceed. These options get narrowed when you conflate a programming language with a configuration language.

                              In fact, even my JSON parser, I want it to be strict when I’m dealing with user input or form submissions or whatever, but if I’m reading a file like ~/.foorc, I want to be lenient. Does your JSON parser have options for that?

                          2. 5

                            Just because there is not one standard it doesn’t mean that JSON is a suitable one. In fact I would even prefer XML over JSON for this particular usecase, just because it has comments.

                        2. 7

                          JSON config files?! Do you have a shrine to Satan in your house as well?

                          1. 1

                            I might not mind if most json parsers sucked totally when it comes to error messages. rather i would prefer a good config parser which generates error messages and documentation WITH EXAMPLES.