1. 6

    Mostly the same old same old, but I do have to agree the inability to reflect on non-public fields can lead to some pretty stupid serialization code.

    1. 3

      you can reflect on unexported fields, but it’s generally a bad idea. The json package doesn’t do it as a design choice for the json package, but it’s not a limitation of the language in general.

      Here’s an example of reflecting on unexported struct fields, which I do not recommend doing for anything other than debug logging: https://play.golang.org/p/swNXd266OVL

      If the json marshaler serialized unexported fields, then any consumers of your library that serialized your data would see their serialized data change when you’ve made changes to unexported fields. That breaks the design philosophy of unexported vs exported identifiers: you pledge to those that import your code that exported identifiers will continue to exist and be supported, and you retain for yourself the right to alter any unexported identifiers as you see fit, without having to coordinate with people who import your code. The json marshaler serializing unexported fields would break the second principle.

      In fact I had a struct with 74 private fields that I wanted to serialize with json and was forced to make all fields public and update all uses throughout the large app.

      Look I’m gonna go out on a limb here and guess that a struct type with 74 fields is not a particularly great design to begin with, or that such code would be easy to work with in any language.

      1. 1

        The consumers of my library don’t serialize its data; they ask it to serialize its own data in a few well-specified ways. Is it actually an expected contract that I can take any arbitrary struct returned from any library, serialize/deserialize it myself, and expect it to work? That seems…a bit too much to expect.

        Conflating code visibility with serialization visibility seems like a non sequitur to me. If I have a library function func (*thing) GenerateJSON() string that’s supposed to generate the JSON for a thing, and that function uses a struct whose only purpose is to drive the JSON library to generate the right JSON, there’s no Go code that needs source-level visibility into that struct. So it just seems goofy that I have to mark those fields “public”.

        1. 1

          Is it actually an expected contract that I can take any arbitrary struct returned from any library, serialize/deserialize it myself, and expect it to work? That seems…a bit too much to expect.

          are you suggesting that json.Marshal, when called from package A, should be incapable of serializing a value of a type defined in package B?

          Conflating code visibility with serialization visibility seems like a non sequitur to me.

          If package A exports A.Thing, and package B imports package A and uses an A.Thing and wants to serialize that to json for whatever reason (let’s say package B defines a type that has a field of type A.Thing in an API response, for example), the author of package A should be able to rename the unexported fields of A.Thing without the author of package B having the output of their program changed. If you want to change the json name of a field you can do so with a struct tag, and if you want to mark an exported field as being hidden from the json, you can also do that with a struct tag. The behavior of json.Marshal is such that a struct defined with no json tags at all will serialize all of its exported fields and none of its unexported fields. The default has to be something, and that seems like a reasonable default. I think it would be more confusing if unexported fields were serialized by default.

          If I have a library function func (*thing) GenerateJSON() string that’s supposed to generate the JSON for a thing

          I’m not sure where you’re going with this. encoding/json defines the json.Marshaler interface for this purpose. If the default doesn’t suit the output that you want, you can readily change the output to be any arbitrary output you want by satisfying the json.Marshaler interface.

          please stop saying public. They’re exported or unexported identifiers. The exported/unexported concept in Go is different than the public/private concept in Java/C++/etc, and the naming reflects that.

          1. 1

            First, to get a terminology problem out of the way, s/public/exported/g in what I said.

            The problem is not with a “default”. It would be fine if the default was to ignore unexported fields. As far as I can tell, it’s impossible to serialize unexported fields using the json package. You just get “struct field xxx has json tag but is not exported”. [0]

            So when marshaling/unmarshaling is not simply a matter of copying exported fields, to use json internally you end up making another unexported struct with exported fields and copying data into that, or some such workaround.

            [0] https://play.golang.org/p/FHvX6x8_61k

            1. 1

              and if you’re serializing something to json, you’re doing so … why? To make the data available externally. The very act of serializing the data to json is an act of exporting; to make it available beyond the confines of the current binary, perhaps to other programs written by yourself, things written by other people, or to even future versions of the same application. That’s the fundamental theory of what exported fields are: things that are a part of public APIs.

              You either have an exported or an unexported struct type. If it’s an unexported struct type, what difference does it make whether the fields are exported or not? If it’s an exported struct type, why would you want to serialize an unexported field? It’s not going to show up in docs for your package, your consumers will be totally confused as to where it comes from, and again, once it gets serialized, you’re no longer free to make unannounced changes to that field.

              This really seems like grasping at straws. I have literally never found this to actually impede my work in seven years of using Go.

              1. 2

                This is kind of getting into the weeds so it should probably move to some esoteric mailing list. :)

                My library writes JSON that only it reads. The JSON itself isn’t a public API; its contents are not documented. The fact my code wants to reveal fields to itself through JSON doesn’t mean it wants to reveal those fields to other Go code. I’m totally free to make changes to the JSON and the code, as long as they’re backward/forward compatible to my code.

                “Exported” does not have an identical meaning in code and in serialization. Consider that I can serialize a gzip.Writer into JSON, because it has some exported fields. I’m pretty sure that is purely accidental as far as the authors of the gzip package are concerned, and obviously if you unmarshal that JSON you don’t get a working gzip.Writer. I don’t think what they meant by exporting the Header field was “please serialize this”; they just meant it’s OK to use it directly in your code at runtime.

                Anyway, I didn’t say this was a deal-breaker, just that it’s goofy. An unexported struct type with exported fields — to me the only non-goofy reason for that would be when the struct is implementing an interface with those fields in it. Instead, I’m doing it because otherwise it would be arbitrarily restricted from being serialized in the most convenient way, that is, by using the json package.

      2. 1

        Just in case you want to upset anyone who looks at your code, you can use https://github.com/zeebo/sudo to remove the read-only restriction on a reflect value.

      1. 13

        I’m currently hiring and if I found that a candidate had written this, I would decline to interview them.

        1. 1

          Are you hiring remote first or in the Dallas area, by any chance?

        1. 7

          you won’t be at the table when the victim-studies majors are defining “hate”

          good grief.

          1. 2

            What’s wrong with this claim? There is absolutely a political faction, characterized in part by association with university soft sciences and humanities subjects, that seeks to stigmatize lots of categories of speech and behavior as socially unacceptable because they think they constitute a type of hatred. I think it’s fair to classify opposition to gun ownership and promotion of the idea that ordinary social institutions should avoid doing business with institutions that sell guns is a policy that this faction generally supports.

            1. 5

              the victim-studies majors

              come on man, that’s not a degree, that’s just blatantly siding with abusers. ESR is a known abuse-apologist who consistently sides with people that actively do harm to others.

              1. 2

                Calling someone a harmful abuser can easily itself be a form of abuse, and the political faction ESR alludes to is fond of claiming that political dissent against them constitutes an extra-political form of abuse which can legitimately be countered with extra-political violence. I don’t believe the people who imply that the political positions ESR has defended constitute actual abuse.

          1. 2

            I’m Jordan Orelli and I work at Etsy, on the Core Platform team. I’m pretty new to the team, though. A handful of other Etsy engineers are around here lurking.

            1. 1

              chugging along on the Udacity Interactive 3D Graphics Programming course. I’m about halfway through; it’s a really great course so far.