The original default was internal, which is the Kotlin-specific modifier scoped to the current compilation unit. That is, it is not Java’s “package private”, which is a fairly useless thing. Kotlin’s internal made much more sense, but, as Kotlin has to be very interoperable with Java, making it default choice would’ve been wrong. So, on the road to 1.0 default was flipped to public.
That said, I really think that Kotlin is onto something here. Making the public API friendly to read and the private bits have local reminders IMO works very well.
The goal is good, but I am not sure default public accomplishes this. greppability is part of readability, and rg 'pub fn' is certainly more intuitive than alternative. Not sure how interface files change the calculus here though. They are a superior substitute for grepping for pub, and having pub in interface files would be redundant, and having different defaults for interface files would be confusing.
It may be Stockholm syndrome but I like the C++ syntax (aside from the struct vs class inconsistency, which is obv for C compatibility.) I appreciate not having to type “public” or “private” over and over, and when reading an interface it keeps the public API segregated.
Go’s mechanism is awful. It’s a clever way of avoiding the need for keywords, but it means that changing access control requires a renaming/refactoring, which is a big pain in an editor without language support and often makes minor changes to a lot of source files, leading to merge conflicts. (Plus, Go has no ability to make things private to a struct or to a source file, so packages easily become grope-fests where everything’s in everything’s business.)
Why yes, I have recently been doing refactoring of Go packages…
+1 on names dictating visibility being awful. It’s a pain in python, too (although visibility is more of a suggestion there).
I think the current iteration of Rust’s visibility system is the sweet spot. Everything defaults to private. To make it public, add pub. To make it crate-public, add pub(crate). And unlike Haskell, you don’t have to duplicate the name to declare it public at the top of the module.
Rust’s visibility rules are significantly better than average, but I don’t think they are even a local optimum. There are at least two improvements to be made:
pub(crate) should be a single, short keyword.
pub should strictly mean “visible outside of the crate”, it currently also plays dual role of “visible in parent”. That is, public-in-private should become deny-by-default
These two issues are more-or-less backwards compatibility sacrifices due to the fact that Rust 1.0 didn’t get visibility exactly right.
I agree. I kinda hate bikeshedding on the syntax but parameterizing pub really hurts usability. It makes it harder to read, and also makes crate level visibility feel like a second class citizen. It doesn’t feel like it’s ever a first-considered option to me even though it’s arguably the most common visibility that I need.
For a new language I like the idea of there not being a raw fn equivalent. I like the cleanliness of having fn_pub, fn_mod, fn_prv keywords that are all top level. My worry is that visually scanning would be hindered, but maybe syntax highlighting is enough to alleviate that?
First, I often end up having to scroll up a moderately long way (especially in classes that contain a lot of doc comments, which good codebases do) to find the access specifier.
Second, I either end with annoying diffs if I want to change an access specifier. I either move a method from one group to another (don’t do this if it’s virtual, that’s an ABI break) or I have to write an access specifier before and after the method.
often makes minor changes to a lot of source files, leading to merge conflicts.
Assuming that private declarations are scoped to a single file, that should never be the case:
If you’re making a private name public, all existing uses of the name should be in only that file since it’s currently private.
If you’re making a public name private, all existing uses of the name most be in only that file because otherwise you can’t make it private since it’s being used externally.
(I work on Dart, which has like Go uses identifier naming for privacy. I refactor all the time and while it is somewhat annoying that it means adjusting the name of every identifier, it’s never a sweeping change because of the above observation.)
The problem with C++ is that a class’s implementation may be spread between many files. The declaration must be in a single place, but method bodies can then be anywhere (in contrast to Objective-C, which puts them all in a block).
This is made worse by the C++ name resolution rules, which often mean that you need to put the definition of a method for one class after the declaration of another, so individual methods of a class may end up being defined across a mix of headers.
If you’re making a private name public, all existing uses of the name should be in only that file since it’s currently private.
Not in Go. That is exactly my complaint. “private” names are accessible within the entire package, which can be any number of source files. (The project I work on is overdue for refactoring and has a package with at least 50 source files in it, and I know a couple structs in it that have sensitive fields that shouldn’t be groped from outside that are being groped.)
changing access control requires a renaming/refactoring, which is a big pain in an editor without language support
Which editors don’t have language support for this kind of thing? (And have meaningful usage numbers.)
and often makes minor changes to a lot of source files, leading to merge conflicts
Changing identifier names should never result in merge conflicts, right?
Go has no ability to make things private to a struct or to a source file
Source files only assert scoping for their imports, which is fine. But it’s entirely possible to make a struct field “private” to the package in which it is defined, by lower-casing its name in the struct. Do you mean something else?
Even if your editor can automate this, churning names creates a really unfortunate code review burden that doesn’t need to be there.
I feel like leveraging naming conventions for this is just a bad workaround for a hesitation about the syntax overhead. I would rather find a syntax overhead that I’m confident in paying.
Which editors don’t have language support for this kind of thing?
Most of them, in 2012 when I started using Go. I agree it’s a lot less annoying today, but I think it’s a bad idea to have such a basic task be dependent on fancy editor refactoring support.
Changing identifier names should never result in merge conflicts, right?
Um, of course it can. Were you being sarcastic? I rename “foo” to “Foo”, meanwhile you change “f := foo()” to “x := foo()”. Bang.
Do you mean something else?
Why yes, I meant “make things private to a struct or to a source file” as I said. A field of a struct that can only be accessed by methods of that struct, as supported by a million other languages. Without that you can’t have safe encapsulation, not unless you make that struct the only thing in its package. Or a function/variable that’s accessible only in its source file, like “static” in C++ or “fileprivate” in Swift.
A field of a struct that can only be accessed by methods of that struct, as supported by a million other languages. Without that you can’t have safe encapsulation, not unless you make that struct the only thing in its package.
Ah, so you’re defining “private” accessibility at the type scope, whereas Go defines it at the package scope. Fair.
I am increasingly of the opinion that there are two meaningful levels of visibility:
Visible from within the same package.
Visible from everywhere.
Anyone who is modifying the code for a package can trivially find all callers within the package and can update them. By restricting visibility further, you gain no additional reduction in fragility (you are not leaking implementation details outside of a self contained unit) and you increase the maintenance burden. C++ ends up with a load of friend definitions to deal with this in common use. Tighter restrictions are simply documentation: anyone who is modifying the package can remove them and so is not bound by them. The compiler can typically see a whole module at a time (even in C++ with ThinLTO) and so gains no benefit from tighter access restrictions.
C++ couldn’t do this because it had no module system and no mechanism for defining a module boundary. Carbon, as a language that adds syntactic sugar to C++, is probably stuck, but other languages have no excuse.
I also like the model where package-private fields / methods are prefixed with an underscore, as long at there’s an attribute to say ‘also expose this as a public identifier with this name’. To @snej’s point, I don’t see a benefit in making it easy to make a public identifier private (it’s an API break for consumers), but I do see a benefit in making it easy to introduce things as private symbols and then make them public once they’re mature. That said, if the only consumers of the private symbol are within a package, a global rename should be easy.
The documentation effect of specifying what is supposed to be a private detail of a type seems useful to me unless your modules are always quite small (which could be fine, in Elixir each type is its own module).
At this level, they don’t even have to be visibilities, if you have interface files like .h or .mli.
Though, I’d say some more restricted visibility for protecting “local” abstractions is also nice to have.
If I were doing a programming language which cares about scalable abstractions, I’d:
use api keyword to mean “visible outside of current multi file unit of compilation, has unknowable callers”
use pub to mean “visible throughout CU”
use no keyword to mean “visible in the current file”, for local abstractions.
Additionally, during compilation compiler would’ve collected all api declarations from all files of CU into a single .api file which is also version controlled and used by compilers for separate compilation, by humans to learn the API and to review API changes.
FWIW, I do think these are the most important sets.
But having the ability to control this at the granularity of a type, while maybe less important, still seems valuable. And coming from C++ in particular, lacking that would be pointlessly frustrating.
So for Carbon, this was a fairly easy call – we need to meet C++ code and C++ developers where they are, especially when it is reasonably easy to do so.
This post though was much more about how to structure the defaults and annotations than which granularity. We plan to add the missing granularity to C++ for package-based private.
I absolutely disagree. You cannot reliably protect the invariants of a class if that class has no control over outside code touching its state. It isn’t ok if that code is in the same package; that’s a different and much looser boundary. There are often multiple engineers working on code in the same package, and without explicit privacy controls it’s easy to assume a field in someone else’s class is OK to access directly.
Any engineer who can write code in the same package has the ability to change the access specifier on a class in the same package. An engineer who will access a field directly, ignoring accessors and documentation about invariants will also change an access specifier to allow them to do the same. If code is being committed in the same package as your class without sufficient review then you have problems in your engineering process that no language feature can rescue you from.
I think the default access control level (if you leave the signifier out) correlates tightly with the default unit of compilation and distribution. In some circumstances, the public interface is the thinnest waist and the contract, so you need to think twice before changing anything public whilst it makes sense to remain open within your package so that different types within your package can freely access most innards of your package even though they themselves may never be exposed to the public interface.
The main benefit of C++‘s region approach is conciseness: A single keyword suffices to set the access control for a long list of member declarations. I do think the lack of that is a big part of Java’s verbosity.
The main pain point with it is that it means class declarations aren’t context-free and compositional. If you invoke a macro inside a class declaration, you don’t know what access control the members produced by the macro will have because it depends on where you call the macro. And the macro can’t itself include an access control modifier because it has no way to “pop” it when it’s done and restore the previous access control level. If I really wanted to over engineer this, I would introduce access blocks like:
private {
// Members...
}
Then you could put a bunch of members in a single block to avoid the repetition of Java. Blocks can nest and the innermost one determines the access control of members, letting you compose macros freely. And you can omit the { ... } and apply a single modifier to a single declaration.
But that’s just way too much machinery for such a small (but important!) feature.
For what it’s worth, Dart uses an underscore prefix on identifiers to mark them private and everything else is public. No one seems to love this, but… I have to admit it works fairly well and is pretty terse.
FWIW, I think the local reminder that something is private via a keyword works surprisingly well in practice.
For example, private things are often fields or fairly short-named APIs. The extra keyword is surprisingly less disruptive here, and it seems easy to afford the verbosity there. The public API, where names are longer and people read it over and over again can be minimal and beautiful.
And I think it is worth a keyword and not just a naming convention. It makes it much more explicit and discoverable for readers.
Historical context for Kotlin’s design.
The original default was
internal, which is the Kotlin-specific modifier scoped to the current compilation unit. That is, it is not Java’s “package private”, which is a fairly useless thing. Kotlin’s internal made much more sense, but, as Kotlin has to be very interoperable with Java, making it default choice would’ve been wrong. So, on the road to 1.0 default was flipped to public.Fun to hear some of the history here…
That said, I really think that Kotlin is onto something here. Making the public API friendly to read and the private bits have local reminders IMO works very well.
The goal is good, but I am not sure default public accomplishes this. greppability is part of readability, and
rg 'pub fn'is certainly more intuitive than alternative. Not sure how interface files change the calculus here though. They are a superior substitute for grepping for pub, and having pub in interface files would be redundant, and having different defaults for interface files would be confusing.For greppability:
rg ^\s*fnshould work, even with a mixture in the API file.But generally, I’ve seen much more using grep and tools to find a name, not the public subset.
It may be Stockholm syndrome but I like the C++ syntax (aside from the struct vs class inconsistency, which is obv for C compatibility.) I appreciate not having to type “public” or “private” over and over, and when reading an interface it keeps the public API segregated.
Go’s mechanism is awful. It’s a clever way of avoiding the need for keywords, but it means that changing access control requires a renaming/refactoring, which is a big pain in an editor without language support and often makes minor changes to a lot of source files, leading to merge conflicts. (Plus, Go has no ability to make things private to a struct or to a source file, so packages easily become grope-fests where everything’s in everything’s business.)
Why yes, I have recently been doing refactoring of Go packages…
+1 on names dictating visibility being awful. It’s a pain in python, too (although visibility is more of a suggestion there).
I think the current iteration of Rust’s visibility system is the sweet spot. Everything defaults to private. To make it public, add
pub. To make it crate-public, addpub(crate). And unlike Haskell, you don’t have to duplicate the name to declare it public at the top of the module.Rust’s visibility rules are significantly better than average, but I don’t think they are even a local optimum. There are at least two improvements to be made:
pub(crate)should be a single, short keyword.pubshould strictly mean “visible outside of the crate”, it currently also plays dual role of “visible in parent”. That is, public-in-private should become deny-by-defaultThese two issues are more-or-less backwards compatibility sacrifices due to the fact that Rust 1.0 didn’t get visibility exactly right.
I agree. I kinda hate bikeshedding on the syntax but parameterizing
pubreally hurts usability. It makes it harder to read, and also makes crate level visibility feel like a second class citizen. It doesn’t feel like it’s ever a first-considered option to me even though it’s arguably the most common visibility that I need.For a new language I like the idea of there not being a raw
fnequivalent. I like the cleanliness of havingfn_pub,fn_mod,fn_prvkeywords that are all top level. My worry is that visually scanning would be hindered, but maybe syntax highlighting is enough to alleviate that?I dislike the C++ style for two reasons:
First, I often end up having to scroll up a moderately long way (especially in classes that contain a lot of doc comments, which good codebases do) to find the access specifier.
Second, I either end with annoying diffs if I want to change an access specifier. I either move a method from one group to another (don’t do this if it’s virtual, that’s an ABI break) or I have to write an access specifier before and after the method.
Assuming that private declarations are scoped to a single file, that should never be the case:
If you’re making a private name public, all existing uses of the name should be in only that file since it’s currently private.
If you’re making a public name private, all existing uses of the name most be in only that file because otherwise you can’t make it private since it’s being used externally.
(I work on Dart, which has like Go uses identifier naming for privacy. I refactor all the time and while it is somewhat annoying that it means adjusting the name of every identifier, it’s never a sweeping change because of the above observation.)
The problem with C++ is that a class’s implementation may be spread between many files. The declaration must be in a single place, but method bodies can then be anywhere (in contrast to Objective-C, which puts them all in a block).
This is made worse by the C++ name resolution rules, which often mean that you need to put the definition of a method for one class after the declaration of another, so individual methods of a class may end up being defined across a mix of headers.
Not in Go. That is exactly my complaint. “private” names are accessible within the entire package, which can be any number of source files. (The project I work on is overdue for refactoring and has a package with at least 50 source files in it, and I know a couple structs in it that have sensitive fields that shouldn’t be groped from outside that are being groped.)
Which editors don’t have language support for this kind of thing? (And have meaningful usage numbers.)
Changing identifier names should never result in merge conflicts, right?
Source files only assert scoping for their imports, which is fine. But it’s entirely possible to make a struct field “private” to the package in which it is defined, by lower-casing its name in the struct. Do you mean something else?
Sublime, vim, Emacs, notepad++, etc etc. Vim and Emacs can do it with plugins, but not with the default configuration.
Even newer editors like helix can only do it if you have the appropriate LSP server installed (and that LSP has support for renaming).
Even if your editor can automate this, churning names creates a really unfortunate code review burden that doesn’t need to be there.
I feel like leveraging naming conventions for this is just a bad workaround for a hesitation about the syntax overhead. I would rather find a syntax overhead that I’m confident in paying.
Most of them, in 2012 when I started using Go. I agree it’s a lot less annoying today, but I think it’s a bad idea to have such a basic task be dependent on fancy editor refactoring support.
Um, of course it can. Were you being sarcastic? I rename “foo” to “Foo”, meanwhile you change “f := foo()” to “x := foo()”. Bang.
Why yes, I meant “make things private to a struct or to a source file” as I said. A field of a struct that can only be accessed by methods of that struct, as supported by a million other languages. Without that you can’t have safe encapsulation, not unless you make that struct the only thing in its package. Or a function/variable that’s accessible only in its source file, like “static” in C++ or “fileprivate” in Swift.
Ah, so you’re defining “private” accessibility at the type scope, whereas Go defines it at the package scope. Fair.
I am increasingly of the opinion that there are two meaningful levels of visibility:
Anyone who is modifying the code for a package can trivially find all callers within the package and can update them. By restricting visibility further, you gain no additional reduction in fragility (you are not leaking implementation details outside of a self contained unit) and you increase the maintenance burden. C++ ends up with a load of friend definitions to deal with this in common use. Tighter restrictions are simply documentation: anyone who is modifying the package can remove them and so is not bound by them. The compiler can typically see a whole module at a time (even in C++ with ThinLTO) and so gains no benefit from tighter access restrictions.
C++ couldn’t do this because it had no module system and no mechanism for defining a module boundary. Carbon, as a language that adds syntactic sugar to C++, is probably stuck, but other languages have no excuse.
I also like the model where package-private fields / methods are prefixed with an underscore, as long at there’s an attribute to say ‘also expose this as a public identifier with this name’. To @snej’s point, I don’t see a benefit in making it easy to make a public identifier private (it’s an API break for consumers), but I do see a benefit in making it easy to introduce things as private symbols and then make them public once they’re mature. That said, if the only consumers of the private symbol are within a package, a global rename should be easy.
The documentation effect of specifying what is supposed to be a private detail of a type seems useful to me unless your modules are always quite small (which could be fine, in Elixir each type is its own module).
+1
At this level, they don’t even have to be visibilities, if you have interface files like .h or .mli.
Though, I’d say some more restricted visibility for protecting “local” abstractions is also nice to have.
If I were doing a programming language which cares about scalable abstractions, I’d:
apikeyword to mean “visible outside of current multi file unit of compilation, has unknowable callers”pubto mean “visible throughout CU”Additionally, during compilation compiler would’ve collected all api declarations from all files of CU into a single
.apifile which is also version controlled and used by compilers for separate compilation, by humans to learn the API and to review API changes.FWIW, I do think these are the most important sets.
But having the ability to control this at the granularity of a type, while maybe less important, still seems valuable. And coming from C++ in particular, lacking that would be pointlessly frustrating.
So for Carbon, this was a fairly easy call – we need to meet C++ code and C++ developers where they are, especially when it is reasonably easy to do so.
This post though was much more about how to structure the defaults and annotations than which granularity. We plan to add the missing granularity to C++ for package-based private.
I absolutely disagree. You cannot reliably protect the invariants of a class if that class has no control over outside code touching its state. It isn’t ok if that code is in the same package; that’s a different and much looser boundary. There are often multiple engineers working on code in the same package, and without explicit privacy controls it’s easy to assume a field in someone else’s class is OK to access directly.
Any engineer who can write code in the same package has the ability to change the access specifier on a class in the same package. An engineer who will access a field directly, ignoring accessors and documentation about invariants will also change an access specifier to allow them to do the same. If code is being committed in the same package as your class without sufficient review then you have problems in your engineering process that no language feature can rescue you from.
I think the default access control level (if you leave the signifier out) correlates tightly with the default unit of compilation and distribution. In some circumstances, the public interface is the thinnest waist and the contract, so you need to think twice before changing anything
publicwhilst it makes sense to remain open within your package so that different types within your package can freely access most innards of your package even though they themselves may never be exposed to the public interface.The main benefit of C++‘s region approach is conciseness: A single keyword suffices to set the access control for a long list of member declarations. I do think the lack of that is a big part of Java’s verbosity.
The main pain point with it is that it means class declarations aren’t context-free and compositional. If you invoke a macro inside a class declaration, you don’t know what access control the members produced by the macro will have because it depends on where you call the macro. And the macro can’t itself include an access control modifier because it has no way to “pop” it when it’s done and restore the previous access control level. If I really wanted to over engineer this, I would introduce access blocks like:
Then you could put a bunch of members in a single block to avoid the repetition of Java. Blocks can nest and the innermost one determines the access control of members, letting you compose macros freely. And you can omit the
{ ... }and apply a single modifier to a single declaration.But that’s just way too much machinery for such a small (but important!) feature.
For what it’s worth, Dart uses an underscore prefix on identifiers to mark them private and everything else is public. No one seems to love this, but… I have to admit it works fairly well and is pretty terse.
FWIW, I think the local reminder that something is private via a keyword works surprisingly well in practice.
For example, private things are often fields or fairly short-named APIs. The extra keyword is surprisingly less disruptive here, and it seems easy to afford the verbosity there. The public API, where names are longer and people read it over and over again can be minimal and beautiful.
And I think it is worth a keyword and not just a naming convention. It makes it much more explicit and discoverable for readers.