I’m the author of the doc in question; you can ask me anything about Fennel.
I found that it was suprisingly rare for a language to have a clear and concise rationale doc explaining why the language exists. The only good examples I could find were for Clojure and Pyret:
The Rails doctrine is not good as in “describes a set of good design ideas” but it is good as in “accurately describes Rails”. I found myself cringing while reading it in several places, but upon reflection that can be a good thing since it tells you ahead of time that you’re probably not going to like it. I’m sure there are folks who feel the same way about Fennel; if you think programming languages should have statements and that operator precedence is a good thing, you should know up front that Fennel’s not your cup of tea!
Totally agree. That was one of the things that made Clojure compelling for me is watching Rich Hickey’s talks about place based programming and what made Clojure different and important.
(It’s on my TODO list to update because it’s over 2 years old.)
A really short summary is: shell and bash in particular are deeply intertwined with almost all modern software, whether at runtime or build time. We need a successor to it. By modern standards, it’s implemented poorly (e.g. missing errors from set -e, confusing error messages, core data structures are broken in multiple significant ways, and more).
Incompatible shells won’t replace bash, because nobody has time to rewrite all that code. It’s getting worse as bash is frequently embedded in YAML these days, Docker files, etc. Shell is more popular than ever, and also as poorly used as ever. (Seems like we didn’t learn anything from the mistake of embedding shell in Make.)
I think these 3 features alone would justify a new shell:
Getting rid of “quoting hell”
Getting rid of ad hoc parsing and splitting
Fixing errexit
But Oil has a lot more than that, including unifying separate ad hoc expression languages like $(( )), [, [[, and ${}.
Writing the idioms doc really crystallized my thinking on that. So yes I recommend everyone try to explain their projects, and especially languages, clearly in words. I have been writing for nearly 4 years and there’s still more to do :)
Not sure what a modifier is in this context; I’m pretty sure it doesn’t mean that pressing the shift key is never required?
Modifier keywords, e. g. public, static, private, deprecated, internal, etc.
wait WHAT
Experience has shown that in typed languages (i. e. fast enough to write your own collection implementation) collection literals are a dead end.
Whatever class you decide in the beginning to grant this special collection literal syntax sugar to – it won’t be the class people want to use 2 years down the line.
Now you are stuck with syntax sugar for a class no one wants to use, and a better implementation that – at best – can get a slightly worse collection literal syntax, and – at the worst – none at all. (Thereby putting it at a distinctive disadvantage to the classes with collection literal syntax.)
Whatever class you decide in the beginning to grant this special collection literal syntax sugar to – it won’t be the class people want to use 2 years down the line.
Collection literal syntax doesn’t have to be limited to a fixed list of collections.
C#, for example, uses a pattern-based approach.
Any class that has a method or methods with the right signature supports collection initializers.
The syntax doesn’t work correctly for immutable collections but that doesn’t seem like an intractable problem.
I agree with the notion that the language shouldn’t be privileged in how it defines things; user-level types should be put on the same level as the built-in ones. Disallowing collection literals altogether is one way of achieving this but certainly not the only way!
Yes, using methods is the right way to go (see comment).
I just think collection initializers hardly qualify as “collection literals”. I’d say that collection literals are things like [1, 2, 3], {1, 2, 3} or something very similar to that.
C# only needs these contortions in the first place, because it blew away constructor calls for rather useless things like new List(length).
It’s similar to properties in that case – both collection initializers and properties look ok if you are stuck with bad decisions of the past, but if you do things better from the start, superior options are available.
It’s similar to properties in that case – both collection initializers and properties look ok if you are stuck with bad decisions of the past, but if you do things better from the start, superior options are available.
Here are some use cases that I think properties are a good fit for.
The paragraph after the first bullet should have the same indentation; I think it’s a formatting bug.
Properties can be a good alternative to a method-heavy API with many parameters.
This is a contrived example; personnel databases often have many more columns.
public sealed class PersonFinder {
// You would probably have other overloads.
public IEnumerable<Person> FindPersons(
string firstName,
string middleName,
string lastName,
string username,
string emailAddress,
string title,
string office,
string department);
}
// Here is the same API with properties.
public sealed class PersonFindOptions {
public PersonFindOptions();
public string FirstName {get; set;}
public string MiddleName {get; set;}
public string LastName {get; set;}
public string Username {get; set;}
public string EmailAddress {get; set;}
public string Title {get; set;}
public string Office {get; set;}
public string Department {get; set;}
}
public sealed class PersonFinder {
public IEnumerable<Person> FindPersons(
PersonFindOptions options);
}
The former either has a lot of overloads or callers have to pass many default values. Many of the parameters are the same type so it’s easy for callers to call the wrong overload by mistake. The latter allows callers to set only the options they care about. It’s also much easier to enhance; I can add additional properties to PersonFindOptions without requiring any new overloads.
Drag-and-drop GUI builders like WinForms and WPF.
The Button class, for example, has dozens of properties to modify its size, location, color, etc.
The code generator creates code to call the parameterless constructor and calls to the appropriate property setters.
Allowing experimentation and code completion-driven development.
Some programmers prefer to experiment with the API instead of reading documentation.
They will invoke the parameterless constructor, set properties on a object, and call methods until they are successful.
The property-based API for PersonFinder and many of the classes in .NET are well-suited to this style of development.
Properties don’t require a lot of skill to understand and are significantly simpler than builders.
What is a better solution to the problem that properties solve?
I’m guessing you’re not concerned with the last use case but am curious about a superior alternative to properties for the other two.
Properties can be a good alternative to a method-heavy API with many parameters.
It’s interesting, because I never even considered this usecase.
Imho the whole design feels more like a workaround for the lack of named arguments and default parameters.
I would design the API completely differently for multiple reasons:
No overloading. Overloading simply does not exist, so it would not have been an alternative to start with.
No mutability. It creates questions, especially for this API, which I don’t even care to answer for users. E. g. you add caching of results – now the user changes some properties – which result will be returned?
Trying to keep as close as possible to your sketch, here is how I would design the API:
class PersonFindOptions(
let firstName: Option[String] = None,
let middleName: Option[String] = None,
let lastName: Option[String] = None,
let userName: Option[String] = None,
let emailAddess: Option[String] = None,
let title: Option[String] = None,
let office: Option[String] = None,
let department: Option[String] = None)
module PersonFinder {
fun find(options: PersonFindOptions): Set[Person] = ...
}
// usage:
PersonFinder.find(PersonFindOptions(firstName = Some("Joe"))
Though I would also consider using a normal method with default parameters instead:
PersonFinder.find(firstName = Some("Joe"))
I would probably only define that PersonFindOptions class, if I also had other options (like PetOptions) I wanted to keep separated.
Drag-and-drop GUI builders like WinForms and WPF.
I would design the API to take all required and immutable arguments in the constructor.
Only settings that can actually be mutated after-the-fact may receive a setter, whose name clearly indicates it mutates something.
Allowing experimentation and code completion-driven development.
Again, I would not expose mutable properties in the first place – I expect that the instance is completely initialized after calling the constructor, and everything necessary to do this needs to go into the constructor (or a factory method that in turn calls the constructor).
In the end, I think everything that’s useful about properties can be done without them, if the language is designed for that from the beginning. Some requirements:
All members (fields, variables, methods) live in the same namespace.
Parameterless methods can be defined and called without the ().
This in turn gives rise to APIs where the implementation details (“is ‘property’ x simply forwarding to a field, or doing some computation?”) are isolated from the callers.
All of that, while retaining this crucial bit of information (stored value vs. computed result) for users reading the code (by giving value access and method access different colors in the IDE).
Here is my favorite example:
class Person(let fullName: String) {
fun firstName: String = fullName.split(" ").get(0)
fun lastName: String = fullName.split(" ").get(1)
}
A user would use such an instance like this:
let somePerson = ...
println(somePerson.firstName)
println(somePerson.lastName)
println(somePerson.fullName)
Now I decide that the way I compute first and last name from the full name is a bit silly, users should simply pass first and last name instead!
Therefore, I change the class to this:
class Person(let firstName: String, let lastName: String) {
fun fullName: String = firstName + " " + lastName
}
… and the user code requires no change at all, because whether a member is a method (fun) or a field (let) does not impact source (or binary) compatibility – it’s the IDE that shows the difference with different colors.
This approach also means that this information is often times the first time available for users to see, because older languages basically wrapped all fields in getters (Java) or properties (C#), making all accesses look the same, and therefore discarding this important bit of information.
I agree with your comments about the API and that mutability. I probably would use either default parameters or immutable classes if I was writing this example at work.
The property-based approach I described isn’t appropriate unless the third-use case is a hard requirement. It’s derived from an example in Framework Design Guidelines. The guidelines are aimed at framework developers and many of them aren’t really suitable for either applications or writing libraries with a small target audience.
Imho the whole design feels more like a workaround for the lack of named arguments and default parameters.
Popular .NET languages such as C#, F#, and VB.NET all have named arguments and default parameters. The .NET platform doesn’t require languages to support default parameters so the class library generally avoids them. Microsoft recommends defining a simpler overload for any overload that takes two or more default parameters; their usability studies have found that many developers are confused by default parameters.
Does your opinion of the design change if PersonFinderOptions has substantially more properties, e.g. 40? This isn’t an unrealistic number; my employer’s personnel database has substantially more columns than that.
Here is a real example for the System.DirectoryServices.AccountManagement library. This code configures a UserPrincipal class with the search criteria. The UserPrincipal class has 21 mutable properties. I wouldn’t want to implement this design but it’s easy enough to use.
// Create the context for the principal object.
var ctx = new PrincipalContext(ContextType.Domain,
"fabrikam",
"DC=fabrikam,DC=com");
// Create a user object to use as the query example.
var u = new UserPrincipal(ctx);
// Set properties on the user principal object.
u.GivenName = "Jim";
u.Surname = "Daly";
// Create a PrincipalSearcher object to perform the search.
var ps = new PrincipalSearcher();
// Tell the PrincipalSearcher what to search for.
ps.QueryFilter = u;
// Run the query. The query locates users that match the supplied user principal object.
var results = ps.FindAll();
The core motivation is to not have both “modifier keywords” and “annotations”. (Things not explicitly annotated are @public.)
Ceylon tried having only “modifiers” (or at least everything had “modifier syntax”) and it was of limited success, due to various–mostly ergonomic–issues.
Instead, my approach is to make everything an annotation – that means that even core things like @private or @final have a source file that defines that annotation with documentation, parameters etc.
The compiler has only a limited set of “known annotations” that get “special” treatment (in the sense of using these annotations during typechecking – e. g. it knows to reject @override-ing a function that is declared as @final).
Imho, the user-level possibilities of annotations (documentation, parameters, being part of a namespace, deprecation, …) are really worth pushing more (formerly hard-coded) things out of the core language.
Fennel’s approach (which is taken from Lua) avoids public/private modifiers by making it so that everything in a file is “private” except the last value in the file, which is treated as the value for the module that you get when you require it from another file. This means no annotations are necessary; if you want something to be exposed to the outside, you put it in the final table; otherwise it’s only visible to whatever’s in scope at that point.
Experience has shown that in typed languages (i. e. fast enough to write your own collection implementation) collection literals are a dead end.
I guess I’ll have to take your word for it since my only experience is about a month’s worth of OCaml. I found the built-in collection syntax to be invaluable, but admittedly I was just a beginner.
But how does pattern matching work without collection literals? I’m confused.
But how does pattern matching work without collection literals? I’m confused.
I don’t think these things are particularly related.
If you have e. g. lists, sets and maps that you would create with List(1, 2, 3), Set(1, 2, 3), Map((1, "a"), (2, "b"), (3, "c")) you could pattern match on them by defining methods (with a certain name determined by the language), like List.match(...), Set.match(...), Map.match(...) and then have pattern matching like
This means you can create your own implementation with the same construction syntax (which are just constructor calls that take varargs), and define the match method as above, and then you are ready to go using pattern matching.
It’s not limited to collections, you can support matching on anything, if the matching is determined by such a method, e. g.:
if string is
... Int(string) { ... }
else { ... }
with
module Int {
fun match(string: String): Option[Int] = string.toInt
}
It’s all hypothetical, as I haven’t implemented it yet, but you can have a look at Scala, which does similar things with its unapply method.
I think that depends on how the map would be implemented – if the map used tuples, creating them there wouldn’t hurt. Otherwise, I think it would be interesting to compile this down to some sort of “mixed-varargs” parameter list.
All assuming that tuples are reference types, though they should probably be value types (structs) whose values are copied/moved and not allocated on the heap.
Thinking about it, it’s basically an array-of-tuple-struct, so no, there shouldn’t be any allocation except the vararg (array) itself.
I had to take out my phone to figure out what the issue is so to spare others the effort:
No margin means on screens with a bevel some of the letters fall of the edge. How did we manage to make our screens so stupid? Are websites now supposed to detect the screen size including what the shape is to make sure the content displays properly?
No margin means on screens with a bevel some of the letters fall of the edge. How did we manage to make our screens so stupid? Are websites now supposed to detect the screen size including what the shape is to make sure the content displays properly?
Text is always more readable with margins. Best example to me: Books also have margins, and you’d probably complain if there wasn’t.
Haha; thanks. I had it set to margin auto, which for some reason made me think it would just do the right thing–silly assumption to make, I know. Pushed out a fix.
I’m the author of the doc in question; you can ask me anything about Fennel.
I found that it was suprisingly rare for a language to have a clear and concise rationale doc explaining why the language exists. The only good examples I could find were for Clojure and Pyret:
Although Rails has a pretty good equivalent document, even though it’s not a language:
The Rails doctrine is not good as in “describes a set of good design ideas” but it is good as in “accurately describes Rails”. I found myself cringing while reading it in several places, but upon reflection that can be a good thing since it tells you ahead of time that you’re probably not going to like it. I’m sure there are folks who feel the same way about Fennel; if you think programming languages should have statements and that operator precedence is a good thing, you should know up front that Fennel’s not your cup of tea!
Totally agree. That was one of the things that made Clojure compelling for me is watching Rich Hickey’s talks about place based programming and what made Clojure different and important.
I haven’t created a single “rationale” for Oil, but many people have read this and liked it:
Why Create a New Unix Shell?
(It’s on my TODO list to update because it’s over 2 years old.)
A really short summary is: shell and bash in particular are deeply intertwined with almost all modern software, whether at runtime or build time. We need a successor to it. By modern standards, it’s implemented poorly (e.g. missing errors from set -e, confusing error messages, core data structures are broken in multiple significant ways, and more).
Incompatible shells won’t replace bash, because nobody has time to rewrite all that code. It’s getting worse as bash is frequently embedded in YAML these days, Docker files, etc. Shell is more popular than ever, and also as poorly used as ever. (Seems like we didn’t learn anything from the mistake of embedding shell in Make.)
Another post I should fold in:
I think these 3 features alone would justify a new shell:
But Oil has a lot more than that, including unifying separate ad hoc expression languages like
$(( ))
,[
,[[
, and${}
.Writing the idioms doc really crystallized my thinking on that. So yes I recommend everyone try to explain their projects, and especially languages, clearly in words. I have been writing for nearly 4 years and there’s still more to do :)
Totally agree on this.
The manifesto page of a language I’m contributing to consists of only two quotes, but I feel it’s a good way to get the design approach across:
The first quote is meant to cause an immediate meltdown¹ that makes it easier to filter out people for whom the language would be a bad fit.
¹ See a recent discussion where I tested this and found some “ideal” victims.
This part is interesting:
I also did something similar, listing the features the languages doesn’t have:
You’re a bit antagonistic in these comments.
Is there a public page for the language?
Seems like this is the one: https://github.com/dinfuehr/dora
Cyber sleuth. :)
No, not at this point yet.
Not sure what a modifier is in this context; I’m pretty sure it doesn’t mean that pressing the shift key is never required?
Awesome.
Obviously! There’s no excuse for having these in a language with a static type system.
Yep; way more trouble than they’re worth.
wait WHAT
Thanks for your thoughts!
Modifier keywords, e. g.
public
,static
,private
,deprecated
,internal
, etc.Experience has shown that in typed languages (i. e. fast enough to write your own collection implementation) collection literals are a dead end.
Whatever class you decide in the beginning to grant this special collection literal syntax sugar to – it won’t be the class people want to use 2 years down the line.
Now you are stuck with syntax sugar for a class no one wants to use, and a better implementation that – at best – can get a slightly worse collection literal syntax, and – at the worst – none at all. (Thereby putting it at a distinctive disadvantage to the classes with collection literal syntax.)
Collection literal syntax doesn’t have to be limited to a fixed list of collections.
C#, for example, uses a pattern-based approach.
Any class that has a method or methods with the right signature supports collection initializers.
The syntax doesn’t work correctly for immutable collections but that doesn’t seem like an intractable problem.
Yeah, Clojure has a similar thing where tagged literals can have various meanings based on extensions to the reader.
https://clojure.org/reference/reader#_built_in_tagged_literals
I agree with the notion that the language shouldn’t be privileged in how it defines things; user-level types should be put on the same level as the built-in ones. Disallowing collection literals altogether is one way of achieving this but certainly not the only way!
Yes, using methods is the right way to go (see comment).
I just think collection initializers hardly qualify as “collection literals”. I’d say that collection literals are things like
[1, 2, 3]
,{1, 2, 3}
or something very similar to that.C# only needs these contortions in the first place, because it blew away constructor calls for rather useless things like
new List(length)
.It’s similar to properties in that case – both collection initializers and properties look ok if you are stuck with bad decisions of the past, but if you do things better from the start, superior options are available.
Here are some use cases that I think properties are a good fit for.
The paragraph after the first bullet should have the same indentation; I think it’s a formatting bug.
Properties can be a good alternative to a method-heavy API with many parameters.
This is a contrived example; personnel databases often have many more columns.
The former either has a lot of overloads or callers have to pass many default values. Many of the parameters are the same type so it’s easy for callers to call the wrong overload by mistake. The latter allows callers to set only the options they care about. It’s also much easier to enhance; I can add additional properties to PersonFindOptions without requiring any new overloads.
The Button class, for example, has dozens of properties to modify its size, location, color, etc. The code generator creates code to call the parameterless constructor and calls to the appropriate property setters.
Some programmers prefer to experiment with the API instead of reading documentation.
They will invoke the parameterless constructor, set properties on a object, and call methods until they are successful.
The property-based API for PersonFinder and many of the classes in .NET are well-suited to this style of development.
Properties don’t require a lot of skill to understand and are significantly simpler than builders.
What is a better solution to the problem that properties solve?
I’m guessing you’re not concerned with the last use case but am curious about a superior alternative to properties for the other two.
Thanks for your insightful reply!
It’s interesting, because I never even considered this usecase.
Imho the whole design feels more like a workaround for the lack of named arguments and default parameters.
I would design the API completely differently for multiple reasons:
Trying to keep as close as possible to your sketch, here is how I would design the API:
Though I would also consider using a normal method with default parameters instead:
I would probably only define that
PersonFindOptions
class, if I also had other options (likePetOptions
) I wanted to keep separated.I would design the API to take all required and immutable arguments in the constructor. Only settings that can actually be mutated after-the-fact may receive a setter, whose name clearly indicates it mutates something.
Again, I would not expose mutable properties in the first place – I expect that the instance is completely initialized after calling the constructor, and everything necessary to do this needs to go into the constructor (or a factory method that in turn calls the constructor).
In the end, I think everything that’s useful about properties can be done without them, if the language is designed for that from the beginning. Some requirements:
()
.This in turn gives rise to APIs where the implementation details (“is ‘property’ x simply forwarding to a field, or doing some computation?”) are isolated from the callers.
All of that, while retaining this crucial bit of information (stored value vs. computed result) for users reading the code (by giving value access and method access different colors in the IDE).
Here is my favorite example:
A user would use such an instance like this:
Now I decide that the way I compute first and last name from the full name is a bit silly, users should simply pass first and last name instead!
Therefore, I change the class to this:
… and the user code requires no change at all, because whether a member is a method (
fun
) or a field (let
) does not impact source (or binary) compatibility – it’s the IDE that shows the difference with different colors.This approach also means that this information is often times the first time available for users to see, because older languages basically wrapped all fields in getters (Java) or properties (C#), making all accesses look the same, and therefore discarding this important bit of information.
I agree with your comments about the API and that mutability. I probably would use either default parameters or immutable classes if I was writing this example at work.
The property-based approach I described isn’t appropriate unless the third-use case is a hard requirement. It’s derived from an example in Framework Design Guidelines. The guidelines are aimed at framework developers and many of them aren’t really suitable for either applications or writing libraries with a small target audience.
Popular .NET languages such as C#, F#, and VB.NET all have named arguments and default parameters. The .NET platform doesn’t require languages to support default parameters so the class library generally avoids them. Microsoft recommends defining a simpler overload for any overload that takes two or more default parameters; their usability studies have found that many developers are confused by default parameters.
Does your opinion of the design change if PersonFinderOptions has substantially more properties, e.g. 40? This isn’t an unrealistic number; my employer’s personnel database has substantially more columns than that.
Here is a real example for the System.DirectoryServices.AccountManagement library. This code configures a UserPrincipal class with the search criteria. The UserPrincipal class has 21 mutable properties. I wouldn’t want to implement this design but it’s easy enough to use.
This pretty much looks like a job for a query language – if you need a query language, use one – would be my recommendation.
Is then everything public or everything private?
The core motivation is to not have both “modifier keywords” and “annotations”. (Things not explicitly annotated are
@public
.)Ceylon tried having only “modifiers” (or at least everything had “modifier syntax”) and it was of limited success, due to various–mostly ergonomic–issues.
Instead, my approach is to make everything an annotation – that means that even core things like
@private
or@final
have a source file that defines that annotation with documentation, parameters etc.The compiler has only a limited set of “known annotations” that get “special” treatment (in the sense of using these annotations during typechecking – e. g. it knows to reject
@override
-ing a function that is declared as@final
).Imho, the user-level possibilities of annotations (documentation, parameters, being part of a namespace, deprecation, …) are really worth pushing more (formerly hard-coded) things out of the core language.
Fennel’s approach (which is taken from Lua) avoids public/private modifiers by making it so that everything in a file is “private” except the last value in the file, which is treated as the value for the module that you get when you require it from another file. This means no annotations are necessary; if you want something to be exposed to the outside, you put it in the final table; otherwise it’s only visible to whatever’s in scope at that point.
That’s a really nice design!
I guess I’ll have to take your word for it since my only experience is about a month’s worth of OCaml. I found the built-in collection syntax to be invaluable, but admittedly I was just a beginner.
But how does pattern matching work without collection literals? I’m confused.
I don’t think these things are particularly related.
If you have e. g. lists, sets and maps that you would create with
List(1, 2, 3)
,Set(1, 2, 3)
,Map((1, "a"), (2, "b"), (3, "c"))
you could pattern match on them by defining methods (with a certain name determined by the language), likeList.match(...)
,Set.match(...)
,Map.match(...)
and then have pattern matching likedesugar to the above-mentioned method calls.
This means you can create your own implementation with the same construction syntax (which are just constructor calls that take varargs), and define the
match
method as above, and then you are ready to go using pattern matching.It’s not limited to collections, you can support matching on anything, if the matching is determined by such a method, e. g.:
with
It’s all hypothetical, as I haven’t implemented it yet, but you can have a look at Scala, which does similar things with its
unapply
method.pyret agrees with not privileging any of the collection types with special literal syntax
I’m curious about this to. @soc, how do you deal with this in your language?
Just use constructors with varargs, e. g.
See this thread.
Hope this helps!
What is (1, “a”) if not a collection literal?
It’s a tuple.
That’s my point. A tuple literal is a collection literal.
Let’s disagree on that. It’s an unnamed struct.
Are there extra allocations for the tuples in the Map call?
I think that depends on how the map would be implemented – if the map used tuples, creating them there wouldn’t hurt. Otherwise, I think it would be interesting to compile this down to some sort of “mixed-varargs” parameter list.
All assuming that tuples are reference types, though they should probably be value types (structs) whose values are copied/moved and not allocated on the heap.
Thinking about it, it’s basically an array-of-tuple-struct, so no, there shouldn’t be any allocation except the vararg (array) itself.
Really cool, thanks for sharing! Some unsolicited css feedback: there’s no margin on the left or right so it’s a bit hard to read on my phone..
I had to take out my phone to figure out what the issue is so to spare others the effort:
No margin means on screens with a bevel some of the letters fall of the edge. How did we manage to make our screens so stupid? Are websites now supposed to detect the screen size including what the shape is to make sure the content displays properly?
Text is always more readable with margins. Best example to me: Books also have margins, and you’d probably complain if there wasn’t.
A link to illustrate the issue
Haha; thanks. I had it set to margin auto, which for some reason made me think it would just do the right thing–silly assumption to make, I know. Pushed out a fix.