One point here is that working with analogy to classic ideas in computing is useful on a bunch of levels (vocabulary, analogies, fun).
Applying this to the area I’ve been thinking about a lot lately, profilers (https://pythonspeed.com/fil/ and https://sciagraph.com): sampling-based profiling can be thought of as a form of compression.
You could record everything your program ever does. This is slow, though, and there’s a lot information, so you need to lose some of that information. Sampling is lossy compression: a JPEG or AVIF losses information, but not the information you care about; sampling to find slow parts of the code is similarly useful (the fast aparts are less likely to be sampled! and you don’t care about them!).
Since I’m dealing with long-running programs (might run for hours!), keeping all the samples gets harder. So you need more compression. Reservoir sampling was one approach, but that doesn’t necessarily lead to evenly spread out samples, so I’m dividing time into N equal slices and picking a random sample from each.
And then when you get to UIs, you can compress multiple samples with a flamegraph. But some samples don’t compress well, like samples from recursion… so maybe you want to apply some semantic compression there…
I agree something seems to be awry with the industry’s popular “little languages”. You can hardly spit without hitting a configuration language that, frankly, deserves it. Devops these days is “yaml hell”. I was on a team years ago that sunk weeks into debugging our monolith’s flaky Makefile, entirely thanks to the Make language’s poor abstractive capabilities and terrible debuggability. In theory, configuration languages that are embedded in a general-purpose host language – i.e. the ruby-based or groovy-based DSLs of Puppet, Chef, Jenkins, and Gradle – should be better than yaml hell
This is a great pitch for Oil ! It’s trying to take all these string-ish languages and make a “real language” out of them [1]. YAML and Makefiles are two of my biggest pet peeves.
Hay Ain’t YAML - Custom Languages for Unix System – based on staged evaluation of a SINGLE language, which is used in an ad hoc way e.g. by the combination of Go Templates / YAML, and CMake / Ninja, etc.
Oil supports internal DSLs (like Ruby), not just calling out to external ones in a string-ish fashion, although the latter can also be useful if the target language is designed for it (Ninja)
Perl’s references are basically pointers. As in, C-style pointers. You know. Addresses. Machine addresses. What in the flip-flop are machine addresses doing in a “very high-level language (VHLL)”, might you ask? Well, gosh, what they’re doing is taking up about 30% of the space in all Perl documentation worldwide.
So the claim is basically that Perl has a badly-shaped heap, and that makes it bad for writing “real algorithms” (e.g. tree and graph traversals).
It started out as a “string-ish” language like shell/awk, and then they hacked on all this reference stuff on top later.
Shell and awk don’t have heaps at all, which is why they don’t require garbage collection, and are limited languages.
Oil has a “proper” garbage-collected heap like Python and JavaScript (and Ruby AFAIK). Not like m4/C preprocessor/awk/etc.
And that is actually more difficult than I thought ! I had to get a grant from NLnet and to hire somebody to help me, but that’s working out pretty well. (I wish I had time for blog posts, but it’s all on #oil-dev on Zulip, and some on #blog-ideas)
And probably a common question is: Does shell really need all this? Isn’t this making it too complicated?
The answer is that you ALREADY have a mess with Go templates/YAML and CMake/Make/Ninja so yes :) Just use the normal tools of 60’s - 70’s computer science (garbage-collected data structures, popularized by Java, Python, JavaScript) to do this stuff.
One point here is that working with analogy to classic ideas in computing is useful on a bunch of levels (vocabulary, analogies, fun).
Applying this to the area I’ve been thinking about a lot lately, profilers (https://pythonspeed.com/fil/ and https://sciagraph.com): sampling-based profiling can be thought of as a form of compression.
You could record everything your program ever does. This is slow, though, and there’s a lot information, so you need to lose some of that information. Sampling is lossy compression: a JPEG or AVIF losses information, but not the information you care about; sampling to find slow parts of the code is similarly useful (the fast aparts are less likely to be sampled! and you don’t care about them!).
Since I’m dealing with long-running programs (might run for hours!), keeping all the samples gets harder. So you need more compression. Reservoir sampling was one approach, but that doesn’t necessarily lead to evenly spread out samples, so I’m dividing time into N equal slices and picking a random sample from each.
And then when you get to UIs, you can compress multiple samples with a flamegraph. But some samples don’t compress well, like samples from recursion… so maybe you want to apply some semantic compression there…
This is a great pitch for Oil ! It’s trying to take all these string-ish languages and make a “real language” out of them [1]. YAML and Makefiles are two of my biggest pet peeves.
[1] I also have a succinct argument about why Perl did not do this, or rather a reference to Steve Yegge.
https://sites.google.com/site/steveyegge2/ancient-languages-perl
So the claim is basically that Perl has a badly-shaped heap, and that makes it bad for writing “real algorithms” (e.g. tree and graph traversals).
It started out as a “string-ish” language like shell/awk, and then they hacked on all this reference stuff on top later.
Shell and awk don’t have heaps at all, which is why they don’t require garbage collection, and are limited languages.
Oil has a “proper” garbage-collected heap like Python and JavaScript (and Ruby AFAIK). Not like m4/C preprocessor/awk/etc.
And that is actually more difficult than I thought ! I had to get a grant from NLnet and to hire somebody to help me, but that’s working out pretty well. (I wish I had time for blog posts, but it’s all on #oil-dev on Zulip, and some on #blog-ideas)
And probably a common question is: Does shell really need all this? Isn’t this making it too complicated?
The answer is that you ALREADY have a mess with Go templates/YAML and CMake/Make/Ninja so yes :) Just use the normal tools of 60’s - 70’s computer science (garbage-collected data structures, popularized by Java, Python, JavaScript) to do this stuff.
Absolutely LOVE that!