1. 34

So, this post was originally just talking about neat features (and ommissions) for non-programmers needing a scripting language.

I kept mulling it over, and with input from friends and colleagues started getting an idea for a language, schlub, together. This post represents a grabbag of the current ideas behind the language.

I think it’d be better served with example programs, but I haven’t written those up quite yet. It’s messy, weird, and hopefully useful. Let me know what you think!

Edit: And yes, in some ways, good chunks of this language are deliberately bad. I think that kicking the search space over by a few whacks into the realm of seeming crazy can yield interesting results.

  1.  

  2. 15

    I think history has shown that there is no warning label large enough to dissuade stubborn users from using a language intended for small-scale work for inappropriately large, complex projects: see PHP, JavaScript, Python, or even to some extent Bash. At a technical level Schlub may be excellent for the quick-and-dirty use case you intend it for, but how do you prevent the social phenomenon of Schlub codebases growing far beyond their intended limits? How do you force users to confront the moment when it’s time to cut their losses and rewrite their prototype using more industrial-strength tools?

    1. 9

      Or even Excel spreadsheets (Excel happens to be the most popular non-programmer programming language ever written).

      1. 5

        A great way would be incentivising the refactor.

        For example, if the language had a “transparent” FFI to Python (kinda disagree about it being on the same level as PHP and JavaScript for this, and it’s flexible enough), then you could move stuff over to a “serious” language without having to rebuild your toolchain.

      2. 6

        This sounds like the kind of language whose UI is a spreadsheet, or at least a Jupyter-style notebook, rather than the traditional plain-text-in-a-generic-editor. Taking inspiration from sgreben, I love the idea of bundling a bunch of example inputs with the program, so that as the program is edited, the IDE (or equivalent) can display how the proposed change affects each of the examples.. ideally, at each step of the process.

        1. 5

          Sounds like a language that doesn’t try to waste my time :) Speaking of which, i think it might be interesting to think about what an “iteration” in the language might look like - something to address “I changed some stuff, what is the result?”.

          Now, there are people who use “tests” for this purpose, but that term comes with a lot of baggage, much of which results in my time being wasted, at least in the short term. So perhaps it’s possible to factor out the quality-assurance/maintainability/code-monkey-metrics aspects and find something that a) is in-between a REPL and a test suite and b) has a simple, baggage-free term for a name.

          Perhaps “examples”? As in, some sort of collection of inputs and their previous/expected/possible results together with short-term time-saving automation. Could be as simple as recording stdout on each run and showing a diff on the next re-run. Or something very clever with generators and fuzzing.

          (Edit) Adding on to the diff idea, having useful “diffs” for (nested) maps, lists, and primitive types out-of-the-box might already go a long way.

          1. 3

            I think this is a fantastic idea. I can’t speak for other developers, but even when I don’t write full tests for a piece of code, I always run it on some example input, in a REPL or a scratch executable file, and “eyeball” the results to see if they match my expectations. Even though these “examples,” as you call them, don’t come with a machine-checkable criterion for their correctness, they are still valuable in their own right as a first step towards validating that a codebase behaves as intend, and also serve as a kind of executable documentation that future developers can use to understand how a project’s APIs work in practice. Current tooling seems to lack first-class support for this kind of example code, and in some cases (e.g. REPLs) actually actively works to ensure it never lasts beyond a single session, but I think it would be valuable to treat it as an important persistent artifact of the development process; an asset to stand alongside code, documentation, and tests.

            1. 3

              So this feels like a referentially-transparent R. I’m curious what do you think your FFI story will look like? Or is the intention to implement things like BLAS, HTTP, CSV, JSON, etc in the language vs in libraries? I feel like new serialization formats and other ways to get an munge data keep appearing and historically they would enter a system is initially through the FFI.

              1. 1

                No FFI!

                That’s tantamount to a user-defined library.

              2. 3

                I just came across a post of mine from a decade ago about mapping functions. The gist is turning this:

                double x[MAX];
                double y[MAX];
                y = sin(x);
                

                into effectively this:

                double x[MAX];
                double y[MAX];
                for (size_t i = 0 ; i < MAX ; i++)
                  y[i] = sin(x[i]);
                

                The information is there—sin() takes a double, so why not have the compiler write the mapping code for you if you are passing in an array and assigning the result to an array?

                1. 2

                  Array languages do this. Literally:

                  q)x
                  0.1958467 0.5615261  0.07043811 0.2124007  0.777882   0.4844727   0.6827999  ..
                  0.8505909 0.8196014  0.09982673 0.8187707  0.6506965  0.03492196  0.03283839 ..
                  0.5281112 0.06494865 0.1167751  0.5464707  0.2569845  0.3529561   0.3923249  ..
                  0.5046331 0.6014692  0.5000071  0.8392881  0.5938637  0.765905    0.8794685  ..
                  0.6354142 0.8709557  0.182261   0.1276624  0.3815986  0.1391596   0.2629808  ..
                  0.2362643 0.3893913  0.4566197  0.9771774  0.3607297  0.9265934   0.307924   ..
                  0.7108719 0.214076   0.1903842  0.4235704  0.01962086 0.1102437   0.08516535 ..
                  0.7242412 0.9886591  0.716507   0.1034303  0.8887313  0.06119115  0.5510458  ..
                  0.6471999 0.2005554  0.3599424  0.7192901  0.4153674  0.831001    0.2386444  ..
                  0.6217243 0.5569152  0.3877172  0.586263   0.5882985  0.7258795   0.2363145  ..
                  0.4420741 0.6962264  0.5392064  0.07121504 0.9690118  0.348649    0.7171755  ..
                  0.6765743 0.9140916  0.9223146  0.1073726  0.5431816  0.4056211   0.2703693  ..
                  0.4602883 0.6513394  0.5250004  0.8667757  0.1200717  0.4089565   0.5700753  ..
                  0.1103875 0.2663224  0.3374996  0.9620906  0.3964715  0.6094688   0.592642   ..
                  0.7661899 0.7618308  0.3613931  0.6256195  0.3929583  0.1120611   0.9880299  ..
                  0.2956769 0.9913145  0.6285445  0.05589389 0.6608354  0.168436    0.9325573  ..
                  0.6891933 0.6467324  0.2952636  0.7418674  0.250098   0.05310517  0.2166502  ..
                  0.798097  0.9955805  0.8605755  0.7959528  0.747095   0.4554808   0.8713021  ..
                  0.7274707 0.6542671  0.9833302  0.8054017  0.9955855  0.8646926   0.1035331  ..
                  0.3144248 0.3295437  0.5278991  0.8646628  0.7115124  0.007967671 0.8627135  ..
                  0.7218258 0.1020001  0.8661184  0.06097264 0.07675494 0.4683239   0.4671813  ..
                  0.8544192 0.2559604  0.7318372  0.04870548 0.4628702  0.5027869   0.4276977  ..
                  ..
                  
                  q)y:sin x;
                  q)y
                  0.1945971 0.5324786 0.07037988 0.2108073  0.7017721 0.4657418   0.6309677  0...
                  0.7516702 0.7308738 0.09966101 0.7303066  0.6057407 0.03491487  0.03283249 0...
                  0.5039027 0.064903  0.1165099  0.5196752  0.2541652 0.3456732   0.3823377  0...
                  0.4834863 0.5658544 0.4794317  0.7441678  0.5595674 0.6931896   0.7704001  0...
                  0.5935109 0.7649448 0.1812535  0.1273159  0.3724045 0.1387109   0.25996    0...
                  0.2340724 0.3796253 0.4409166  0.8289218  0.352957  0.7995787   0.3030809  0...
                  0.6524948 0.2124446 0.1892361  0.4110179  0.0196196 0.1100205   0.08506243 0...
                  0.6625673 0.8352895 0.6567546  0.103246   0.7762726 0.06115297  0.5235786  0...
                  0.6029549 0.1992136 0.3522203  0.6588508  0.4035261 0.7386065   0.2363857  0...
                  0.5824376 0.5285701 0.378076   0.5532519  0.5549464 0.6637935   0.2341212  0...
                  0.4278151 0.6413269 0.5134551  0.07115486 0.8243267 0.3416284   0.6572586  0...
                  0.6261256 0.7920083 0.7970017  0.1071664  0.5168623 0.3945895   0.2670873  0...
                  0.4442064 0.6062521 0.5012134  0.7622459  0.1197834 0.3976521   0.5396954  0...
                  0.1101634 0.2631853 0.3311288  0.8203888  0.386166  0.5724319   0.5585544  0...
                  0.6933949 0.6902473 0.3535777  0.5855995  0.3829229 0.1118267   0.8349434  0...
                  0.2913875 0.8367465 0.5879681  0.0558648  0.6137766 0.1676407   0.8031462  0...
                  0.6359148 0.6025819 0.2909921  0.6756657  0.247499  0.05308021  0.2149594  0...
                  0.7160289 0.8390749 0.7582179  0.7145305  0.6795103 0.4398941   0.7651679  0...
                  0.6649827 0.6085778 0.8323478  0.721109   0.8390776 0.7608959   0.1033482  0...
                  0.3092696 0.3236114 0.5037196  0.7608765  0.65298   0.007967587 0.7596102  0...
                  0.6607563 0.1018233 0.7618202  0.06093486 0.0766796 0.4513913   0.4503714  0...
                  0.7541897 0.2531746 0.6682375  0.04868622 0.4465181 0.4818694   0.414777   0...
                  ..
                  
                2. 3

                  Have you taken a look at Frink?

                  1. 1

                    Tell me why you landed on no user-defined types?

                    1. 8

                      Sure!

                      So, there’s a difference between having no user-defined types at all (which is kinda nuts) and not having the ability to create your own types that are viewable from the type system.

                      First, users can use functions to map parameters into an associative array or other composite record, and theoretically they could then write functions that take the resulting records as parameters and return a mutated version. So, in the strictest sense, you could have your own user-defined types and defined operations on them.

                      That’s a bit besides the point, though.

                      When thinking about the problem space for schlub, I deliberately removed support for programming-in-the-large. Here, this means no user-defined libraries. The reason for that choice is:

                      • Having user-defined libraries means that people write abstractions that they have to fight against later when refactoring or tweaking programs
                      • Having user-defined libraries means that, when they update the library, they end up creating bugs in programs dependent on that library that previously worked correctly (or, at least, well-enough).
                      • Having user-defined libraries means it is important to have user data types, to be able to encapsulate and hide the information and operations that a user might want to do. If there are no libraries, there is no encapsulation, and if there is no encapsulation, there is no need for user-defined type support in the language.

                      Half the time on OOP projects (or FP projects that handle business objects), we model out the various business entities (this is a customer, this is a machine, this is a bike, etc.) and we can end up getting really quite carried away in the metaproblems that that process creates:

                      • is a bike a type of machine? should it respond to the same operations as a machine?
                      • if I have a person and i add it to a bike, do i get a cyclist? is cyclist a person?
                      • if I eschew inheritance and just go the straight composition route, can I really make any guarantees about a type and its operations? How much machinary is required for it?

                      I guess I’d sum up my reasoning as:

                      At the end of the day, every language that supports arbitrary associative maps or strings (and you have to do this, because real-world data almost always shows up that way) ends up breaking their careful user-defined type systems and subverting the safety of those systems anyways, so we might as well just embrace that and spend innovation tokens elsewhere.

                      1. 2

                        This actually sounds like a great language to have as an embedded scripting language (as in Lua within Redis)! I was actually just looking for one that I could embed within a Go project recently that could operate on a limited API and was sad to find a scarcity of good options.