Threads for bogdan

  1. 13

    Ahahahaha yes, does Fennel run on it?

    1. 8

      Not yet! It looks light right now it can parse almost all the way through to the end of the Fennel compiler, but it fails on the one use of long brackets in the file:

      ~/s/Fennel (main) [1]> racket fennel.lua
      fennel.lua:5187:25: expected expression but found [
        context...:
         /Users/bogdan/sandbox/racket/racket/collects/syntax/readerr.rkt:15:2: -raise-read-error
         /Users/bogdan/sandbox/racket-lua/lua-lib/lang/parser.rkt:304:0: parse-expr
         /Users/bogdan/sandbox/racket-lua/lua-lib/lang/parser.rkt:293:0: parse-exprs
         /Users/bogdan/sandbox/racket/racket/collects/racket/match/compiler.rkt:548:40: f112
         /Users/bogdan/sandbox/racket-lua/lua-lib/lang/parser.rkt:78:0: parse-block
         /Users/bogdan/sandbox/racket-lua/lua-lib/lang/parser.rkt:99:0: parse-statement
         /Users/bogdan/sandbox/racket-lua/lua-lib/lang/parser.rkt:78:0: parse-block
         /Users/bogdan/sandbox/racket-lua/lua-lib/lang/parser.rkt:74:0: parse-chunk
         /Users/bogdan/sandbox/racket-lua/lua-lib/lang/reader.rkt:23:2: custom-read-syntax
         /Users/bogdan/sandbox/racket/racket/collects/syntax/module-reader.rkt:211:2: wrap-internal
         .../syntax/module-reader.rkt:76:9: wrap-internal/wrapper
      

      That should be pretty easy to fix. After that, I’m guessing it’ll fail because it relies on some of the missing stdlib things I mentioned in the post. I’ll try to spend some time in the next couple of weeks getting this to work though, since it would be pretty neat to be able to run Fennel from Racket.

      1. 7

        Aha, you found the one place in all of Fennel that uses Lua’s bracket-string syntax, for loading built-in macros. You could cut that part out and load a macroless variant of Fennel. =)

        Come by the #fennel channel on Libera if you need any more help! This does sound very interesting. Among niche Lua implementations the most common cause of incompatibilities are bugs from require/package and incompatibilities in string.find and other pattern functions. (I think one of the JVM ones just doesn’t even try and treats patterns exactly the same as Java patterns which obviously breaks a ton of things.)

      2. 3

        On the Racket discord the author mentioned something along the lines of ‘passed to antifennel and the result back through fennel.’ but you will have to ask the author on the Racket discord https://discord.gg/6Zq8sH5 or discourse https://racket.discourse.group/

        The repo for #lang lua is at https://github.com/Bogdanp/racket-lua

      1. 10

        From TFA:

        type Interval(x, y) <: Int {
          x <= self <= y; 
        }
        
        var i: Interval(1, 10) = 3
        i += 7 #ok
        i += 1 #error!
        

        You can do this in Ada, and it will even detect it at compile-time in most cases:

        procedure foo is
            type Interval is range 1..10;
            I: Interval := 3;
        begin
            I := I + 7;
            I := I + 1;
        end foo;
        

        Gives me

        foo.adb:6:12: warning: value not in range of type "Interval" defined at line 2
        foo.adb:6:12: warning: "Constraint_Error" will be raised at run time
        

        Also from TFA:

        A language designed around having first-class GUI support

        I miss VB6.

        I miss REBOL. :(

        1. 9

          I miss REBOL. :(

          Isn’t Red the logical successor of REBOL? Not sure how much community has formed around it (I mean, I’ve never encountered RED in the wild…), but it always struck me as interesting, at least.

          1. 3

            Red is super neat. Last I looked (which was a long time ago) at it they didn’t support a View-like dialect on anything but Windows but it looks like that’s changed! That’s awesome. Good for them. I’ll need to look at it again.

          2. 4

            The interval example is just one example, I also want to be able to do something like

            type SortedList[T] <: List[T] {
              T is Comparable
              forall x, y in 0..Len(self):
               x < y => self[x] < self[y]
            }
            
            def binary_search(l: SortedList[T], elem: T) {
             ...
            }
            
            1. 9

              Possible in Ada 2012, without using SPARK. You can also use Preconditions/Postconditions in Ada 2012 without requiring proofs, as a runtime assertion mechanism.

              -- Rough, probably doesn't compile as is, but this is the idea:
              generic
                  type T (<>) is private;
              package Lists
                  type List(Length : Natural) is record
                      Elements : array(Natural range <>) of T;
                  end record
                      with Type_Invariant => Is_Sorted;
              
                  function Is_Sorted(L : List) return Boolean;
              end Lists;
              

              Actual Example of Type_Invariant Usage, which has Pre/Post usage in the same file

              1. 4

                Racket makes this sort of stuff trivial:

                #lang racket/base
                
                (require racket/contract)
                
                (define (sorted-listof ctc [less-than? <])
                  (and/c
                   (listof ctc)
                   (flat-named-contract
                    `(sorted-listof ,(object-name ctc))
                    (λ (xs)
                      (or (null? xs)
                          (for/and ([x (in-list xs)]
                                    [y (in-list (cdr xs))])
                            (less-than? x y)))))))
                
                (define/contract (binary-search xs elem)
                  (-> (sorted-listof integer?) integer? (or/c #f exact-nonnegative-integer?))
                  #f)
                
                (binary-search '() 20)
                (binary-search '(1 2 3) 5)
                (binary-search '(2 1 3) 1)
                

                Produces:

                binary-search: contract violation
                  expected: (sorted-listof integer?)
                  given: '(2 1 3)
                  in: an and/c case of
                      the 1st argument of
                      (->
                       (and/c
                        (listof integer?)
                        (sorted-listof integer?))
                       integer?
                       (or/c #f natural?))
                  contract from: (function binary-search)
                  blaming: /Users/bogdan/tmp/sorted-list.rkt
                   (assuming the contract is correct)
                  at: /Users/bogdan/tmp/sorted-list.rkt:16:18
                Context (errortrace):
                   /Users/bogdan/tmp/sorted-list.rkt:22:0: (binary-search (quote (2 1 3)) (quote 1))
                

                And blame works in the opposite way, too:

                (define/contract (sort-list xs)
                  (-> (listof integer?) (sorted-listof integer?))
                  '(1 3 2))
                
                (sort-list '(1 2 3))
                

                Produces:

                sort-list: broke its own contract
                  promised: (sorted-listof integer?)
                  produced: '(1 3 2)
                  in: an and/c case of
                      the range of
                      (->
                       (listof integer?)
                       (and/c
                        (listof integer?)
                        (sorted-listof integer?)))
                  contract from: (function sort-list)
                  blaming: (function sort-list)
                   (assuming the contract is correct)
                  at: /Users/bogdan/tmp/sorted-list.rkt:24:18
                Context (errortrace):
                   /Users/bogdan/tmp/sorted-list.rkt:28:0: (sort-list (quote (1 2 3)))
                

                Note how the same contract now blames the sort-list function, not the caller.

                  1. 1

                    No, I’m not. Dependent types are push you into the formal methods space. I specifically want to try using these kinds of things without having to also play Fight The Theorem Prover. I do enough of that for work!

                    1. 1

                      Well, that’s the name for this sort of thing.

                      1. 1

                        To my knowledge, dependent types are very static. Here is the context for the interval example in Hillel’s article:

                        Static types are great! There’s lots of cool stuff being done with static typed languages right now. Not so much with dynamic types, though. Most of the research and interest is in adding gradual typing to dynamic languages. I think there’s a lot of interesting experiments we can do with dynamic types. I mean, you can generate new types at runtime! Combine that with contracts and we can attach constraints to variables, like this:

                        1. 1

                          I don’t think that’s a very useful distinction. While the types are technically static, the actual types of objects may not be resolved until runtime (potentially resulting in type errors). This is pretty dynamic behavior for a “static” type system.

                  2. 1

                    I think you could get close to something like that with Ada/SPARK. I’m no expert though.

                    EDIT: See @pyj ’s much better response above.

                    1. 1

                      If you’re already willing to accept dynamic types, and contracts, doesn’t mixing the two get you all the way there? You need contracts on the class / structure, not just pre/post conditions on functions and methods. But this is already established, in say, Eiffel. Are you merely asking for more expressive invariants?

                      1. 2

                        A more interesting idea would be adding and removing types to values as they move around. Like calling sort on a list returns a SortedList, and then shuffling the SortedList returns a regular list. I dunno. I’m spitballing here, I think there’s interesting ideas you could do with dynamic types that haven’t been done yet, but maybe I need to think more about those actually are.

                        Something else I’ve wanted to play with are “contagious types”: 2 * ViralNum(2) == ViralNum(4). Don’t think this would be useful for anything, but I just want to play with it.

                        EDIT: actually I think the thing I really want is to be able to say “TestableObj is a subtype of Obj, now replace *every instance of Obj with TestableObj in this program run, kthx”. I think it’s a similar idea to the “semantic relations”. I’d really like to see the contracts/semantic relations/wild metaprogramming all in one language.

                        1. 3

                          “contagious types”: 2 * ViralNum(2) == ViralNum(4). Don’t think this would be useful for anything

                          One application would be to have a number type where you mark numbers as “inexact” if they come from special inexact numerals, or if they are the result of a floating point operation where the overflow / underflow / inexact flags were set, or if an argument to the operation is inexact. Scheme uses this same terminology for a feature that is less useful than what I just described.

                          This is related to tainted values in Perl, which are useful for enforcing security policies. Say you want to fix Javascript to prevent fingerprinting. Any Javascript operation that returns a value that can be used for fingerprinting is tainted. If a tainted value is used to construct another value, the second value is also tainted. Then you have a security restriction that prevents tainted values from being transmitted to another host over a network connection (or added to a cookie, same thing).

                          1. 1

                            EDIT: actually I think the thing I really want is to be able to say “TestableObj is a subtype of Obj, now replace *every instance of Obj with TestableObj in this program run, kthx”. I think it’s a similar idea to the “semantic relations”. I’d really like to see the contracts/semantic relations/wild metaprogramming all in one language.

                            Can’t this be achieved today with something like Guice, or other Dependency Injection frameworks?

                            1. 1

                              It looks like that requires you to modify the original class to get the desired behavior? I was thinking of applying it to, say, objects coming from a third party library.

                              1. 1

                                These frameworks are kind of viral, but you can create “modules” — a rule set, basically — and swap per execution. Something, somewhere is going to need to make a choice about which type / constructor to use instead, so it’s unlikely that you’ll get away without modifying something. In the case you mention, you’d likely have to use a proxy, so, you’re still modifying something.

                            2. 1

                              Re: contagious types, they can be useful for expressing measurement units. Here’s an example in Julia using the Unitful package:

                              10 * 3.6u"km" == 36u"km"
                              

                              Some Julia packages also use “contagious types” for tracking the error on a calculation, derivatives, etc.

                      1. 1

                        It astonishes me how many people claim to care about lisps parenthesis. It really seems to be a trained knee jerk rejection as part of the culture, popularized and enshrined for the web generation of developers thanks to xkcd. Meanwhile it seems to make no sense!

                        1. 1

                          I think the parens would bother me a bit less if they were paired with sensible whitespace.

                          1. 1

                            I’m not sure what it is, either, but I’m one of those people. It may be that I learned C young enough that other styles are like learning a second language? But I feel like I can scan Python or ML family pretty easily, and I didn’t encounter those until college.

                            I’m trying still, it may just take more effort, and more familiarity with good tooling. I wonder if there’s a correlation with those of us who use fairly basic text editors. At the same time I’m experimenting with graphical structure editors and paren-less syntaxes, because I suspect I could make it easier on myself. Some things may just be more work for some humans to parse, and there’s plenty of equivalent syntaxes.

                            1. 2

                              I use vim without syntax highlighting and I enjoy writing Clojure.

                              My disdain was not aimed at those without sufficiently powerful workflows, but at those who write off a language because they’ll spend much more time with shift 9.

                              1. 1

                                Everyone has a limited lifespan and an essentially unlimited number of potential things to learn; it’s only natural that people develop dumb heuristics for disqualifying the vast majority of things based on surface-level appearance. If you learned enough about everything to make a non-surface-level judgement to find the best thing you would have no time to actually use the thing because you’d be spending it all gathering enough data to make the judgements.

                                The only thing that’s surprising is that people pretend they’re making said judgements based on logic rather than knee-jerk reactions.

                                It’s OK for knees to jerk in response to stimulous; they evolved that behavior for a reason.

                            2. 1

                              Your comment comes off as awfully arrogant and can easily be flipped around (“C-like syntax is just a trained knee jerk reaction …”). I like s-expressions because they’re easy to structurally manipulate when editing code, and once you get used to structural editing of programs, editing other kinds of syntax is cumbersome.

                              1. 1

                                C syntax is a trained comfort. That’s exactly my point.

                                1. 3

                                  Ah! OK, my bad. I read your comment as complaining about people who like s-exprs, not the other way around.

                              2. 1

                                I have a lot of trouble reading lisps, and wonder (but am unsure) if it’s related to my dyslexia. If so, that would be a personally compelling argument against the parens!

                                1. 4

                                  If it is dyslexia related, maybe using a Rainbow Parens plugin might help?

                                  I don’t have dyslexia, but using Rainbow parens in Neovim and Kate helped me read Janet much more easily, myself.

                                  1. 1

                                    I don’t have dyslexia either, but for I have found that while they might be helpful early on, once you learn to read it, drawing attention to the parens is counterproductive. The parens exist in order to show the compiler what to do and to get the editor to indent the code correctly; you shouldn’t be paying attention to them while you’re reading or writing code. On my own system I have it set up so they fade into the background, and I read the code based on its indentation.

                                    The one exception to this rule is if you pair program, it can be really useful to be able to say “add foo right after the purple paren on line 41”; that kind of thing is difficult to do with faded parens.

                                  2. 2

                                    The developer of Wisp syntax suggests [1] it might be related to the phenomenon where the letters at the beginning and end are relatively more important in recognizing words (a form of the transposed letter effect). This is interesting but probably overstated: it seems unlikely to be reduced by whitespace after ( (though this would be interesting to test), and the transposed letter effect is not that strong.

                                    [1] Why Wisp?, slide 4

                                1. 3

                                  wow, this is great! :D

                                  I might convert Fafi Gemini Browser to use this instead of direct racket/gui calls. This looks so much easier.

                                  1. 2

                                    Thanks! Support for pasteboards is lacking right now so let me know if you run into any issues with that. I’m planning to write some documentation on making custom views soon, but I’m sure you’ll be able to figure it out. The hn.rkt example has an example of exactly that.

                                  1. 1

                                    Thank you for sharing.

                                    What kind of layout engine does it have? Can I change the layout (not the code) based on screen size (eg when the user resizes/screen rotation/etc) ? Is there a notion of horizontal/vertical , left/center/end, etc layout primitives ?

                                    1. 3

                                      The library is built on top of racket’s built-in racket/gui lib so it shares the same layout management. It is possible to control things like alignment, stretch and size and any view that supports setting those properties also supports having them passed in as observables (so, for example, you could make an observable that tracks the current window size then derive values to control the layout of things based on it).

                                      1. 1

                                        ah, thank you. I constrained my search just in your docs, and did not realize the above. Will study this.

                                    1. 7

                                      Below is a copy of a comment I made on Reddit re. the post.

                                      Some patches that Racket needed were detri­mental for Chez Scheme at large.

                                      I’m curious which patches MB is referring to here. “Detrimental” to me implies making CS worse, but I’d be surprised if there were any such patches. There are definitely patches containing changes that the CS folks aren’t interested in, but that doesn’t mean they’re not good changes. That seems to me more of a difference in values (i.e. a subjective thing) rather than a judgement on whether the changes are good or bad. It can also be a matter of the effort required by the CS team to understand and merge some of the more complicated patches.

                                      But the actual project under­taken by the Racket team was to preserve the existing behavior of Racket and all its libraries. […] And instead of capit­u­lating to the Chez Scheme way—say, by removing or changing certain features of Racket—these Racket-compat­i­bility patches became the basis of the Racket-specific fork of Chez Scheme.

                                      I’ve been a Racket user for far less than MB, but to me the opposite would’ve been a lot more surprising. Of course a language that values backwards-compatibility is going to do its best not to break it.

                                      Single flonums were removed because nobody was using them so Racket CS did break backwards-compatibility in areas where the costs outweighed the benefits. I think there were a couple more small examples like this, but I can’t remember them off the top of my head. In areas where changes were needed to preserve the same level of performance as Racket BC (eg. unboxed flonums), those changes also benefited CS. Parallel and incremental GC is another one like that, but that’s one that hasn’t yet made it into CS.

                                      I thought the broader objec­tive was to perma­nently combine the greatest assets of Chez Scheme (= its speed and stability) with the greatest assets of Racket (= its language-oriented program­ming facil­i­ties, including its macro system and module system). This didn’t happen.

                                      I think this did happen. Racket CS is generally faster than Racket BC, it is more maintainable by more people and it’s still Racket.

                                      ~/s/racket (master)> git shortlog -s racket/src/{cs,ChezScheme} | wc -l
                                            37
                                      ~/s/racket (master)> git shortlog -s racket/src/bc | wc -l
                                            15
                                      
                                      1. 5

                                        one thing that annoys me when upgrading Racket is that all of a sudden I’ll be getting errors on my current projects because the version mismatch between the new Racket version and whatever version was used to generate the stuff in the “compiled” folders. Also, reinstalling every single package when I upgrade is a chore. I’m sure there might be an easier way, but damn when upgrading a language I don’t expect this kind of friction.

                                        If there is a version mismatch between what is in compiled and the current running runtime, then just recompile. Why can’t packages be installed in a way that multiple Racket versions can access, why are they siloed per version?

                                        1. 4
                                          read-compiled-linklet: version mismatch  expected: "7.9.0.3"  found: "7.8.0.6"  in: .../afile_rkt.zo
                                          

                                          Is this the error you are seeing? The 3480 issue might resolve your problem.

                                          https://github.com/racket/racket-lang-org/issues/110

                                          https://github.com/racket/racket/issues/3253

                                          https://github.com/racket/racket/issues/3480

                                          1. 1

                                            will check the bugs, thanks for the links.

                                          2. 4

                                            reinstalling every single package when I upgrade is a chore

                                            raco pkg migrate <previous-version-here> will automatically migrate your old packages to the new version.

                                            Re. re-compiling modules as needed, I think someone just needs to put in the time/effort to get that working. I don’t think there’s anything inherent to prevent it from working. In the mean time, you should be able to run raco setup after an upgrade to get it to re-compile all your stale packages (assuming you install your projects as packages).

                                            1. 2

                                              thanks @bogdan, as usual your content saves my day.

                                          1. 2

                                            Racket:

                                            (for/fold ([p #f]
                                                       [cnt 0]
                                                       [counts null]
                                                       #:result (cdr (reverse (cons `(,p ,cnt) counts))))
                                                      ([c (in-string "aaaabbbcca")])
                                              (if (equal? p c)
                                                  (values c (add1 cnt) counts)
                                                  (values c 1 (cons `(,p ,cnt) counts))))
                                            
                                            1. 1

                                              Also using fold, with mnm.l :

                                              (def challenge (IN)
                                                  (foldr (\ (C ACC)
                                                           (let ((((V . N) . TL) . ACC))
                                                             (if (= C V)
                                                               (cons (cons C (+ N 1)) TL)
                                                               (cons (cons C 1) ACC))))
                                                    IN NIL))
                                              
                                              1. 1

                                                Racket’s group-by is wonderful but I usually want to group consecutive equal items into clumps as they arise rather than a single monolithic group.

                                                (define (group xs)
                                                  (match xs
                                                    [(cons x _)
                                                     (define-values (ys zs) (splitf-at xs (curry equal? x)))
                                                     (cons ys (group zs))]
                                                    [_ null]))
                                                
                                                (define (encode xs)
                                                  (for/list ([x xs])
                                                    (list (first x) (length x))))
                                                
                                                (encode (group (string->list "aaaabbbcca")))
                                                
                                              1. 0

                                                I’d argue that types are a terrible solution to the actual problem.

                                                I’d argue the actual problem is, we need to check variables in our languages, for instance for someone’s age, we almost certainly don’t want -32768 as a possible age, as that makes zero sense. Also an age of 32768 is also probably equally stupid.

                                                we want age to be an integer between 0 and 120 or so. Certainly < 200. That’s what we actually want.

                                                We can argue about the upper and lower bounds of the age variable, depending on the program being written(i.e. an age of 120 for a program about the romans would be laughable, since they never lived remotely that long). But I hope we can all agree there should be upper and lower bounds for our variables.

                                                Types, basically don’t do that, and practically every language ever written skips over any sane way to have limits on our variables besides basically int8, int16 and string.

                                                There are some attempts, Liquid Haskell has a way. Postgres does this pretty well with the check constraint(though it’s not a programming language, obviously). Nim recently got ‘DrNim’ that attempts the same thing. Most other languages fail.

                                                1. 8

                                                  Types do that. Commonly the pattern looks like having a constructor function that validates the machine type, like smart constructors in Haskell. (Though, as cgenschwap already noted, dependent types can do this in a quite different way.)

                                                  1. 2

                                                    I think you meant types might be able to do that. so I agree, in theory, types can grow to specify constraints, but other than Haskell(with multiple competing solutions apparently) and now Ada(thanks @weinholt), nobody’s type system has this ability that I’m aware of(I’m sure there are some weird esoteric languages out there that do..).

                                                    But no commonly available/used language has anything resembling a way to sanely do constraints. The best you can do is writing a check function and calling it all over the place, hoping you don’t miss a place and your constraint fails to hold, with no warnings if you miss a spot. Languages aren’t immune to the problem, even most databases, except some SQL ones(like PG’s check) pretty much punt on constraints, so it’s a very wide problem.

                                                    Even web frameworks, mostly fail at checking constraints, and it’s rare to see web code in the wild do much beyond the occasional JavaScript-based client-side validation of inputs. Otherwise the most you will see is ‘yes input validation is important’, but almost nobody is doing it. It’s hard, it’s not taught in programming classes or books or documentation(for the most part) and languages by and large punt on the issue.

                                                    I’ve read a few haskell learning sites, and a book, though I’ve never programmed anything useful in Haskell, and I’d never heard of smart constructors before, so even when it’s available, nobody apparently uses it.

                                                    Anyways, the goal here, from my perspective, is we desperately need to validate inputs, and store data with known constraints, and with rare exception, no code does this, and no commonly used languages make this remotely easy. so I think my point stands, constraint checking is still very much not done in the wild.

                                                    Thanks for teaching me about Smart Constructors! :) I <3 learning new things!

                                                    1. 8

                                                      Pascal also supports it:

                                                      type
                                                        Age = 0..200;
                                                      
                                                      1. 4

                                                        Smart constructors are definitely not in the “nobody uses it” category. They are one of the most pervasive patterns in strongly typed languages (especially ML descendants like Haskell or Scala).

                                                        1. 2

                                                          Smart constructors are just functions that build values of the required type, but perform some extra checks when the value is constructed…

                                                          It seems like you could employ the same smart constructor pattern in many languages. For the age example, you’d create a type for the valid age range and use this type elsewhere in your code:

                                                          class Age {
                                                              final int value;
                                                              Age(int value) {
                                                                  if (value < 0 || value > 120) {
                                                                      ...raise exception
                                                                  }
                                                                  this.value = value;
                                                              }
                                                          }
                                                          

                                                          This is along the lines of cgenschwap’s recommendation of preferring parsing over verifying.

                                                          1. 1

                                                            You’d have to make sure the setters and getters did the checking as well along with making sure there was never any access to the internal value other than by the setters and getters. Which is a lot more faff than having it supported natively.

                                                      2. 7

                                                        Aren’t you just talking about dependent types? I don’t think this is a situation where languages “fail” per se, but more that it is a technically difficult thing to do (while also having a language that is easy to use for regular programmers/programs).

                                                        Dependent types are just an extension of types – so they are certainly a good solution to the actual problem!

                                                        1. 1

                                                          Dependent types is one way to do this, though so far I’ve never seen that built into a language and usually a very complicated way.

                                                          But who uses dependent types? no languages really have them built-in, certainly no commonly available/used languages. Haskell, has Liquid Haskell and per pushcx smart constructors also does this, apparently. Are Smart Constructors built-in? It seems like they are, but my Haskell on this machine won’t compile it, but it’s also a really old Haskell, so that could be why.

                                                          I agree bolting constraints onto our existing type system(s) is very complicated and messy.

                                                          I’ve never seen any code out in the wild that does anything like this.

                                                          We all know we can’t trust whatever the user gives us, and the giant list of security vulns just enforce the point, and yet basically no language in common use has any nice sane way to handle these sorts of issues.

                                                          1. 3

                                                            If dependent types interest you, the answer to “who uses dependent types” is idris

                                                            1. 2

                                                              It is unfortunate that few languages have dependent types – I completely agree with you here. However, putting constraints on user input isn’t really an issue in my opinion. You should always parse the info from the user rather than verify it.[1]

                                                              Raw user input is stored in a Raw{var} type (which can’t be trusted) and is parsed (with constraints) into a {var} which can be used without fear. Of course, this doesnt guarantee the data is valid – your code might still screw it up – but this concept can just be used every time the data is mutated.

                                                              Dependent types are fantastic because they turn this into a compile-time guarantee. Otherwise I don’t really see the benefit of having a language feature like this at runtime, it is very easy to do yourself.

                                                              [1] https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/

                                                              1. 1

                                                                I think you and I are in complete agreement here about the end goal. I’ve read that post before, and while what they write is possible in Haskell, basically never used(from my very limited Haskell experience), in other languages it’s a giant PITA.

                                                                No popular JSON parser will let you add constraints to your parsing of JSON(that I’m aware of).. The most they will do is sort of let you say this must be an int64, or this must be a string. Obviously JSON is just one example here. I’m sure there is one or two very generic parsers that allow for constraints at parsing time, but I’m not aware of any in popular languages that make adding actual constraints easy, or even possible.

                                                                1. 1

                                                                  Certainly with strong, static typing this is easier to enforce (functions only take the parsed version of input). However every language can do this, it just becomes a matter of code hygeine. If JSON parsers dont let you add constraints then you just add another parsing step which can apply constraints.

                                                                  Sure this adds boilerplate and complexity, but I would argue that this is fairly easy to do.

                                                          2. 6

                                                            I’d argue that types are a terrible solution to the actual problem.

                                                            I’d argue the actual problem is, we need to check variables in our languages…

                                                            …we want age to be an integer between 0 and 120 or so. Certainly < 200. That’s what we actually want.

                                                            Let me make sure that I understand what you’re asserting here: you believe that there’s only value in type systems insofar as they can allow for constraining numeric types to a bounded range? And, all languages with static type systems which fail to do this are fundamentally failures?

                                                            1. 0

                                                              No, Types are awesome for compilers, to make things go fast and optimize their implementation. There is a reason types got invented in the first place. It’s hugely easier for a compiler to optimize number math if they don’t have to concern themselves with if it might also be a string “happy”.

                                                              I argue though that they are pretty terrible for end programmers, because they lack constraints. There is a reason Python and friends, who mostly ignore types are hugely popular.

                                                              I only picked age as a very easy, hard to dispute example, but I believe constraints should apply to all data types, not just numeric types. pay period date constraints on a timesheet would be another easy use-case that doesn’t use numbers, but dates.

                                                              PostgreSQL’s check syntax is what I’m after here, before anything can get stored in the table, it has to be verified for sanity. but PostgreSQL is the wrong place to do this, it needs to happen way out near the user, so it’s way, way easier to say “hey user, “first” is not a valid number in the range 0-100, since this is an input for age.” So when we go to store the variable in our program, we need to be able to parse it and constrain it and verify it’s sane before anything else can happen. We as an industry completely fail at this, for the most part.

                                                              1. 3

                                                                Yes and no. There is a lot of benefits to types, they help when writing code by pointing out bugs before the code is running (contrary to Python for example). Depending on the type model you even get theorems for free, i.e. you can prove theorems about functions just from it’s type signature! That’s pretty neat.

                                                                Another problem is the overly structured data you’re proposing: What if I’m 106 and want to register for your service? What if I don’t have a first name? What if I don’t have a street address and live “2 mi N then 3 mi W of Jennings, OK 74038”? What if, in a family tree program, a person wants to input a child which he had with his daughter?

                                                                I’m not saying no constraints is a better approach, but there are a lot of edge cases to be thought of, even for something as simple as an age or a name, which maybe shouldn’t be restricted at all.

                                                                1. 2

                                                                  There are new things being added to types to make it better, I agree, dependent types one of them. Like the OP post says, many type systems are hard to reason about.

                                                                  I agree gross errors are caught trying to shove a string into an int8 variable. But academic papers disagree over how useful types are in practice, it’s mostly just yelling opinions across the internet these days, as far as I can tell. Perhaps over time, academic papers will have better science involved and eventually figure it out, but I think we can all agree the answer , scientifically speaking, is murky at best[0]. That said proving theorems sounds pretty neat!

                                                                  Of course there are lots of edge cases with data validation/parsing, the known issues here, are multitude. I don’t have a street address most of the time. That doesn’t mean we should punt and give up on the problem, and let everything be free-form text fields. :) It’s very, very hard to reason about free-form text fields.

                                                                  Every application will likely need unique constraints around particular data, depending on the context, but I think sane defaults can be found for many use cases. In my OP I covered this already. We can argue where the limits need to be, but for most applications I think we can agree negative ages, and ages over 150[1] are probably sane defaults.

                                                                  If you buy into the parse vs. validate debate[2], we can’t even currently parse with any sanity. OpenBSD mostly punts on sane parsers and just assumes they are totally broken by sandboxing the parsers from the rest of the application(sshd, tcpdump as examples). I also mentioned in a different comment about JSON parsers as an example.

                                                                  0: http://danluu.com/empirical-pl/

                                                                  1: https://en.wikipedia.org/wiki/Oldest_people

                                                                  2: https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate

                                                            2. 5

                                                              Types, basically don’t do that, and practically every language ever written skips over any sane way to have limits on our variables besides basically int8, int16 and string.

                                                              I believe that Ada handles this in a good way. You can declare an age type that has a valid range of 0 .. 120. I don’t have direct knowledge of this, but AFAIK it uses a combination of static and dynamic checking. Those cases which are impossible to verify with a static checker are verified at runtime.

                                                              1. 3

                                                                Pascal and Common Lisp are two other languages with this feature, just for completeness. They’re called “subrange types” in Pascal.

                                                                1. 2

                                                                  Yep, Ada can do this. You can also use a regular integer type with contracts and use Spark (the Ada subset) to prove that the integer is in the range you expect it to be, thus guaranteeing that your program is free from runtime exceptions.

                                                                  Ada has some pretty cool features, it’s sad that nobody knows about them.

                                                                2. 4

                                                                  You might find Racket’s contract system interesting. Also this talk if you have ~40 minutes (assuming 1.5x speed).

                                                                  1. 1

                                                                    Neat! So I agree in non-popular languages, this is sort of only recently being addressed, to some degree or other, with varying degrees of implementation. Most documentation ignores the problem. I’ve never used Racket, but Haskell is the other commonly known language that has this ability to some degree, yet I’ve never seen any Haskell code in production that uses any of the features available. Is using the contract system actively used in Racket?

                                                                    I went looking through github, seems Pollen is popular for something written in racket, and it uses ‘provide’! [0]

                                                                    It seems however, it’s limited to modules only.

                                                                    0: https://github.com/mbutterick/pollen/search?q=provide&unscoped_q=provide&type=Code

                                                                    1. 3

                                                                      Yes, the majority of Racket code uses contracts extensively and they are also used in documentation. Some examples:

                                                                      provide is just Racket’s way of exposing bindings from one module so that they can be required by another. The contract library provides the contract-out provide transformer that attaches contracts to bindings when they are provided. You’re right that when someone uses contract-out, then that contract will only be enforced at module boundaries. However, that’s not the only way to attach a contract to a value. The same library also provides define/contract among other things so you can do stuff like this within a single module:

                                                                      #lang racket
                                                                      
                                                                      (define smallint/c (integer-in 0 255))
                                                                      
                                                                      (define/contract (f x)
                                                                        (-> smallint/c smallint/c)
                                                                        x)
                                                                      
                                                                      (f 42)
                                                                      (f 1024)
                                                                      

                                                                      Running the above yields:

                                                                      42
                                                                      f: contract violation
                                                                        expected: (integer-in 0 255)
                                                                        given: 1024
                                                                        in: the 1st argument of
                                                                            (-> (integer-in 0 255) (integer-in 0 255))
                                                                        contract from: (function f)
                                                                        ...
                                                                      

                                                                      Or even stuff like:

                                                                      #lang racket
                                                                      
                                                                      (define smallint/c (integer-in 0 255))
                                                                      
                                                                      (define/contract (f x)
                                                                        (->i ([in smallint/c])
                                                                             [out (in) (=/c (add1 in))])
                                                                        x)
                                                                      
                                                                      (f 42)
                                                                      

                                                                      That yields:

                                                                      f: broke its own contract
                                                                        promised: (=/c 43)
                                                                        produced: 42
                                                                        in: the out result of
                                                                            (->i
                                                                             ((in (integer-in 0 255)))
                                                                             (out (in) (=/c (add1 in))))
                                                                        contract from: (function f)
                                                                        blaming: (function f)
                                                                         (assuming the contract is correct)
                                                                      

                                                                      It’s been a while since I’ve done any Haskell, but I don’t think there’s anything quite like this in the language. I definitely recommend watching the talk I linked to because it explains some of these ideas in a visual way and much better than I could do here.

                                                                      I believe Clojure is also adopting some of these ideas now via spec, but I haven’t looked into it.

                                                                      1. 1

                                                                        <3 Thanks!! This looks very awesome! Go Racket!

                                                                        1. 1

                                                                          <3 Thanks!! This looks very awesome! Go Racket!

                                                                    2. 2

                                                                      I believe you’re looking for dependent types as in eg Idris: https://en.wikipedia.org/wiki/Dependent_type

                                                                    1. 2

                                                                      I want to see a language with a built-in mechanism for this kind of ambient context. Something strongly-typed, that allows application developers to supply context values that “follow” across things like async “callLater” type deferrals, as well as being useful for structural “container” hierarchies like eg something like a react context.

                                                                      But I don’t want it just as an alternative to explicit call parameters, I’d also like to be able to push/pop contextual information for debugging, which would be available when running in debug-mode alongside a regular stack trace.

                                                                      1. 2
                                                                        1. 1

                                                                          Common Lisp has special variables, which are exactly for dynamic context. Regarding type, you can always just DECLAIM or PROCLAIM a type for the special variable.

                                                                        1. 15

                                                                          Every now and then I’ll visit a website, notice there’s nothing new since I last visited and then absent-mindedly hit cmd+L, type the website URL in and visit that same website again within the span of 30 seconds. It’s, uh, a problem.

                                                                          1. 3

                                                                            SELECT ... FOR UPDATE SKIP LOCKED was introduced in Postgres 9.5 to solve (some of) these issues. With it, dequeueing a set of jobs looks like this. No need to hang on to a long running transaction either as long as you have some sort of process to requeue dropped jobs.

                                                                            1. 1

                                                                              The article was published in 2015.

                                                                              1. 1

                                                                                Oh, I wasn’t aware it got fixed in Postgres 9.5. However, I still do like the article as it goes into depth on various intricacies of Postgres and gives a good insight on how things work. A good, albeit outdated, learning material.

                                                                                1. 1

                                                                                  It doesn’t however solve the “dead tuples problem” mentioned in the article. Correct me if I’m wrong.

                                                                                  1. 2

                                                                                    That’s right. That’s an issue for any table that has high churn in terms of updates and deletes and that’s something you’d work around by tuning auto vacuum (to run frequent, fast vacuums – the goal being predictable performance over very high throughput) and things like the fillfactor for the jobs table specifically.

                                                                                    Another option I’ve heard about but never tried is having two jobs tables and periodically switching between them and truncating the other. Presumably, that’d involve turning off autovacuum for the two tables since you’re effectively performing a full vacuum “in software” at every switch. Personally, I think that if you get to the point where you need to do something like that, it’s probably time to move on to a real message queue (like RabbitMQ).

                                                                                1. 4

                                                                                  Interesting. Any experience with cross-compilation? Apparently there’s:

                                                                                  https://docs.racket-lang.org/raco/cross-system.html

                                                                                  But doesn’t appear to have a lot of convenience around it (eg: no “raco add-target win64; easily compile for win64”)?

                                                                                  1. 2

                                                                                    I’d never tried it before, but it seems to work great. I downloaded a build of Racket CS for linux to ~/Downloads/racket and then ran the following:

                                                                                    $ racket -C -G ~/Downloads/racket/etc -X ~/Downloads/racket/collects -l- raco exe -o app app.rkt
                                                                                    $ file app
                                                                                    app: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=7fbdabb07e54037e2ede76daf74d7ecfdcc5b486, stripped
                                                                                    
                                                                                    $ racket -C -G ~/Downloads/racket/etc -X ~/Downloads/racket/collects -l- raco distribute dist app
                                                                                    $ tree dist/
                                                                                    dist/
                                                                                    ├── bin
                                                                                    │   └── app
                                                                                    └── lib
                                                                                        └── plt
                                                                                            ├── app
                                                                                            │   └── exts
                                                                                            │       └── ert
                                                                                            │           ├── r0
                                                                                            │           │   └── example.txt
                                                                                            │           ├── r1
                                                                                            │           │   └── error.css
                                                                                            │           ├── r2
                                                                                            │           │   └── dh4096.pem
                                                                                            │           └── r3
                                                                                            │               └── bundles
                                                                                            │                   ├── es
                                                                                            │                   │   └── srfi-19
                                                                                            │                   └── srfi-19
                                                                                            └── racketcs-7.7.0.9
                                                                                    
                                                                                    12 directories, 7 files
                                                                                    

                                                                                    Someone could definitely make this easier by providing a package to add a raco command like the one you’ve suggested.

                                                                                  1. 6

                                                                                    Thank you for writting this.

                                                                                    With regards to distribution options with racket,

                                                                                    Is there a way to ship platform independent, but hard to reverse engineer ‘precompiled’ binaries ?

                                                                                    For example for deploying closed-source software into a private enterprise (where the servers are controlled by customers)

                                                                                    1. 6

                                                                                      Racket CS modules are compiled to native code so you can ship those around and reverse-engineering them would be about as hard as reverse-engineering any other natively-compiled code. You’d have to compile these separately for every platform you want to target. I believe there’s a way to ship modules in “linklet” form, which is platform-independent, and Racket CS will convert those to native code on first run, but I’ve never tried it.

                                                                                      Here’s the output of running raco decompile on the compiled version of the app in the post: https://gist.github.com/Bogdanp/64a7ef659d6376fff918e3cb23518f9b

                                                                                    1. 2

                                                                                      Shameless plug: there’s also nemea which I wrote and use for all my stuff.

                                                                                      1. 1

                                                                                        cc @soapdog. This seems relevant to your Gemini browser!

                                                                                        1. 13

                                                                                          https://defn.io - I mostly write about Racket.

                                                                                          1. 3

                                                                                            Your blog helped me a lot with Racket. Thanks a ton.

                                                                                            1. 2

                                                                                              Subscribed to your RSS feed a while ago, I learned a lot about racket thanks to you!

                                                                                              1. 2

                                                                                                I remember your post about using Racket to write an e-commerce platform. I thought that was awesome!

                                                                                                https://defn.io/2019/08/20/racket-ecommerce/

                                                                                                1. 1

                                                                                                  @jee @Artemix @soapdog glad to hear it! Thank you!

                                                                                                1. 8

                                                                                                  Finishing up my talk and tutorial in preparation for Racketfest next week!

                                                                                                  1. 9

                                                                                                    This is really good news. I will admit I’m a bit disappointed that it’s currently at best about as fast as the original. I’m glad to hear they have a plan to help address this though. The future does look bright.

                                                                                                    1. 1

                                                                                                      currently at best about as fast as the original

                                                                                                      Based on the note about the relative performance of indirect function calls and continuations, I think the impact for Real™ applications is probably understated. Anything that makes heavy use of exceptions and threads (both implemented using continuations), should be faster than in Racket BC.

                                                                                                      To test this, I ran some local perf. tests to measure the difference in throughput of the web-server-lib between CS and BC and I saw a significant increase in throughput under CS. If I find some time next week, I might clean up and publish the results.

                                                                                                    1. 5

                                                                                                      Does Chez compile to native code like Chicken? Would it be possible to produce statically linked binaries with musl or similar for the C runtime?

                                                                                                      1. 1

                                                                                                        Yes to the first question. I think the second should be possible, but I’m not sure what the level of effort required is.