Threads for jnb

  1. 10

    This has been done — or rather, independently discovered as a technique — many times, and appears to be an interesting and powerful technique. I researched this possibility some time ago and categorised “C in S-expressions” implementations into two types:

    1. “C” code in S-expression form is interpreted by a Lisp interpreter in an environment with appropriate symbols. Execution of the program constructs an AST on which further transformations can be performed, or causes C code to be output. (This allows you to use macros and generate code programmatically, etc. You’re generating C by executing a Lisp program.)

    2. “C” code in S-expression form is parsed and analyzed by a program, but the program is not executed as a Lisp program. The parser may still be implemented in Lisp to take advantage of Lisp’s affinity for S-expressions and macro processing. (Essentially, the S-expressions are input to a compiler supporting a custom — but non-Lisp — language designed for translation to C.)

    In any case, examples of S-expression-to-C systems (some of these links may have dropped off the internet by now):

    I have some very rough and unpublished notes about this sort of thing and other ‘preprocessing’ technologies which people might find interesting: https://www.devever.net/~hl/lr

    Another use for S-expressions or DSLs built in full-on Lisp environments is typesetting. I have my own typesetting system which is basically just a set of macros and functions in a Guile Scheme environment, which generates an SXML S-expression in a custom XML schema then translated either to XHTML or TeX, or fed directly into ConTeXt using its XML support. And again, I’m far from the only one to discover that Scheme makes a nice typesetting environment (nicer than XML, certainly); you can find many such Lisp-based typesetting systems. One nice feature of these systems allow direct lexical references to terminology, preventing typos:

    ;; Define terminology.
    (dt pci-device "PCI device" "A PCI, PCI-X or PCI Express device.")
    (define (some-chapter)
      (p "A "pci-device" supports blah blah blah:")
      (p "..."))
    

    (This takes advantage of the fact that the S-expression syntax doesn’t mind symbols appearing next to quoted strings without whitespace; I tend to tell my eyes to “forget” I’m looking at a string constant, and read this code as though the quote marks referencing a term are opening a string, rather than ending a string and starting a new one. Lawyers seem to like to put terminology in quotes, so this is a vaguely familiar kind of ‘English’ to read.)

    1. 3

      This has been done — or rather, independently discovered as a technique — many times

      Thank you for the excellent summary. To your list, I’ll add: sph-sc built with Guile.

      1. 1

        Regarding the typesetting, there is also something like Pollen, implemented in racket. Is there a reason you prefer your own implementation to it?

        I really love that approach to using sexprs for generating html, it just works so well together.

        1. 2

          Nope, I just decided to make my own to do exactly what I want. I subsequently learned of Pollen and many similar systems, which I’m sure are great. It’s a very powerful approach and SXML is probably the most pleasant way to generate and work with XML.

          If people want an interesting rabbit hole to dive down, take a look at ConTeXt and its XML support. There be dragons!

      1. 21

        For such a use case I’d really recommend tqdm, which does basically all of that (and more) automatically.

        https://tqdm.github.io/

        1. 4

          A progress library with custom merch. This is where we’re going?

          1. 3

            I recommend this lib in general too! Main problem is just having the foresight to have it installed in whatever env I’m working with. Of coures tqdm is Python-specific but you can replicate stuff like that in 20 lines or so if you want to.

            1. 1

              Yes, I think especially because it’s a nice-to-have dependency it’s often not included in a dependency list or environment explicitly. It also “feels” a bit heavy-weight, although it hardly adds a performance penalty to the execution itself.

              While tqdm is python-specific, if you go via the command line, then you can use tqdm as an executable, eg from the website:

              seq 9999999 | tqdm --bytes | wc -l
              

              Otherwise the stuff you outlines in the post is more generally applicable.

            2. 2

              There is also the language agnostic pv, but it’s limited in what it can output.

            1. 3

              It looks really neat, although I have to admit as a person that doesn’t compose music I wouldn’t exactly know what to use it for. I can certainly imagine that the traditional music notation is limiting, but I don’t quite understand how a written score (which looks beautiful!) would be translated to actual audible music.

              Would love to see more of this in use, but I suppose it’s a rather young project so it might need more time to build up a larger gallery of scores.

              1. 7

                Non-standard notation can more clearly convey a lot of things that are not easily expressed in the standard one, for example compositions that makes use of improvisation or particular aural effects that are not heard very often in conventional music. It’s almost always accompanied by written notes explaining the notation more or less clearly depending on the effect that the composer wants to convey.

                For example R. Murray Schafer experimented a lot with graphical notation in order to render particular soundscapes; his scores make some use of conventional notation but also include lots of weird ad hoc notation, and if you listen to them you’ll certainly understand why they had to be written that way and how impossible it would be to write them using only conventional notation. There’s a lot of videos by the Vancouver Chamber Choir on youtube that also show the scores like this one: https://www.youtube.com/watch?v=dzUXzu7JYFc

                1. 2

                  I’ve had several conversations with Sam Aaron (author of SonicPi) about this. Western musical notation today largely dates back to the Gutenberg press. It was a compromise between the various hand-drawn notations that could be printed with hand-carved printing blocks in a single colour (a lot of the older notations used colour to convey meaning). We’ve kept this notation for centuries, not because it is a good way of communicating semantics from the composer to the performer but because it works with a Gutenberg-era printing press.

              1. 1

                A more flexible approach to this is shackle. I have been meaning to try this out for quite some time (as I share the opinion in the blog post), but I haven’t come around to it yet.

                1. 4

                  This a long-standing discussion. You may want to read Paul Prescod’s XML is not S-expressions to see why this is subtley wrong and not really the best of ideas.

                  1. 2

                    Thank you for sharing that link, that was the first time I’ve noticed someone speaking out against using S-expressions for HTML/XML. I’d disagree and say that S-expressions are easier to work with and there exist something like Scribble for documentation in Racket.

                    1. 1

                      There are also strong mappings like SXML

                    2. 2

                      I’ve been generating HTML from S-expressions for decades and I’ve got to say none of the downsides mentioned in the article have ever come up. Being able to use tools like paredit for authoring documents is a huge leap over the editing tools I’ve tried that are available for HTML.

                      Granted I’m typically using an s-expression flavor which allows for key/value data structures with {} since I do kind of agree that forcing associative data into parens isn’t very clear.

                      The XML one defaults to treating random characters as text, not as markup.

                      This might be considered an advantage when you’re just manually authoring documents directly, but when you’re writing application code that emits HTML (which is the case in the original article) this property is very bad.

                      In the S-expression version, the software does not know that there is a missing parenthesis until it gets to the end of the document. […] Smart editing tools can help but the cannot solve the problem.

                      Incorrect; paredit and other structural editors completely solve this.

                      the canonical documentation of the Scheme and Lisp standards is maintained not in S-expression syntax but in LaTeX

                      This is a simple case of bootstrapping; you can’t define something using the thing that’s in the process of being defined. Obviously bootstrapped lisps do exist, you always have to have some starting point for a bootstrap.

                      DTDs, RELAX and XML Schema define constraints on individual instances of XML documents.

                      I guess Clojure spec didn’t exist when this document was written, but it does now.

                      1. 1

                        Not having to escape quotes in XML was the killer feature for me over Sexps.

                        (Well, that and inline markup.)

                      1. 2

                        This approach is also one of the core ideas behind a book-publishing domain-specific language called Pollen, which is written in Racket. The documentation also dives a bit into the relationship between S-expressions and HTML.

                        1. 2

                          Trying to eat healthier, to become fit again. How do you guys find inspiration to make your meals at home, with few ingredients?

                          1. 2

                            For me, I got quite a range of spices since they don’t go bad and then you can get write far with dishes, a curry is almost always just a few vegetables (and rice) away. Then there is the evergreen pasta, making tomato sauce from pulp is quite easy or make pesto yourself, that stores reasonably well and can be frozen. Other than that, rice cooked in broth with eggs cracked into the boiled rice is super easy to make and requires five or less ingredients.

                            In general you don’t have to cook every day and reheating things can save you a lot of time, too. For me it’s a nice process to do, you can try out new stuff and refine recipes once you figure out what you like.

                          1. 3

                            Start migrating from Arch to NixOS.

                            1. 2

                              A write up of that process would be interesting!

                              1. 2

                                Hope all goes smooth and well. I went through the same migration around 6 months ago. Haven’t looked back!

                              1. 3

                                I also have built my blog with pollen and really enjoyed the process of doing so. One issue I frequently ran into is debugging while developing a new feature, that experience is not yet fine-tuned for a developer, especially one that doesn’t have extensive experience in racket. But asking in the forum can be very helpful and did help me in more than one case.

                                In general I don’t like the pollen server since it doesn’t include anything for live-reloading, which I find very important. As such, I went a completely different route and now use redo together with watchexec and a Firefox plugin for reloading. Specifying the dependencies is a bit awkward since racket does not work well with calling redo from racket and as such my build tool is currently a hodgepodge of shell and racket scripts, but it works on my machine.™

                                All in all, the idea of using sexprs for representing a blog post is simply beautiful and I wouldn’t want to miss it anymore.

                                1. 2

                                  I finally got a keyboard with US keys at work so I don’t have to switch back and forth between the German layout and the American one anymore. I tried to type more accurate (mostly pressing the keys with the correct finger) and noticed that the pinky and index finger have to do quite a bit more work than the others.

                                  In general I’m interested in more ergonomic layouts, especially if the thumbs can be put to better use. But I can hardly stand the backslash having a slightly different location on the keyboard at work compared to mine at home, so I am afraid that the switch between more uncommon keyboards will be even worse. How do people deal with this in general?

                                  1. 2

                                    I finally got a keyboard with US keys at work so I don’t have to switch back and forth between the German layout and the American one anymore. I tried to type more accurate (mostly pressing the keys with the correct finger) and noticed that the pinky and index finger have to do quite a bit more work than the others.

                                    In general I’m interested in more ergonomic layouts, especially if the thumbs can be put to better use. But I can hardly stand the backslash having a slightly different location on the keyboard at work compared to mine at home, so I am afraid that the switch between more uncommon keyboards will be even worse. How do people deal with this in general?

                                    I typically just use my native (Swedish) layout and make sure to use a programmable keyboard controller. That way I can have the exact layout I want (utilizing my thumbs more, custom layers with symbols et c) without having to switch layouts all the time. QMK is really nice.

                                    And if I can’t use a custom keyboard, I just use ANSI layout and switch between layouts when needed.

                                    1. 2

                                      I use:

                                      • a MacBook Pro keyboard that has half-height T-arrow, weird mutating function keys in a touchbar, and a Fn key in the left corner.

                                      • a conventional 90% board with a numlock key to switch between numpad and cursor/text keys, and a Fn key that does different things near the right-side alt/ctrl.

                                      • an X-Bows Knight, columnar ergo QMK board with a weird split section and a messed up far-right hand section, customized significantly with QMK; many Fn keys.

                                      My hands touch the board and move to the right positions. It just happens. They’re all basically QWERTY, though.

                                      1. 2

                                        I tried different keyboards, and have an ErgoDox but I need to use standard keyboards from time to time so the one I’ve been using for the last couple of years is a Dygma Raise that has thumb keys, but is otherwise standard and when you join both sides everything is where you would expect it to.

                                        One thing I was going to miss was the cursor keys, so while I waited for it to arrive (it was originally crowdfunded so it took a while) I prepared by mapping the CapsLock key to work as Fn (layer shift) key with cursor on I,JKL, Home/End on UO, PgUp/Down on YH, and it was so convenient that I use those keys even in full-size keyboards. Now I have the Fn key in thumb keys at both sides.

                                      1. 4

                                        There are many similarities between the two, though I don’t have any experience with Nix to comment further here.

                                        I don’t understand why one would use Guix without trying Nix first.

                                        1. 9

                                          What I gathered is that the author has a preference for Scheme and thus also for Guix, which I think is a sufficient argument.

                                          However, I’d be very interested in reading an in-depth comparison between the two projects, as they target the same niche and are quite related.

                                          1. 3

                                            This is basically my position. I’ve used Nix a couple times but Guix is preferable.

                                          2. 7

                                            Early on Nix had the opposite problem. You would ask it to install firefox, and completely unprompted it would install the adobe flash plugin to go with it. They told me if I wanted firefox without that awful shit to install a separate “firefox-no-plugins” package or something ridiculous like that.

                                            It hasn’t done that for a while, but it’s taken like a decade to recover from the lost trust. I couldn’t handle the idea of running an OS managed by people capable of making such a spectacularly bad decision.

                                            1. 2

                                              Doesn’t that depend on your affinities? If you like lisp languages then Guix seems like a logical choice. I guess it also depends on what software you depend on in your day-to-day activities.

                                              1. 5

                                                There is more to consider here:

                                                Guix is a GNU project with the unique lenses and preferences that come along with that. It is also a much smaller community that Nix, which means less packages, less eyes on the software, and I would argue also less diversity of thought.

                                                I personally prefer Scheme to Nix’s DSL-ish language, but as a project I think Nix is in a much better position to deliver a reasonable system of such levels of ambition.

                                                It also frustrates me that Guix tries to act like it’s not just a fork of Nix, when in reality it is, and it would be better to embrace this and try to collaborate and follow Nix more closely.

                                                Unfortunately the GNU dogma probably plays a role in preventing that.

                                                1. 4

                                                  It is also a much smaller community that Nix, which means less packages

                                                  If you convert Nix packages to Guix packages, you can get the best of both worlds in Guix. But that’s admittedly not a very straightforward process, and guix-import is being/has been removed due to bugginess.

                                              2. 2

                                                I’m aware of both, but I tried Guix first because people I know use it, I like the idea of using a complete programming language (even if I dislike parens), and the importers made getting started easy. Also guix is just an apt install away for me.

                                              1. 1

                                                I’m getting more and more interested in Guix (and Nix) as they keep getting brought up around here quite often nowadays. The first post I read was packaging pytorch for Guix (lobsters discussion), which turns out to be astonishingly complex. Since I only recently set up my new PC I don’t think I’ll switch anytime soon (mostly because I don’t want to deal with yet another package manager), but possibly for the next time I’m going to install a distro I’ll keep it in mind and try to evaluate the project.

                                                Discussion for this on HN.

                                                1. 3

                                                  I clicked through to the Speck system since I haven’t heard of it before. Its package description looks so much more sane than nix’s: https://spack-tutorial.readthedocs.io/en/latest/tutorial_packaging.html#id7

                                                  I wonder if we can have some nicer “compiles to nix” alternatives in the future.

                                                  1. 2

                                                    You don’t need to change your OS to try nix or guix. They can be installed on other OSes.

                                                  1. 2

                                                    It saddens me when open source projects/languages move to a proprietary solution for community support. It makes me reluctant to sign up and I worry that the project not owning the data will become a problem later in the future. I feel similarly about Julia, which has the same approach. Maybe I just don’t understand the advantage of going into that direction. Also those websites do not work without Javascript, which I don’t understand why that is a requirement.

                                                    1. 10

                                                      I think this is using the Discourse discussion platform, which is open source: https://github.com/discourse/discourse

                                                      1. 2

                                                        Well, now my statement makes a lot less sense, for some reason I thought this was the same as disqus. Thank you for pointing it out!

                                                      2. 6

                                                        Well, the main community links are to the mailing lists: https://lists.racket-lang.org/

                                                        Understandably many people prefer Discourse to mailing lists though.

                                                        Also those websites do not work without Javascript,

                                                        Discourse instances usually also support a mailing list mode.

                                                        1. 2

                                                          I did not know this, too. Although I don’t get why the text content cannot be rendered in a browser without Javascript it’s nice to know there is at least some alternative.

                                                      1. 3

                                                        I did not expect the story from the title, but nevertheless enjoyed it. I am also thinking that the Telegram API is not a bad way at all to have interactive cross-platform content/media. It’s nice to see that other people also have that impression. Although I am wondering whether there are any drawbacks to it that I am not seeing. Would love to hear other opinions on this!

                                                        I have used a Telegram bot in the past in order to notify me of experiment results, which was a great way to know when a model has finished training or just generally to have the results at hand instead of extracting them with sftp from the server. Although admittedly this isn’t the most interactive process since it just send me an unsolicited image, but it worked surprisingly well!

                                                        1. 7

                                                          I think the idea they detail last in the article is pretty big. Having the vulnerability information as part of the language server and at the granularity of the function level is an interesting approach in making developers more aware of security vulnerabilities. Because in my experience this information is not (as) easily available anywhere, at least for most languages that I use regularly.

                                                          1. 4

                                                            I partially disagree with the “quantity over quality” advice regarding topics one is passionate about. There’s so much content now that posting often hardly helps one stay visible. Deep topics do require deep posts, and for practicing writing it may be better to take small topics that wouldn’t make big posts anyway.

                                                            There’s an inherent problem with the blogging paradigm, that posts are complete and if you have something new to say, you write another one. It’s easy to see where it’s coming from: there are no protocols whatsoever for announcing content updates, and everything from RSS/Atom to ActivityPub is focused on notifying people about new content only.

                                                            But perhaps there should be. Or we should be ok with making write-ups on topics we are passionate about living, updatable pages rather than fire-and-forget blog posts, and be ok with having to promote them outside of the blogging paradigm.

                                                            1. 3

                                                              I thought about this as well recently and think it might be feasible to try and generate some sort of diff of an article and use that as another feed element. This could then link to a chapter of an already existing blog Post which has been added. I’m not yet sure where the issue will be yet as I haven’t tried implementing it yet, but it definitely makes the feed generation a lot harder than it currently is.

                                                            1. 1

                                                              Another related things that I only found out because I got to talk to the people working on it is SCION. Unfortunately I don’t know much about this domain and how those two things compare to each other.

                                                              1. 4

                                                                I think that the human-level intelligence is also partially a way to enable more people to “access” APIs, especially the people with less affinity towards technology.

                                                                Just as an example, consider ordering a pizza. If you are able to order a pizza via a website I have never not seen the option to add custom text regarding the order. You simply will have people that do not have their needs met by the ordering mask or do not find the option they really want. In this case it can make sense to have a computer attempt to decipher the customers wish.

                                                                This could also be applied to ordering via phone, which is still very common in Germany. This would be another avenue where voice recognition could make sense and improve efficiency. But ultimately, those problems are intricately connected to UX and not necessarily to fancy ML models, too.

                                                                In the end these types of ML models reach more people and it’s important that they do not feel left behind, which can cause rejection and opposition to modern technology. That is a very social problem and I am not convinced that there is a technological solution to it.

                                                                1. 3

                                                                  I think I agree with this, it might be that the “left behind” angle is one I don’t consider enough when thinking about this, ultimately the vast majority of people, can’t use google maps, heck, even many professionals like cab drivers which would benefit immensely from it.

                                                                  I guess I’m not sure how much of this can be fixed by “education” once technology becomes to useful to be ignored, but it’s the above google map example that made me hedgy on this, after all it’s a prime example of a life-transforming technology that many people just ignore.

                                                                  Maybe in some cases it’s just more fun to live life without technology? I’ve heard this argument made by literate people that preferred using paper maps and compass navigation just because it was exciting. But under that scenario I’m not sure simpler UX would help, or that any help is needed per se

                                                                  1. 8

                                                                    I have a personal theory that many people firmly believe, in a deep, maybe even unconscious way, that they are entitled to stop learning new things at some point in their lives. It’s something I’ve been noticing more and more with the pandemic cutting off so many “non-tech” ways of doing things.

                                                                    1. 3

                                                                      I have noticed this too. It really pains me. I personally favour the hypothesis that our education systems are to blame. All humans start off with curiosity as a primary motivator, perhaps the primary motivator after survival stuff. It seems there is a strong correlation between attending a few years of school and this no longer being a strong motivator for many people. As far as causation goes there are reasons to favour this hypothesis - the degree to which curiosity is actively suppressed in most education systems, and also the degree to which education is made to be associated with bland, boring and generally unpleasant hard work, rather than discovery. Unfortunately there is not a lot of good data regarding well-educated individuals that have not been subjected to a classical western style education. There are alternative systems like Montessori but there is no clear way to quantify cognitive ability, we use western school performance for that in most instances. The Finnish school system is an interesting case study though.

                                                                      This may all seem off topic, but to tie it in to the main theme: It is actually trivial to create a human level intelligence. Many people do it by accident when they are drunk. This makes creating them artificially doubly redundant. I think that parallel to improving inhuman AI as the article suggests, we should be looking into improving the human level intelligence we have easy access to, which is mainly a software problem. Most of the software that is widely available for the human brain is decades out of date and designed for applications that are no longer needed.

                                                                      1. 1

                                                                        To some extent though this might be a good conservative instinct.

                                                                        Don’t get me wrong, I like being in some “privileged” 0.01% of the world’s population that knows the technology well enough to make 6 figures writing stuff on a laptop, I’m sure most people here feel the same way.

                                                                        But if I had to choose between being in the top 20% or in the bottom 20% I’d choose to bottom.

                                                                        Technology very quickly becomes a tool that most people end up using to fulfil destructive patterns more than anything.

                                                                        That’s not to say it has no uses, it does, but it’s way too easy to get into pitfalls if you spend too much time on computers and don’t understand how good certain programs are at manipulating your preferences.

                                                                        1. 1

                                                                          I’ve noticed it myself with myself. After 30 years in the computer industry, I’m just sick and tired of having to relearn the hot new development methodology. Again. For the umteenth time. Can’t things just settle down for a few years? It seems as if the singularity has hit and no one can keep up.

                                                                        2. 3

                                                                          I suppose to some people it is more fun. I write my day-to-day plans still on paper because I prefer it to electronic bookkeeping. The battery of a sheet of paper cannot run out, although my phone hardly does either these days. But there still is something that makes me prefer to write them by hand, it also helps me remember what I actually wrote down. To some extent it is also control over the medium; I know that I am responsible if I lose it or it somehow is destroyed. I guess the aspect of control and being in charge can also translate to navigation or in general other aspects of life (although I have to agree with you, navigating via online maps is much better).

                                                                          Potentially education could help with furthering usage of technology. But it could also be that most people are just stubborn and want to continue with their way of life indefinitely. Depending on how big that share of people is, it is important to make new non-human intelligence backwards compatible, so to say. Then once most people stop relying on the old technology (die out?) it can be deprecated.

                                                                      1. 2

                                                                        The project looks really nice, but whenever I look at trailer data I reach for pandas with the occasional plot, which suffices for my needs most of the time. While I like the idea of VisiData a lot, it’s value just doesn’t seem to justify learning how to use the software for me.

                                                                        1. 5

                                                                          I dunno, it’s pretty easy for basic use cases: arrow keys, Shift+F for freq table, ‘q’ to quit. Seems like it would be less effort to try it out than to post a comment about why it doesn’t seem worth it to try it out.

                                                                          1. 3

                                                                            I use VisiData to massage csv and excel files before importing them elsewhere. Best tool I found so far for this use case.

                                                                            1. 2

                                                                              Thank you for VisiData. I’ve only discovered it in the past 18 months, I’m certainly no expert user, and the 45 minutes I’ve spent squinting at youtube demos and cheatsheets has conservatively saved me 3 or 4 days worth of scripting against crappy CSVs and workbooks.

                                                                              Also, I feel like, lurking somewhere around the corner, I could integrate this with the django shell and similarly shorten my approach to getting data into a form I can help business people manipulate and use. I think I’m super close to finding that path but not there yet.

                                                                              1. 1

                                                                                This could be pretty easy, depending on how the django shell is designed. Ping me if you’d like to talk or try to bang through it some afternoon!

                                                                          1. 1

                                                                            I found this article interesting as it seems that this way you can strive a good balance between having images on your site and reducing the size of the website.

                                                                            Additionally, for the use case of adding a picture to the about page, it adds a bit of privacy to an image as the dithering obscures some details in an image. As such I feel less hesitant about putting an image of myself on an about page. The inspiration comes from Matthew Butterick’s Beautiful Racket, although his rendtiion seems like it was not automatic dithering.