Threads for yogsototh

  1. 4

    I remember a course from a Ph. D Student of Alain Colmerauer the creator of Prolog. She added a new method to solve a new class of problems with Prolog. She explained her work for a whole month to us until the last day when she concluded by the complexity of her algorithm which was a tower of 2 from the size of the problem.

    So you could only use her method to solve this class of problems up to size 5.

    1. 12

      Once upon a time, we invented regular expressions as a comfortable midpoint between expressive power and efficiency on 1970s computers. We invented tools to search for regexes (grep), tools to automatically edit streams based on regexes (sed), programming languages designed to run code when a regex was matched in the input stream (awk) and even interactive editors with regexes as first-class UI primitives (vi).

      Here in the 21st century, text processing has advanced a long way. In particular, parsing context-free grammars no longer requires days of swearing and wrestling with shift/reduce errors; you can write grammars nearly as succinctly as regexes, and more readably. Using packrat parsing or Earley parsing or some other modern algorithm, I’d love to have a suite of text-processing tools like grep, sed, awk and vi based on grammars rather than regexes.

      1. 4

        you can write grammars nearly as succinctly as regexes, and more readably

        Do you have any links or examples? I struggle at this!

        1. 3

          In most cases, I end up doing text processing using (neo)vim macros rather than regexp. It can run surprisingly fast even on large datasets (in headless mode or with lazyredraw).

          Feels like a very modern/ergonomic/incremental/less abstract approach compared to regular expressions.

          I do like the premise of clearly defined grammars, however! Could compound nicely with macros, too.

          Made me think of this recent Structural Search & Replace plugin that uses treesitter (basically grammars under the hood).

          Now that I think of it, treesitter is essentially a database of grammars that can be used for data annotating and processing. 🙃 I guess the next step is to have a more on-the-fly way to describe and use these things.

          1. 2

            semgrep?

            1. 2

              I kind of got that feeling back when I played with turtle:

              You can see some examples in the comment here: https://github.com/Gabriella439/turtle/blob/main/src/Turtle/Pattern.hs

              hackage seems down right now, but there is a tutorial there: https://hackage.haskell.org/package/turtle/docs/Turtle-Tutorial.html

              1. 1

                That’s interesting. I think you’d be looking at matching/querying an AST-like structure?

                For matching elements of specific kinds of tree-like structure, we have jq and xpath.

                Is that the kind of thing you mean (but perhaps for general grammars?)

                If not, how do these differ from your thoughts/vision?

              1. 31

                Nix for normies:

                NixOS for normies:

                • Working graphical installer (almost there last I tried)
                • Consolidate tools. See nix build vs nix-build, for example
                • Either stabilise flakes or get rid of them

                nixpkgs for normies:

                • Quality assurance up the gazoo. If it isn’t formatted, linted, tested, and documented, it shouldn’t go into an enormously important repo like this.
                • Simple instructions for new contributors, like which branch to base new stuff on
                1. 2

                  I would add to NixOS for normies:

                  • A straightforward way to run binaries without making a derivation or any patchelf business

                  That’s what broke the camel’s back for me. One too many not-packaged binary where writing a derivation was cumbersome, and the patchelf business too much complexity.

                  Though this was in 2018, things might be better now.

                  1. 1

                    and the patchelf business too much complexity.

                    I’ve found that buildFHSUserEnv works for me in some cases. It took me literally years too understand how to use that, due to the usual docs problem. Basically, I have this in my config

                    https://github.com/matklad/config/blob/49ace0891e4566a3199dba08ba49e909ed3ad9ad/hosts/default.nix#L19-L36

                    This adds a fhs utility to my PATH, which drops me into a “normal linux”. It’s up to me to add all the required libraries here (and that huge block of packages is 99% copy-pasted from somewhere), but it gets the thing going:

                    $ cd downloads
                    $ aunpack sublime_text_build_4143_x64.tar.xz 
                    $ ./sublime_text/sublime_text 
                    exec: Failed to execute process '/home/matklad/downloads/sublime_text/sublime_text': The file exists and is executable. Check the interpreter or linker?
                    $ fhs
                    (fhs) $ ./sublime_text/sublime_text
                    # gui fires up and seems to work
                    

                    Not saying that this is an acceptable solution, just sharing something which was useful for me after I’ve finally grokked how to hold it

                    1. 1

                      There’s so much in nixpkgs, I think I’ve only done this once. Of course, YMMV.

                      1. 1

                        I never tried it, but there’s nix-ld.

                        The readme also mentions nix-autobahn and nix-alien, which seem to try to solve the same problem.

                      2. 1

                        Or Nix but localized so I stop getting errors like

                        error: ‘optimize’ is not a recognised command

                        1. 1

                          Isn’t https://devenv.sh the Nix for normies, or at least a very good first step?

                          1. 2

                            Looks like too much magic under the hood. Like languages.python.enable = true;, that’s all well and good, but how nasty does it get when I need a specific version of Python? Or when I need to start generating devenv.yaml because it’s so complex?

                        1. 7

                          I’m especially curious about existing implementations anyone uses here on Lobsters. My Lobsters-based POSSEs have been manual so far; the only automated POSSE-ing I do is via a shell script that calls toot to post to the Fediverse.

                          1. 3

                            I use brid.gy a lot

                            1. 3

                              I am using my own[1] to cross-post to mastodon[2], pleroma[3], pinboard[4] and twitter[5] accounts. The last via https://crossposter.masto.donte.com.br/.

                              Currently rewriting to be an activitypub instance on it’s own[6].

                              [1] http://mro.name/shaarligo
                              [2] https://digitalcourage.social/@mro
                              [3] https://pleroma.tilde.zone/@mro
                              [4] https://pinboard.in/u:mro/
                              [5] https://twitter.com/mrohrmoser
                              [6] https://seppo.app/en/

                              1. 2

                                I use this for my blog. I post to my blog and then have it poke another service that spreads out notifications. I wrote about it in my most recent talk.

                                1. 1

                                  I personally self-host an espial instance (https://github.com/jonschoning/espial). Then I also self-host a node-red instance (https://nodered.org) where I created 4 workflows:

                                  • blog2twitter
                                  • espialBookmark2twitter
                                  • espialNopte2twitter
                                  • espialBookmark2pinboard

                                  You get the idea, the workflows looks for RSS feed from my blog, my public espial bookmarks and my public espial notes. On any new item, it is published to twitter (and the bookmarks are copied to pinboard).

                                  More details:

                                  https://her.esy.fun/posts/0004-how-i-internet/index.html

                                  1. 1

                                    I did some manual POSSE to Lobsters in the past for comments, and when I’ve posted blog posts I’ve made sure I’ve recorded the syndication / cross post to Lobsters, but would prefer to move to more of a backfeed approach where I can write posts in the Lobsters UI then later they’ll automagically sync back to my site

                                    1. 2
                                      1. 2

                                        The problem with PESOS is that it prevents original-post-discovery, undermining the “own your content” goal of the IndieWeb. POSSE-over-PESOS also fits nicely into the logic behind POSSE before sending Webmentions.

                                        Backfeeding does make sense for aggregating responses, though. It’d be cool if Lobsters gave replies permalinks (rather than anchor-links) so we could enable that functionality…maybe I’ll file a ticket.

                                    2. 1

                                      I can’t say it would be truly automatic, but if I had to spend a lot of time copying and pasting text across multiple sites, that’s when I’d start using some level of copy/paste and browser tab automation. Nothing fancier than echo $text | xclip -sel clipboard and using a firefox $url invocation to spawn each site’s URL to make a new post on. This is what I did for a job with not-so-great automation aspects.

                                    1. 2

                                      I have a different approach, I use GitHub as this is a social network. But as every system, after a while GitHub will probably not be a good platform for me anymore. So I also self-host on Gitea (which I greatly recommend in favor of Gitlab for personal use).

                                      And I when I push, I push on both github and gitea.

                                      1. 9

                                        I’m still sad about the state of messengers. One one hand you have Telegram and Signal which are two walled gardens with a closed-source server. (Signal being slightly worse as the author does not want it to be packaged for fdroid, and released a few versions without publishing the source code prior to integrating his cryptocurrency-scam in it.)

                                        On the other hand, the decentralised alternatives are miles behind WhatsApp/Telegram/Signal. Jami just does not work for me and my friends (messages are not going through) and is a battery killer. (Messages are not being received) Tox has one of the worst UI/UX I’ve ever experienced (let alone push notifications being broken) And Matrix’s support for E2EE is not widespread.

                                        I appreciate the community effort, but unfortunately I still use Telegram :( .

                                        1. 25

                                          I’m not sure it’s fair to place Telegram and Signal in the same category, at least not if E2EE is important to you.

                                          The vast majority of Telegram’s chats are stored in plaintext on their servers. Group chats cannot be end-to-end encrypted, and you have to manually start a secret chat between two people if you want E2EE. In spite of this, Telegram continues to be seen as a secure messenger – which is why a 2017 study found that many Telegram users mistakenly think their chats are end-to-end encrypted when they’re not.

                                          1. 2

                                            I did not realise this and now it makes sense to me that they can delete group chats.

                                            1. 2

                                              Yes. Telegram has to be used with over mitts, I agree. I don’t do any group chat, and only secure chats.

                                            2. 10

                                              I have no expectation that Telegram is actually secure. I just find it moderately interesting it’s one of the few modern IM services with open client APIs.

                                              1. 9

                                                I’m with you on the Telegram and Signal, though I still use both for some contacts. What has been working better for me, though, was setting up a hosted Snikket instance and then I got my family using that. It’s XMPP/Jabber (with OMEMO for the e2ee, which is based on the Signal protocol), and actually has good mobile clients. It’s one of the few ways the iOS family members get consistent notifications and experience. Might be worth a try if you’re ever up for trying another one.

                                                1. 8

                                                  matrix works great for me.

                                                  1. 12

                                                    People keep saying this, but I don’t know when it’ll actually be the case. Every time I try it, I get disappointed by mediocre clients, resource hogging server, and inadequate moderation tools for groups.

                                                    1. 6

                                                      I tried matrix about 3 years ago I think, it worked, but it wasn’t great. In fact I self-hosted a matrix server, and while playing it filled my server with garbage from IRC-bridges, etc… So after one year or so, I uninstalled matrix, and just used a self-hosted IRC server, and for my group of geeky friends it was ok.

                                                      Then, after a while, IRC show some limits, so we’ve tried to use matrix again, about 1 year ago, and there were a lot of progress. The client is a lot better, the server too, I just forgot about it, and it “just works”. The e2ee user interface is a lot better than a few years ago.

                                                      So if you haven’t tried matrix in a while you should really try it again. Really, now I think matrix is ready.

                                                      1. 4

                                                        resource hogging server

                                                        conduit.rs has been running flawlessly for me, for a couple months now - way less resource-hungry than the Synapse stack (albeit, with some missing features).

                                                        1. 2

                                                          If you haven’t tried element in, say the last 3-4 months, then you aren’t up to date.

                                                          1. 7

                                                            I have and it’s still not great. The interface is a bit confusing (i.e. the multiple levels of hierarchy for chat, the weird spaces thing that’s trying to paper over federation, etc.) and lacks this je ne sais quoi other clients have that makes it feel wrong for lack of a better term.

                                                        2. 6

                                                          just wanted to say: matrix/element has come a long way. Due to it being used across orgs without federation (work vs family) I’d say the biggest downside is that there is no recommended way for multiple accounts on one device (especially mobile). Otherwise it works great.

                                                          1. 5

                                                            It REALLY has. But sadly I think the fact that @acatton did a whole ‘state of messengers’ and didn’t even mention Matrix is a great example of how it’s gaining mindshare, but VERY slowly and not with non technical users.

                                                            The clients are IMO finally ready for that to happen, now we just need more people to actually try the updated versions and figure out how good it can be.e

                                                            Meanwhile sadly 99% of the communities I actually want to chat on are Discord or Slack :(

                                                        3. 6

                                                          If you’re ever in the mood to try again, I suggest exploring Snikket.

                                                          1. 3

                                                            I use prosody xmpp server with Conversations from f-droid on mobile. It works great for me and my friends and family. It is also very fast and easy to setup. Server to server works flawlessly so I can connect with others easily and without creating an account.

                                                            1. 3

                                                              Can you provide more context or links around Signal’s cryptocurrency-scam that you mention? I could only locate a Verge article, but nothing more technical. Thanks.

                                                                1. 2

                                                                  Thank you. Yes, it does.

                                                              1. 3

                                                                miles behind WhatsApp/Telegram/Signal.

                                                                I know it wasn’t your intention, but just to clarify, this makes it seem as if these three are of similar quality. However, Telegram is miles ahead of WhatsApp.

                                                                1. 2

                                                                  Signal being slightly worse as the author does not want it to be packaged for fdroid

                                                                  This is pretty valid though. F-droid’s whole build process is really strange. It also isn’t built with security in mind.

                                                                1. 44

                                                                  Name popular OSS software, written in Haskell, not used for Haskell management (e.g. Cabal).

                                                                  AFAICT, there are only two, pandoc and XMonad.

                                                                  This does not strike me as being an unreasonably effective language. There are tons of tools written in Rust you can name, and Rust is a significantly younger language.

                                                                  People say there is a ton of good Haskell locked up in fintech, and that may be true, but a) fintech is weird because it has infinite money and b) there are plenty of other languages used in fintech which are also popular outside of it, eg Python, so it doesn’t strike me as being a good counterexample, even if we grant that it is true.

                                                                  1. 28

                                                                    Here’s a Github search: https://github.com/search?l=&o=desc&q=stars%3A%3E500+language%3AHaskell&s=stars&type=Repositories

                                                                    I missed a couple of good ones:

                                                                    • Shellcheck
                                                                    • Hasura
                                                                    • Postgrest (which I think is a dumb idea, lol, but hey, it’s popular)
                                                                    • Elm
                                                                    • Idris, although I think this arguably goes against the not used for Haskell management rule, sort of

                                                                    Still, compare this to any similarly old and popular language, and it’s no contest.

                                                                    1. 15

                                                                      Also Dhall

                                                                      1. 9

                                                                        I think postgrest is a great idea, but it can be applied to very wrong situations. Unless you’re familiar with Postgres, you might be surprised with how much application logic can be modelled purely in the database without turning it into spaghetti. At that point, you can make the strategic choice of modelling a part of your domain purely in the DB and let the clients work directly with it.

                                                                        To put it differently, postgrest is an architectural tool, it can be useful for giving front-end teams a fast path to maintaining their own CRUD stores and endpoints. You can still have other parts of the database behind your API.

                                                                        1. 6

                                                                          I don’t understand Postgrest. IMO, the entire point of an API is to provide an interface to the database and explicitly decouple the internals of the database from the rest of the world. If you change the schema, all of your Postgrest users break. API is an abstraction layer serving exactly what the application needs and nothing more. It provides a way to maintain backwards compatibility if you need. You might as well just send sql query to a POST endpoint and eliminate the need for Postgrest - not condoning it but saying how silly the idea of postgrest is.

                                                                          1. 11

                                                                            Sometimes you just don’t want to make any backend application, only to have a web frontend talk to a database. There are whole “as-a-Service” products like Firebase that offer this as part of their functionality. Postgrest is self-hosted that. It’s far more convenient than sending bare SQL directly.

                                                                            1. 6

                                                                              with views, one can largely get around the break the schema break the API problem. Even so, as long as the consumers of the API are internal, you control both ends, so it’s pretty easy to just schedule your cutovers.

                                                                              But I think the best use-case for Postgrest is old stable databases that aren’t really changing stuff much anymore but need to add a fancy web UI.

                                                                              The database people spend 10 minutes turning up Postgrest and leave the UI people to do their thing and otherwise ignore them.

                                                                              1. 1

                                                                                Hah, I don’t get views either. My philosophy is that the database is there to store the data. It is the last thing that scales. Don’t put logic and abstraction layers in the database. There is plenty of compute available outside of it and APIs can do precise data abstraction needed for the apps. Materialized views, may be, but still feels wrong. SQL is a pain to write tests for.

                                                                                1. 11

                                                                                  Your perspective is certainly a reasonable one, but not one I or many people necessarily agree with.

                                                                                  The more data you have to mess with, the closer you want the messing with next to the data. i.e. in the same process if possible :) Hence Pl/PGSQL and all the other languages that can get embedded into SQL databases.

                                                                                  We use views mostly for 2 reasons:

                                                                                  • Reporting
                                                                                  • Access control.
                                                                                  1. 2

                                                                                    Have you checked row-level security? I think it creates a good default, and then you can use security definer views for when you need to override that default.

                                                                                    1. 5

                                                                                      Yes, That’s exactly how we use access control views! I’m a huge fan of RLS, so much so that all of our users get their own role in PG, and our app(s) auth directly to PG. We happily encourage direct SQL access to our users, since all of our apps use RLS for their security.

                                                                                      Our biggest complaint with RLS, none(?) of the reporting front ends out there have any concept of RLS or really DB security in general, they AT BEST offer some minimal app-level security that’s usually pretty annoying. I’ve never been upset enough to write one…yet, but I hope someone someday does.

                                                                                      1. 2

                                                                                        That’s exactly how we use access control views! I’m a huge fan of RLS, so much so that all of our users get their own role in PG

                                                                                        When each user has it its own role, usually that means ‘Role explosion’ [1]. But perhaps you have other methods/systems that let you avoid that.

                                                                                        How do you do for example: user ‘X’ when operating at location “Poland” is not allowed to access Report data ‘ABC’ before 8am and after 4pm UTC-2, in Postgres ?

                                                                                        [1] https://blog.plainid.com/role-explosion-unintended-consequence-rbac

                                                                                        1. 3

                                                                                          Well in PG a role IS a user, there is no difference, but I agree that RBAC is not ideal when your user count gets high as management can be complicated. Luckily our database includes all the HR data, so we know this person is employed with this job on these dates, etc. We utilize that information in our, mostly automated, user controls and accounts. When one is a supervisor, they have the permission(s) given to them, and they can hand them out like candy to their employees, all within our UI.

                                                                                          We try to model the UI around “capabilities”, all though it’s implemented through RBAC obviously, and is not a capability based system.

                                                                                          So each supervisor is responsible for their employees permissions, and we largely try to stay out of it. They can’t define the “capabilities”, that’s on us.

                                                                                          How do you do for example: user ‘X’ when operating at location “Poland” is not allowed to access Report data ‘ABC’ before 8am and after 4pm UTC-2, in Postgres ?

                                                                                          Unfortunately PG’s RBAC doesn’t really allow us to do that easily, and we luckily haven’t yet had a need to do something that detailed. It is possible, albeit non-trivial. We try to limit our access rules to more basic stuff: supervisor(s) can see/update data within their sphere but not outside of it, etc.

                                                                                          We do limit users based on their work location, but not their logged in location. We do log all activity in an audit log, which is just another DB table, and it’s in the UI for everyone with the right permissions(so a supervisor can see all their employee’s activity, whenever they want).

                                                                                          Certainly different authorization system(s) exist, and they all have their pros and cons, but we’ve so far been pretty happy with PG’s system. If you can write a query to generate the data needed to make a decision, then you can make the system authorize with it.

                                                                                  2. 4

                                                                                    My philosophy is “don’t write half-baked abstractions again and again”. PostgREST & friends (like Postgraphile) provide selecting specific columns, joins, sorting, filtering, pagination and others. I’m tired of writing that again and again for each endpoint, except each endpoint is slightly different, as it supports sorting on different fields, or different styles of filtering. PostgREST does all of that once and for all.

                                                                                    Also, there are ways to test SQL, and databases supporting transaction isolation actually simplify running your tests. Just wrap your test in a BEGIN; ROLLBACK; block.

                                                                                    1. 2

                                                                                      Idk, I’ve been bitten by this. Probably ok in a small project, but this is a dangerous tight coupling of the entire system. Next time a new requirement comes in that requires changing the schema, RIP, wouldn’t even know which services would break and how many things would go wrong. Write fully-baked, well tested, requirements contested, exceptionally vetted, and excellently thought out abstractions.

                                                                                      1. 6

                                                                                        Or just use views to maintain backwards compatibility and generate typings from the introspection endpoint to typecheck clients.

                                                                                2. 1

                                                                                  I’m a fan of tools that support incremental refactoring and decomposition of a program’s architecture w/o major API breakage. PostgREST feels to me like a useful tool in that toolbox, especially when coupled with procedural logic in the database. Plus there’s the added bonus of exposing the existing domain model “natively” as JSON over HTTP, which is one of the rare integration models better supported than even the native PG wire protocol.

                                                                                  With embedded subresources and full SQL view support you can quickly get to something that’s as straightforward for a FE project to talk to as a bespoke REST or GraphQL backend.. Keeping the schema definitions in one place (i.e., the database itself) means less mirroring of the same structures and serialization approaches in multiple tiers of my application.

                                                                                  I’m building a project right now where PostgREST fills the same architectural slot that a Django or Laravel application might, but without having to build and maintain that service at all. Will I eventually need to split the API so I can add logic that doesn’t map to tuples and functions on them? Sure, maybe, if the app gets traction at all. Does it help me keep my tiers separate for now while I’m working solo on a project that might naturally decompose into a handful of backend services and an integration layer? Yep, also working out thus far.

                                                                                  There are some things that strike me as awkward and/or likely to cause problems down the road, like pushing JWT handling down into the DB itself. I also think it’s a weird oversight to not expose LISTEN/NOTIFY over websockets or SSE, given that PostgREST already uses notification channels to handle its schema cache refresh trigger.

                                                                                  Again, though, being able to wire a hybrid SPA/SSG framework like SvelteKit into a “native” database backend without having to deploy a custom API layer has been a nice option for rapid prototyping and even “real” CRUD applications. As a bonus, my backend code can just talk to Postgres directly, which means I can use my preferred stack there (Rust + SQLx + Warp) without doing yet another intermediate JSON (un)wrap step. Eventually – again, modulo actually needing the app to work for more than a few months – more and more will migrate into that service, but in the meantime I can keep using fetch in my frontend and move on.

                                                                              2. 2

                                                                                I would add shake

                                                                                https://shakebuild.com

                                                                                not exactly a tool but a great DSL.

                                                                              3. 21

                                                                                I think it’s true that, historically, Haskell hasn’t been used as much for open source work as you might expect given the quality of the language. I think there are a few factors that are in play here, but the dominant one is simply that the open source projects that take off tend to be ones that a lot of people are interested in and/or contribute to. Haskell has, historically, struggled with a steep on-ramp and that means that the people who persevered and learned the language well enough to build things with it were self-selected to be the sorts of people who were highly motivated to work on Haskell and it’s ecosystem, but it was less appealing if your goals were to do something else and get that done quickly. It’s rare for Haskell to be the only language that someone knows, so even among Haskell developers I think it’s been common to pick a different language if the goal is to get a lot of community involvement in a project.

                                                                                All that said, I think things are shifting. The Haskell community is starting to think earnestly about broadening adoption and making the language more appealing to a wider variety of developers. There are a lot of problems where Haskell makes a lot of sense, and we just need to see the friction for picking it reduced in order for the adoption to pick up. In that sense, the fact that many other languages are starting to add some things that are heavily inspired by Haskell makes Haskell itself more appealing, because more of the language is going to look familiar and that’s going to make it more accessible to people.

                                                                                1. 15

                                                                                  There are tons of tools written in Rust you can name

                                                                                  I can’t think of anything off the dome except ripgrep. I’m sure I could do some research and find a few, but I’m sure that’s also the case for Haskell.

                                                                                  1. 1

                                                                                    You’ve probably heard of Firefox and maybe also Deno. When you look through the GitHub Rust repos by stars, there are a bunch of ls clones weirdly, lol.

                                                                                  2. 9

                                                                                    Agree … and finance and functional languages seem to have a connection empirically:

                                                                                    • OCaml and Jane St (they strongly advocate it, mostly rejecting polyglot approaches, doing almost everything within OCaml)
                                                                                    • the South American bank that bought the company behind Clojure

                                                                                    I think it’s obviously the domain … there is simple a lot of “purely functional” logic in finance.

                                                                                    Implementing languages and particularly compilers is another place where that’s true, which the blog post mentions. But I’d say that isn’t true for most domains.

                                                                                    BTW git annex appears to be written in Haskell. However my experience with it is mixed. It feels like git itself is more reliable and it’s written in C/Perl/Shell. I think the dominating factor is just the number and skill of developers, not the language.

                                                                                    1. 5

                                                                                      OCaml also has a range of more or less (or once) popular non-fintech, non-compiler tools written in it. LiquidSoap, MLDonkey, Unison file synchronizer, 0install, the original PGP key server…

                                                                                      1. 3

                                                                                        Xen hypervisor

                                                                                        1. 4

                                                                                          The MirageOS project always seemed super cool. Unikernels are very interesting.

                                                                                          1. 3

                                                                                            Well, the tools for it, rather than the hypervisor itself. But yeah, I forgot about that one.

                                                                                        2. 4

                                                                                          I think the connection with finance is that making mistakes in automated finance is actually very costly on expectation, whereas making mistakes in a social network or something is typically not very expensive.

                                                                                        3. 8

                                                                                          Git-annex

                                                                                          1. 5

                                                                                            Not being popular is not the same as being “ineffective”. Likewise, something can be “effective”, but not popular.

                                                                                            Is JavaScript a super effective language? Is C?

                                                                                            Without going too far down the language holy war rabbit hole, my overall feeling after so many years is that programming language popularity, in general, fits a “worse is better” characterization where the languages that I, personally, feel are the most bug-prone, poorly designed, etc, are the most popular. Nobody has to agree with me, but for the sake of transparency, I’m thinking of PHP, C, JavaScript, Python, and Java when I write that. Languages that are probably pretty good/powerful/good-at-preventing-bugs are things like Haskell, Rust, Clojure, Elixir.

                                                                                            1. 4

                                                                                              In the past, a lot of the reason I’ve seen people being turned away from using Haskell based tools has been the perceived pain of installing GHC, which admittedly is quite large, and it can sometime be a pain to figure out which version you need. ghcup has improved that situation quite a lot by making the process of installing and managing old compilers significantly easier. There’s still an argument that GHC is massive, which it is, but storage is pretty cheap these days. For some reason I’ve never seen people make similar complaints about needing to install multiple version of python (though this is less off an issue these days).

                                                                                              The other place where large Haskell codebases are locked up is Facebook - Sigma processes every single post, comment and massage for spam, at 2,000,000 req/sec, and is all written in Haskell. Luckily the underlying tech, Haxl, is open source - though few people seem to have found a particularly good use for it, you really need to be working at quite a large scale to benefit from it.

                                                                                              1. 2

                                                                                                hledger is one I use regularly.

                                                                                                1. 2

                                                                                                  Cardano is a great example.

                                                                                                  Or Standard Chartered, which is a very prominent British bank, and runs all their backend on Haskell. They even have their own strict dialect.

                                                                                                  1. 2

                                                                                                    GHC.

                                                                                                    1. 1

                                                                                                      https://pandoc.org/

                                                                                                      I used pandoc for a long time before even realizing it was Haskell. Ended up learning just enough to make a change I needed.

                                                                                                    1. 23

                                                                                                      I enjoyed this quite a bit when I first saw it, and I still do kinda enjoy the original - it was never meant to be taken too seriously of course and it succeeds as a joke - but since then, the word “wat” has become a trigger of rage for me.

                                                                                                      These things tend to have a reasonable explanation if you take the time to understand why it does what it does. They tend to be predictable consequences of actually generally useful rules, just used in a different way than originally intended and coming out a bit silly. You might laugh but you can also be educated.

                                                                                                      But you know how often I see people taking the time to try to understand the actual why? Not nearly as often as I see people saying “WAT” and just dismissing things and calling the designers all kinds of unkind things. And that attitude is both useless and annoying.

                                                                                                      1. 38

                                                                                                        These things tend to have a reasonable explanation if you take the time to understand why it does what it does. They tend to be predictable consequences of actually generally useful rules, just used in a different way than originally intended and coming out a bit silly.

                                                                                                        For me the most important takeaway is that rules might make sense by themselves, but you have to consider them in the bigger picture, as part of a whole. When you design something, you must keep this in mind to avoid bringing about a huge mess in the completed system.

                                                                                                        1. 4

                                                                                                          Exactly. It is generally underappreciated how incredible hard language design is. The cases Bernhardt points out are genuine design mistakes and not just the unfortunate side effects of otherwise reasonable decisions.

                                                                                                          That’s why there are very few languages that don’t suffer from ugly corner cases which don’t fit into the whole or turn out to have absurd oddities. Programming languages are different, contrary to the “well, does it really matter?” mindset.

                                                                                                          1. 3

                                                                                                            I don’t know. What I always think about JS is that R4RS Scheme existed at the time JS was created and the world would be tremendously better off if they had just used that as the scripting system. Scheme isn’t perfect but it is much more regular and comprehensible than JS.

                                                                                                            1. 3

                                                                                                              I think one have to remember the context in which JavaScript was created; I’m guessing the main use case was to show some funny “alert” pop-ups here and there.

                                                                                                              In that context a lot of the design decisions start to make sense; avoid crashes whenever possible and have a “do what I mean” approach to type coercion.

                                                                                                              But yeah, I agree; we would’ve all been better off with a Scheme as the substrate for maybe 80% of today’s end-user applications. OTOH, if someone would’ve told Mozilla how successful JS would become we could very well have ended up with some bloated, Java-like design-by-committee monstrosity instead

                                                                                                            2. 2

                                                                                                              I don’t think I know a single (nontrivial - thinking about brainfuck/assembly maybe) programming language with no “unexpected behaviour”.

                                                                                                              But some just have more or less than others. Haskell, for example, has a lot of these unexpected behaviours but you tend not to fall on these corner cases by mistake. While in javascript and Perl, it is more common to see such a “surprise behaviour” in the wild.

                                                                                                              Another lesson I gather from this talk is that you should try to stick as much as possible in your well-known territory if you want to predict the behaviour of your program. In particular, try not to play too much with “auto coercion of types”. If a function expects a string, I tend not to give it a random object even if when I tried it, it perform string coercion which will most of the time be what I would expect.

                                                                                                              1. 1

                                                                                                                Well, there are several non-trivial languages that try hard not to surprise you. One should also distinguish betweem “unexpected behaviour” and convenience features that turn out to be counterproductive by producing edge-cases. This is a general problem with many dynamically typed languages, especially recent inventions: auto-coercion will remove opportunities for error checking (and run-time checks are what make dynamically typed languages type-safe). By automatic conversion of value types and also by using catch-all values like the pervasive use of maps in (say) Clojure, you effectively end up with untyped data. If a function expects a string, give it a string. The coercion might save some typing in the REPL, but hides bugs in production code.

                                                                                                            3. 3

                                                                                                              In javascript, overloading the + operator and the optional semicolon rules I would call unforced errors in the language and those propagate through to a few other places. Visual Basic used & for concatenation, and it was very much a contemporary of JS when it was new, but they surely just copied Java’s design (which I still think is a mistake but less so given the type system).

                                                                                                              Anyway, the rest of the things shown talk I actually think are pretty useful and not much of a problem when combined. The NaNNaN Batman one is just directly useful - it converts a thing that is not a number to a numeric type, so NaN is a reasonable return, then it converts to string to join them, which is again reasonable.

                                                                                                              People like to hate on == vs === but…. == is just more useful. In a dynamic, weakly typed language, things get mixed. You prompt for a number from the user and technically it is a string, but you want to compare it with numbers. So that’s pretty useful. Then if you don’t want that, you could coerce or be more specific and they made === as a shortcut for that. This is pretty reasonable. And the [object Object] thing comes from these generally useful conversions.

                                                                                                              1. 3

                                                                                                                == vs ===

                                                                                                                It definitely makes sense to have multiple comparison operators. Lisp has = (numeric equality), eq (object identity), eql (union of the previous two), equal (structural equality).

                                                                                                                The problem is that js comes from a context (c) in which == is the ‘default’ comparison operator. And since === is just ==, but more, it is difficult to be intentional about which comparison you choose to make.

                                                                                                                1. 1

                                                                                                                  Well, a lot of these things boil down to implicit type coercion and strange results due to mismatched intuitive expectations. It’s also been shown time and again (especially in PHP) that implicit type coercions are lurking security problems, mostly because intuition does not match reality (especially regarding == and the bizarre coercion rules). So perhaps the underlying issue of most of the WATs in here simply is that implicit type coercion should be avoided as much as possible in languages because it results in difficult to predict behaviour in code.

                                                                                                                  1. 1

                                                                                                                    Yeah, I perfer a stronger, static type system and that’s my first choice in languages. But if it is dynamically typed… I prefer it weaker, with these implicit coercion. It is absurd to me to get a runtime error when you do like var a = prompt("number"); a - whatever; A compile time error, sure. But a runtime one? What a pain, just make it work.

                                                                                                                    1. 3

                                                                                                                      Lots of dynamic languages do this (e.g. Python, Ruby, all Lisp dialects that spring to mind), and IME it’s actually helpful in catching bugs early. And like I said, it prevents security issues due to type confusions.

                                                                                                              2. 10

                                                                                                                Yeah, I think that this talk was well-intentioned enough, but I definitely think that programmers suffer from too much “noping” and too little appreciation for the complexity that goes into real-world designs, and that this talk was a contributor… or maybe just a leading indicator.

                                                                                                                1. 6

                                                                                                                  There was a good talk along these lines a couple of years ago, explaining why javascript behaves the way it does in those scenarios, and then presenting similar ‘WAT’s from other languages and explaining their origins. Taking the attitude of ‘ok this seems bizarre and funny, but let’s not just point and laugh, let’s also figure out why it’s actually fairly sensible in context’.

                                                                                                                  Sadly I can’t find it now, though I do remember the person who delivered it was associated with ruby (maybe this rings a bell for somebody else).

                                                                                                                  1. 1

                                                                                                                    Isn’t the linked talk exactly the talk you’re thinking about? Gary is ‘associated with’ Ruby and does give examples from other languages as well.

                                                                                                                    1. 2

                                                                                                                      No. I was thinking of this, linked else-thread.

                                                                                                                  2. 3

                                                                                                                    While things might have an explanation, I do strongly prefer systems and languages that stick to the principle of least surprise: if your standard library has a function called ‘max’ that returns the maximum value in an array and a function called ‘min’ that returns the position of the minimum element instead, you are making your language less discoverable and putting a lot of unnecessary cognitive load on the user.

                                                                                                                    As someone who has been programming for over 20 years and is now a CTO of a small company that uses your average stack of like 5 programming languages on a regular basis I don’t want to learn why anymore, I just want to use the functionality and be productive. My mind is cluttered with useless trivia about inconsistent APIs I learned 20, 15, 10 years ago, the last thing I need is learning more of that.

                                                                                                                  1. 3
                                                                                                                    ( brilliant(,)
                                                                                                                    	( if
                                                                                                                    		( only
                                                                                                                    			( it
                                                                                                                    				( wasn\'t
                                                                                                                    					( lisp :))))))
                                                                                                                    
                                                                                                                    1. 13

                                                                                                                      I don’t know why people have this learned hatred of parenthesis.

                                                                                                                      1. 2

                                                                                                                        LISP value lies beyond its syntax. But when your world is only C-like/Python-like syntax it is hard to look beyond just the syntax of any new language.

                                                                                                                        Mainly: https://wiki.haskell.org/Wadler's_Law

                                                                                                                        In any language design, the total time spent discussing a feature in this list is proportional to two raised to the power of its position.

                                                                                                                              0. Semantics
                                                                                                                              1. Syntax
                                                                                                                              2. Lexical syntax
                                                                                                                              3. Lexical syntax of comments
                                                                                                                        
                                                                                                                        1. 2

                                                                                                                          LISP value lies beyond its syntax. But when your world is only C-like/Python-like syntax it is hard to look beyond just the syntax of any new language.

                                                                                                                          My world is not only C-like/Python-like syntax. and LISP is not a new language to me. It was my first real exposure to programming to get real work done (not counting some hello-world stuff in other languages and copy/paste POSIX/BAT script script stuff I didn’t really understand at the time) and I kept up with it for years through my next couple languages (Perl being my next major stop). I realize it has value, but even knowing where some of that value lies it’s hard to to make fun of how obtuse it can be.

                                                                                                                          1. 2

                                                                                                                            I don’t know why people have this learned hatred of parenthesis.

                                                                                                                            Experience.

                                                                                                                            Mine with LISP being from before the days when editors had syntax highlighting of any kind, much less nested parenthesis color matching.

                                                                                                                            1. 2

                                                                                                                              i often use my editors in monochrome mode: vi has the % command to check on matching parens, emacs has the full suite of s-expr commands.

                                                                                                                        1. 8

                                                                                                                          You should learn a new language only if it learns you something along the way. Some language can be put in a “bag” of similar patterns:

                                                                                                                          • Python/Javascript/Ruby/PHP/Perl; common scripting language spirit
                                                                                                                          • C/C++/Java/Go; compiled in spirit, feel “not modern” quite low level
                                                                                                                          • Clojure; has my preference to have fun while learning new concepts along the way, should make you quite productive
                                                                                                                          • Haskell; if you prefer never finish your work but learn a lot of things along the way trying and feel good about yourself because you mastered the beast (partially)
                                                                                                                          • Rust; if you still like to learn a low level imperative languages but with a lot better compiler that checks common mistakes for you so your CRUD API will probably be a lot faster and use a lot less resources than using Python
                                                                                                                          1. 2

                                                                                                                            if you prefer never finish your work

                                                                                                                            I’m not sure this is fair, I’ve been developing using Haskell for about 6 years now and we’ve completed plenty of projects. In fact, one of Haskell best features is the ability to fearlessly refactor; when new requirements come up, we just make the obvious change and the compiler guides up to all the places we forgot about. This makes experimentation also very cheap, but only once you’ve learned the language enough to be proficient and know the patterns to use and avoid to help the compiler help you (big one for me is always using total case statements, unless it’s a situation where you know there’s a single thing you want to match and everything else is invalid).

                                                                                                                            1. 1

                                                                                                                              Learning Haskell to the level of pragmatic productivity for an average software developer sometimes literally takes years.

                                                                                                                              Not a single experienced Haskellite would argue with that sentence. Anyone who would - probably hasn’t ventured deeply enough into quite complicated ecosystem of Haskell.

                                                                                                                          1. 1
                                                                                                                            • my blog (a gemini server too)
                                                                                                                            • espial a pinboard clone written in Haskell
                                                                                                                            • git repositories I duplicate most of my repos here and in github
                                                                                                                            • pihole
                                                                                                                            • A Matrix server to talk securely between friends
                                                                                                                            • An IRC server
                                                                                                                            • the Loundge for IRC web interface
                                                                                                                            • a private image upload service to be used via IRC
                                                                                                                            • a minimal poll service to find event dates between friends
                                                                                                                            • node-red to have a self-hosted IFTTT to POSSE
                                                                                                                            • znc
                                                                                                                            1. 1

                                                                                                                              For my own website I have a more complex system. By default it renders html but I also generate ASCII as well as pdf files.

                                                                                                                              Another important aspect, my RSS contain my full article. And I also use a slightly more complex script to generate RSS from files and not generate the RSS directly from my static page generator.

                                                                                                                              And latest but nice touch, I use a generic build system that is incredibly smarter and faster than all previous specialized static site generators (https://hackage.haskell.org/package/shake).

                                                                                                                              That being said I fully agree with the spirit of this blog post. We should self host as much as we can because you could always be put out of big centralized products. But instead of just keeping things in my personal area I duplicate them. I have github and self-hosted gitea. I use both espial and pinboard. I use espial notes to tweet, etc…

                                                                                                                              If you are curious here is a post about it https://her.esy.fun/posts/0004-how-i-internet/index.html

                                                                                                                              I haven’t written about how I use shake yet.

                                                                                                                              1. 9

                                                                                                                                I built Org-roam, which i had initially intended to be a simple layer on top of Org-mode that added backlinks to regular Org-mode files. A bunch of tools such as org-roam-bibtex and org-roam-server have since been built by community users to work with citations and provide a graphical overview of the notes. My notes are automatically published via netlify here.

                                                                                                                                Org-mode is unparalleled as a plain-text system, which beginners can use as a simple outliner, and power users can use it to build complex workflows (GTD, literate programming etc.). It’s simply a gift that keeps on giving.

                                                                                                                                1. 2

                                                                                                                                  Thanks jethro for org-roam. It’s been a few months now that I use it daily and I really love it.

                                                                                                                                  As far as I can tell, this is the ultimate note taking tool.

                                                                                                                                  Along the rest of org-mode (org-agenda, org-capture, etc…) this is a life changer tool for me.

                                                                                                                                  1. 3

                                                                                                                                    you’re welcome!

                                                                                                                                  2. 1

                                                                                                                                    After hearing about org-mode and org-agenda for a while and then org-roam yesterday, I’ve finally decided to dive into Emacs. I’m starting from the basics with a vanilla installation and reading through a few people’s config files and the docs before I attempt to use org-roam though; I’ve heard it’s a challenge to work with.

                                                                                                                                    My notes are automatically published via netlify here.

                                                                                                                                    That’s incredibly similar to what someone sent me yesterday, which was the final straw that convinced me to try Emacs. Is the output a template from a specific package or something you’ve created yourself?

                                                                                                                                    1. 3

                                                                                                                                      As a new user I was glad to start to use emacs using a configuration framework like doom-emacs or spacemacs. In fact, after a few years getting used to emacs I now believe that doom emacs make a better job than I could ever do myself to create an emacs configuration.

                                                                                                                                      That being said, let just say that diving into org-mode was probably one of the best use of my time, and I hope you’ll enjoy it as much as I do.

                                                                                                                                      1. 2

                                                                                                                                        I’m starting from the basics with a vanilla installation and reading through a few people’s config files and the docs before I attempt to use org-roam though; I’ve heard it’s a challenge to work with.

                                                                                                                                        It’s hard if you fight it, easier if you are ready to learn – seeing that you have the right stance, you’ll probably be fine.

                                                                                                                                        1. 1

                                                                                                                                          I’m starting from the basics with a vanilla installation and reading through a few people’s config files and the docs before I attempt to use org-roam though; I’ve heard it’s a challenge to work with.

                                                                                                                                          Yes it is. Emacs is a complex beast, and so is Org-mode, and Org-roam, it really does take some time to get used to. Maybe this guide can help you: https://github.com/nobiot/Zero-to-Emacs-and-Org-roam

                                                                                                                                          That’s incredibly similar to what someone sent me yesterday, which was the final straw that convinced me to try Emacs. Is the output a template from a specific package or something you’ve created yourself?

                                                                                                                                          That’s my hugo theme, cortex, which that website in that link had modified for use directly with org-publish. I had since taken some of the modifications (javascript, mostly) and placed them back into my theme :)

                                                                                                                                      1. 31

                                                                                                                                        I have recently gone down the rabbit-hole of note-taking apps, since none of them seem to meet my criteria:

                                                                                                                                        • works offline with my local files (ideally in a human-readable format, if not compliant Markdown)
                                                                                                                                        • excellent math support (see my prosemirror-math extension for an example)
                                                                                                                                        • wysiwym (this is crucial for documents with lots of math)
                                                                                                                                        • support for tags + bidirectional links, for easy categorization and interlinking between notes
                                                                                                                                        • citations
                                                                                                                                        • custom css themes
                                                                                                                                        • free and open source, for extensibility

                                                                                                                                        Here’s a summary of the different workflows I’ve tried over the years:

                                                                                                                                        • Initially, I simply took notes on paper. Writing math was easy, but it was too difficult to stay organized or to make changes later.
                                                                                                                                        • Next, I switched to a system where I’d take rough notes on paper, then polish them into a nice LaTeX document with overleaf that I could reference later. This works out well for some things, but when I’m writing LaTeX I feel to pressured to make everything look pretty. This also isn’t ideal because of the lack of links, difficulty inserting images, etc..
                                                                                                                                        • For a while I used Jupyter notebooks, since they enable a nice mix of code / math / markdown. Eventually it just grew too cumbersome to start a notebook server every time I wanted to write. (however, these days I think notebooks are built into VS Code–so maybe it’s better).
                                                                                                                                        • Next, I started using Typora for rough notes, and I’d eventually synthesize the most important ones into a LaTeX document with Overleaf. This was fine, but I had a lot of trouble with organization. Typora doesn’t support anything like tags / wikilinks.
                                                                                                                                        • Next, I started using OneNote to keep a daily work journal, in linear order. If I’ve forgotten something, I can usually remember what else I had going on the same month/week, so having everything in a linear order really helped when I wanted to search over my past notes. It also helps remind me of my thought process when I go on a long depth-first tangent.
                                                                                                                                        • Unfortunately, OneNote has terrible math support. So at this point, my notes were spread betweeen paper, OneNote, Typora, and Overleaf. I had no idea where to look for the most “up to date” version of anything.

                                                                                                                                        When the pandemic started, I found myself with a lot of free time, so I decided it was time to make my own note-taking app called Noteworthy! I’ve been using it exclusively for my notes the past 3-4 months and it’s almost ready for public release!

                                                                                                                                        In the process of making Noteworthy I’ve been inspired by all the other great note-taking apps out there. Here are just a few of my favorites:

                                                                                                                                        • Typora, a nicely polished Markdown editor – has the best support for math input I’ve seen
                                                                                                                                        • Obsidian, a split-pane Markdown editor focused on bidirectional linking
                                                                                                                                        • Zettlr a Markdown editor focused on publishing / academics
                                                                                                                                        • RemNote converts your notes into spaced-repetition flash cards, similar to Anki
                                                                                                                                        • foambubble, a family of VS Code extensions to help search + organize your notes
                                                                                                                                        • logseq, a GitHub-hosted alternative to Roam
                                                                                                                                        • Neuron Notes a neat Zettelkasten system written in Haskell, based on GitHub repos
                                                                                                                                        • R Studio includes an awesome Markdown publishing experience, similar to Jupyter Notebooks
                                                                                                                                        • (coming soon) Athens Research, an open-source alternative to Roam
                                                                                                                                        • (coming soon, made by me) Noteworthy, which aims to be an extensible, open-source alternative to Obsidian and Typora, with a focus on wikilinks and excellent math support

                                                                                                                                        Some honorable mentions:

                                                                                                                                        • Dendron, a hierarchical note-taking editor based on VS Code
                                                                                                                                        • kb, a minimal text-oriented command-line note manager
                                                                                                                                        • Notebag a minimal Markdown app with tag support
                                                                                                                                        1. 4

                                                                                                                                          I would add to your list Joplin, which I’ve had very good experiences with. I think it ticks a lot of your boxes, and it also has quite good mobile support which can come in handy.

                                                                                                                                          1. 4

                                                                                                                                            I can’t believe I left out Joplin! It’s multiplatform (mobile. pc, terminal) has more features than practically every other note-taking app out there.

                                                                                                                                            It’s been a while since I last tried Joplin, but I think I remember choosing not to use it for my own notes since it uses split-pane (code+rendered) editing instead of wysiwym, which isn’t ideal for notes with lots of math. I believe there’s also no support bidirectional links, but I could be misremembering.

                                                                                                                                            1. 4

                                                                                                                                              I currently use Joplin, which works decently well for me. I have a few complaints about it: it’s a heavy Electron app that doesn’t exist in my distro’s package manger, so I have to build it from source. I don’t like the distinction it makes between “notes” and “notebooks” - I wish that notes could have arbitrarily-deeply-nested children, like some other notetaking software I’ve used has had. I do appreciate that it has a mobile app, but I’ve run into a few useability nits inputting text in that mobile app. And I wish there was a browser version.

                                                                                                                                              This Noteworthy project looks interesting, and if can solve some of these problems better than Joplin, while still doing everything that I do like out of Joplin, I would consider switching to it.

                                                                                                                                              1. 4

                                                                                                                                                I wish that notes could have arbitrarily-deeply-nested children

                                                                                                                                                A quick skim of the Joplin site didn’t really make it clear what a “notebook” is – are you just talking about being able to easily define hierarchies of notes? Or do you mean a full-on, infinitely-nested-list style app like Athens / Logseq / Roam, where every list bullet is considered to be a separate note? And where all the notes are connected as a big graph?

                                                                                                                                                With Noteworthy, the goal is not to impose too much structure on your notes – you shouldn’t have to change how you think just to work with a new app. I decided that an approach based on tags would give the most freedom, similar to how Obsidian does it.

                                                                                                                                                • (done!) Include tags anywhere using [[wikilink]] , #tag, or @[citation] syntax.
                                                                                                                                                • (done!) easily search for all documents referencing a specific tag
                                                                                                                                                • (done!) By default, filenames are tags and tags are filenames! Each file can additionally define a list of aliases, which allows for e.g. an abbreviation and its expansion to point to the same file.
                                                                                                                                                • (planned) define your own tag hierarchies for easier search / disambiguation
                                                                                                                                                • (planned) use logical operations (and/or/etc) in tag searches

                                                                                                                                                I’d like to experiment with a Datalog-esque syntax for tag search as well. Roam and Athens both use Datalog internally to facilitate searches, and I believe it has worked out well for them. It would be super cool to expose some kind of tagging system based on predicate logic to the user.

                                                                                                                                                This Noteworthy project looks interesting, and if can solve some of these problems better than Joplin, while still doing everything that I do like out of Joplin, I would consider switching to it.

                                                                                                                                                What would you say are your most-used features? Regarding Electron vs native vs browser,

                                                                                                                                                • Noteworthy is an Electron app, and I’m trying to keep it as lightweight as possible. This is mostly by necessity, since projects like KaTeX and ProseMirror have no native counterparts, afaik. I’ll happily re-write it if an alternative emerges – I’ve been following React-Native and Rust GUI for that reason. I also plan to delegate some of the heavy lifting to Rust programs like ripgrep.

                                                                                                                                                • Browser version – I can definitely imagine Noteworthy running in a local server, accessible through a browser.

                                                                                                                                                If all goes as planned, there will be a public beta of Noteworthy in a couple months, where I’ll try to gather feedback about what’s working / what’s missing. Keep an eye out :)

                                                                                                                                                1. 1

                                                                                                                                                  In Joplin a notebook is a collection of notes, which map to single markdown files. Notebooks can be arbitrarily nested, but a notebook can only contain notes, not have raw text associated with it. So there’s effectively two types of node in the tree structure it exposes to you. I would prefer it if you could have a tree of notes, all of which contain text, and may or may not have any kind of nesting under them.

                                                                                                                                              2. 1

                                                                                                                                                IIRC, they have an editor that lets you edit the markdown as it is rendered (one pane). I think this feature is still experimental though.

                                                                                                                                                I’m not sure what you mean by bidirectional? In the sense that linking from node A to note B also creates a back-link in note B to note A? That’s not a thing in Joplin to my knowledge.

                                                                                                                                                I’ve got the skeleton of a graph viewer inspired by Obsidian which talks to Joplin over it’s REST API, but it’s not currently working and I haven’t had the time to finish a PoC yet. I’m far enough into it to determine that creating such a companion app is definitely do-able – Joplin’s API is quite nice.

                                                                                                                                            2. 1

                                                                                                                                              This is a great resource, thank you.

                                                                                                                                              1. 1

                                                                                                                                                I think org-roam fill all your checkboxes.

                                                                                                                                                The author answered in this thread here

                                                                                                                                                1. 1

                                                                                                                                                  I haven’t personally tried org-roam due to a phobia of emacs, but it looks like a great alternative to the other roam-likes. One thing that’s not clear – does it support some kind of instant math preview?

                                                                                                                                                  1. 2

                                                                                                                                                    Due to how ridiculously extensible emacs is, you can be certain that the answer will be “yes, with some elisp”.

                                                                                                                                                    1. 1

                                                                                                                                                      and if you want a packaged solution, org-fragtog :)

                                                                                                                                              1. 14

                                                                                                                                                Why did Haskell’s popularity wane so sharply?

                                                                                                                                                What is the source that Haskell’s popularity are declining so sharply? Is there really some objective evidence for this, I mean numbers, statistics, etc.?

                                                                                                                                                It’s anecdotal and just my personal impression by observing the Haskell reddit 1 for 10 years, but I have never seen so many Haksell resources, Conferences, Books and even postings for jobs as now. I have not at all the impression that the language is dying. It has accumulated cruft, has some inconsistencies, is struggling to get a new standard proposal out, but other than that I have the impression that it attracts quite some people that come up with new ideas.

                                                                                                                                                1. 2

                                                                                                                                                  Haskell had glory days when SPJ/Marlow were traveling to various conferences talking about the new language features. Mileweski’s posts, LYAH, Parsec, STM, and Lenses are from that era. The high-brow crowd was of course discussing Lenses. Sure, these things drove adoption, and there’s a little ecosystem for the people who went on the Haskell bandwagon back then.

                                                                                                                                                  What innovation has it had over the last 5 years? The community couldn’t agree on how to implement any of the features of a respectable dependent-type system, so they invented a bunch of mutually incompatible flags, and destroyed the language. Thanks to the recent hacking, GHC is plastered with band-aids.

                                                                                                                                                  It’s true that you can’t explain these things with some points on a pretty graph, but that doesn’t make it anecdotal. Look at the commits going into ghc/ghc, and look at the activity on the bread-and-butter Haskell projects: lens, trifecta, cloud-haskell. Maintenance mode. Where are the bold new projects?

                                                                                                                                                  1. 23

                                                                                                                                                    These assertions about Haskell are all simply false. There are plenty of problems with Haskell, we don’t need to add ones that aren’t true.

                                                                                                                                                    The community couldn’t agree on how to implement any of the features of a respectable dependent-type system, so they invented a bunch of mutually incompatible flags, and destroyed the language. Thanks to the recent hacking, GHC is plastered with band-aids

                                                                                                                                                    The reason GHC didn’t just turn on all flags by default is that many of them are mutually incompatible, so your individual .hs file has to pick a compatible set of language features it wants to work with.

                                                                                                                                                    You keep saying this in multiple places, but it’s not true. Virtually no GHC extensions are incompatible with one another. You have to work hard to find pairs that don’t get along and they involve extremely rarely used extensions that serve no purpose anymore.

                                                                                                                                                    The community is also not divided on how to do dependent types. We don’t have two camps and two proposals to disagree about. The situation is that people are working together to figure out how to make them happen. GHC also doesn’t contain bad hacks for dependent types, avoiding this is exactly why building out dependent types is taking time.

                                                                                                                                                    That being said, dependent types work today with singletons. I use them extensively. It is a revolution in programming. It’s the biggest step forward in programming that I’ve seen in 20 years and I can’t imagine life without them anymore, even in their current state.

                                                                                                                                                    Look at the commits going into ghc/ghc, and look at the activity on the bread-and-butter Haskell projects: lens, trifecta, cloud-haskell. Maintenance mode. Where are the bold new projects?

                                                                                                                                                    Haskell is way more popular today than it was 5 years ago, and 10 years ago, and 20 years ago. GHC development is going strong, for example, we just got linear types, a huge step forward. There’s been significant money lately from places like cryptocurrency startups. For the first time I regularly see Haskell jobs advertised. What is true, is that the percentage of Haskell questions on stack overflow has fallen, but not the amount. The size of Stack Overflow exploded.

                                                                                                                                                    Even the community is much stronger than it was 5 years ago. We didn’t have Haskell Weekly news for example. Just this year a category theory course was taught at MIT in Haskell making both topics far more accessible.

                                                                                                                                                    Look at the commits going into ghc/ghc

                                                                                                                                                    Let’s look. Just in the past 4 years we got: linear types, a new low-latency GC, compact regions, deriving strategies & deriving via, much more flexible kinds, all sorts of amazing new plugins (type plugins, source plugins, etc.) that extend the language and provide reliable tooling that was impossible 5 years ago, much better partial type signatures, visible type applications (both at the term level and the type level), injective type families, type in type, strict by default mode. And much more!

                                                                                                                                                    This totally changed Haskell. I don’t write Haskell the way I did 5 years ago, virtually nothing I do would work back then.

                                                                                                                                                    It’s not just GHC. Tooling is amazing compared to what we had in the past. Just this year we got HLS so that Haskell works beautifully in all sorts of editors now from Emacs, to vscode, to vim, etc.

                                                                                                                                                    look at the activity on the bread-and-butter Haskell projects: lens, trifecta, cloud-haskell. Maintenance mode. Where are the bold new projects?

                                                                                                                                                    lens is pretty complete as it is and is just being slowly polished. Haskell packages like lens are based on a mathematical theory and that theory was played out. That’s the beauty of Haskell, we don’t need to keep adding to lens.

                                                                                                                                                    I would never use trifecta today, megaparsec is way better. It’s seen a huge amount of development in the past 5 years.

                                                                                                                                                    There are plenty of awesome Haskell packages. Servant for example for the web. Persistent for databases. miso for the frontend. 5 years ago I couldn’t dream of deploying a server and frontend that have a type-checked API. For bold new ideas look at all the work going into neural network libraries that provide type safety.

                                                                                                                                                    I’m no fanboy. Haskell has plenty of issues. But it doesn’t have the issues you mentioned.

                                                                                                                                                    1. 1

                                                                                                                                                      Right. Most of my Haskell experience is dated: from over five years ago, and the codebase is proprietary, so there are few specifics I can remember. I’m definitely not the best person to write on the subject. In any case, I’ve rewritten the Haskell section of the article, with more details. Thanks.

                                                                                                                                                      1. 6

                                                                                                                                                        From my definition of “dying language” it means losing popularity, or losing interest. For Haskell this is absolutely not clear. Also your section is about “why Haskell is bad” not “why it is dying”. People do not talk about Haskell as they used to in my opinion, but I still see a lot of activity in Haskell ecosystem. And it doesn’t really look like it’s dying.

                                                                                                                                                        I think it is easier to agree about Clojure dying looking at Google trends for example: https://trends.google.com/trends/explore?cat=5&date=all&geo=US&q=haskell,clojure

                                                                                                                                                        But Haskell looks more like a language that will never die but still probably never become mainstream.

                                                                                                                                                        1. 5

                                                                                                                                                          I’m definitely not the best person to write on the subject. In any case, I’ve rewritten the Haskell section of the article, with more details. Thanks.

                                                                                                                                                          Great! Although there are still many issues that are factually untrue.

                                                                                                                                                          I think this is just a sign that you’ve been away from the community for many years now, and don’t see movement on the things that were hot 5-10 years ago. Like “The high-brow crowd was obssessed with transactional memory, parser combinators, and lenses.” Well, that’s over. We figured out lenses and have great libraries, we figured out parser combinators, and have great libraries. The problems people are tackling now for those packages are engineering problems, not so much science problems. Like how do we have lenses and good type errors? And there, we’ve had awesome progress lately with custom error messages https://kodimensional.dev/type-errors that you would not have seen 5 years ago.

                                                                                                                                                          The science moved on to other problems.

                                                                                                                                                          The issue is that different extensions interact in subtle ways to produce bugs, and it’s very difficult to tell if a new language extension will play well with the others (it often doesn’t, until all the bugs are squashed, which can take a few years).

                                                                                                                                                          This still isn’t true at all. As for the release cadence of GHC, again, things have advanced amazingly. New test environments and investments have resulted in regular GHC releases. We see several per year now!

                                                                                                                                                          In Atom, the Haskell addon was terrible, and even today, in VSCode, the Haskell extension is among the most buggy language plugins.

                                                                                                                                                          That was true a year ago, it is not true today. HLS merged all efforts into a single cross-editor package that works beautifully. All the basic IDE functionality you would want is a solved problem now, the community is moving on to fun things like code transformations.

                                                                                                                                                          Then there’s Liquid Haskell that allows you to pepper your Haskell code with invariants that it will check using Z3. Unfortunately, it is very limited in what it can do: good luck checking your monadic combinator library with LH

                                                                                                                                                          Not true for about 3 years. For example: https://github.com/ucsd-progsys/liquidhaskell/blob/26fe1c3855706d7e87e4811a6c4d963d8d10928c/tests/pos/ReWrite7.hs

                                                                                                                                                          The worst case plays out as follows: the typechecker hangs or crashes, and you’re on the issue tracker searching for the issue; if you’re lucky, you’ll find a bug filed using 50~60% of the language extensions you used in your program, and you’re not sure if it’s the same issue; you file a new issue. In either case, your work has been halted.

                                                                                                                                                          In 15 years of using Haskell I have never run into anything like this. It is not the common experience. My code is extremely heavy and uses many features only available in the latest compiler, with 20-30 extensions enabled. Yet this just doesn’t happen.

                                                                                                                                                          There is almost zero documentation on language extensions. Hell, you can’t even find the list of available language extensions with some description on any wiki.

                                                                                                                                                          Every single version of GHC has come with a list of the extensions available, all of which have a description, most of which have code: https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html You can link to the manual that neatly explains everything, rather than to the git repo.

                                                                                                                                                          Looking at the big picture: first, this is a poor way to do software development; as the number of language extensions increase, your testing burden increases exponentially.

                                                                                                                                                          This is only true if you can’t prove how extensions interact, or more fundamentally, that they don’t interact.

                                                                                                                                                          Second, the problem of having a good type system is already solved by a simple dependent type theory; you study the core, and every new feature is just a small delta that fits in nicely with the overall model.

                                                                                                                                                          That’s totally untrue. There is no such general-purpose language today. We have no idea how to build one.

                                                                                                                                                          As opposed to having to read detailed papers on each new language extension. And yes, there’s a good chance that very few people will be able to understand your code if you’re using some esoteric extensions.

                                                                                                                                                          Again, that’s just not true. You don’t need to know how the extensions are implemented. I have not read a paper on any of the extensions I use all the time.

                                                                                                                                                          In summary, language extensions are complicated hacks to compensate for the poverty of Haskell’s type system.

                                                                                                                                                          That’s just the wrong way to look at language extensions. Haskell adds features with extensions because the design is so good. Other languages extend the language forcing you into some variant of it because their core is too brittle and needs fundamental changes. Haskell’s core is so solid we don’t need to break it.

                                                                                                                                                          However, PL research has shifted away from Haskell for the most part

                                                                                                                                                          That’s again totally factually untrue. Just look at Google Scholar, the number of Haskell papers per year is up, not down. The size of the Haskell workshop at ICFP is the same as 5 years ago.

                                                                                                                                                          Moreover, there are no tools to help you debug the most notorious kind of bug seen in a complicated codebase: memory blowups caused by laziness.

                                                                                                                                                          Again, that’s not factually true.

                                                                                                                                                          We have had a heap profiler for two decades, in the past few years we got ThreadScope to watch processes in real time. We have systematic ways to find such leaks quickly, you just limit the GC to break when leaks happen. https://github.com/ndmitchell/spaceleak We also got stack traces in the past few years so we can locate where issues come from. In the past few years we got Strict and StrictData.

                                                                                                                                                          As for the code examples. I can pick 2 lines of any language out of context and you’ll have no idea what they do.

                                                                                                                                                          Who cares what every extension does for every example? That’s the whole point! I have literally never looked at a piece of Haskell code and wondered what an extension does. I don’t need to know. GHC tells me when I need to add an extension and it tells me when an extension is unused.

                                                                                                                                                          How many more language features are missing?

                                                                                                                                                          Extensions are not missing language features.

                                                                                                                                                        2. 1

                                                                                                                                                          GHC also doesn’t contain bad hacks for dependent types, avoiding this is exactly why building out dependent types is taking time.

                                                                                                                                                          Honestly, I’d much rather prefer a simple core model, like that of HoTT.

                                                                                                                                                          1. 3

                                                                                                                                                            Honestly, I’d much rather prefer a simple core model, like that of HoTT.

                                                                                                                                                            I’d love that too! As would everyone!

                                                                                                                                                            But the reality is, we don’t know how to do that. We don’t even know how to best represent computations in HoTT. It might be decades before we have a viable programming language. We do have dependent types that work well in Haskell today, that I can deploy to prod, and that prevent countless bugs while making code far easier to write.

                                                                                                                                                            1. 1

                                                                                                                                                              I think HoTT with computations is “cubical type theory”? It’s very active currently.

                                                                                                                                                              As for the dependent types as the backend for advanced type level features, I think it’s what Dotty/scala 3 is about. It’s definitely not the only way to do it, but it’s also not decades away. Idris 2 is also an interesting effort.

                                                                                                                                                        3. 4

                                                                                                                                                          Dependent types aren’t that useful for production software, and full blown dependent types are really contrary to the goals of Haskell in a lot of ways. Any language that’s >20 years old (basically 30) is gonna have some band-aids. I’m not convinced that Haskell is waning in any meaningful way except that people don’t hype it as much on here/hn. Less hype and more doing is a good thing, imho.

                                                                                                                                                          1. 3

                                                                                                                                                            Reminds me of the days when people said FP and complete immutability weren’t useful for production software. It is true that there is no decent general purpose language that implements dependent types, but that’s besides the point.

                                                                                                                                                            It’s true, hype is a poor measure.

                                                                                                                                                            1. 4

                                                                                                                                                              Yeah, that’s an interesting comparison, but I think it’s a totally different situation. Immutability and dependent types both are things you do to make certain assumptions about your code. In that, immutability allows you to know that some underlying value won’t change. Dependent types allow you to make more general statements/proofs of some invariant. The big difference is that immutability is a simplification. You’re removing complexity by asserting some assumption throughout your code. Generally, dependent types are adding complexity. You have to provide proofs of some statement externally or you have to build the proof of your invariants intrinsically into your constructions. IMHO, that’s a huge difference for the power to weight ratio of these two tools. Immutability is really powerful and fairly light weight. Dependent types are not really that powerful and incredibly heavy. I’m not saying dependent types are worthless. Sometimes you really really want that formal verification (eg. compilers, cryptography, etc). The vast majority of code doesn’t need it, and you’re just adding complexity, something I think should be avoided in production software.

                                                                                                                                                              1. 3

                                                                                                                                                                Tl;dr I have a good amount of experience with dependently typed languages, and I write Haskell for a living. After all of my experience, I have come to the conclusion that dependent types are over hyped.

                                                                                                                                                                1. 1

                                                                                                                                                                  I’ve started writing a post on dependent types. Here’s early draft: https://artagnon.com/articles/dtt

                                                                                                                                                                2. 3

                                                                                                                                                                  What about Ada?

                                                                                                                                                          1. 1

                                                                                                                                                            Druid

                                                                                                                                                            I think this deserve a word here. I worked with it around 2014-2016. Now the project is passed to apache.

                                                                                                                                                            This was like a “secret weapon” for our team at that time. I worked on make a social media analytics (think a dashboard for twitter/FB etc…)

                                                                                                                                                            With that we were able to provide analysis in real time about a ridiculously big amount of data. It was very hard to configure correctly. But once done, it was incredible. You could click on a button and hundreds of aggregations were processed about ten of millions big json object in less than 200ms. That was a really cool tool.

                                                                                                                                                            Here is a short presentation about it: http://yogsototh.github.io/mkdocs/druid/druid.reveal.html#/

                                                                                                                                                            1. 1

                                                                                                                                                              Very cool I was thinking about building something similar

                                                                                                                                                            1. 1

                                                                                                                                                              I use org-publish + a few scripts to generate an RSS feed and optimize the size of the website by taking advantage of a full HTML+CSS minimizer. I previously used hakyll, and nanoc. I tried to have a CSS that look like a markdown in a terminal.

                                                                                                                                                              my blog: https://her.esy.fun

                                                                                                                                                              1. 2

                                                                                                                                                                Having tried just about all kinds of static site generators under the sun—from the more mainstream ones like Jekyll and Hugo to more exotic ones like ssg, ox-hugo, org-page, Org publish, org-static-blog, Haunt, and even a custom one written in Haskell—I’m now back to hand-written HTML files + SSI rules (for simple templating), and love the simplicity. The only thing missing right now is an Atom feed. I wonder if I could use GNU M4 for that, like @technomancy does.

                                                                                                                                                                Result at https://bandali.eu.org, “sources” at https://git.bandali.eu.org/site.

                                                                                                                                                                1. 2

                                                                                                                                                                  If your page structure is more or less consistent (looks like it is), you can extract metadata from pages with an HTML parser and generate a feed from it. That approach allows some things that are impossible or unwieldy in the traditional paradigm, such as using an arbitrary paragraph for post excerpt, not the first one.

                                                                                                                                                                  My own generator automates that process: the blog index of soupault.neocities.org/blog is produced by this config. It can dump exported metadata to JSON, which isn’t hard to produce Atom from. I’m still to complete the JSONFeed/Atom generator script good for public release.

                                                                                                                                                                  That said, making a custom page to Atom script using an HTML parsing library that supports querying the data with CSS selectors (BeautifulSoup, lambdasoup etc.) isn’t that hard if making it work for anyone else’s site isn’t a goal.

                                                                                                                                                                  1. 1

                                                                                                                                                                    Indeed; that’s one of the approaches I’m considering. Also, soupalt seems quite interesting, thanks for the links; I’ll check it out!

                                                                                                                                                                  2. 1

                                                                                                                                                                    I also missed a tool to generate RSS from a tree of HTML files. This is how I generate my RSS with a very basic shell script using html-xml-utils:

                                                                                                                                                                    https://her.esy.fun/posts/rss-gen.html

                                                                                                                                                                    1. 1

                                                                                                                                                                      Nice, thanks for sharing!

                                                                                                                                                                    2. 1

                                                                                                                                                                      Any particular reason you switched back to handwriting html files?

                                                                                                                                                                      1. 2

                                                                                                                                                                        Yeah, a few, actually:

                                                                                                                                                                        • I’d like my site setup to be very lightweight, both on the server and on my machine when editing, and use tools that are nearly universally available on all GNU/Linux systems. This rules out things like my custom static site generator written Haskell, since Haskell and its ecosystem are arguably quite the opposite of my two criteria above.

                                                                                                                                                                        • I’d like to have convenient complete control over the generated HTML output, and this is rather hard to come by in most existing static site generators and rules out most markup formats, since almost none are as expressive and flexible as HTML.

                                                                                                                                                                        Aside from the repetitive pre/postamble bits in each file, I find writing plain HTML quite alright, actually. The annoyance of HTML’s lack of built-in templating and simple conditionals can be solved by using SSI directives, which are fairly widely supported across web servers. Alternatively, I’m considering using GNU M4 in place of SSI, if it results in a simpler and cleaner setup. And it fits the two criteria in my first point above too.

                                                                                                                                                                        1. 3

                                                                                                                                                                          These are rather valid arguments. I used to write in plain html too, it’s perfectly fine, especially with an editor’s auto-completion like emmet. Nevertheless, I now mostly write in markdown and enjoy it. Whenever I need something more complex, I just embed html.

                                                                                                                                                                    1. 1

                                                                                                                                                                      My current main website is: https://her.esy.fun

                                                                                                                                                                      Interesting aspects:

                                                                                                                                                                      • org-mode instead of markdown (it’s way better IMHO)
                                                                                                                                                                      • use org-publish with a bit of magic but not too much (more infos here
                                                                                                                                                                      • different CSS choices (all support light/dark theme, I’m particularly fond of the sci dark one when there are images)
                                                                                                                                                                      • no tracker of any sort
                                                                                                                                                                      • no js (except for pages displaying formula with Mathjax)
                                                                                                                                                                      • RSS is generated via a self made shell script (I was quite surprised no tool like that existed before more info here
                                                                                                                                                                      • Optimize the size by using all CSS and HTML, surprisingly it was very efficient (~30% better than classics minimizers) even using quite naive approach more info here

                                                                                                                                                                      My older one was https://yannesposito.com. I used hakyll for this one.

                                                                                                                                                                      The hardest part is always to produce content, and not to lose too much time optimizing its blog tech. Still I love doing that time to time :).