1. 7

    Plus, when I see year as the first element I immediately know it’s this format.

    1. 2

      I wonder what’s informations critical mass. Professor A. Dońda did some research on that.

      1. 7

        The Bekenstein Bound. That’s the maximum amount of information you can store in a given volume of space before it collapses into a black hole.

      1. 4

        An interesting attempt to analyse the data by someone who has completely grabbed the wrong end of the stick.

        The age of citations is quite interesting. It definitely doesn’t reflect the velocity of cases through the system, but it might say something about memory of the aggregate legal profession.

        1. 2

          the wrong end of the stick

          What would be the correct end then?

          1. 8

            The end of the stick that doesn’t mistake ‘doing research-like actions’ for ‘doing research’, nor ‘graphs and statistics’ for ‘an understanding of the system’.

            To quote xkcd: “Liberal-arts majors may be annoying sometimes, but there’s nothing more obnoxious than a physicist first encountering a new subject.” In this case, for physicist read ‘data scientist’. [edit 17:25 CET: use ‘data scientist’ instead of another phrase –SBB]

            The dataset is incomplete in a very non-random way: it only includes transitive citations of Supreme Court decisions. Nowhere does the author betray any signs of realising that this affects the statistics, and therefore the conclusions they so confidently draw about, say, average duration of a judicial procedure.

            I don’t have the time for a full fisking, so I’ll only address the most important idiocy:

            The writer says “somewhere between 33% and 50% of all cases are completely trivial and could be easily automated away”. They base that only, I kid you not, only on seeing that somewhere between 33% and 50% of the judgements they gathered cite only one previous case or judgement.

            No, the author doesn’t discuss how to recognise those supposedly trivial cases; nor whether the right to appeal can even be fulfilled by an automated system; and nowhere do they wonder whether ‘the judgement cites only one case’ is, in fact, a reliable sign that a case was trivial.

            You have to understand the system you’re talking about! Even if you’re a data scientist! Especially if you’re a data scientist!

          2. 1

            I’d also like to hear more about the correct end of the stick from @Student .

            I’m not a data engineer at all, could you describe what you consider a better approach?

            1. 6

              For starters, citations don’t mean what you think they mean. Cases are cited because they are a source of law under discussion.

              You need to actually work with a domain expert to understand what the data mean.

          1. 3

            @denismerigoux A friend of mine pointed out that Poland bought something like Mlang(called Poltax) in 1990’s from BULL for 10 milion franks - and currently they are trying to replace it with a new version(PoltaxPlus). Nice niche.

            1. 8

              I would love to see a side-by-side with Catala Lang, which was posted here awhile back, though only the Git repo: https://lobste.rs/s/b74svy/catalalang_catala

              1. 27

                Hi! Author of both Mlang and Catala here :) So Catala is basically an evolution/reboot of the M language, but this time done right using all the PL best practices.

                1. 7

                  Wait, are you for real? That is absolutely fascinating! My wife is a lawyer (which makes me not a lawyer) and I am very interested in these types of intersections. Namely where a highly regimented and regulated domain gives rise to some type of formalism once exposed to CS through some “interdisciplinary process”.

                  I have studied DSL design peripherally but would really like to pick your brain about some things. I did once, long ago, design a policy language. Are you open to additional discussions and collaboration?

                  1. 10

                    Ha ha ha yes this area is fascinating. I have the impression that there’s a lot of people in legaltech that are all trying to make a DSL to express parts of the law but have no clue about how to properly make a DSL.

                    I am open to discussions and collaboration, moreover both Mlang and Catala are open-source and accept contributions. Hit me up using the email in the Mlang paper for instance :)

                    1. 2

                      As a lawyer designing my own DSL ;) I would love to know how using of Mlang has affected legislation. For example how do you deal with law being changed? Does your parliament creates updates as “diffs” or as already “merged” texts? Do you use lawxml? Soo many questions!

                      1. 4

                        The French laws are usually written in terms of “diff”. Also I had made a prototype that warned which articles of law your program was relying on were about to expire https://twitter.com/DMerigoux/status/1252914283836473345?s=19. I don’t use any form of XML, I just copy paste the law text to start writing a Catala program. XML would not improve the way Catala programs are written since the XML structure does not follow the logical structue of the law but rather its formatting structure, which we don’t care when translating it to executable code.

                  2. 3

                    Hi Denis - nothing constructive to say except that I am a British CS student and my friends and I are big fans of your work! In fact I think a friend of mine will be basing his undergraduate thesis on your ideas :-)

                    1. 3

                      Thanks Jack! Well if your friend does end up basing his undergrad thesis on Catala or else please drop me an email, I’ll be happy to give feedback or suggest interesting things to look at.

                    2. 2

                      but this time done right using all the PL best practices.

                      Does this mean that DGFiP is migrating to something one of the implementers considers not done right?

                      1. 4

                        I suppose it’s easier to migrate step by step: Improve the tooling, so that everything can be in the open without security concerns and so the system can evolve more easily from its apache cgi-bin roots. That’s what MLang seems to offer.

                        Once that’s in place, there can be further steps to improve the language (e.g. by introducing Catala) because the foundations are state of the art again. And even if that doesn’t happen, the system is still better off than before because it’s a single system instead of a single system + 25 years of wrappers that extend it ad-hoc.

                        1. 3

                          I could not have said it better!

                        2. 2

                          Migrating to Mlang improves the compiler but the M language stays the same. For instance, in DGFiP’s M, there are no user-defined functions. And the undefined value in M is a contant reminder of the “billion dollar mistake”. So yes we can definitely improve the M language from its 1990 design :)

                        3. 2

                          I want to just praise you for the time and effort you put into this space. I’ve recently got into “hobbyist” law myself, specifically Canadian law (http://len.falken.ink/law/101.txt), and instantly had the same thoughts: where are the formal proofs? :) Sure there are tax calculators, and some will creators, but are they rigorous? Can they tell us other properties of a situation?

                          I’m 100% going to play with Catala. This is technology worth spending time on because law governs our every day lives.

                          1. 1

                            I’m just curious: who is driving all this? Is this simply something you one day decided to go and implement, or were you approached by someone to do this seemingly huge project? How do you get it financed, did you have backing from the start?

                            Fascinating stuff!

                            1. 10

                              I started looking into this after watching this talk: https://youtu.be/EshxZVMURt4. I always wondered whether it was possible for me to play with formal methods outside the traditional application domains like security or safety-critical embedded systems. Then I fell into a rabbit hole :) I started with a Python prototype of French law encoded into SMT, then moved to try and use the DGFiP code and ended up coding Mlang, then created Catala as a next logical step. I created these on my spare time during my PhD and was helped by some friends who contributed to the open source repos. I’m only starting now to have institutional backing! During a French PhD, your funding is secured for the whole duration from the start so I didn’t have to worry about it and could focus on other things. I would say stable and long-term unconditional funding enabled me to create all this. In my opinion research should promote that instead of the myriad of tiny little funding sources, each of them requiring a lot of paperwork to fill. But in that regard I go against the zeitgeist.

                        1. 12

                          A post announcing a vulnerability in a C program, and a comment thread on “rewrite in rust yes/no”

                          Also, such comments could be created by default.

                          1. 5

                            That’s a good idea. Maybe we could offload all bike shedding to adversarial neural networks. We can check in every once in a while to see if anyone has won.

                          1. 20

                            I just have been fired, so I think I will take some time to set back and enjoy my newborn.

                            1. 6

                              Sorry to hear that. May better opportunities come your way

                              1. 3

                                That is life. I will survive and come out stronger.

                              2. 3

                                Congratulations!

                                1. 1

                                  Thanks

                                2. 2

                                  Ouch, hope that doesn’t put you in a difficult situation financially.

                                  1. 3

                                    No, I have some savings, I am left with generous additional month of payment, so I will manage. With my skillset I think that I will find next job within a month.

                                  2. 2

                                    Wow that’s rough; I’m sorry to hear that.

                                    1. 1

                                      It isn’t that bad. I am not bragging, just answering the question ;) I would say it is mild inconvenience, nothing to be worried about. With my skillset I think I will find another gig soon.

                                    2. 1

                                      Sorry for that, but in the other hand, congrats!

                                    1. 5

                                      People seem to miss the point of OO. The point is about names. The example defines save_items, but realistically, to avoid name clashes, it would be client_save_items. Similarly, “bag of functions” leads to names like treemap_add and hashmap_add, which is redundant, because it can be inferred from the runtime type of the first argument. The point of OO is to avoid redundancies of TreeMap and treemap_add and HashMap and hashmap_add and collapse them to add.

                                      1. 6

                                        For purely namespacing purposes I personally prefer modules over classes in Python, in a lot of cases. Put a collection of functions into foo.py, then import foo from another file, and all those functions are nicely namespaced as foo.name. Admittedly classes vs modules depends a lot on the project and programming style.

                                        1. 2

                                          treemap.add is not really an improvement over treemap_add. The point is you write m.add(k, v), not m.treemap_add(k, v). Compare treemap_add(m, k, v) and treemap.add(m, k, v).

                                        2. 3

                                          I think I can safely speak for Alan Kay when I say that this is not the point of OO.

                                          The points of OO are dynamic/late binding, encapsulation, and inheritance, IMHO and IIRC.

                                          However, I agree that OO-style naming is a good thing. You can have it without objects using function overloading, which pairs well with a universal-function-call syntax as found in Nim and some other languages.

                                          1. 5

                                            “Inheritance” was definitely not included in Kay’s famous quote:

                                            OOP to me means only messaging, local retention and protection and hiding of state-process, and extreme late-binding of all things.

                                            I argue that inheritance is an anti-pattern, and is in large part responsible for OO hate. OO with composition only and immutable value objects can be pretty nice.

                                            EDIT: This got me to search for something more specific on Kay’s feelings about inhertiance, and I found this great post by him on Quora

                                            1. 3

                                              These days Alan Kay says that OO is about message passing. I’ve seen some debate about whether that was always his message. hwayne probably has a more informed opinion on this than I do.

                                              1. 5

                                                Given the history, I suspect that Alan Kay did a very poor job of explaining what he did and has been yelling at us ever since for failing to read his mind correctly.

                                                1. 1

                                                  Dealers of Lightning has a chapter on how Kay was disappointed with Smalltalk and wanted to start again, but the team basically told him no.

                                                  BTW: When I asked him about Self, he told me:

                                                  I liked Self. “Good OOP” is still waiting for a much better notion to replace the idea of a “Class”

                                              2. 1

                                                If OO is about dynamic/late binding etc, C++ without virtual functions shouldn’t be OO at all. But to the contrary, C++ without virtual functions looks and feels like C with OO, and virtual functions feel like a superfluous addition.

                                                1. 3

                                                  C++ without virtual functions shouldn’t be OO at all.

                                                  Alan Kay would agree: “I made up the term object-oriented, and I can tell you I did not have C++ in mind.”

                                                2. 1

                                                  BTW: Only when I’ve read the book Moldable tools, I’ve finally got some of Kay’s points of OOP and “modeling”.

                                                3. 1

                                                  The point was about “enterprise environment”, design patterns and Java eating our lunch. Ohh, how many times have I seen people complaining about perl using an arrow instead of a dot.

                                                  1. 1

                                                    An arrow? That’s so silly — ooRexx uses a ~ twiddle.

                                                1. 1

                                                  Why would a developer choose to use this editor over something like, say, NeoVim?

                                                  1. 2

                                                    Neovim is a modal editor with lots of legacy and complexity (from vim), lite is a simple editor with little to no legacy and low complexity.

                                                    It’s hard to compare them to each other as they are fundamentally different. It’s not something inherently good or bad.

                                                    I’m very familiar with using vim but I can’t imagine using vim for anything more than minor edits in textfiles on remote systems. For development I’d use something else.

                                                    I’m using lite on my work-work machine as a replacement for notepad (as we are a Windows shop). It works great for that, at least this far.

                                                    1. 1

                                                      I think now it’s just a fun little project for doing things the other way.

                                                    1. 25

                                                      From time to time this seems to pop up, and they talked as if it was never tried at big scale. But it’s wrong, it has been already tried. In my country, Spain, every citizen has a digital certificate (request process is tedious, you need a Java program and a going physically to some places) but you can use it at any bureaucratic thing, including paying taxes, buying national debt, request certificates from the judge, … But also you can use it on some banks and even in my university. I, as a user, think that it has some real benefits but also some cons.

                                                      However, in the end, most people don’t like the workflow. Administrations have put login/password systems for the most commonly used services because of the request process, how to secure the certificates, how to move them between computers and so on it’s complicated. Also, the web browser interfaces are bad and scary, and for a lot of the time, some browsers (Firefox looking at you) rejected the certificates as invalid (Chrome and IE did just fine).

                                                      Some other people prefer the electronic ID card, which is similar but you need special hardware, but in the end people reason about that as easier than a pure digital certificate.

                                                      1. 3

                                                        Do you think it would be better with a USB smart-card such as a Yubikey or something similar? IMO any system that expects a user to know how to secure and use a certificate file is a bad system, but it’s a usability problem, not really a technical one.

                                                        1. 7

                                                          That’s basically WebAuthentication.

                                                          1. 1

                                                            I’m talking like a government issued USB smartcard or hardware token, not the APIs that would use those tokens.

                                                            1. 2

                                                              I believe this is what Estonia does with “e-Estonia”; I don’t really know of the details though.

                                                              1. 2

                                                                Yes – we get an ID card which is also a smartcard. You use it with something like this, and you can authenticate and sign using two different PINs on it. It uses the normal smartcard APIs so it tends to Just Work in browsers on major operating systems. They recently introduced a smartphone version that you can use alongside, but you need to register first using either the smartcard or visiting a bank.

                                                          2. 3

                                                            Sweden’s Bank ID makes managing certificates really easy. I use it to authenticate to everything from my bank, the tax authority, my company’s payroll software, and our kid’s school’s attendancy interface.

                                                            The certificates are handled via my bank. The initial startup requires an personal visit with ID, after that you get a reminder every few years that it’s time to renew.

                                                            https://www.bankid.com/en/

                                                            1. 2

                                                              Same in Luxembourg with https://www.luxtrust.lu/ it works really well and you have the start card version (mostly for corporate) and the 2FA version.

                                                              1. 1

                                                                Looks very similar but instead of government-issued, they’re bank-issued :)

                                                                1. 2

                                                                  Here in Sweden, the traditional issuers of ID cards have been banks. It’s changed now since the rules around passports have tightened, so the government offers a “national ID card” in addition to a passport (which is now accepted as ID - it used not to be up to the standards of ID cards).

                                                                  You do need a bank “partnership” (usually a checking account) but these are easy to get, and free up until you’re starting to earn some money. Our kid got an account, debit card and a Bank ID at 16.

                                                              2. 1

                                                                Yes, the electronic ID card I was talking at the end is a SmartCard with password protection, but you need a reader and (another) Java program in the middle. If you forget your password you can go to the police office and they have machines that reset the password. It is not perfect but it feels more natural for regular users.

                                                              3. 1

                                                                It may not have worked out that well in Spain, but in Estonia it seems to be working well (from the outside, at least. Any Estonians can feel free to correct me). I don’t like how much information is stored in the ID, but the digital signature and ID verification seems to work well, at least according to Wikipedia https://en.wikipedia.org/wiki/Estonian_identity_card#Uses_for_identification

                                                                1. 1

                                                                  The identity card of Estonia and the one from Spain are the same model! When a security bug was discovered in Estonia they needed to invalidate a whole bunch of cards here, because they were the same :) However, the physical smartcard and the pure client-side certificate (which the post was talking about) are two related but different systems (which I think complicates more its understanding for the normal citizen). The smartcard flow is not really standard, needs a Java program in the middle.

                                                                2. 1

                                                                  In Poland it’s other way around - you may use your bank account as some sort of id. It works surprisingly well, some formalities may be done using your bank’s interface.

                                                                1. 7

                                                                  For many centuries, the existence of zero was not even acknowledged. Even after it was, it was highly controversial and took a long time to come into vogue.

                                                                  People are culturally prejudiced against 0. It makes sense that people find the use of 0 as an index less ‘natural’ than 1, but the reasons for that aren’t inherent to our psyche or to the world; they’re purely cultural.

                                                                  Zero was an excellent innovation and I whole-heartedly welcome it: into the domain of mathematics, into the domain of programming languages, everywhere else. It’s a generalisation of magnitude, and a more natural way to count.

                                                                  1. 2

                                                                    We totally should use Roman numerals for indexing.

                                                                    1. 2

                                                                      INTERCAL didn’t go far enough, huh?

                                                                  1. 2

                                                                    Does anyone have any recommendations for good epaper/eink devices? A lot of what I’ve seen are ereaders/tablets, which I’m not as interested in.

                                                                    1. 1

                                                                      What size/interface/battery/input requirements do you have?

                                                                      Just a monitor, monitor + touch, hdmi/spi, mobile/fixed?

                                                                      1. 2

                                                                        Because it’s a single company who owns the patents, I think you can get an exhaustive list of products here: https://shopkits.eink.com/

                                                                        1. 1

                                                                          I’m curious about any that are an assembled thing. Monitors, laptops, phones.

                                                                          1. 1

                                                                            There used to be an hdmi display made by Waveshare and there were a few phones - Yotaphones.

                                                                        2. 1

                                                                          I got one from Waveshare this month, but you’ll have to get/build the enclosure yourself. My 7” one fits nicely in a 4x6 photo frame.

                                                                        1. 2

                                                                          Some links don’t work:

                                                                          http://foldoc.doc.ic.ac.uk/foldoc/foldoc.cgi?query=continuation+passing+style
                                                                          http://foldoc.doc.ic.ac.uk/foldoc/foldoc.cgi?query=closure
                                                                          
                                                                          1. 4

                                                                            It could be a bit more responsive, it’s pain to use on mobile. Otherwise very cool!

                                                                            1. 2

                                                                              @isthatforreal thank you very much for your feedback! I knew it was pretty janky on mobile, but… I think I was just too lazy to read a couple responsive design tutorials :) I’ve gotten imgs and pre blocks to scale down correctly, so I think it’s a vastly better experience now on mobile.

                                                                              1. 2

                                                                                Thank you for giving me another reason to give Janet a try.

                                                                            1. 3

                                                                              Queue management for inbound digital content Essentially information overload how do we deal with the non-stop barrage of information in a way that is effective, efficient and still valuable. IMO an untapped problem that is only getting worse.

                                                                              1. 4

                                                                                RSS is ok in that role. I find news using RSS, select them on my mobile, then push to my Kindle. Often awesome content turns out to be just a clicky title and very little content.

                                                                                1. 3

                                                                                  Yeah, the “read later” flag that most newsreaders have is very useful. I use it constantly in Feedly. I also like Feedly’s “Boards”, which are just customizable tags for articles, since you can add any URL to them, making them a nice personal bookmark-tagging system.

                                                                                2. 2

                                                                                  I don’t find the queue to be much of a problem. Yes, it is annoying to have so many “unsynchronized input queues” but on the other hand the flexibility to add and remove queues all the time is also necessary.

                                                                                  Currently, I focus on my digital brain and I find the act of connecting informations more valuable than the ability to trace it back to its source. If I can it link, it works fine. For video, personal communication, chat, it does not work well, but at least the information is in my system.

                                                                                1. 18

                                                                                  I like to think about these things, but don’t have much hope. Here are my points:

                                                                                  • Networking shouldn’t be an afterthought. Distributed computing should not be as difficult as it is. Transparently interacting or using resources from other systems should be something you don’t have to think about. I don’t care about hardware. I don’t care about CPU architectures. I don’t care about GPUs. I don’t care about drivers. All computers form an transnational turing machine.
                                                                                  • Object capabilities should be a primitive concept. Imagine sharing a screen: That shouldn’t be the hassle it is, you should just be able to give someone read access to a segment or the whole display. The same applies to Files (but we probably shouldn’t have files), Hardware access, etc.
                                                                                  • Hypertext should be a everywhere. The web has shown how attractive the idea is, but browsers are cursed to contain it, which is getting harder and harder. Project Xandu had good ideas about this, and HTTP is a shallow copy. We need complex links that can point to miscelanious parts of the system, and ideally also have back-references. You probably want a lof of cryptography for something like thise, to avoid the centralisation of power.
                                                                                  • Logic and UI should be separate. Unix programms regard the standard output and input as the default UI, everything else is a side effect. Instead we should have the ability for a program (or procedure, algorithm, …) to produce complex data, that doesn’t only mean something in a specific environment (Powershell), but is universally understood. A terminal-like environment could display the results line-by-line, but it should be transformed into a graphical representation using a table, or a graph (or whatever one might come up with later).
                                                                                  • Programming should not be a specialist’s affair. We have two classes of people, those who are at the mercy of computers, and those who can use them. This shouldn’t be the case, because the former are in a much weeker position, getting lost, getting overwhelmed, and sometimes even abused by those who know better. A proper operating system cannot be based on the lie, that you don’t need to know anything to use a computer: To be a responsible user, you need to know some basics. A simple programming language (I would like something like Scheme, but that’s just be) should be integrated into the system, and the user shouldn’t fear it. It’s a direct link to the raw computational power that can be used.

                                                                                  In some sense, I like to think of it like Plan 9, without the Unix legacy, but that seems to simplistic. The interesting thing about Unix, is that despite it’s limitations, it creates the fantasy of something better. Inbetween it’s ideal power and it’s practical shortcomings, one can imagine what could have been.

                                                                                  1. 14

                                                                                    Programming should not be a specialist’s affair. We have two classes of people, those who are at the mercy of computers, and those who can use them. This shouldn’t be the case, because the former are in a much weeker position, getting lost, getting overwhelmed, and sometimes even abused by those who know better. A proper operating system cannot be based on the lie, that you don’t need to know anything to use a computer: To be a responsible user, you need to know some basics. A simple programming language (I would like something like Scheme, but that’s just be) should be integrated into the system, and the user shouldn’t fear it. It’s a direct link to the raw computational power that can be used.

                                                                                    I think the ultimate problem is that most people don’t want to program. They want to accomplish a task, and for the most part, someone else has programmed the tool to accomplish the task. They don’t want to build the tool. Us freaks who want to write tools are few and far. It’s trhe same reason cars have mechanics.

                                                                                    1. 5

                                                                                      I don’t think that programming has to be the same as “building the tool”, but more along the lines of what @spc476 mentions with Excel. Especially when you take “Logic and UI should be separate”, one can imagine that programming doesn’t even have to mean “writing text in a text editor”, but could be a GUI afair, where you work on connection tools in a graphical representation, trivially connecting components of your system, without depending on another tool.

                                                                                      Yes, not everyone want’s to be a car mechanic, nor do I, but to drive a car you need to get a drivers license, and that is the reason we can assume people can take some basic responsibility. We don’t have that for computers, and that’s why the responsibility has to be delegated to Microsoft or Apple. If we want to think of a computer as a tool, not a toy, I argue that a basic understanding for computational thinking should be assumable, and would help everyone.

                                                                                      1. 3

                                                                                        I sincerely believe enso (née Luna) has a serious fighting chance to fill this gap. Though they’re taking their time :)

                                                                                    2. 11
                                                                                      • Networking: QNX was network transparent. It was wild running a command on computer 1, referencing a file from computer 2, piping the output to a program on computer 3 which sent the output to a device on computer 4. All from the command line. The IPC was fast [1] and network transparent, and used for just about everything.
                                                                                      • Hypertext: The only operating system I know of that uses extensive form of hypertext is TempleOS (I don’t think it’s HTML but it is a form of hypertext) that extends pervasively throughout the system.
                                                                                      • Logic and UI: There are bits and pieces of this in existence. AmigaOS has Rexx, which allows one to script GUI programs. Apple has (had?) something similar. Given that most GUI based programs are based around an event loop, it should be possible to pump events to get programs to do stuff.
                                                                                      • Programming: True, but there is Excel, which is a programming language that doesn’t feel like one. Given an easy way to automate a GUI (similar to expect on the command line), and teaching people that computers excel (heh) at repeated actions could go a long way in giving non-programmers power.

                                                                                      [1] In the early-to-mid 90s, I had friends that worked at a local software company that wrote and sold custom X Window servers. Their fastest X Window server ran on QNX.

                                                                                      1. 3

                                                                                        Programming: True, but there is Excel, which is a programming language that doesn’t feel like one. Given an easy way to automate a GUI (similar to expect on the command line), and teaching people that computers excel (heh) at repeated actions could go a long way in giving non-programmers power.

                                                                                        One program idea I’ve had was a spreadsheet that users could “compile” into a simple gui. Analysts already use it as an adhoc RAD tool. Why not give them an actual custom GUI for their efforts?

                                                                                        1. 3

                                                                                          There was something like that in KDE, it was called krusader or something like that.

                                                                                      2. 8

                                                                                        Transparently interacting or using resources from other systems should be something you don’t have to think about.

                                                                                        Then everyone will run headlong into the fallacies of distributed computing, unfortunately. This is why things like CORBA and DistriibutedObjects failed. Networking is not transparent, much as we would like it to be.

                                                                                        At least not in a normal imperative programming paradigm, like RPC. You can get a lot of transparency at a higher level through things like async replication, e.g. Dropbox or [plug] Couchbase Mobile. But even then you have to be aware & tolerant of things like partitions and conflicts.

                                                                                        1. 4

                                                                                          Your first point is pretty much what Barrelfish is designed for, go check it out!

                                                                                          1. 4

                                                                                            Programming should not be a specialist’s affair. We have two classes of people, those who are at the mercy of computers, and those who can use them.

                                                                                            Indeed, the power dynamics are way out of control in software today. On the orange site, @akkartik describes this well via an analogy to food production: nearly all software today is restaurant-style, while almost none of it is home-cooked.

                                                                                            For anyone interested in this topic, I would suggest looking into the Malleable Systems Collective (the Matrix room is most active) and Future of Coding communities, as it comes up in those places regularly.

                                                                                            1. 3

                                                                                              Programming should not be a specialist’s affair.

                                                                                              Jonathan Edwards has been working on this problem for a long time. It goes well beyond the OS.

                                                                                              1. 2

                                                                                                The same applies to Files (but we probably shouldn’t have files)

                                                                                                Could you elaborate on this? Why no files?

                                                                                                Logic and UI should be separate. Unix programms regard the standard output and input as the default UI, everything else is a side effect. Instead we should have the ability for a program (or procedure, algorithm, …) to produce complex data, that doesn’t only mean something in a specific environment (Powershell), but is universally understood. A terminal-like environment could display the results line-by-line, but it should be transformed into a graphical representation using a table, or a graph (or whatever one might come up with later).

                                                                                                There was an interesting newsletter post about emacs being interface independent. I’m not too familiar with emacs, but it struck me as an intriguing and beautiful idea.

                                                                                                1. 5

                                                                                                  Could you elaborate on this? Why no files?

                                                                                                  Maybe it’s clearer, if I say file system. It might be too much to throw out the concept of a digital document, but I have come to think that file systems, as we know them on POSIX systems, are too low level. Pure text, without hyperlinks would be a wierd thing in an operating system where everything is interconnected, and why directories shoudln’t have to be a simple tree (because tools like find(1) couldn’t do proper DFS in the 70’s), but instead could be any graph structure of sets, or even computed.

                                                                                              1. 2

                                                                                                It’s probably the best option for text processing. Unicode, pcre, marpa. Mojolicious is awesome.

                                                                                                1. 65

                                                                                                  Giving up social media and HN.

                                                                                                  1. 16

                                                                                                    It was Twitter for me.

                                                                                                    1. 10

                                                                                                      lobste.rs except ~once a week for me, sadly. But having a reading diet, especially an online one, is in any event a good thing.

                                                                                                      1. 5

                                                                                                        When I stopped binging lobsters I started reading reddits r/all, so that didn’t improve anything for me - only made it worse ?

                                                                                                        1. 3

                                                                                                          Oof, yeah – r/all is a timesuck cesspool. I’m still working on getting off that.

                                                                                                          1. 8

                                                                                                            How can someone get stuck on r/all ? It’s full of nothing but pictures and memes. The current state of r/all was what cured my reddit addiction!

                                                                                                            1. 2

                                                                                                              endless scrolling + sometimes something interesting to read - mostly like watching bad TV stuff (except I haven’t watched TV since 8 years)

                                                                                                    2. 3

                                                                                                      HN

                                                                                                      I still go back to HN, for the wide variety of content under the “intellectual curiosity” umbrella that gets posted. It’s something I miss if I don’t go on HN and I wish there were lobste.rs-like communities for this type of thing.

                                                                                                      1. 1

                                                                                                        I even would pay for such thing. Going through all low quality stuff posted on hn takes time and time is the money. I had some hopes with amazon’s “Singles Classic”, they have published “Secrets of the Little Blue Box” but it seems to be dead now.

                                                                                                      2. 3

                                                                                                        The social media resistance intensifies. I believe the early adopters will be some of the first to exit the platforms.

                                                                                                        That is, if they’re willing to torch their follower count to be free. My two cents: it’s worth it; and the influence you believe you have via those mechanisms is not quite what you think it is.

                                                                                                      1. 21

                                                                                                        Personally, I think this is a good thing, though of course it’s so easily circumventable by apps.

                                                                                                        Whenever I’m in the position where I’m in an app using sign-in-with-some-other-service and it’s showing me an internal web view, I’m usually very anxious about who else is getting access to my password: The makers of the app? The analytics SDK of the makers of the App? The advertising SDK of the makers of the app?

                                                                                                        By forcing the sign-in to work through an external browser, I can be sure (to some extent) that the password will go only to the site itself (barring malware or bad extensions, of course)

                                                                                                        1. 3

                                                                                                          Microsoft has tried it and they failed. And I bet chrome in gmail will not be affected.

                                                                                                          1. 6

                                                                                                            And I bet chrome in gmail will not be affected.

                                                                                                            Oh - I’m sure it won’t. But google can trust their own gmail app.

                                                                                                            Or rather: If you don’t want to give your gmail password to the gmail app because you don’t trust your gmail app with your gmail password, then I don’t see how you could still be able to access gmail through the app.

                                                                                                            Or: If google wants access to your password, it can just take the password you submit through the embedded web view of their own app. It doesn’t need to exfiltrate it using JS injected into their own web view by their own app.

                                                                                                            1. 1

                                                                                                              I was thinking about logging into other google services using chrome embedded into gmail app. But it also makes little sens. Silly me!

                                                                                                        1. 2

                                                                                                          Some other awk-like languages from the Oil wiki [1]

                                                                                                          (1) Miller: https://miller.readthedocs.io/en/latest/reference.html

                                                                                                          Impressive in many ways, but it looks like it started as “cut” for CSV/TSV, and then grew a programming language …

                                                                                                          (2) TXR, a Lisp-based language, although it also has relatively confusing syntax: http://nongnu.org/txr/

                                                                                                          I get the feeling that these languages are “too different to be worth it”, and tend to grow beyond their domain of applicability.

                                                                                                          More specifically, I would draw a distinction between streaming lines of text and operating on tables. The former feeds into the latter. In other words, clean your data with awk, then analyze it with SQL or R.

                                                                                                          SQL and R are both extremely powerful and can do basically anything these languages can. But sometimes you do want a small sharp tool to cut down a big data set, since it’s easy to use both SQL and R in extremely slow ways.

                                                                                                          But it also means that you don’t want these languages to grow too many features, like mathematical libraries, etc. because that is better done with the “real thing” as a separate step.

                                                                                                          What is a Data Frame? (In Python, R, and SQL)

                                                                                                          [1] https://github.com/oilshell/oil/wiki/ExternalResources

                                                                                                          1. 6

                                                                                                            Perl is also a pretty good replacement for awk. It’s installed everywhere, runs on everything.

                                                                                                            1. 7

                                                                                                              And it behaves the same on Linux and BSD ;)

                                                                                                              1. 5

                                                                                                                You reminded me of a2p!

                                                                                                            2. 1

                                                                                                              Re: What is a Data Frame? (In Python, R, and SQL), your SQL is awkward because it’s approximating an imperative style.

                                                                                                              It would be nice if SQL had Python-like scalar variables (i.e. variables that aren’t tables).

                                                                                                              You can avoid this using window functions:

                                                                                                              with total_hits_per_url as (
                                                                                                                select url, sum(hits) as hits
                                                                                                                from traffic
                                                                                                                group by url
                                                                                                              )
                                                                                                              select url, hits * 100.0 / sum(hits) over () as percentage
                                                                                                              from total_hits_per_url
                                                                                                              order by percentage desc;
                                                                                                              

                                                                                                              Of course this style is more verbose in this simple case, but it preserves the sequential pipeline of transformations that makes dplyr code so readable.

                                                                                                              1. 1

                                                                                                                Yes that’s basically what the linked run.sh code does. The comment there shows that update came from the original lobste.rs discussion!

                                                                                                                https://github.com/oilshell/blog-code/blob/master/data-frames/run.sh#L79

                                                                                                                I didn’t change the blog post code since it is less familiar – I just mentioned the possibility. Although I should probably have both styles in run.sh for comparison.

                                                                                                                1. 1

                                                                                                                  Not quite: run.sh approximates a total_hits variable definition by joining with a 1 row 1 column CTE. Logically speaking, a join has nothing to do with dividing the total hits per url by the sum of all hits. Using a window function more closely matches the logical intent.

                                                                                                                  In this simple example there isn’t much difference. But for larger queries, particularly those already involving many joins, using joins to implement window functions leads to confusing SQL. At every join you must ask, are these tables intermediate result sets, or are they implementation details?

                                                                                                                  1. 1

                                                                                                                    Hm I tried to dig deeper into this and am now very confused… I haven’t really used window functions or CTE before.

                                                                                                                    https://github.com/oilshell/blog-code/tree/master/data-frames

                                                                                                                    1. I get a syntax error in your example. Try this:

                                                                                                                      ./run.sh csv2sqlite ./run.sh with-window

                                                                                                                      Error: near line 1: near “(”: syntax error

                                                                                                                    Is that because sqlite uses a different syntax? They seem to have a WINDOW function.

                                                                                                                    https://sqlite.org/windowfunctions.html

                                                                                                                    I would appreciate a patch to make it work; I couldn’t do it.

                                                                                                                    1. I noticed the with-cte version is actually broken!

                                                                                                                    The correct answer is done by with-sql:

                                                                                                                    Popular Pages:
                                                                                                                    /data-frames.html|59.8563447725459
                                                                                                                    /blog/|32.7214684756584
                                                                                                                    /site.html|7.42218675179569
                                                                                                                    

                                                                                                                    The with-cte version prints:

                                                                                                                    /data-frames.html|59.8563447725459
                                                                                                                    /blog/|14.365522745411
                                                                                                                    /site.html|2.47406225059856
                                                                                                                    

                                                                                                                    I don’t know why that is! This is why I don’t use SQL for analytics :-/

                                                                                                                    1. 2

                                                                                                                      OK I fixed the CTE query – it was missing a SUM()!

                                                                                                                      https://github.com/oilshell/blog-code/commit/75d4f86fc7b6ee8d32236089e9b147375cb5f46a

                                                                                                                      Looks like the bug was present in the original suggestion!

                                                                                                                      https://lobste.rs/s/hnfc6a/what_is_data_frame_python_r_sql#c_0os5e8

                                                                                                                      This is kinda why I like having intermediate results to “debug” … I think modern SQL engines should be good at optimizing those?

                                                                                                                      I get that you want to do things “in one step” with SQL, but I think that sequence of transformations is easier to debug, and the whole point of the optimizer should be to “erase” those boundaries.

                                                                                                                    2. 1

                                                                                                                      OK it looks like the problem is that sqlite3 from 2016 doesn’t support window functions!

                                                                                                                      I get the same syntax error on the example here with row_number(), etc.

                                                                                                                      $ sqlite3 --version
                                                                                                                      3.11.0 2016-02-15 17:29:24 3d862f207e3adc00f78066799ac5a8c282430a5f
                                                                                                                      

                                                                                                                      Looks like they were only added in 9/2018 as of version 3.25 !

                                                                                                                      https://sqlite.org/windowfunctions.html


                                                                                                                      edit: I got the window function working. I agree this is a nicer style (especially since the other one had a bug!)

                                                                                                                      https://github.com/oilshell/blog-code/blob/master/data-frames/run.sh#L136

                                                                                                                      I’m still a bit confused about OVER (), but it doesn’t work without that. To me there is no “Windowing” going on here? It just looks like a Common Table Expression?