1. 13

    Genuine comment (never used Nix before): is it as good as it seems? Or is it too good to be true?

    1. 51

      I feel like Nix/Guix vs Docker is like … do you want the right idea with not-enough-polish-applied, or do you want the wrong idea with way-too-much-polish-applied?

      1. 23

        Having gone somewhat deep on both this is the perfect description.

        Nix as a package manager is unquestionably the right idea. However nix the language itself made some in practice regrettable choices.

        Docker works and has a lot of polish but you eat a lot of overhead that is in theory unnecessary when you use it.

      2. 32

        It is really good, but it is also full of paper cuts. I wish I had this guide when learning to use nix for project dependencies, because what’s done here is exactly what I do, and it took me many frustrating attempts to get there.

        Once it’s in place, it’s great. I love being able to open a project and have my shell and Emacs have all the dependencies – including language servers, postgresql with extensions, etc. – in place, and have it isolated per project.

        1. 15

          The answer depends on what are you going to use nix for. I use NixOS as my daily driver. I am running a boring Plasma desktop. I’ve been using it for about 6 years now. Before that, I’ve used windows 7, a bit of Ununtu, a bit of MacOS, and Arch before. For me, NixOS is a better desktop than any of the other, by a large margin. Some specific perks I haven’t seen anywhere else:

          NixOS is unbreakable. When using windows or arch, I was re-installing the system from scratch a couple of times a year, because it inevitably got into a weird state. With NixOS, I never have to do that. On the contrary, the software system outlives the hardware. I’ve been using what feels the same instance of NixOS on six different physical machines now.

          NixOS allows messing with things safely. That’s a subset of previous point. In Arch, if I installed something temporarily, that inevitably was leaving some residuals on the system. With NixOS, I install random on-off software all the time, I often switch between stable, unstable, and head versions of packages together, and that just works and easy rollbackabe via entry in a boot menu.

          NixOS is declarative. I store my config on GitHub, which allows me to hop physical systems while keeping the OS essentially the same.

          NixOS allows per-project configuration of environment. If some project needs a random C++ package, I don’t have to install it globally.

          Caveats:

          Learning curve. I am a huge fan of various weird languages, but “getting” NixOS took me several months.

          Not everything is managed by NixOS. I can use configuration.nix to say declaratively that I want Plasma and a bunch of applications. I can’t use NixOS to configure plasma global shortcuts.

          Running random binaries from the internet is hard. On the flip side, packaging software for NixOS is easy — unlike Arch, I was able to contribute updates to the packages I care about, and even added one new package.

          1. 1

            NixOS is unbreakable. When using windows or arch, I was re-installing the system from scratch a couple of times a year, because it inevitably got into a weird state. With NixOS, I never have to do that. On the contrary, the software system outlives the hardware. I’ve been using what feels the same instance of NixOS on six different physical machines now.

            How do you deal with patches for security issues?

            1. 8

              I don’t do anything special, just run “update all packages” command from time to time (I use the rolling release version of NixOS misnamed as unstable). NixOS is unbreakable not because it is frozen, but because changes are safe.

              NixOS is like git: you create a mess of your workspace without fear, because you can always reset to known-good commit sha. User-friendliness is also on the git level though.

              1. 1

                Ah I see. That sounds cool. Have you ever had found an issue on updating a package, rolled back, and then taken the trouble to sift through the changes to take the patch-level changes but not the minor or major versions, etc.? Or do you just try updating again after some time to see if somebody fixed it?

                1. 4

                  In case you are getting interested enough to start exploring Nix, I’d personally heartily recommend trying to also explore the Nix Flakes “new approach”. I believe it fixes most pain points of “original” Nix; two exceptions not addressed by Flakes being: secrets management (will have to wait for different time), and documentation quality (which for Flakes is now at even poorer level than that of “Nix proper”).

                  1. 2

                    I didn’t do exactly that, but, when I was using non-rolling release, I combined the base system with older packages with a couple of packages I kept up-to-date manually.

            2. 9

              It does what it says on the box, but I don’t like it.

              1. 2

                I use Nixos, and I really like it, relative to how I feel about Unix in general, but it is warty. I would definitely try it, though.

              1. 1

                I’ve been using massren[0] for a long time. It lets you rename all the files in CWD with $EDITOR.

                1. https://github.com/laurent22/massren
                1. 2

                  vidir, which comes with moreutils, also lets you do exactly this.

                  1. 1

                    IIRC, moreutils also has qmv and qcp. Great tools.

                  1. 10

                    Considerably better than I was expecting. High-level summary: computer vision company founded before the deep learning revolution outperforms neural nets via huge amounts of baked-in domain knowledge.

                    1. 4

                      baked-in domain knowledge

                      Bravo, sir!

                      1. 1

                        Fortunately, not everything is “deep” learning

                      1. 18

                        Other posts in this series:

                        • Why All My Servers Have a Process that Allocates 1 GB of RAM and Then Sleeps Forever
                        • Why All My Servers Have a Process that Runs an Infinite Loop on One of the Cores
                        • Why All My Databases has a Table With 1 GB of NUL Strings
                          1. 11

                            You can rely on Go picking up stupid ideas and doing them for real ..

                            1. 3

                              Thanks for sharing – that was a fun read! “Ballast” is such a perfect name for this.

                          1. 2

                            until recently GHC’s runtime relied on a mix of volatile variables and explicit memory barriers to enforce memory consistency

                            This is kind of shocking. One of the first things I learned about memory ordering is that I shouldn’t use volatile for it. Maybe this just shows GHC’s age.

                            1. 4

                              Yeah the fact that GHC’s runtime was written well-before memory consistency models were widely understood, and nearly two decades before the standardization of C11 atomics (emphasis mine) really stood out to me.

                            1. 18

                              the user may specify extra, language-specific dependencies to model the edits more accurately, for example the dependency between introducing a function and using it in another file or another part of the same file.

                              Wow. That’s the first time I’ve heard about this feature. That’s super interesting! I guess that relationship will have to be stored in the patch, then?

                              1. 11

                                Yes: in Pijul, dependencies are already stored in patches, and there is already a way to add them manually.

                                1. 8

                                  FWIW, you might want a hat for making it clear when you talk from a position of authority about the subject.

                                  1. 3

                                    I’ve requseted one, thanks.

                                    1. 1

                                      That’s actually a really interesting feature. I never paid those hats much attention. I just assumed they were like user-set Reddit flairs.

                                    2. 3

                                      I really like the idea going somewhere behind current (D)VCS. The associativity is great. I would like to know how far your goals are. Are you going towards semantic diff/merge? Because some operations are not possible without having support for particular programming language or data format.

                                      With Pijul, developers can be 100% confident that the code they reviewed is the code that gets merged, which is not necessarily the case in Git and Mercurial.

                                      I am in doubt whether this is possible at all. If I want to be really sure, I have to do (a second) review after the merge. There are also cases where the merge runs flawlessly from the text/line point of view, but the result is wrong on the semantic level (two methods with same name/signature, two enum items with same value etc.)

                                      The question is whether these semantic operations belong to the VCS or should be done separately (and possibly on top of any VCS)…

                                      1. 4

                                        There are also cases where the merge runs flawlessly from the text/line point of view, but the result is wrong on the semantic level (two methods with same name/signature, two enum items with same value etc.)

                                        Sure, but at least with Pijul you can predict what the merge will be with absolute confidence. You can also be sure that merging changes one by one and all at once will yield the same.

                                        The question is whether these semantic operations belong to the VCS or should be done separately (and possibly on top of any VCS)…

                                        You can model dependencies between changes in Pijul, and one could totally imagine adding hooks to add dependencies.

                                      2. 2

                                        Are there any papers / specifications that you relied on for writing pijul that you could share? The ‘towards 1.0’ post gave a nice idea what is going on, and the posts on initialcommit are also nice but I’d also be interested in seeing some of the research underlying the design.

                                        1. 5

                                          It started with “A categorical theory of patches” (Mimram, Di Giusto), but there is a lot more in Pijul than just that paper. I still have to write the papers. I do have some of the proofs written up, but it’s not a paper yet.

                                          1. 2

                                            Cool, eagerly awaiting the writeups, in the meanwhile there is the code :)

                                            Thanks for all your hard work.

                                            1. 1

                                              Out of curiosity, do you know what the relationship is between academic papers and FOSS licenses? Theoretically, could someone read the papers you publish and then turn around and write an MIT-licensed DVCS similar to pijul?

                                              To be clear, I have absolutely no interest (is negative interest a thing?) in doing so. I’m just wondering how academia views licensing and public information and what the standards are.

                                        2. 7

                                          This is really interesting. When using git I try to write small, atomic commits where each change is as small as possible and passes CI (and could be deployable to production without breaking anything). I feel like, with git, it takes a lot more effort to track dependencies between changes than it should and this kind of functional dependency tracking system would make it easier.

                                        1. 11

                                          There’s a great blog post by Conal Elliott about the false belief that “everything is a function” in Haskell:

                                          There a belief about Haskell that keeps popping up in chat rooms and mailing lists — one that I’ve been puzzling over for a while. One expression of the belief is “everything is a function” in Haskell.

                                          Of course, there are all of these non-functions that need to be accounted for, including integers, booleans, tuples, and lists. What about them? A recurring answer is that such things are “functions of no arguments” or functions of a one-element type or “constant functions”.

                                          I wonder about how beliefs form, spread, and solidify, and so I asked around about how people came to this notion and how they managed to hold onto it. I had a few conjectures in mind, which I kept to myself to avoid biasing people’s responses. Of the responses I got, some were as I’d imagined, and some were quite surprising to me, revealing some of my blind spots about others’ thinking and about conversation dynamics.

                                          http://conal.net/blog/posts/everything-is-a-function-in-haskell

                                          1. 19

                                            This interview with the (former) Helm maintainer is really good. Turns out he’s a alpine guide who never programmed for a living.

                                            https://sachachua.com/blog/2018/09/interview-with-thierry-volpiatto/

                                            1. 19

                                              Thanks for sharing this! I’m the author of Gleam (and this post). Very happy to answer any questions :)

                                              1. 6

                                                Thank you for your work on Gleam! It looks really promising, and it’s been great seeing it progress from the sideline.

                                                Is it easy to integrate it (e.g. writing one module in Gleam) in an existing Erlang + rebar3 project? (Is this documented somewhere?)

                                                1. 7

                                                  Yes for sure. Currently we don’t have a dedicated build tool so all Gleam projects are rebar3 projects with a project plugin (https://github.com/gleam-lang/rebar_gleam), so compilation of Erlang works as per usual.

                                                  There’s also a mix plugin for Elixir projects (https://github.com/gleam-lang/mix_gleam).

                                                  The tooling is a bit rough-and-ready at the moment, I’m hoping to improve it in the near future.

                                              1. 9

                                                What is your favorite pitfall in Date?

                                                Has to be toISOString(). Claims to return ISO8601, which contains the timezone offset, but instead it just gives you the GMT string, even though it’s perfectly aware of the timezone information:

                                                // It's 15.44 in Europe/Warsaw
                                                > dt.getTimezoneOffset()
                                                -120
                                                > dt.toISOString()
                                                '2020-08-02T13:44:03.936Z'
                                                
                                                1. 5

                                                  That is a valid ISO 8601 timestamp. The ‘Z’ (“zulu”) means zero UTC offset, so it’s equivalent to 2020-08-02T15:44:03.936+02:00.

                                                  1. 3

                                                    Oh, it is valid, yes. It’s just less useful than one containing the TZ information that is stored in that Date object. It’s correct, but less useful than it could be (and with little extra effort).

                                                    1. 3

                                                      Ah, I misunderstood you, then. When you wrote “claims to return ISO 8601” I thought you meant that it wasn’t actually an ISO 8601 string.

                                                      So what you mean is that the “encoding” of the of the ISO 8601 string should reflect the local timezone of the system where you call .toISOString()? I.e. 2020-08-02T15:44:03.936+02:00 if you called .toISOString() on a CEST system and 2020-08-02T09:44:03.936-04:00 if you called it on an EDT system?

                                                      1. 2

                                                        I’d expect it to not lose the timezone information, given that it already uses a format that supports that information. It’s not incorrect, it’s just less useful that it could be. Perhaps that’s just the implementation, not the spec – but I’m yet to see it implemented differently. It’s not a huge deal, it’s just frustrating that it could’ve been better at a little cost and yet no one bothered, apparently.

                                                        It’s not about the system it’s called on – that determines the timezone that’s already in the object, as my code snipped showed. I’d expect the data that’s already there to be included in the formatting, instead of being converted to UTC, lost and disregarded. If implemented correctly better, toISOString could’ve been a nice, portable, lossless serialization format for Dates – but as it is, a roundtrip gives you a different date than you started with, because it will now always come back as UTC.

                                                        1. 2

                                                          I would actually assume that getTimezoneOffset is a class method that just looks at your system’s configured time zone and does not read anything from the Date object. I’m pretty sure the object does not store information about the timezone of the system in which it was generated, because it’s never needed. You can always convert to the timezone you want at read time.

                                                          This is also what PostgreSQL does. If you create a column for “timestamps with timezone” it will discard the timezone information at write time and just use UTC (because why not?). The only thing that is different when you choose a timestamp column with timezone is that at read time it will convert values from columns to the configured timezone. All it stores is the number of seconds since the epoch.

                                                          If you look at Firefox’s JS source, it looks like they also just store the seconds since the Unix epoch in a Date object, no timezone information: https://github.com/mozilla/gecko-dev/blob/d9f92154813fbd4a528453c33886dc3a74f27abb/js/src/vm/DateObject.h

                                                      2. 3

                                                        I don’t believe Date contains a time offset. As far as I’m aware, like many languages, the problem is not that the APIs ignore the time offset - they would have to silently reach into the client locale to get it, which would be misleading and make it easy to create bugs. the problem is that they named it “Date” when it’s really just a point in absolute time. Combine a Date with the client locale’s time offset and you’ve got yourself a date, but a Date is not a date.

                                                    2. 5

                                                      This is a namespacing error that’s common when methods are on objects like this. getTimezoneOffset is a property here of the client locale, not of the date time object.

                                                    1. 5

                                                      With all the enthusiasm for zettelkasten/second-brain like systems (roam, org-roam, now this), I’m surprised that nobody has been working on I haven’t heard of an external format/tool that various UI’s can interface. VSCode, at least that’s my impression, is the kind of editor that gets displaced from it’s throne every few years by the next new thing, as has happened to Sublime and Atom before, so I certainly wouldn’t be too confident in making my “second brain” depend on it, except maybe if it’s used as a brainstorming tool for projects, but then it would have to be distributable too – but from skimming the article that doesn’t seem to be the case.

                                                      Edit: Fixed the first sentence, sorry for my ignorance. Also I missed that this is markdown based, so I guess the rest of the comment isn’t quite right either, but I guess/hope my general point is still legitimate.

                                                      1. 6

                                                        I’m surprised that nobody has been working on an external format/tool that various UI’s can interface

                                                        Checkout neuron which is editor-independent, has native editor extensions, but can also interface (in future) with editors through LSP.

                                                        Some examples of neuron published sites:

                                                        Easiest way to get started (if you don’t want to install yet): https://github.com/srid/neuron-template

                                                        1. 3

                                                          That sounds cool, but I don’t really get why LSP would help? I (personally) would much prefer a native client, in my case for Emacs, than something that forces itself into a protocol for program analysis.

                                                          1. 2

                                                            Well, neuron does have native extensions for emacs and vim (see neuron-mode and neuron.vim) - but LSP support just makes multiple editor support easier by shifting common responsibility to a server on neuron.

                                                            EDIT: I’ve modified the parent comment to clarify this.

                                                          2. 1

                                                            Is there any easier way to install (i.e. without nix?) I’m on a laptop and installing new toolchains is prohibitive for the low storage I have.

                                                            1. 1

                                                              Nix is the only way to install neuron (takes ~2GB space including nix and deps), until someone contributes support for building static binaries.

                                                              But I’d encourage you give Nix a try anyway, as it is beneficial even outside of neuron (you can use Nix to install other software, as well as manage your development environments).

                                                              1. 2

                                                                I got a working binary with nix-bundle, that might be a simpler option. It’s a bit slow though, especially on first run when it extracts the archive. nix-bundle also seems to break relative paths on the command line.

                                                                1. 1

                                                                  Interesting. Last time I tried nix-bundle, it had all sorts of problem. I’ll play with it again (opened an issue). Thanks!

                                                          3. 3

                                                            Isn’t the markdown that this thing runs on exactly that external format, and one that has been getting adoption across a wide range of platforms and usecases at that?

                                                            1. 3

                                                              There is tiddlywiki and the tiddler format.

                                                              1. 2

                                                                I wish the extension used the org format instead of markdown (so if something happens to vscode, I can use it with emacs), but otherwise I totally agree with your comment!

                                                                1. 2

                                                                  You can use markdown files with org-roam in emacs by using md-roam. I prefer writing in Markdown most of the time, so most of my org-roam files are markdown files.

                                                              1. 6

                                                                Is there a video link for this talk? I want to watch it!

                                                                1. 10

                                                                  This one has me conflicted. On the one hand, I understand this reasoning and agree with it, in the specific case discussed — a local University computing system. I think the author is probably making the right choice for their users.

                                                                  On the other hand, I will still almost always recommend people just use UTC because it’s a safe long term default. I’ve worked with multiple companies now where all the servers are still on Pacific Time despite opening many global offices over the years. Many of their users and developers are now doing time zone math anyway, because they don’t all live in California anymore. But now they have the added adventure of daylight savings time! 😉

                                                                  Granted, if your whole mission is focused on serving a given locality, like a University, you’re probably safe with local time. But even then… as soon as you look at computers intended for research collaboration, that might go out the window too. I’ve seen plenty of academic HPC systems that eventually have more external users than internal, as they get linked into wider research programs.

                                                                  1. 5

                                                                    On the other hand, I will still almost always recommend people just use UTC because it’s a safe long term default.

                                                                    Anything should be a safe long-term default, as long as you’re consistent or store the corresponding time zone. The problems usually happen when you have something like 2020-05-15 17:56:34 and no idea which TZ that refers to, or need to get the user’s configured TZ (which may change). But if it’s stored as 2020-05-15 17:56:34 +0800 then it’s always safe and easily convertible to whatever timezone.

                                                                    IMO “always use UTC” is much better phrased as “always know which TZ a time is”. Using UTC internally everywhere is often a convenient way of doing that, but not always.

                                                                    1. 1

                                                                      But if it’s stored as 2020-05-15 17:56:34 +0800 then it’s always safe and easily convertible to whatever timezone.

                                                                      I think that’s a lucky example, since there’s little daylight savings out that way, but much of the world moves their clocks around, so you might share a DST-offset part of the year with someone when it matters, and you’re using these timestamps specifically for correlation.

                                                                      The reason UTC is better is because we know where 0° longitude is and we know they don’t practice daylight savings. Was that the result of a car crash into a telephone pole at almost six o’clock? Time zone doesn’t tell you that in parts of the world, but UTC-reported dates will.

                                                                      The reason UTC is worse, of course, is because sometimes people don’t provide UTC dates, and sometimes people confuse the time in London or Greenwich with UTC so their reports are bad, and people rarely make this mistake with local time (as the author points out).

                                                                      1. 4

                                                                        Won’t a format like that take care of DST? For example in Western Europe it would be +0100 in the winter, and +0200 in the summer.

                                                                        1. 2

                                                                          Not always. Consider an event in the future planned before a decision to change daylight savings time. It’s still going to open at 9am whatever-is-local- on some given future date.

                                                                          1. 2

                                                                            Assuming the DST change is communicated clearly and in good enough time to the TZ database maintainers… you’d be surprised how often this is not the case.

                                                                            Off the top of my head, I can mention Jordan, Turkey, Armenia, Egypt and Russia announcing changes to their DST change schedules with very short notice.

                                                                            My fear is that this will happen in the EU too, considering that most politicians don’t really seem to understand the implications of changing the DST schedule…

                                                                            1. 1

                                                                              Even with past dates, calculations will only be correct iff a library is using a correct timezone database that includes all change history and is using it correctly. It also may not be the case.

                                                                              Using UTC and converting to local time only when needed saves one from a lot of “if’s”.

                                                                              1. 2

                                                                                Only if you’re using a format that doesn’t save the UTC offset, right? I don’t see how the interpretation of an ISO-8601 datetime like 2020-05-18T08:42:57+02:00 can change in the future.

                                                                                1. 1

                                                                                  It can change if you’re planning to schedule something at 09:00 (AM) on 15 Jun 2022 in Berlin, and the current schedule for DST tells you that Germany will be observing DST at that time.

                                                                                  Right now we don’t know what EU countries are going to do with DST - abolish it, and stay on normal time? Abolish it and stay on summer time?

                                                                                  If Germany decides to stay on normal time your appointment at 2022-06-15T09:00:00+02:00 will be 1 hour later than expected (which is 9AM CET in this case).

                                                                                  1. 2

                                                                                    Yeah, that makes sense, but the comment above mine was talking about calculations on past dates.

                                                                                    1. 1

                                                                                      Sorry, I misinterpreted your comment! Yeah, past dates are generally “safe”.

                                                                    1. 23

                                                                      It only works in Google Chrome and Microsoft Chrome, unfortunately:

                                                                      For the best experience with Codespaces, we recommend using a Chromium-based browser, like Google Chrome or Microsoft Edge. Firefox is currently unsupported, and there are known issues using Safari.

                                                                      1. 12

                                                                        Codespaces allows you to develop in the cloud instead of locally. Developers can contribute from anywhere, on any machine, including tablets or Chromebooks

                                                                        …and on iOS all browsers including Chrome use the Safari rendering engine so this doesn’t really open up development on the most popular tablet platform at all.

                                                                        1. 1

                                                                          I imagine they will add that.

                                                                        2. 4

                                                                          Before that note is this paragraph, though:

                                                                          During the beta, functionality is limited.

                                                                          So hopefully once it’s actually released it will be usable in every browser.

                                                                          1. 1

                                                                            It only works in Google Chrome and Microsoft Chrome, unfortunately:

                                                                            To be honest, it’s quite scary to run all of that inside a browser. Can you imagine the performance on that?

                                                                            1. 1

                                                                              It probably performs fine on most development machines, to be fair.

                                                                          1. 1

                                                                            I’d honestly entirely forgotten about Ogg. Is it still a thing in a meaningful sense, what with the MP3 patents having expired?

                                                                            1. 6

                                                                              Ogg is a container format, able to hold different codecs inside, Vorbis is the codec designed to replace MP3. Now Ogg is being used with the Opus codec, with quite success.

                                                                              1. 3

                                                                                Ugh, I thought that Opus had its own container format/was using MKV already …

                                                                                Tried to parse Ogg once, wouldn’t recommend.

                                                                              2. 6

                                                                                Spotify uses ogg vorbis for streaming to their apps: https://en.wikipedia.org/wiki/Spotify#Technical_information

                                                                                1. 5

                                                                                  Ogg vorbis file size pretty regularly beat out vbr mp3’s at the max setting I could distinguish in a blind listening test. If a lossless source is available I always prefer encoding vorbis myself for use on my own (non internet) music player! The criticisms of the Ogg container make sense though. I’ve never really seen Vorbis in any other container tbh.

                                                                                  1. 3

                                                                                    Old-style WebM files used Vorbis in a Matroska container.

                                                                              1. 10

                                                                                The mastodon discussion that was linked to in the bottom, about the evergreen rust compiler and how that challenges “traditional systems” was really interesting.

                                                                                1. 2

                                                                                  Kornel definitely thought about this and related issues for a while, e.g. see his #Rust2020 post: https://users.rust-lang.org/t/rust-2020-growth/34956

                                                                                1. 1

                                                                                  398 days is a year and 33 days, a little over a month. Is there a particular reason for choosing 398 as the limit, versus rounding down to 365 or 366 (a year), 395 or 396 (a year and a month) or rounding up to 400?

                                                                                  1. 2

                                                                                    Apparently it’s a “renewal grace period” - not sure why that’s 32/33 days though.

                                                                                    1. 3

                                                                                      Maybe 2 days to account for weekends/holidays, then 30 days to allow a cert nenewal PR[1] to wend its way through the corporate bureacracy.

                                                                                      [1] Purchase Request

                                                                                    2. 2

                                                                                      Pure speculation:

                                                                                      One year is ⌈365.24⌉ = 366 days. One month is ⌈31 days + 1 leap second⌉ = 32 days.

                                                                                      1. 3

                                                                                        At least for ISO dates, the leap second is included in the preceding day (i.e. it’s extended by one second)

                                                                                        2016-12-31T23:59:59
                                                                                        2016-12-31T23:59:60
                                                                                        2017-01-01T00:00:00
                                                                                        
                                                                                    1. 37

                                                                                      I think this will not succeed for the same reason that RSS feeds has not (or REST). The problem with “just providing the data” is that businesses don’t want to just be data services.

                                                                                      They want to advertise to you, watch what you’re doing, funnel you through their sales paths, etc. This is why banks have never ever (until recently, UK is slowly developing this) provided open APIs for viewing your bank statement.

                                                                                      This is why businesses LOVE apps and hate web sites, always bothering you to install their app. It’s like being in their office. When I click a link from the reddit app, it opens a temporary view of the link. When I’m done reading, it takes me back to the app. I remain engaged in their experience. On the web your business page is one click away from being forgotten. The desire to couple display mechanism with model is strong.

                                                                                      The UK government is an exception, they don’t gain monetary value from your visits. As a UK citizen and resident, I can say that their web site is a fantastically lucid and refreshing experience. That’s because their goal is, above all, to inform you. They don’t need to “funnel” me to pay my taxes, because I have to do that by law anyway. It’s like reading Wikipedia.

                                                                                      I would love web services to all provide a semantic interface with automatically understandable schemas. (And also terminal applications, for that matter). But I can’t see it happening until a radical new business model is developed.

                                                                                      1. 6

                                                                                        This is why banks have never ever (until recently, UK is slowly developing this) provided open APIs for viewing your bank statement.

                                                                                        This has happened in all EU/EEA countries after the Payment Services Directive was updated in 2016 (PSD2). It went into effect in September 2019, as far as I remember. It’s been great to see how this open banking has made it possible for new companies to create apps that can e.g. gather your account details across different banks instead of having to rely on the banks’ own (often terrible) apps.

                                                                                        1. 6

                                                                                          The problem with PSD2 to my knowledge is that it forces banks to create an API an open access for Account Information Service Providers and Payment Initiation Services Providers, but not an API to you, the customer. So this seems to be a regulation that opens up your bank account to other companies (if you want), but not to the one person who should get API access. Registration as such a provider costs quite some money (I think 5 digits of Euros), so it’s not really an option to register yourself as a provider.

                                                                                          In Germany, we already seem to have lots of Apps for management of multiple bank accounts, because a protocol called HBCI seems to be common for access to your own account. But now people who use this are afraid that banks could stop this service when they implement PSD2 APIs. And then multi-account banking would only become possible through third-party services - who probably live from collecting and selling your data.

                                                                                          Sorry if something is wrong. I do not use HBCI, but that’s what I heard from other people.

                                                                                          1. 1

                                                                                            I work on Open Banking APIs for a UK credit card provider.

                                                                                            A large reason I see that the data isn’t made directly available to the customer is because if the customer were to accidentally leak / lose their own data, the provider (HSBC, Barclays etc) would be liable, not you. That means lots of hefty fines.

                                                                                            You’d also likely be touching some PCI data, so you’d need to be cleared / set up to handle that safely (or having some way to filter it before you received it).

                                                                                            Also, it requires a fair bit of extra setup and the use of certificate-based authentication (MTLS + signing request objects) means that as it currently sits you’d be need one of those, which aren’t cheap as they’re all EV certs.

                                                                                            Its a shame, because the customer should get their data. But you may be able to work with intermediaries that may provide an interface for that data, who can do the hard work for you, ie https://www.openwrks.com/

                                                                                            (originally posted at https://www.jvt.me/mf2/2019/12/7o91a/)

                                                                                        2. 4

                                                                                          Yes, this does seem like a naive view of why the web is what it is. It’s not always about content and data. For a government, this makes sense. They don’t need to track you or view your other browsing habits in order to offer you something else they’re selling. Other entities do not have the incentive to make their data easier to access or more widely available.

                                                                                          1. 6

                                                                                            That’s very business centric view of the web, there’s a lot more to the internet than businesses peddling things to you. As an example, take a look at the ecosystem around ActivityPub. There are millions of users using services lile Mastodon, Pleroma, Pixelfed, PeetTube, and so on. All of them rely on being able to share data with one another to create a federation. All these projects directly benefit from exposing the data because the overall community grows, and it’s a cooperative effort as opposed to a competitive one.

                                                                                            1. 3

                                                                                              It’s a realistic view of the web. Sure, people who are generating things like blogs or tweets may want to share their content without monetizing you, but it’s not going to fundamentally change a business like a bank. What incentive is there for a bank to make their APIs open to you? Or an advertiser? Or a magazine? Or literally any business?

                                                                                              There’s nothing stopping these other avenues (like the peer-based services you are referring to) from trying to be as open as possible, but it doesn’t mean the mainstream businesses are ever going to follow suit.

                                                                                              I think it’s also noteworthy that there is very little interesting content on any of those distributed systems, which is why so many people end up going back to Twitter, Instagram, etc.

                                                                                              1. 1

                                                                                                My point is that I don’t see business as the primary value of the internet. I think there’s far more value in the internet providing a communication platform for regular people to connect, and that doesn’t need to be commercialized in any way. Businesses are just one niche, and it gets disproportionate focus in my opinion.

                                                                                          2. 3

                                                                                            Aye, currently there is little motivation for companies to share data outside silos

                                                                                            That mind-set isn’t really sustainable in the long term though as it limits opportunity. Data likes to date and there are huge opportunities once that becomes possible.

                                                                                            The business models to make that worth pursuing are being worked on at high levels.

                                                                                            1. 1

                                                                                              Ruben Verborgh, one of the folks behind the Solid initiative 1, has a pretty good essay 2 that details a world in which storage providers compete to provide storage, and application providers compete on offering different views to data that you already own.

                                                                                              Without getting into Solid any more in this post, I will say that there are a ton of websites run by governments, non-profits, personal blogs, or other situations where semantically available data would be a huge boon. I was looking through a page of NSA funded research groups the other day for McMurdo station 3, and finding what each professor researched on took several mouse clicks per professor. If this data was available semantically, a simple query would be enough to list the areas of research of every group and every professor.

                                                                                              One can think of a world where brick-and-mortar businesses serve their data semantically on their website, and aggregators (such as Google Maps, Yelp, and TripAdvisor) can aggregate them, and enable others to use the data for these businesses without creating their own scrapers or asking a business to create their own API. Think about a world where government agencies and bureaucracies publish data and documents in an easy to query manner. Yes, the world of web applications is hard to bring the semantic web to due to existing incentives for keeping data siloed, but there are many applications today that could be tagged semantically but aren’t.

                                                                                              1. 1

                                                                                                The web has been always used mostly for fluff since day 1, and “web assembly” is going to make it more bloated, like the old browser-side java.

                                                                                                The world needs user-centric alternatives once again.