1. 3

    Are there other examples of SQLite being used as a website backend database in production? What kind of scale could you reach with this approach? And what would be the limiting resource?

    1. 10

      Expensify was based exclusively on sqlite for a long time, then they created a whole distributed database thing on top of it.

      1. 7

        Clojars used SQLite for a good 10 years or so, only recently moving away to Postgres for ease of redeployment and disaster recovery. The asset serving was just static assets, but the website and deployments ran against SQLite pretty well.

        1. 3

          If I remember correctly, the trouble that Clojars ran into had more to do with the quality of the JVM-based bindings to SQLite than they did with SQLite itself, at least during the portion of time that I was involved with the project.

          1. 2

            Yeah, looking back at the issues, “pretty well” is maybe a little bit generous. There were definitely settings available later on which would have helped the issues we were faxing around locking.

        2. 4

          I can’t remember whom but at least one of the well funded dynamoDB style distributed database-y products from the mid 10s used it as the storage layer.

          So all the novel stuff that was being done with data was the communication and synchronisation over the network, and then for persistence on individual nodes they used sqlite instead of reinventing the wheel.

          1. 6

            That was FoundationDB, purchased by Apple in 2013, then gutted, and then returned as open-source in 2018. I’m a bit annoyed, because it was headed to be CockroachDB half a decade earlier, and was taken off the market with very little warning.

            1. 1

              Thanks!

          2. 3

            You probably will get really fast performance for read-only operations. The overhead of client/server and network stack could be more than10x times of function calls from same address space. The only real limitation might be single server, since you cannot really efficiently scale sqlite beyond single system. But when you reach that scale, you usually needs much more than sqlite.

            1. 3

              The sqlite website claims to run entirely on sqlite.

              They also have this page, though most of those aren’t websites: https://sqlite.com/mostdeployed.html

            1. 1

              Nix has makeWrapper that should help with this. It creates wrapper that sets environment to correct values, so all subprograms would be executed using correct paths. Maybe that would be useful for you?

              1. 1

                They are sort of complementary.

                makeWrapper is used in a lot of places in nixpkgs and works as intended. It requires the packager to know all of the dependencies in advance and put them use wrapProgram to append them to the PATH.

                What happens in practice is that not all code paths are executed during manual testing and some hidden dependency is only found out later. Or the package gets upgraded and the reviewer doesn’t spot the additional dependency.

                As I understand it, resholve allows to scan the program and look ahead, increasing the likelihood that all the runtime dependencies are being found.

                1. 1

                  What zimbatm said. Concretely, this is really useful in cases where you’re writing scripts that execute in stage-1/stage-2 of boot, or other annoying-to-test places. Guaranteeing you have identified and included all your dependencies is very powerful, and can save an hour plus in cycle time for each problem it catches.

                  1. 1

                    Author here. Mostly echoing zimbatm, but: makeWrapper is great for simple executable scripts, especially those you wrote yourself, where you have strong knowledge about what they’ll run under what conditions.

                    The case that motivated me, personally, is shell libraries, in particular some shell profile modules I’ve been working on. Since a shell library shares environment/state with anything that sources it, idiomatic PATH manipulation can obviously cause trouble, and you can (at least in theory) have issues mixing libraries that intend to use different sources for the same command (the resholve repo/CI has a contrived demo using openssl/libressl).

                    It’s also, as zimbatm notes, useful when packaging complex scripts that you don’t know like the back of your hand. It’s easy to miss them when they’re off the obvious path (or even just beyond <command> --help). I’ve found undeclared dependencies in all 4 of the shell packages that I’ve identified in nixpkgs and started converting to use resholve.

                  1. 1

                    Great read! (I read the entire thing! :O)

                    Should we be worried/concerned that you cannot actually have reproducibility in docker build? I tried a few other OCI Image build tools (img for example) and they all suffer from the same problem.

                    Part of me thinks this is something worth solving, but OTOH if you read some of the other related articles in this post you can see arguments against cryptographically reproducible builds…

                    1. 3

                      Nix’s support for building Docker images allows for reproducible images, here is a talk I gave on the issue: https://www.youtube.com/watch?v=pfIDYQ36X0k

                    1. 9

                      You mention similarity to PHP already, but you may not know that in 1999 PHP supported <script language="php"></script> too! I’m not sure if PHP still does.

                      1. 4

                        ASP had runat="server".

                        1. 3

                          Thankfully that was removed in the 7-series. Possibly 7.0.0?

                        1. 3

                          It seems nice ; but that looks like a lot of work to generate 10 lines of systemd config. or even just one line in a cron job?

                          1. 3

                            It’s about declaring what your system config / app deployment should look like in a single place, rather than changing details across a running system. You can combine that systemd unit and cronjob in a single NixOS module that you keep in a git repo, and that’s just a very small example. More complicated applications can also include virtual host config, build steps, etc. in a single file, if the developer feels they logically belong together.

                            Some of this is comparable to what tools like Ansible achieve. What if you have to reinstall the machine? Or what if you want to share some configuration across machines? You don’t want to figure out how to setup your server / app all over again, especially if the situation is unexpected and you’re pressed for time.

                            NixOS goes a bit further than Ansible et al by not describing steps to apply to an Ubuntu system (for example), but by being its own Linux distribution built entirely with Nix. (Benefits of that are a separate discussion, I think.)

                            1. 2

                              That was kind of my point. We now need a custom Linux distribution with its own (new) programming language to achieve very simple things. Well, at least it’s not YAML again.

                              1. 4

                                The language and distribution are about 17 years old, so I guess new is relative here.

                                1. 2

                                  A matter of perspective, maybe? I’ve tried often to get comfortable with Debian packaging for my own (also work) applications and smaller tools, but couldn’t. Instead, it’s always just ‘treat the OS like a black box’ and deploy to /opt or similar. NixOS is a lot more approachable to me, despite the learning curve. (I guess a different learning curve.)

                              2. 3

                                It’s not just 10 lines of systemd config. It’s also putting that config in the right place and implicitly activating it so you can’t mess it up. One line of cronjob doesn’t give you logs for that cronjob when it fails. The basic process here can also be adapted to other things like backup scripts.

                                This blogpost was cherry-picked from this config: https://github.com/Xe/nixos-configs/blob/master/hosts/chrysalis/tulpachat.nix, which uses Nix functions to dynamically create discord webhook timers so I can add an arbitrary number of them in the future.

                                For a more complicated example, see here: https://github.com/Xe/nixos-configs/blob/master/common/services/mi.nix, this handles a service called mi and exposes it at mi.within.website. I could “just write the systemd units by hand”, but that doesn’t handle pushing the units and scripts to the machine and making sure they are enabled. This allows me to rest assured that I can trivially move the config to other machines if I need to, such as if I get a new home server. Not to mention automatically building and installing all the services on the machine and then making sure the systemd units point to the correct binaries.

                                Sure, I can write 10 lines of systemd config today and I will be fine. However tomorrow it may end up not working out when circumstances and facts change.

                                1. 1

                                  One line of cronjob doesn’t give you logs for that cronjob when it fails.

                                  Well, I used to get emails from failed cron jobs, and systemd logs are still a thing if you’re on Linux.

                                  I completely understand the appeal of having the state/config integrated in one repository, but it looks like we only reach as far as tools like Ansible/Chef/Salt do for now. I’ve only seen a few bits of nix config here and there so far, and there’s probably a bigger picture that makes it all very exciting, but I guess I’d need to dig myself into that hole to find out if I like digging.

                                  1. 4

                                    The key difference is Ansible, Chef, and Salt are all still saying what you want to do, not what you want to end up with. What I mean by this is with each of these tools:

                                    1. start with a new system
                                    2. define a service you want running
                                    3. deploy
                                    4. delete that service from your configuration repository
                                    5. deploy

                                    Chef and friends will just stop managing that service, but the service will still exist. NixOS, though, won’t have that service anymore.

                                    1. 1

                                      The key difference is Ansible, Chef, and Salt are all still saying what you want to do, not what you want to end up with.

                                      That should be tag line for nix in general. For whatever reason, that really resonates with me.

                                      Chef and friends will just stop managing that service, but the service will still exist. NixOS, though, won’t have that service anymore.

                                      I don’t want to be pedantic, but if not having the service anymore is your goal, that’s still totally doable with chef and friends.

                                      1. 1

                                        Yes it is possible, but you have to make it your goal. Again, you’re so used to having to think about what you want to do, not just what you want. Nix lets you skip past that and just write down what you want.

                              1. 1

                                Maybe it’s time to google “zfs” for once in my life. The encryption sounds cool as. (Is encrypted drives on a remote server like this safe? Can you trust that the drives are encrypted securely, or can the server host decrypt them in some evil way?)

                                1. 5

                                  Zfs is really cool. There have been a few filesystem + volume manager “in one” systems - and there are some arguments to be made about cross cutting concerns - like handling SSD discard with an encrypted file system.

                                  As for software encryption - à remote server isn’t safe if you don’t trust the provider. They even let you know they’ll run your is that mount the disks in a vm - so they cold just dum the ram and read the encryption keys.

                                  If your threat model is more: I’d rather not anyone get the data from the disks after they’re spun down - it’s OK. (remote trusted computing is a really high bar to clear, with hardware enclaves that you can communicate with over the network, encrypted ram, worrying about encrypted cpu cache…).

                                  1. 6

                                    You don’t have to send the encryption key to the receiving server with ZFS.

                                    1. 2

                                      Ah, I was mostly responding to the question about mounting encrypted data on a remote server.

                                      For just backup, no “remote” reading, it is indeed possible to send snapshots - and no need to decrypt/mount (but you can, if you want to).

                                1. 18

                                  IMO, give up on the bizarrely macho idea that hjkl is uniquely amazing and use the arrow keys.

                                  1. 18

                                    To me the advantage of using hjkl instead of the arrow keys is that I don’t have to move my fingers away from the home row to move the cursor around. I don’t see how that is a “bizarrely macho idea”.

                                    1. 1

                                      I cannot imagine how that helps. Can you explain?

                                      1. 7

                                        You can keep your fingers in the middle of the typing area (home row) instead of going over to the cursors.

                                        1. 0

                                          Oh, hi ane!

                                          You can keep your fingers in the middle of the typing area (home row) instead of going over to the cursors.

                                          I cannot imagine how that helps. Can you explain?

                                          1. 4

                                            Hi!

                                            I cannot imagine how that helps. Can you explain?

                                            You… move around less? It saves time. Like a keyboard shortcut or macro does, basically. It depends on the form factor of the keyboard, but usually the arrow keys are further away from the regular text input keys.

                                            1. 1

                                              Perhaps 10 years of active piano practice makes this a moot point for me but not so much for others. Or I’m just being an asshole. Could be both ;)

                                            2. 2

                                              What I find helpful about it, is that I don’t have to look away from the screen to see where the arrow keys are, and likewise when going back to the home row. If there’s a lot going on on your screen it’s easy to lose your place, especially when reading lots of text.

                                              Also it avoids the physical motion of moving your hand, it just feels more comfortable I think.

                                      2. 16

                                        I use a 60% keyboard without arrow keys

                                        1. 1

                                          By choice, though?

                                          1. 2

                                            Yes, it’s more portable and means i don’t have to reach as far for my mouse

                                        2. 10

                                          I used to think this way. Then I learned hjkl. Now I am in the cult of the ancient keyboard warriors

                                          1. 2

                                            Ancient keyboard warriors who didn’t have arrow keys? I used arrow keys, learned hjkl, thought it was nice, moved to dvorak, ditched hjkl for arrow keys. Nothing is magic about it. Why contort yourself to use hjkl on non-qwerty? Would hjkl be dhtn if the standard were dvorak at the time? Probably, but you can’t just remap dhtn now.

                                            1. 5

                                              There was once a really good reddit thread of a guy playing counter-strike with zqsd movement keys on a qwerty keyboard cause he copied a French (AZERTY) player’s keybinds. He did not realize this and posted about how much better these movement keys made him.

                                              1. 5

                                                Pedantic note: ScreaM is a Belgian, not French, player. The keyboard layout is still French though

                                              2. 1

                                                Would hjkl be dhtn if the standard were dvorak at the time? Probably, but you can’t just remap dhtn now.

                                                If you’re using Dvorak in the “suggested sense”, even dhtn would be awkward since you’d be using your right index finger for both d and h. Maybe htns (or QWERTY jkl;) would be better.

                                            2. 9

                                              How did you come to the conclusion hjkl is “bizarrely macho”?

                                              1. 8

                                                Why is that macho?

                                                I mean some people tried it, liked it and share the good experience of how great it feels when you get used to it.

                                                1. 3

                                                  I used to use hjkl. Then I started getting RSI. Now I appreciate the break my fingers naturally get every time I have to move my hands to the arrow keys.

                                                  I can still use hjkl at a pinch, and probably use them many times a day without noticing. But yeah, it’s not worth getting worked up about.

                                                  1. 8

                                                    I can see this being the case, but I think it’s worth noting that RSI covers a large number of distinct problems, and that most RSI would only get worse by more frequently moving your hands away from the home position.

                                                1. 3

                                                  Man I wish I could use Mako, but unfortunately it requires systemd or elogind. Just no. I’d rather just use dunst via XWayland.

                                                  1. 4

                                                    Yes, depending on the sd-bus library sucks. However that’s the only half-decent D-Bus library out there. The plan is to get basu up and running on non-systemd distributions.

                                                    1. 1

                                                      mako doesn’t require systemd or elogind. It does however require dbus (as does dunst if I’m not mistaken).

                                                      Edit: sd-bus is required so you do need libelogind at a minimum. You do not need to be using elogind however.

                                                      1. 3

                                                        I don’t want anything from elogind nor systemd on my computer. So I guess for now, whenever I’m in a wayland session I’ll stick to using dunst.

                                                        1. 13

                                                          It uses systemd’s dbus library, that is all. But, if that is the line you want to draw, by all means.

                                                          1. 5

                                                            It is the line I want to draw, I personally have a no-tolerance policy for systemd, and that extends to elogind.

                                                            Why couldn’t it just use dbus without systemd or elogind?

                                                            dunst uses dbus but doesn’t haven’t a hard dependency on systemd/elogind,

                                                            1. 12

                                                              It does, indeed, just use dbus. It happens to use the dbus library from systemd’s repository for it. If you forked systemd, deleted everything but the sd-bus bits, it would work.

                                                              1. 8

                                                                That’s exactly my point, that shouldn’t be necessary. dunst can use d-bus on it’s own without elogind/systemd.

                                                                Having a hard dependency on an init system, or a portion of it, is beyond stupid for something like a notification daemon.

                                                                1. 11

                                                                  Okay.

                                                                  1. 3

                                                                    Having a hard dependency on an init system, or a portion of it, is beyond stupid for something like a notification daemon.

                                                                    I disagree, it’s part of the operating system for many distributions, a specifc component they want to rely on. There are certainly people who want to use the Linux kernel, but are not interested in systemd, but that is not everyone.

                                                                    And setting that aside, it’s not a hard dependency, as others have already said. It’s a dependency on an interface, at best.

                                                                    1. 2

                                                                      It is a hard dependency as it is literally required at this point in time.

                                                                      You can not compile it without having elogind or systemd.

                                                                      But also I think you’re missing the point.

                                                                      There are plenty of other notification daemons, that do no require any certain init system or a portion of one. That can do the same, or even more than, Mako can. The only difference is that in regards to Mako there’s not really any major competing software for it.

                                                                      By requiring elogind/systemd, there’s basically a boundry set in place for people who want to rid themselves of using the garbage known as systemd. And that’s totally fine, it’s their software. But they could probably gain a lot more users by not using a dbus interface tied to any one init system.

                                                                2. 4

                                                                  Might be pretty easy to change if you make a PR or request for it. There’s other dbus interfaces out there.

                                                                  1. 1

                                                                    I would make a PR, but I am garbage at both reading and writing C.

                                                                    I can try requesting, but I really doubt they would even consider it.

                                                              2. 1

                                                                Wait, why not? Any good writeups I could read about the downfalls of systemd?

                                                                1. 1

                                                                  I don’t really have any resources but my own list of things that drive me insane about systemd.

                                                              3. 2

                                                                Every time I’ve tried to compile it, it fails due to missing systemd/elogind. I can try compiling it again.

                                                                Is there some sort of like build-time configuration that I’m missing? I have not found any way to compile it without systemd/elogind.

                                                                1. 4
                                                                  Building
                                                                  
                                                                  Install dependencies:
                                                                  
                                                                      meson (build-time dependency)
                                                                      wayland
                                                                      pango
                                                                      cairo
                                                                      systemd or elogind (for the sd-bus library)
                                                                      gdk-pixbuf (optional, for icons support)
                                                                      dbus (runtime dependency, user-session support is required)
                                                                      scdoc (optional, for man pages)
                                                                      jq (optional, runtime dependency)
                                                                  

                                                                  This is directly from the README.

                                                            1. 34

                                                              There are no inline links (every link goes on its own line), no formatting, and no inline images

                                                              So it’s borderline useless for all but the simplest use cases, and even there it does a very subpar job. I’d like to have an inline image every now and then, or organize things in some multi-column layouts or the like, or a table. 19th century newspapers could do this, and if your technology can’t emulate typography produced on a daily basis – often in a hurry – 150 years ago then your tech is rather limited.

                                                              These protocols keep shooting themselves in the head with stuff like this, not even in the foot because this entire thing is dead on arrival for all but a few niche tech folk.

                                                              1. 17

                                                                I see this a lot, and I really, really wish that the people making this case would actually try to use Gemini for what it is before bringing their preconceptions into it. Easily a full third of the people I’ve spoken to about Gemini immediately start talking about some $FEATURE that they would add to it - and I’ve heard a lot of different features. Do you see the problem with this? This is just a recipe for re-inventing the bloat of the web.

                                                                You must first use, then understand, then change the software. You can’t skip steps one and two. It is possible to express yourself within these constraints (you’re already expressing yourself within some fairly arbitrary constraints right now, for instance, imposed by Lobsters’ strict markdown rules), and you should first apply a little bit of creativity to the new medium before you try to bend the medium to your will. We don’t really want it bent, we’re doing something different for a reason.

                                                                By the way, your comment can be losslessly represented in Gemtext.

                                                                1. 7

                                                                  Easily a full third of the people I’ve spoken to about Gemini immediately start talking about some $FEATURE that they would add to it - and I’ve heard a lot of different features. Do you see the problem with this? This is just a recipe for re-inventing the bloat of the web.

                                                                  I’d argue that this is because the problems it solves are inherently complex, so any good solution to it will be complex as well. That doesn’t mean I don’t think the web can’t be improved or made simpler – it’s an organically grown mess of 25 years – but it will never be replaced by a dead-simple solution such as Gemini.

                                                                  And yeah, my Lobsters comments can be expressed in Gemini without much loss (except the inline links thing), but Lobsters isn’t my website though and I wouldn’t be able to express my fairly simple website on Gemini, or a fairly simple site as Wikipedia. And even the entire Lobsters website would be a worse UX in Gemini as it precludes any sort of design.

                                                                  1. 1

                                                                    Gemini does not preclude any sort of design, not at all - it just gives the responsibility for design to the client, not to the server.

                                                                    1. 2

                                                                      But isn’t this very limiting? Like, it would work for your blog, but in its current iteration adding a table would already be difficult, never mind something more advanced like the fivethirtyeight election forecast and results.

                                                                      I suppose it all depends what your vision for Gemini is; if you want it to replace large parts of the web then it’s got basically 0 chance in its current state; if you just want some alternative to use yourself and like-minded people, then sure, go for it.

                                                                      1. 1

                                                                        Of course Gemini doesn’t preclude design: It’s a networking protocol. You can use it to move HTML and JS and CSS and images and whatever else just fine.

                                                                        Attempting to conflate a protocol with a document format is wrong.

                                                                        1. 1

                                                                          The specification also includes a document format, as well as a networking protocol.

                                                                          Arguably, things would have been clearer if they were separated out more clearly, but this would kind of defeat the point of Gemini. The networking protocol and document format are intended to be used together.

                                                                    2. 6

                                                                      It seems interesting; to make it clear, I do hate tracking too, as well as inaccessibility & various Flash-equivalents of today. And dependency on the complexity tower of Chrome/Firefox/… So I’m certainly curious about alternatives to the current status quo.

                                                                      I do understand that the protocol seems to allow for a lot of expression, and more “slow-food” style, relaxed consumption. Yet, I cannot escape immediately having a feeling of loss. If I understand this correctly, with no mathematical equations support, I would not be able to write or consume an article trying to explain Shor’s algorithm. With no graphs support, I assume whenever I’d want to display graphs, I’d have to resort to ASCII-Art as a workaround, with necessarily limited precision (for example, no smooth curves or complex 3D surfaces). Finally, I assume I would not be able to write or consume an article about typography, or bysantine icons. Or am I wrong?

                                                                      Having the concerns mentioned above, I’m immediately starting to think about possible workarounds. I am assuming people would start resorting to “rich clients” - for example, with support for rendering of some kinds of mathematical equations. This, I am guessing, would quickly lead to on-site recommendations like: “This page requires support for LaTeX-Super-Renderer-3.7”. De-facto becoming in-band signaling of the kind Gemini apparently tries to dispell by “only allowing one field in response, the mimetype”. But wouldn’t people start doing that anyway, and start writing such plugins anyway, and start de-facto reinventing HTML anyway? With the likes of Flash plugins of yore raising from the dead in new clothes?

                                                                      With regards to just graphs and images, I believe it could be somewhat easier actually. I am now reminded of the days of my childhood, when books in my country were mostly in black and white, with a few pages in the center being in color, and being referenced to by the b&w pages (“See illustration 7.b.”). I imagine something like that could work with images being listed, or linked, or attached (?) at the end of a Gemini page. But even then, I don’t think that would be usable for math-heavy documents. And from glancing through the few links around here, it’s not fully clear to me if any images at all, even at the end of a document, are expected to be supported? (Though mimetypes support kinda seems to hint at allowing this?)

                                                                      So, with the ideal of simplicity sounding theoretically interesting and alluring, at the same time I’m concerned about losing access to, especially, maths-related content. (Assuming that some image-heavy, yet comparatively sparse in that regard, content could be possibly approximated with a list of image links at the end, referenced from the main article. Which kinda already sounds like a hack, that would probably soon lead to reintroducing inline images via specialized clients…) That feels like a pity to me.

                                                                      1. 2

                                                                        I do understand that the protocol seems to allow for a lot of expression, and more “slow-food” style, relaxed consumption. Yet, I cannot escape immediately having a feeling of loss. If I understand this correctly, with no mathematical equations support, I would not be able to write or consume an article trying to explain Shor’s algorithm. With no graphs support, I assume whenever I’d want to display graphs, I’d have to resort to ASCII-Art as a workaround, with necessarily limited precision (for example, no smooth curves or complex 3D surfaces). Finally, I assume I would not be able to write or consume an article about typography, or bysantine icons. Or am I wrong?

                                                                        I’ve brought this issue about math up in the past, but remember, nothing is stopping you from using HTML or PDF. Gemini uses Gemtext as a lowest-common-denominator form of markup, but if you are writing a client, you can always built in HTML support or shell out to something else that does.

                                                                      2. 5

                                                                        I think that the reason that people keep trying to suggest features is that Gemini is marketed as a replacement for “the web”, but designed (and only useful) as just a plain text delivery mechanism - there’s (rightly) some cognitive dissonance going on.

                                                                        That is, “the web” includes plain text, hypertext, rich documents, audio/video media, and interactive content. I don’t think that most people expect Gemini to implement the last two, but there’s some reasonable expectation that something trying to replace “the web” should at least be able to represent reasonably interesting documents - such as scientific papers, which Gemini clearly cannot.

                                                                        Perhaps if proponents of Gemini stopped trying to claim that it could actually replace anything but the tiniest slice of the web that is pure, plain text (not hypertext, as inline links are really important), you would stop getting people talking about adding features.

                                                                        Edit: that is, there’s no “preconception” going on here. GP post is responding to the argument that Gemini is a viable replacement for the web by pointing out that it is missing critical features for actually acting as a sane replacement, which has no “preconception” - it’s just a statement of fact. You cannot replace the web without inline links, formatting, or inline images, full stop.

                                                                        1. 1

                                                                          You cannot replace the web without inline links, formatting, or inline images, full stop.

                                                                          As a matter of fact, you can. Full stop.

                                                                          1. 7

                                                                            …that’s so obviously false that I shouldn’t need counterexamples, but here they are:

                                                                            https://graphicallinearalgebra.net/2015/04/26/adding-part-1-and-mr-fibonacci/

                                                                            http://foldr.org/mw/emacs/

                                                                            http://507movements.com/

                                                                            https://lobste.rs/

                                                                            https://www.av8n.com/physics/clifford-intro.htm

                                                                            None of these sites, or the majority of the thousands of other bookmarks that I have, can be replaced without inline links, formatting, or images.

                                                                            Linked files do. Not. Cut. It.

                                                                            Not only are linked (vs. inline) files objectively and meaningfully different (and, therefore, you cannot replace the web with them), but they are inferior from an information-processing standpoint. From both a practical and theoretical level, you cannot replace the web with Gemini as it currently stands. People will not use it, they are not equivalent, and I, personally, will discourage people from using it - it is a cognitive anti-tool, similar to a reverse printing press. Useful for implementors, terrible for anyone who actually wants to read documents and exchange information that will actually help them to learn and reason.

                                                                            1. 0

                                                                              I can provide another counter-example: you had 5 links in your comment, none of which were inlined, despite having access to inline links on lobsters.

                                                                              You can’t place identical content on Gemini as you can on the web. You can reformat your content for the new medium. Nothing transitions between mediums without change.

                                                                              Gemini replaces a subset of the web, which I stated in the opening article and have re-iterated dozens of times.

                                                                              1. 4

                                                                                I can provide another counter-example: you had 5 links in your comment, none of which were inlined, despite having access to inline links on lobsters.

                                                                                …that’s not a counter-example. I never claimed that you can’t replace any of the content on the internet, which is what that statement would be a counter-example to. Perhaps you should take a logic course. Edit: that was too aggressive, my apologies.

                                                                                Edit: I claimed that you cannot replace “the web”, which implies replacing the majority of the content at least, and all of the content at most.

                                                                                You can’t place identical content on Gemini as you can on the web. You can reformat your content for the new medium.

                                                                                …and, as it should be very clear, that re-formatting process is (1) lossy and (2) causes a significant decrease in readability and information processing - meaning that it is not a replacement. The format of the web is part of the web, therefore you cannot replace the web with Gemini.

                                                                                Gemini replaces a subset of the web, which I stated in the opening article and have re-iterated dozens of times.

                                                                                You have tried to claim that Gemini can replace anything but the tiniest trivial slice of the web that is pure, un-hyperlinked plain-text - which, as shown above, is false. The format of the web is part of the web. The content and the medium are entwined.

                                                                                1. 2

                                                                                  Anything with “less features” is lossy, but that’s not correlated with a “significant decrease in readability”. On the other hand, putting the presentation entirely into the client’s control is a significant win for readability, because the user can configure fonts, contrast, and layout to their preferences and usability needs. The screen reader experience, for instance, is WAY better. Inline links and inline formatting are less accessible to some.

                                                                                  1. 4

                                                                                    Anything with “less features” is lossy

                                                                                    Only true in the most needlessly pedantic sense. You can easily build systems with a small set of carefully-chosen features that still implement the vast majority of the functionality of larger systems that were poorly-built. This…does not describe Gemini. Gemini picked a basis of features that do not allow it to implement more complex functionality or richer documents.

                                                                                    that’s not correlated with a “significant decrease in readability”

                                                                                    Not in general, sure - but in the case of Gemini, it’s pretty clear that there is a significant decrease in readability (and, therefore, cognitive processing ability). Requiring users to click through to a separate document for each image, animation, or individual mathematical equation is extremely unreadable for anything with a reasonable density of those things - while “only” being merely slightly less readable for documents with only a few of those objects.

                                                                                    On the other hand, putting the presentation entirely into the client’s control is a significant win for readability the user can configure fonts, contrast, and layout to their preferences and usability needs

                                                                                    You can already almost entirely control presentation with major browsers for the current web, through userscripts and stylesheets.

                                                                                    Moreover, HTML is actually structured, unlike Gemtext, so I have anchor points and document metadata that I can use to exert more control over my content than with unstructured Gemtext - so your statement is actually untrue.

                                                                                    The screen reader experience, for instance, is WAY better.

                                                                                    Modern web gives you more options to make a good screen reader experience - such as image alt text. Is it really inconsistently used? Sure, but that’s not a technical problem, that’s a social one - a social one that manifests itself in an equally bad way with Gemini, because Gemtext doesn’t have any way to bind accessibility-related metadata to document structure, because there is no document structure.

                                                                                    Inline links and inline formatting are less accessible to some.

                                                                                    Expanding links and controlling inline formatting are very basic features for screen readers.

                                                                                    So, to summarize, modern web gives you a much richer interface for screen readers, as well as a better interface for people who don’t use screen readers (which is the majority).

                                                                                    Gemtext is straight-up less conducive to thought and information transfer.

                                                                                    1. 2

                                                                                      I think “significant decrease in readability” is interpreted a little too literally here.

                                                                                      Many of the documents linked by fouric would be way more inconvenient to read if they were adapted to fit in gemini’s limitations because of how heavily they rely on tables, anchor links to a specific part of the same document, and inline images (especially with how inconsistent most clients available today are at dealing with non-gemtext content types).

                                                                                      I personally am a big fan of how gemtext files only ever result in a single request due to the lack of stylesheets, scripting, and inline resources, and have pushed back on the adoption of HTML and Markdown as alternate document content types because I think that is a valuable part of the protocol. I’m also not delusional: the stance I hold also directly limits the mass appeal of gemini as a technology, because normal people who just want to read these kinds of documents are unlikely to accept what seems like massive inconvenience for technical reasons they don’t fully understand. They’ll just stick to the Web.

                                                                                      I am willing to put up with that inconvenience as a reader and writer of gemtext content because I think the other benefits of gemini are worth it, but I don’t expect anyone who isn’t another disillusioned Web developer to get on board quite as easily.

                                                                          2. 2

                                                                            How would metafilter-style writing look, with lots of inline links? Examples appreciated.

                                                                            1. 7

                                                                              You just wouldn’t have lots of inline links. You’d have a separate list of references, perhaps. Here are some examples of Gemini posts I have written with lots of links:

                                                                              https://portal.drewdevault.com/2020/11/01/What-is-Gemini-anyway.gmi

                                                                              https://portal.drewdevault.com/2020/10/22/Firefox-the-embarassment-of-FOSS.gmi

                                                                              https://portal.drewdevault.com/2020/10/01/Spamtoberfest.gmi

                                                                              Let me put it this way: how would inline links work in a newspaper, book, or magazine? The answer is: they wouldn’t, and you just work within those limitations. That doesn’t mean the medium is bad.

                                                                          3. 6

                                                                            19th century newspapers could do this, and if your technology can’t emulate typography produced on a daily basis – often in a hurry – 150 years ago then your tech is rather limited.

                                                                            Not arguing against you, but when it comes to web browsers, then they still haven’t learnt to properly hyphenate text. Another thing we could do 150 years ago. One of the reasons (another being rather poor support for rendering maths, or, in the case of Chromium, complete lack of it) I’m relieved whenever I get to read a PDF generated from LaTeX instead of a webpage.

                                                                            1. 5

                                                                              You can manually hyphenate with soft hyphens like you did 150 years ago just fine.

                                                                              Automating hyphenation is nearly impossible. LaTeX certainly requires some manual help.

                                                                              1. 2

                                                                                If I use soft hyphens, then copying He&shy;re are rhi&shy;nos. to terminal results in He re are rhi nos. visually, because it copied the text with the soft hyphens even though everything was on a single line.

                                                                                LaTeX certainly requires some manual help.

                                                                                It never required it when I used it. When you enable correct language support, it should just work.

                                                                                1. 3

                                                                                  That’s because your terminal is 1970s computer technology.

                                                                                  1. 1

                                                                                    No. That’s because I copied text, but received bytes instead.

                                                                                    FWIW it doesn’t happen with hyphenate: auto in CSS, but that’s only supported by Firefox, so it’s irrelevant.

                                                                                  2. 2

                                                                                    150 years ago you couldn’t copy manually-hyphenated text from your newspaper in to your terminal.

                                                                                    1. 2

                                                                                      I thought this analogy was cute for a little bit. But seriously, why is 150 years ago the standard? Are we really that painfully nostalgia-blind that we can’t recognize there’s a medium point between the old and the new?

                                                                                      1. 3

                                                                                        It isn’t my standard, and frankly I don’t really care what we could / couldn’t do 150 years ago because I don’t believe it matters in the context of what are we doing, trying to do, and can’t do today.

                                                                                2. 4

                                                                                  Firefox does hyphenation with hyphens: auto; if you set the document language.

                                                                                  1. 1

                                                                                    Does Gemini support setting the document language?

                                                                                    1. 2

                                                                                      A quick search on the specification shows it does. So hyphenation working on gemini or not would be a matter of the client implementing it.

                                                                                  2. 2

                                                                                    Well, 150 years ago it was a manual process, and not an automatic one :-) I regularly insert &shy for this on my website by the way. But yeah, I agree that the lack of a universally supported hyphenate: auto has been long overdue (only Firefox supports it, and it doesn’t even do a great job at it, although I don’t know if there are technical reasons that prevent it from doing a better job; I think I saw someone on HN detail some issues with automatic hyphenation on the web years ago, but I forgot what they were).

                                                                                    Math rendering has also been supported by Firefox for 15 years with MathML, but for some reason Chrome doesn’t (but Safari does). Actually, looks like they removed it in Chrome years ago, although it looks like there’s work to add it back soon-ish.

                                                                                    1. 1

                                                                                      Actually, along with Firefox, some versions of Chrome (Android & Mac) support auto hyphenation, and Safari has it hidden behind a prefix: https://caniuse.com/css-hyphens I feel that together these may cover enough of the market to make it worth adding to web pages.

                                                                                      If you want to see a quick and stupid demo, here’s my very unfinished Hugo theme I’m working on developing: https://www.maximumethics.dev/blog/2020/07/my-first-post/ If you shrink the viewport width until it’s skinny like a phone, you’ll see the longer 90 chars “alphabet” line break to hyphenate (on the aforementioned supported browsers), as well as random words in the text.

                                                                                      1. 2

                                                                                        Ah yeah, it’s just the desktop version that doesn’t support it for some weird reason; I forgot about that detail.

                                                                                        I suppose it depends what you use it for: most of the time I really wanted auto hyphenation is for comparatively narrow pieces of text and/or for text-align: justify (e.g. a sidebar or info box). In those cases the hyphenation (manual or automatic) is pretty much a must-have rather than a nice-to-have, because without it it just looks ugly. Your site, on the other hand, looks fairly decent without hyphenation even on smaller screens, but it won’t hurt to add it for browsers that support it.

                                                                                      2. 1

                                                                                        Math rendering has also been supported by Firefox for 15 years with MathML, but for some reason Chrome doesn’t (but Safari does). Actually, looks like they removed it in Chrome years ago, although it looks like there’s work to add it back soon-ish.

                                                                                        That’s what I was on about. If something isn’t available in Chrome, it’s irrelevant these days. I’ve been aware that “it looks like there’s work to add it back soon-ish” for quite some time now. I don’t have hope.

                                                                                    2. 5

                                                                                      I think the idea is that the publisher delivers only the information, not the presentation. This constraint allows for greater user control over presentation, for example one might configure Gemini to lay out news articles (maybe on a site by site basis) in a column style, to suit their preferences. I think it’s an interesting idea as a way of making better typesetting more widely available to content published on sites without the budget / know-how to tune their CSS for readability.

                                                                                      I agree that Gemini would be better off if it supported inline images. That’s pretty often part of the information layer.

                                                                                      1. 6

                                                                                        I think the idea is that the publisher delivers only the information, not the presentation.

                                                                                        Wait! That is HTML! (as compared to PDF)

                                                                                        1. 6

                                                                                          Not really, HTML still focuses a lot on presentation. Headers, side bars, footers, nagivation line, footnotes, etc. are all concepts foreign to the browser, that have to be manually re-implemented in the markup language, again and again. That’s actually what I think that protocols like Gemini misunderstand: HTTP/HTML is not too systamatically complicated (at most it’s “organically” complicated), it’s actually too simple. If for example the <head> section had made it possible to describe how a web-page is navigated, it would have been possible for the browser to render that part of the site, leaving the <body> to be actual content. It’s something you see with other simple languages too: Scheme has various implementations, that implement a common core, but diverge in their additional libraries, that are regarded as useful, but do not allow for code to be portable. So too, had HTTP/HTML has the foresight to recognize what people would want from websites (which is probably too much to ask), or at least the foresight to allow a certain degree of extensibility, a lot of implementation-specific complexity could have probably been avoided.

                                                                                          So the issue might not be “simplicity vs compexity”, but “what kind of complexity”.

                                                                                          1. 3

                                                                                            Years ago I came across a browser extension that would build a navigation bar if it found certain <link> tags in the header. I modified my blogging engine to automatically add said links. Sadly, the extension no longer runs (because of course the extension mechanism changed—at least twice—since then) but I still generate the data in the vain hope it’ll be useful again.

                                                                                        2. 5

                                                                                          A strict separation of information and presentation is something that sounds great in theory, but doesn’t really work all that well in practice once you move beyond the simple use cases, or use cases that are pre-defined and supported in some way.

                                                                                          1. 1

                                                                                            I don’t know if this was part of the design goal - from what I can see, Gemini doesn’t have any sort of mechanism to describe or annotate what content/information is being delivered, which would make any presentation strategy necessarily heuristic, as there would be no way to delineate e.g. where the “author” of the article is.

                                                                                            1. 1

                                                                                              I guess you’d just put that information into the document itself.

                                                                                              One underappreciated tag in HTML is the <address> tag, which is a nice place to put stuff like author, publication date, etc.

                                                                                              1. 1

                                                                                                I’m not quite sure what you mean - when you say “just put that information into the document itself” do you mean in a machine-readable format? Because if so, that’s not part of the Gemini text spec - which is the thing being discussed. Adding tags like <address> would be extending the Gemini spec to add features, which is what proponents of it are fiercely against.

                                                                                                1. 1

                                                                                                  No, I just meant to add something like

                                                                                                  • Author: Gustaf Erikson
                                                                                                  • Date: 3 Nov 2020

                                                                                                  in the body of the text somewhere.

                                                                                          2. 4

                                                                                            As it has been stated on the mailing list, “you can always serve up text/html or text/markdown, you aren’t limited to text/gemini.” There are sites out there serving up images and even music.

                                                                                            1. 12

                                                                                              Well, that’s not really what’s advocated for in this post (or on the Gemini website as far as I could see), and serving HTML over Gemini just sounds like a s/HTTP/Gemini/, and you can serve text or Markdown over HTTP as well. I don’t really see the point if you’re going to serve HTML over Gemini.

                                                                                            2. 4

                                                                                              I’m just going to drop a link to the FAQ page, in the hopes that it’ll answer some questions here.

                                                                                              1. 3

                                                                                                I’m sick of how complex html+css+js gets, and how complicated extracting the actual content can be, in order to view it on “unusual” platforms (small screens. ereaders, etc.), where a good experience depends on the client controlling the presentation of the content. Something like gemini could be the solution to that, but I have to say, the lack of inline images is a deal-breaker for me.

                                                                                                I view multi-column layouts as part of the presentation, not the content (I don’t want two columns of text on a phone screen: if I do want two columns of text on the desktop, I can write that into the client), but the images are part of the content.

                                                                                                I did wonder whether technically a gemini-ish client could fetch urls which look like they point to images and display them inline, but this is explicitly banned by the specification:

                                                                                                Clients can present links to users in whatever fashion the client author wishes, however clients MUST NOT automatically make any network connections as part of displaying links whose scheme corresponds to a network protocol (e.g. gemini://, gopher://, https://, ftp://, etc.).

                                                                                                I can sort of see what they’re aiming for, presumably trying to avoid the sort of nonsense some websites have become, where the initial request just gets a jumble of html elements that require the client to fetch and execute tons of javascript before they can be assembled into something resembling content, but I think they should have made an exception for images (They could still leave it up to the client whether it displays inline images or not).

                                                                                                Interestingly, gemini probably works really well with screen readers, until you get to an image link (or any link for that matter), at which point it depends entirely on whether the author bothered to provide a caption, or explain what the link is as part of the text (Although that’s arguably no worse than html, where the existence of alt text doesn’t mean that it is always used or used correctly).

                                                                                                1. 5

                                                                                                  Another reason to not allow non-user-initiated network connections is that’s abused to track users (Facebook pixel etc).

                                                                                                  IMHO this could be sufficiently solved by allowing additional requests to the same domain as the document request.

                                                                                                  1. 4

                                                                                                    One big reason I’m excited about Gemini is the possibility of escaping the web’s bloat, which we need to do in order to reduce energy usage / emissions in the climate crisis. For this reason, I support Gemini’s decision to ban inline images. If you have inline images, next people will want inline video, and soon even if Gemini is “simpler” it’s using as much storage+bandwidth+processing as a web page.

                                                                                                    Inlining images, especially from other locations, is also a plausible method for sneaking user tracking back into the protocol, and I also approve of Gemini’s strong efforts towards privacy. I also hate ads even when they don’t track people, I’d love to live someplace where billboards are banned so I can escape the constant background hum of marketing. Eliminating inline images is also a great way to reduce the annoyance of advertising.

                                                                                                    I’m going to have to agree with everyone in this thread that refusing to implement inline links was a mistake, though. That was one of the biggest problems I had with Gopher, and I was hoping Gemini would do a better job with hypertext.

                                                                                                    1. 4

                                                                                                      Inlining images, especially from other locations, is also a plausible method for sneaking user tracking back into the protocol, and I also approve of Gemini’s strong efforts towards privacy.

                                                                                                      It’s a very difficult balance to strike between simplicity and functionality. You could probably avoid trackers by, as @m_eiman says, only allowing additional requests from the same domain as the original document.

                                                                                                      Alternatively, you could create/use a format which embeds the images directly in the original document (similar to how pdf or epub does). Some might argue that this is inefficient, as you need to serve multiple copies of the image data if it appears on multiple pages. However, in this context, it could be seen as a feature, as we don’t want the kind of images (such as ads) which aren’t actually “part of the document”.

                                                                                                      There will always be some people who abuse any protocol, but not having inline images is very limiting and means it isn’t any good for a significant portion of (what I consider) “good content that doesn’t need javascript/css”.

                                                                                                  2. 1

                                                                                                    I’d like to have an inline image every now and then

                                                                                                    Might I ask what’s wrong with merely providing a link to the image?

                                                                                                    or a table.

                                                                                                    This can be easily done with some ASCII art (which is what you do for Markdown, anyways) coupled with a preformatted block.

                                                                                                    1. 10

                                                                                                      This can be easily done with some ASCII art…

                                                                                                      But ASCII art is pretty much the quintessence of mixing presentation and content, which is very specifically what Gemini seeks to avoid. It can’t wrap or scroll differently on small devices, it can’t be extracted (easily and reliably) to a machine-readable data format, and it isn’t accessible because there’s no semantic information about the table structure.

                                                                                                      1. 3

                                                                                                        You’re right. I kinda hope some kind of table syntax does make it into the Gemini spec after all.

                                                                                                      2. 3

                                                                                                        I need to click a link to an image? It seems pretty obvious to me that this is worse UX than just displaying it.

                                                                                                        ASCII art relies on a monospaced font, which I don’t really like for reading text. I think Lobsters would be significantly worse if it was all monospaced text for example. It also relies on a certain screen size, which doesn’t work for smaller screens (or very big ones, for that matter). I mean, it kinda works but it’s a hack.

                                                                                                        1. 2

                                                                                                          I need to click a link to an image? It seems pretty obvious to me that this is worse UX than just displaying it.

                                                                                                          Yeah. You’re completely right, this is worse UX. I’m beginning to think of proposing an inline-image syntax for the gemini spec.

                                                                                                          ASCII art relies on a monospaced font, which I don’t really like for reading text.

                                                                                                          I admit I was wrong. ASCII tables are a hack, I suppose. Hopefully the simple table syntax I’m thinking of right now makes it into the spec!

                                                                                                          I think Lobsters would be significantly worse if it was all monospaced text for example. It also relies on a certain screen size, which doesn’t work for smaller screens (or very big ones, for that matter). I mean, it kinda works but it’s a hack.

                                                                                                          FYI there are gemini clients that support variable-width fonts.

                                                                                                          1. 4

                                                                                                            FYI there are gemini clients that support variable-width fonts.

                                                                                                            I tried to use it with Gopher years ago (I hacked up an existing client), and many sites don’t really work all that well because they assume stuff will be displayed as monospaced text (such as tables you mentioned, but also things like code examples, or other alignment things).

                                                                                                            Gemini is a bit of an improvement because the spec at least recognizes that proportional text exits and has “code blocks”, but it also leaves much to the clients. In other words: I wouldn’t be surprised if people would just keep using it as they used Gopher, and the weird (and IMO, broken) way line wrapping works in Gemini doesn’t help either. I suppose time will tell how it will be used in practice (but as mentioned, I don’t think it will be used much in practice beyond a few “Hacker blogs” like Drew’s).

                                                                                                            One of the big mistakes of HTML was, ironically, to “leave it up to the clients” which resulted in a big mess which had to be unified so that people could get a reasonably consistent experience irregardless of which browser they use, but that just unified the mess instead of clearing it up.

                                                                                                        2. 1

                                                                                                          I use HTML to add tables to my Markdown content.

                                                                                                          1. 1

                                                                                                            I personally find HTML’s table syntax difficult to read and use.

                                                                                                            1. 4

                                                                                                              Yep, I usually whip up a quick Perl script with the (deprecated) CGI module to generate it from plain text ;)

                                                                                                        3. 1

                                                                                                          Gemini is only a protocol, but the specification also propose a basic text/gemini format as a de-facto document type. It however supports any mimetype, including text/html:

                                                                                                          5 The text/gemini media type
                                                                                                          5.1 Overview

                                                                                                          In the same sense that HTML is the “native” response format of HTTP and plain text is the native response format of gopher, Gemini defines its own native response format - though of course, thanks to the inclusion of a MIME type in the response header Gemini can be used to serve plain text, rich text, HTML, Markdown, LaTeX, etc.

                                                                                                          You could totally sent the following HTML document over gemini:// and enjoy inline images:

                                                                                                          <html>
                                                                                                            <body>
                                                                                                            <p>My favorite cat picture:</p>
                                                                                                            <img src="gemini://domain.tld/cat.png" />
                                                                                                            </body>
                                                                                                          </html>
                                                                                                          

                                                                                                          It would then be up to the gemini client to process this document, fetch required resources (the cat picture) and render it nicely for you. The advantage of the gemini protocol here is that it would put a limit on the HTML subset one could put in a document. For example, <form> tags won’t be any useful. There would also be no need for <script> tags, as they wouldn’t be able to “phone home” over gemini.

                                                                                                          One of the biggest problem of the web is that the HTTP protocol itself grew in an enormous monster that’s impossible to understand, where sometimes the headers of a requests can be 10 times bigger than the content itself.

                                                                                                        1. 5

                                                                                                          Hmm… Pretty cool! NixOS recently fixed an issue with 32MB large shebangs. I wonder if this does actually work on macOS? I thought macOS prohibited the program in a shebang from being a script with a shebang? already in the README. It might be interesting to note that Perl already re-parses its shebang on startup, to handle very large shebangs.

                                                                                                          1. 1

                                                                                                            I saw that story – is there hope that they’ll actually improve the shebang mechanism with whatever the solution to this is?

                                                                                                            There is work being done to achieve the original goal (preventing the kernel from possibly running the wrong interpreter) while not breaking existing users; that is proving harder than one might expect and will almost certainly have to wait for 5.1.

                                                                                                            It’s too bad that the -I magic nix does with perl is perl-specifc (and that it requires 32MB shebangs). Does nix similarly isolate other interpreted languages this way?

                                                                                                          1. 27

                                                                                                            We’ve learned that having a function that deletes your database is too dangerous to have lying around. The problem is, you can never really test the safety mechanisms properly, because testing it would mean pointing a gun at the production database.

                                                                                                            I’d argue that you should have this function “laying around”, for development use – and it should be the only way you drop and create tables. Otherwise, you’ll fall back to dropping tables by hand, where it is impossible to add more safety. At least with this function you can add checks like “don’t do it if any table has 1000 records” or something. This is called poka yoke.

                                                                                                            1. 2

                                                                                                              Thank you for sharing the link to poka yoke! I had never heard this expressed in such formal terms, but it makes a lot of sense after it is.

                                                                                                              1. 1

                                                                                                                This is interesting. I’m always surprised at how frequently software development leans on learnings from manufacturing. I guess it’s just another form of manufacturing.

                                                                                                              1. 1

                                                                                                                If you liked this post, do me a favor and tell me if you can run the dev build of Oil, which I believe takes less than 2 minutes, although people appear to have problems with it:

                                                                                                                https://github.com/oilshell/oil/blob/master/README.md

                                                                                                                It’s basically:

                                                                                                                1. A Linux machine – Debian/Ubuntu-ish is most likely to work – bash, coreutils, etc. (Other distros should also work, but Debian/Ubuntu are frequently tested)
                                                                                                                2. git clone (with a submodule)
                                                                                                                3. build/dev.sh minimal
                                                                                                                4. bin/osh.
                                                                                                                5. Now you have a shell and you can even modify it with pure Python code!

                                                                                                                I think that is easy, but I’m interested in feedback otherwise. And let me know if there instructions are unclear.

                                                                                                                If you have a Linux machine, and are not able to do that in 2 minutes, let me know where you are at the 2 minute mark … did something not work?

                                                                                                                1. 4

                                                                                                                  I gave this a go, but I don’t have a /bin/bash and got stuck at:

                                                                                                                  Removing _devbuild/gen/*
                                                                                                                  asdl/hnode.asdl -> (asdl/tool) -> _devbuild/gen/hnode_asdl.py
                                                                                                                  frontend/types.asdl -> (asdl/tool) -> _devbuild/gen/types_asdl.py
                                                                                                                  core/runtime.asdl -> (asdl/tool) -> _devbuild/gen/runtime_asdl.py
                                                                                                                  tools/find/find.asdl -> (asdl/tool) -> _devbuild/gen/find_asdl.py
                                                                                                                  ./build/dev.sh: build/codegen.sh: /bin/bash: bad interpreter: No such file or directory
                                                                                                                  

                                                                                                                  I do have a /bin/sh (which is dash) and I do have bash in my PATH and /usr/bin/env, so #!/usr/bin/env bash would work.

                                                                                                                  edit I gave it more than two minutes. I ran:

                                                                                                                  rg -l '#!/bin/bash' | xargs sed -i -e 's%#!/bin/bash%#!/usr/bin/env bash%'
                                                                                                                  

                                                                                                                  and got further, but found gcc was needed:

                                                                                                                  build/setup.py -> libc.so
                                                                                                                  unable to execute 'gcc': No such file or directory
                                                                                                                  error: command 'gcc' failed with exit status 1
                                                                                                                  

                                                                                                                  I added gcc and readline to my environment, and ran:

                                                                                                                  (export PATH=$(dirname $(which zcat)):$(dirname $(which git)):$(dirname $(which gcc)):$(dirname $(which bash)):$(dirname $(which coreutils)):$(dirname $(which python2)); ./build/dev.sh minimal)
                                                                                                                  

                                                                                                                  and it seems to work, and the build is super fast. Really nice!

                                                                                                                  1. 3

                                                                                                                    Excellent, the shebangs are indeed a mistake and I appreciate the followups! Thanks for trying it.

                                                                                                                  2. 2

                                                                                                                    On Arch it works as advertised.

                                                                                                                    1. 1

                                                                                                                      ‘’’Removing _devbuild/gen/* /usr/bin/env: ‘python2’: No such file or directory’’’

                                                                                                                      Fwiw, it’s issues like this that steer me away from Python projects. I’ve been burnt out from maintaining Python dependencies, and even though Py2 is now deprecated, it still exists. If I really need a Python project, I’ll typically try to encapsulate it within a docker container or go out of my way to make sure it only support Py3.

                                                                                                                      I’ve been following Oil since it feels like it’s inception, but Python has always kept me from trying it, and still does apparently.

                                                                                                                      1. 3

                                                                                                                        OK thanks for trying it.

                                                                                                                        BTW you don’t need Python (either 2 or 3) to use Oil. You only need it to change the code.

                                                                                                                        The tarballs don’t depend on Python: https://www.oilshell.org/release/0.8.1/ (and that’s been true since the very first release, over 3 years ago)

                                                                                                                        Or it is also packaged in a number of places: https://github.com/oilshell/oil/wiki/Oil-Deployments

                                                                                                                        Also, for developers, there is a nascent shell.nix that declares the Python dep: https://github.com/oilshell/oil/blob/master/shell.nix

                                                                                                                        (I don’t use Nix personally, but others may want to run with it.)

                                                                                                                        Filed an issue for this: https://github.com/oilshell/oil/issues/833


                                                                                                                        Also FAQ about Python: http://www.oilshell.org/blog/2018/03/04.html#faq

                                                                                                                      1. 4

                                                                                                                        Good to know. Guess I won’t be using wayland anytime soon.

                                                                                                                        1. 2

                                                                                                                          There has recently been some work on implementing a “pure GTK3” Emacs: https://github.com/masm11/emacs

                                                                                                                          But other than that, I’m currently using Wayland and haven’t had any real issues with Emacs.

                                                                                                                      1. 3

                                                                                                                        Suggest tag: historical

                                                                                                                        1. 24

                                                                                                                          Don’t. They’re loud, heavy, inconvenient, and expensive.

                                                                                                                          1. 3

                                                                                                                            To add to this: unless you specifically want to learn about enterprise technologies (RAID, SFPs, etc.), go with consumer hardware. You’ll save money on parts, electricity, mounting, etc. and won’t have to deal with the noise as much. NUCs are great if you want something small and powerful(-ish), or Mini-ITX, mATX for small form-factor custom builds. The consumer space has a lot more flexibility than enterprise hardware, which lets you fine-tune your build for whatever usecase you have.

                                                                                                                            1. 1

                                                                                                                              I second the NUC comment. I have two set up at home and they’re awesome.

                                                                                                                            2. 2

                                                                                                                              Agreed.

                                                                                                                              I had a rack-mount 2U Compaq DL380 with all the bells and whistles that I got for free during the dot-com bust. It was a pain to live with:

                                                                                                                              • Weird boot process that only worked with Windows and Red Hat Linux
                                                                                                                              • So many fans that even though it was in the basement I could hear it on the ground floor
                                                                                                                              • It cost $10/mo in electricity, even after I removed one of the two power supplies
                                                                                                                              • Not that fast, except for the disks which were 15k RPM

                                                                                                                              Do the planet and yourself a favor and go with consumer-grade hardware.

                                                                                                                              1. 1

                                                                                                                                Do the planet and yourself a favor and go with consumer-grade hardware.

                                                                                                                                I’m not an environmentalist, but the prospective impact to my power bill has me concerned. $10 / month to run a fast machine would be okay, though. I’ll have to do some more research into TCO I think.

                                                                                                                              2. 2

                                                                                                                                They also eat power. See vermaden’s posts (e.g. https://vermaden.wordpress.com/2019/04/03/silent-fanless-freebsd-server-redundant-backup/ ) on running a simple server if you really need something.

                                                                                                                                (bias: I’ve done the freenas thing and paid into the myths of server hardware/ecc ram/etc. While it’s come in handy at times (ipmi is convenient) it’s also been a time burden when it comes to doing maintenance or trying to do any slight customization in setup from what freenas wants. If your needs are largely just storage, consider just leveraging your fast internet connection and paying a cloud provider. Remember even with a server-grade NAS you’re arguably still going to be paying someone for off-site backup.)

                                                                                                                              1. 25

                                                                                                                                Suggest -a11y

                                                                                                                                I also use set -e quite a lot. There is a very large and useful BashFAQ describing ways that set -e doesn’t do what authors might expect: https://mywiki.wooledge.org/BashFAQ/105

                                                                                                                                1. 9

                                                                                                                                  Suggest -a11y

                                                                                                                                  If you mean anything other than “accessibility”, could you please define what you’re talking about, or provide a link to a write-up of it? The abbreviation “a11y” is so universally used for “accessibility” (in the same way “i18n” is “internationalization”) that it’s effectively impossible to perform a search for other potential meanings.

                                                                                                                                  1. 4

                                                                                                                                    The article was originally tagged a11y, and:

                                                                                                                                    Suggest -a11y

                                                                                                                                    is as far as I know common syntax to instruct other lobsters to make a suggestion to remove the a11y tag.

                                                                                                                                    You can see the moderation log here: https://lobste.rs/moderations?utf8=%E2%9C%93&moderator=%28Users%29&what%5Bstories%5D=stories

                                                                                                                                    Edit: Seeing as the automatic moderation event took place 5 minutes after my comment, it appears that my understanding of it being common syntax is true.

                                                                                                                                    1. 26

                                                                                                                                      Here I was thinking -a11y was a Bash option.

                                                                                                                                      1. 3

                                                                                                                                        Same here :)

                                                                                                                                      2. 3

                                                                                                                                        OK. I didn’t realize this was trying to suggest something about the metadata of the post; the context of the rest of the comment made it seem as if you were suggesting some strange set of bash flags that I’d never heard of.

                                                                                                                                        1. 1

                                                                                                                                          My bad. I don’t know how I ended up adding -a11y

                                                                                                                                        2. 3

                                                                                                                                          This is exactly why I think these type of abbreviations are i5c [1]. It’s just a way to obfuscate words and confuse people. (Sorry for the overly salty reaction, but it’s a pet peeve.)

                                                                                                                                          [1] idiotic

                                                                                                                                          1. 0

                                                                                                                                            Same here, I would love to learn about this as well.

                                                                                                                                          2. 5

                                                                                                                                            Thanks for that link, I never suspected there were so many undefined corner cases with set -e

                                                                                                                                            1. 4

                                                                                                                                              Yes, great link. I think especially example 5 demonstrates that set -e might be more trouble than it’s worth.

                                                                                                                                              1. 4

                                                                                                                                                I wonder how many security issues are lurking underneath these inconsistent and weird semantics. This is the kind of muddled thinking that used to cause (and probably still causes) problems in PHP scripts.

                                                                                                                                            2. 2

                                                                                                                                              I also use set -e quite a lot. There is a very large and useful BashFAQ describing ways that set -e doesn’t do what authors might expect: https://mywiki.wooledge.org/BashFAQ/105

                                                                                                                                              Thanks. This is great learning for me. Fortunately, I switch to Python before such complications usually arise but you are right, one should not blindly add “set -e” to an existing BASH script they didn’t write.

                                                                                                                                            1. 3

                                                                                                                                              I’m a big fan of luadns.com. They have a simple Lua based DSL for authoring zone data and the records are kept in Git. It isn’t so expensive, either.

                                                                                                                                              1. 2

                                                                                                                                                This is a great, favorite trick of mine. It has some surprising edge cases, however! For example:

                                                                                                                                                [grahamc@Petunia:~]$ cat <(date)
                                                                                                                                                Sat Dec 14 21:08:17 EST 2019
                                                                                                                                                
                                                                                                                                                [grahamc@Petunia:~]$ sudo cat <(date)
                                                                                                                                                cat: /proc/self/fd/13: No such file or directory