1. 8

    These kinds of posts tend to take one out of two stances on ‘network transparency’ depending on what the author actually wants to attack, and if there is a reference to the “standup over substance” Daniel Stone talk on X11 vs Wayland - you can be sure it will be the ‘haha, X isn’t actually transparent’ one, followed by perplexed users saying that it is actually very useful to them. The reddit thread referencing this article is one such example.

    The first camp talking about the drawing primitives and commands as such and saying that the actual drawing commands should be sent and treated locally in the same way as they are treated remotely and since the toolkits have all made data type and parameters opaque pixmaps, opaque transfers are needed.

    The ‘my use experience’ camp talks about the convenience in just saying ‘run graphics program’ from a remote shell and it seem to appear on the local machine similarly enough to how it would behave had it actually been run locally. That’s the meaning of transparency to them.

    The later perspective is, in my opinion, more interesting since it can be improved on in many more ways than just switching compression schemes. Incidentally, I might just have something for that in the works…

    1. 3

      Most basic applications work fine with X11 forwarding. Stone’s talk stating X isn’t transparent is really referring to specific use cases (direct rendering and opengl), which, of course, isn’t going to work over the network.

      I agree with his talk that we need to move forward from X, but you can’t just hand wave away a lot of features that people currently use. Usability matters. It took Pulseaudio a long time to get to a usable state where it’s not the first thing I try to disable (I think it works pretty well now, especially with bluetooth devices). Systemd is still terrible in terms of its usability with its terrible CLI interface.

      1. 4

        tone’s talk stating X isn’t transparent is really referring to specific use cases (direct rendering and opengl), which, of course, isn’t going to work over the network.

        Gaming over the network has been around for some time. Some products were around before the cloud became a thing. These days, some are doing gaming desktops and rendering workstations with cloud VM’s. Probably more like X is an outdated design being made to do things in totally different context it can’t handle.

        1. 2

          specific use cases (direct rendering and opengl), which, of course, isn’t going to work over the network.

          Couldn’t OpenGL, at least, be made to work over a network? It’s just commands which have larger effects; I’d think some sort of binary protocol could transmit it fairly effectively.

      1. 15

        Spiritually similar to Baudelaire and “the finest trick of the devil is to persuade you that he does not exist”, the biggest deception search engines played was switching the meaning of ‘search’ from ‘select from what it knows, that which bests matches the criterion you provided’ into ‘select from what it knows, influenced by what it knows about you, that which <some deity - advertisers, hostile governments, …> wants you to know’.

        The saving grace is still how embarrassingly crap they are at it ;-)

        1. 6

          I’m all over the place this week. Some minor things to fix on the OpenBSD port of Arcan, along with a writeup on the porting experience as such. Then there is some low level stuff when it comes to accelerated/compressed handle passing/mapping/drawing for multi-planar YUV formats that falls in ‘tedious, please kill me, has to be done..’. To not burn out on that I’m playing around with support for 180/360 stereoscopic video playback on HMDs.

          1. 4

            Your best bet is probably to avoid modifying the environment at all, if possible.

            This is sound advice. Consider the environment immutable and your sanity will be preserved.

            1. 3

              This starts getting a bit rough when you’re using libraries that are relying on stuff like locale.

              I don’t know if the solution is to edit the environment, but the fact that so many C libraries change behavior based on env bubbles up in so many places. Bit of a rough foundation

              1. 2

                I treat the environment as immutable, but often call sub-processes with altered/augmented environments, e.g.

                #!/usr/bin/env bash
                doSomethingWith "$FOO" "$BAR"
                FOO="A different foo" someOtherThing
                

                For LOCALE in particular, HaskeIl (compiled/interpreted with GHC, at least) will crash on non-ascii data if LOCALE isn’t set to an appropriate value, so I often invoke Haskell programs with LOCALE="en_US.UTF-8" myHaskellProgram.

                I run into this a lot in build scripts (e.g. using Nix), since their environments are given explicitly, to aid reproducibility (rather than, say, inheriting such things from some ambient user or system config).

                I imagine this would be extra painful if using libraries which need conflicting env vars.

                Racket handles this quite nicely using dynamic binding (AKA “parameters”), which we can override for the evaluation of particular expressions. It feels very much like providing an overridden env to a sub-process. For example, I wrote this as part of a recent project (my first Racket project, actually):

                ;; Run BODY with VARS present in the environment variables
                (define (parameterize-env vars body)
                  (let* ([old-env (environment-variables-copy (current-environment-variables))]
                         [new-env (foldl (lambda (nv env)
                                           (environment-variables-set! env (first  nv)
                                                                       (second nv))
                                           env)
                                         old-env
                                         vars)])
                    (parameterize ([current-environment-variables new-env])
                      (body))))
                

                Looking at it now, it might have been nicer as a macro, like (with-env (("FOO" "new foo value")) body1 body2 ...).

                1. 2

                  Locale- defined behaviour is indeed an ugly little duckling that will never become a swan. One of the weird bugs I’ve been called into over the years was a C+Lua VM + script that was outputting CSV with “,” as element separator in a longer processing chain (system was quite big, simplified for the sake of story).

                  The dev had been observant and actually checked the radix point and locale before relying on its behaviour in printf- related functions. Someone had linked in another library though, that indirectly called some X+Dbus nonsense, received language settings that way, and changed the locale mid-stream - asynchronously. Sometimes the workload was finished, sometimes a few gigabytes had gone by and corruption was a fact as floats turned into a,b rather than a.b after a while…

                1. 1

                  To add to your pile, Systemic Software Debugging - free, almost entirely unknown and written by yours truly along with a colleague. It was not really intended as a book as such - the writing project had a few odd constraints given that the target was course material for mostly self-taught seniors that worked almost exclusively with debugging difficult problems, and we needed some common ground for the other parts of the course that I, sadly, can’t go into. Got some stories out of it though :-) …

                  Some of my thesis work a while back was also in this direction, though it had to be disguised because academia and funding disapprove when you get too practical (though there is ‘retooling and securing systemic debugging’ as an article). From that dark period I’ve gone through a few books as well. Some that comes to mind:

                  “The Science of Debugging” (telles/hsieh), “Debugging by thinking” (metzger), “If I only changed the software, why is the phone on fire?” (simone), the bland one by Zeller, Advanced Windows Debugging (hewardt, pravat), “How debuggers work” (rosenberg), “Linkers & Loaders” (levine), … but in general I’d give the literature (including my own modest contributions) a grunting sound as a kind-of review and summary.

                  1. 1

                    Awesome. Thank you! I’ve added it to my queue. It looks like a fun read.

                    I’ve read or own all the other books you recommended, except for Rosenberg’s book, strangely enough. The advice is helpful, though, because it’s good to know the common/popular set of books that are typical for the topic.

                  1. 6

                    Recently there’s been a lot of discussion of keyboard latency, as it is often much higher than reasonable. I’m interested in how much the self-built keyboard community is aware of the issue. Tristan Hume recently improved the latency of his keyboard from 30ms to 700µs.

                    1. 2

                      The Planck that Dan and I tested had 40ms of latency - not sure how much that varies from unit to unit though.

                      1. 3

                        I would expect very little, using the QMK firmware with a custom keymap. There’s typically only a handful of C with a couple ifs, no loops.

                      2. 2

                        Why are those levels of latency problematic? I would think anything under 50ms feels pretty much instantaneous. Perhaps for people with very high typing speeds or gamers?

                        1. 1

                          The end-to-end latency on a modern machine is definitely noticeable (often in the 100s of ms). Many keyboards add ~50 ms alone, and shaving that off results in a much nicer UX. It is definitely noticeable comparing, say, an Apple 2e (~25ms end-to-end latency) to my machine (~170ms end-to-end latency, IIRC).

                        2. 1

                          I recall reading about that. I’ll see about getting some measurements made, and see what it’s like on my Planck.

                          I’m interested in how much the self-built keyboard community is aware of the issue

                          I haven’t really seen much about it :/ If we could find an easy way of measuring latency without needing the RGB LEDs and camera, that would be good.

                          1. 2

                            a simple trick - use a contact microphone (piezo), jack it into something like https://www.velleman.eu/products/view/?id=435532

                        1. 1

                          How long until scammers leverage this in phishing campaigns as foreplay to extortion…

                          1. 2

                            Playing around with VR desktop code, likely get around to implement some geometry projections for 360/180 stereoscopic video source playback.

                            1. 15

                              Such pointless posturing and inflammatory piece of nonsense all in one (we’re talking about ~300 lines of code here).

                              The whole schtick that Nvidia does this out of spite is just short-fused idiocy. It doesn’t take much digging into the presentations that Nvidia has held at XDC2016/2017 (or just by asking aritger/cubisimo,… directly) to understand that there are actual- and highly relevant- technical concerns that make the GBM approach terrible. The narrative Drew and other are playing with here - that there’s supposedly a nice “standard” that Nvidia just don’t want to play nice with is a convenient one rather than a truthful one.

                              So what’s this actually about?

                              First, this is not strictly part of Wayland vs X11. Actually, there’s still no accelerated buffer passing subprotocol accepted as ‘standard’ into Wayland, the only buffer transfer mechanism part of the stable set is via shared memory. Also, the GBM buffer subsystem is part of Xorg/DRI3 so the fundamental problem applies there as well.

                              Second, this only covers the buffer passing part of the stack, the other bits - API for how you probe and control displays (KMS) is the same, even with the Nvidia blobs, you can use this interface.

                              Skipping to what’s actually relevant - the technical arguments - the problem they both (EGLStreams, GBM) try to address is to pass some kind of reference to a GPU-bound resource from a producer to a consumer in a way that the consumer can actually use as part of its accelerated drawing pipeline. This becomes hairy mainly for the reason that the device which produces the contents is not necessarily the same as the device that will consume it - and both need to agree on the format of whatever buffer is being passed.

                              Android has had this sorted for a long while (gralloc + bufferQueues), same goes for iOS/OSX (IOSurface) and Windows. It’s harder to fix in the linux ecosystem due to the interactions with related subsystems that also need to accept whatever approach you pick. For instance, you want this to work for video4linux so that a buffer from a camera or accelerated video decoding device can be directly scanned out to a display without a single unnecessary conversion or copy step. To be able to figure out how to do this, you pretty much need control and knowledge of the internal storage- and scanout formats of the related devices, and with it comes a ton of politics.

                              GBM passes one or several file descriptors (contents like a planar YUV- video can have one for each plane-) from producer to consumer with a side-channel for metadata (plane sizes, formats, …) as to the properties of these descriptors. Consumer side collects all of these, binds into an opaque resource (your texture) and then draws with it. When there is a new set awaiting, you send a release on the ones you had and switch to a new set.

                              EGLStreams passes a single file descriptor once. You bind this descriptor to a texture and then draw using it. Implicitly, the producer side signals when new contents is available, the consumer side signals when it’s time to draw. The metadata, matchmaking and format conversions are kept opaque so that the driver can switch and decide based on the end use case (direct to display, composition, video recording, …) and what’s available.

                              1. 4

                                The whole schtick that Nvidia does this out of spite is just short-fused idiocy. It doesn’t take much digging into the presentations that Nvidia has held at XDC2016/2017 (or just by asking aritger/cubisimo,… directly) to understand that there are actual- and highly relevant- technical concerns that make the GBM approach terrible.

                                I think they may have had more of a point if they had been pushing EGLStreams back when these APIs were initially under discussion, before everyone else had gone the GBM route. And I question the technical merit of EGLStreams - it’s not like the Linux graphics community is full of ignorant people who are pushing for the wrong technology because… well, I don’t know what reason you think we have. And at XDC, did you miss the other talk by nouveau outlining how much Nvidia is still a thorn in their side and is blocking their work?

                                The narrative Drew and other are playing with here - that there’s supposedly a nice “standard” that Nvidia just don’t want to play nice with is a convenient one rather than a truthful one.

                                What, you mean the interfaces that literally every other gfx vendor implements?

                                To be able to figure out how to do this, you pretty much need control and knowledge of the internal storage- and scanout formats of the related devices, and with it comes a ton of politics.

                                Necessary politics. A lot of subsystems are at stake here with a lot of maintainers working on each. Instead of engaging with the process Nvidia is throwing blobs/code over the wall and expecting everyone to change for them without being there to support them doing so.

                                Your explanation of how EGLStreams and GBM are different is accurate, but the drawbacks of GBM are not severe. It moves the responsibility for some stuff but this stuff is still getting done. Read through the wayland-devel discussions if you want to get a better idea of how this conversation happened - believe it or not, it did happen.

                                1. 1

                                  I think they may have had more of a point if they had been pushing EGLStreams back when these APIs were initially under discussion, before everyone else had gone the GBM route. And I question the technical merit of EGLStreams - it’s not like the Linux graphics community is full of ignorant people who are pushing for the wrong technology because… well, I don’t know what reason you think we have.

                                  Well, the FOSS graphics “community” (as in loosely coupled tiny incestuous fractions that just barely tolerate each-other) pretty much represents the quintessential definition of an underdog in what is arguably the largest, most lock-in happy, control surface there is. The lack of information, hardware and human resources at every stage is sufficient explanation for the current state of affairs without claiming ignorance or incompetence. I don’t doubt the competence of nvidia driver teams in this regard either and they hardly just act out of spite or malice - their higher level managers otoh - that’s a different story with about as broken a plot as the corresponding one had been at AMD or Intel in other areas where they are not getting their collective asses handed to them.

                                  Your explanation of how EGLStreams and GBM are different is accurate, but the drawbacks of GBM are not severe. It moves the responsibility for some stuff but this stuff is still getting done. Read through the wayland-devel discussions if you want to get a better idea of how this conversation happened - believe it or not, it did happen.

                                  I have, along with the handful of irc channels where the complementary passive-aggressive annotations are being delivered (other recommended reading is ‘luc verhaagen/libvs blog’ starting with https://libv.livejournal.com/27799.html - such a friendly bunch n’est ce pas?), what I’m blurting out comes from rather painful first hand experiences - dig through the code I’ve written on the subject and the related wayland-server stuff and you can see that I’ve done my homework, if there’s one hour of code there’s one hidden hour of studying/reversing prior use. I’ve implemented both (+ delivered evaluations on three others in a more NDAy setting) and experimented with them since initial proposal and bothered nvidia about it (~dec 2014) - neither deliver what I think “we” need, but that’s a much longer thread.

                                  On GBM: The danger is splitting up an opaque ‘handle’ into a metadata- dependent set of file descriptors, passing the metadata over a side channel, recombining them again with no opportunity for compositor- validation at reassembly. That is validation both for >type/completion<, for >synchronization< and for >context-of-use<. It’s “try and hope” for asynch fail delivery, crash or worse. There’s ample opportunity here for race conditions, DoS and/or type confusions (got hard-to-reproduce such issues with wild writes both in gpu-space and kernel-space in this very layer with Xorg/DRI3 bound as producer via a render node). Look at the implementation of wl_drm and tell me how that approach should be able to bring us back from buffer bloat land to chase-the-beam producer-to-scanout (the VR/AR ideal and 8k/16k HDR bandwidth scaling necessity) - last I checked Mesa could quad-buffer a single producer just to avoid tearing - FWIW, Nvidia are already there, on windows.

                                  What, you mean the interfaces that literally every other gfx vendor implements?

                                  Yes, the Mali and Exynos support for GBM/KMS is just stellar – except “Implements” range from half-assed, to straight out polite gestures rather than actual efforts, so about the quality of compositor ‘implementations’ on the Wayland side. Out of 5 AMD cards I run in the test rig here right now, 2 actually sort of work if I don’t push them too hard or request connector level features they are supposed to offer (10bit+FreeSynch+audio for instance). To this day, Xorg is a more reliable “driver” than kms/gbm - just disable the X protocol parts. I started playing with KMS late 2012, and the experience have been the same every step of the way. If anything, Gralloc/HWC is more of a widespread open “standard”. Then again, Collabora and friends couldn’t really excel selling consulting for Android alternatives if they’d use that layer now could they?

                                  Necessary politics. A lot of subsystems are at stake here with a lot of maintainers working on each. Instead of engaging with the process Nvidia is throwing blobs/code over the wall and expecting everyone to change for them without being there to support them doing so.

                                  and

                                  I think they may have had more of a point if they had been pushing EGLStreams back when these APIs were initially under discussion, before everyone else had gone the GBM route

                                  Again, gralloc/hwc predates dma-buf, and are half a decade into seamless multi-gpu handover/composition. For nvidia, recall that they agree on KMS as a sufficiently reasonable basis for initial device massage (even though it’s very much experimental- level of quality, shoddy synchronisation, no portable hotplug detection, no mechanism for resource quotas and clients can literally priority-invert compositor at will) - It’s the buffer synchronisation and the mapping to accelerated graphics that is being disputed (likely to be repeated for Vulkan, judging by movements in that standard). Dri-devel are as much the monty python ’knights who say NIH’ as anyone in this game: Unless my memory is totally out of whack EGLStreams as a mechanism for this problem stretch as far back as 2009, a time when khronos still had “hopes” for a display server protocol. Dma-buf (GEM is just a joke without a punchline) was presented in 2012 via Linaro and serious XDC @ 2013 with Xorg integration via DRI”3000” by keithp the same year. Nvidia proposed Streams @ 2014, and the real discussion highlighted issues in the span 2014-2015. The talks have then been how to actually move forward. Considering the “almost enough for a decent lunch” levels of resource allocations here, the pacing is just about on par with expectations.

                              1. 3

                                Sounds about right. I’m stuck with a 2K resolution on a 4K screen and no HDMI sound. Even picked a pricey GPU that’s supposed to be well supported. Can anyone suggest an AMD card that supports the above?

                                1. 3

                                  I currently have an AMD RX580 and use dual 1440p monitors without issue. I have also tested another 2x 1080p simultaneously with those 2x 1440p. All at 60Hz. So it can definitely push the necessary pixels for 4K on Linux. You need a more recent Linux kernel (4.8 or above iirc, I’m on 4.12) for the newer AMDGPU driver. There are some improvements to be made on certain codec playback support (I get occasional tearing on VP9 at 1440p for instance) but ever since the switch to the AMDGPU stack I have a lot of confidence in AMD fixing things as time goes on.

                                  My previous Nvidia card was supposed to support 2x 1440p, but wouldn’t run the second one at the correct resolution, instead limiting it at some odd resolution of 2160x1200 (iirc?). Would not work in nouveau, would not work in the latest proprietary drivers.

                                  As for HDMI audio on the RX 580, I am traveling but will try and test when I return home. I don’t know if I have an HDMI audio device though.

                                  1. 2

                                    I believe HDMI audio is still not available ootb until a set of AMD patches (that have been around for quite a while now) are mainlined.

                                    https://www.phoronix.com/scan.php?page=news_item&px=AMDGPU-Linux-4.15-1

                                    1. 1

                                      Yeah, HDMI audio with the latest AMD cards got pushed back when their big code dump was refused back in the spring. They’ve been rewriting it to use kernel APIs instead of wrappers ever since & the main code merge was accepted by Dave Arlie a week or two ago IIRC. So, barring some ironing out of wrinkles, 4.15 looks hopeful.

                                  2. 2

                                    I have an R9 390X that drives 3840x2160 + 1920x1080 all at 60Hz in Linux without issue. I’d do some research, though; I don’t even come close to stressing it in Linux, it’s there so the Windows install on that machine can run games.

                                    No idea about sound, my monitors are connected over displayport and dvi, respectively, and my speakers are plugged into my sound card.

                                    1. 1

                                      Even my old intel i915/i5 testing setups can push 60Hz@4K, it’s likely that it’s not about your card but rather about the combination of (GPU, connector, cable and display) or bandwidth saturation. KVMs and similar stuff in between can mess with the negotiation. For 4k@60Hz you want a chain capable of HDMI-2.0 or DisplayPort. It might simply be that Xorg probes the connector, gets the choice of 2k@60Hz or 4k@30Hz and picks the former.

                                      AMD cards won’t give you HDMI sound on the open source drivers unless you manually integrate their HAL patches. The relevant fixes are slowly trickling in, and chances are that in 4.15 the support will get there. But really, “recent” features like display-port chaining, event-driven synch like FreeSync etc. all have spotty support and its a gamble if it will work on any given card.

                                      1. 2

                                        I’m using the DisplayPort cable that came with my monitor. I can set it to 4K but the DE becomes unusably sluggish on both Wayland and Xorg. Thanks for the pointers.

                                    1. 5

                                      Can we please just mark the API/ABI as GPL-only and be done with it?

                                      Nvidia has received considerable leeway for years and nothing good has come out of it.

                                      1. 2

                                        I’m not sure what you mean here. The Nvidia binary module is already infringement, just no one interested in enforcement/compliance efforts :)

                                        1. 2

                                          Agree, but in fact there is a difference between EXPORT_SYMBOL and EXPORT_SYMBOL_GPL and it matters as copyright law usually takes an author’s intention into account.

                                        2. 1

                                          It would be interesting to see the outcome, since NVIDIA definitely relies on Linux for its HPC market.

                                          1. 1

                                            They maintain a FreeBSD port of their driver for some reason, guessing that such a move would empower more HPC on FreeBSD so yes please :)

                                          1. 1

                                            Also very much recommend multi-channel signed distance fields, code and reading at: https://github.com/Chlumsky/msdfgen

                                            1. 1

                                              Okay, so the video is cheesy but the tech is good. Why downvoted?

                                              1. 1

                                                Not from me, but I suspect that it’s due to a kickstarter with an ‘open source if we reach our goals’ knife to the throat.

                                                A non-kickstarter option is to take some adafruit neopixels, recycle a USB charger or two, a wifi-module, a pullup resistor and capacitor, stick to one of the many arduinos laying around and reuse the code from one of the many hackaday.com projects or something like playing with LEDs.

                                              1. 1

                                                The purpose is to use cli without terminal? Looks like it is a Rube Goldberg machine

                                                1. 1

                                                  The point is that the terminal protocols are a Storm P machine and CLIs should integrate with the toolkits or display server protocols that are in use and not go through a ‘tower of babel’- like translation layer like VtXXX. Lots of OSes us CLIs without a terminal emulator and are much better off because of that.

                                                  1. 1

                                                    Do you mean all curses program will not work in your system? Which os has not terminal emulator and much better than unixes?

                                                    1. 1

                                                      Curses can be implemented 1:1 over the existing API, so that’s just a matter of writing the mapper, and the current implementation has a terminal emulator that can be switched on for legacy support.

                                                      Terminal Emulator != Command Line/Shell or the tools you use from it. The argument is against the Protocol (how all these tools communicate, the ESC, OSC, … stuff) and how it makes it impossible to write something better than what you can do with curses right now.

                                                      Some non-terminal emulator based CLIs: Plan9? Windows(PowerShell)? TempleOS? or OS independent ones like Notty. I assume some language problems here, but “better off” != “much better than” - only that they have the potential to be.

                                                1. 3

                                                  There is arguably a rift here between those that prefer the ‘shove it in a browser’ or flashy UIs that animate and morph as you interact, and those that prefer staring into a text editor.

                                                  I have been thinking about starting a project similar to this, but I cannot decide between those two approaches. I love and use my terminal a lot, but once you go down the multimedia path and display images, you soon want to play audio, show videos, render tables, etc. The easiest way to achieve this is to use a browser. At least a browser widget.

                                                  The other aspect is security. If we tackle this, we should as well replace ssh and suddenly security is insanely important.

                                                  1. 1

                                                    If you go with the browser you’ve already surrendered every other tangible gain in favour of “easiness” (and especially security, it’s infinite attack surface already), you might just write your shell as a plugin and live there.

                                                    If you take the time to study the underlying system in a bit more detail here, you’ll find that both audio and video content is actually there, but as an extra and not a necessity.

                                                    I don’t think SSH would need to be replaced as such, recall how it is used to tunnel X, even though there are some other benefits from doing so - and the task is somewhat simpler these days with so many good crypto primitives around.

                                                  1. [Comment from banned user removed]

                                                    1. 3

                                                      Depends on what you mean by “mass produced.” If you mean built in high volume at a single factory, then $300 doesn’t sound unreasonable. If you mean produced by every factory that makes keyboards like most commodity keyboards, then <$50 would sound more reasonable.

                                                      You can read the blog posts over at http://blog.keyboard.io about producing a mechanical keyboard in China. There’s a huge amount of work in bringing a unique product like that to market. The reason a random keyboard from Fry’s is so cheap is that every company is practically selling the same keyboard with a different brand label on it.

                                                      Regarding the comment “it’s not even wireless,” adding a BT IC and antenna to a keyboard hardly has any impact on the per unit cost compared to the mechanicals when you’re building something unique like this.

                                                      1. 3

                                                        Keyboards should be moving in the direction of “tactile but silent”.

                                                        Not really. Auditory feedback is processed much faster than tactile feedback and this matters when you start typing faster. Cherry MX brown switches are the best combination as far as I’m concerned.

                                                        1. 2

                                                          or some HRTF binaural ASMR that emits the typing sound based on posture, limb lengths and key being pressed. But I have a sneaking suspicion that this guy wouldn’t consider it authentic enough. As much as I liked buckling spring, modern day mechanical are tactile enough that the sound pollution tradeoff just isn’t worth it.

                                                          1. 2

                                                            You think there is enough demand to justify mass producing this?

                                                            I think the only nonsense here is the $50 you just pulled out of thin air.

                                                          1. 12

                                                            I really like a lot of these ideas. I didn’t quite get the compatibility story though:

                                                            alternative platform output target that undoes all this work and outputs into a VT100 compliant mess again – all for the sake of backwards compatibility

                                                            Do bash, readline, and ls --color work inside your new system?

                                                            If you want to kill something you have to reimplement it. That’s how bash killed almost all previous shells – it subsumed all their features and ran scripts targeted for them.

                                                            That is pretty much the reasoning behind OSH in Oil (http://www.oilshell.org/). I mean part of it is that I don’t want to implement something worse than bash, so I have to understand it, and implementing it is how you understand it. I learned many things that made the Oil language better, regardless of whether it was compatible with bash.

                                                            And the other part is that running bash scripts and auto-converting them to Oil is a pretty reliable way to get rid of bash. But it’s obviously a huge amount of work.

                                                            I don’t know how big terminal emulators are, like xterm, or I use ROXTerm. I would guess 100K lines of code?
                                                            There are definitely a few minimal ones that are 10K or so.

                                                            So anyway, I think if you want to kill terminal protocols, which I think is a great idea, you have to implement them. That’s how Microsoft Excel killed all competitors too – it just opened all their file formats.

                                                            Another thing that occurred to me: it might be better have apps dynamically switch protocols using feature detection rather than require build time options. I think you were saying that building a new shell might require #ifdef or something, to use a new Arcan API (I might have missed that part.)

                                                            But I think being able to dynamically switch would provide a better compatibility story. I could just type “bash” or “oil” in both the old context and new context, and the binary is the same, and the same config files are read, and my user experience is kinda similar. At least for awhile.

                                                            1. 2

                                                              The quoted part of the compatibility story is for the other direction, running legacy things has been in place for many years. Say, for instance, that someone likes the API and uses it instead of curses to build their fancy new TUI for communicating via Matrix or managing files. The problem is that any users would be forced to run a terminal that speaks this API right? forcing people to give up their old ways like that isn’t very nice, and that’s where the compatibility layer would possibly become handy, though there are possibly some other interesting parts of shmif- that can be leveraged to avoid even that.

                                                              Do bash, readline, and ls –color work inside your new system?

                                                              Yes, the terminal emulator that’s in there is very much functional, with most of the normal stuff (bracketed paste, mouse, unicode, 24bit color, …). The shell itself is would be responsible for starting something in compatibility mode.

                                                              That is pretty much the reasoning behind OSH in Oil (http://www.oilshell.org/). I mean part of it is that I don’t want to implement something worse than bash, so I have to understand it, and implementing it is how you understand it. I learned many things that made the Oil language better, regardless of whether it was compatible with bash.

                                                              Oil looks highly interesting, I’ll be sure to remember that one when I evacuate bash from my systems :-)

                                                              I don’t know how big terminal emulators are, like xterm, or I use ROXTerm. I would guess 100K lines of code? There are definitely a few minimal ones that are 10K or so.

                                                              Xterm is something like 70kloc, everything built using libVTE adds at least 95kloc, ‘st’ has a decent amount of features and is in the 2k range I believe. But they are all also specialized, i.e. xterm needs an xserver, same with ‘st’, libVTE needs GTK (if I remember correctly) and so on. The solution I advocate here can work both in place of a kernel provided one (depending on the availability of low-level interfaces), or routed via a display server. It can even switch between the two at runtime.

                                                              Another thing that occurred to me: it might be better have apps dynamically switch protocols using feature detection rather than require build time options. I think you were saying that building a new shell might require #ifdef or something, to use a new Arcan API (I might have missed that part.)

                                                              I might have phrased that poorly, the detection is dynamic and you can switch on and off whenever you like. You can see a minimal example. The same use-/pattern is the same regardless if you’re writing a shell or some other kind of TUI.

                                                              1. 3

                                                                OK, it’s great to see that you are thinking about compatibility in both directions.

                                                                I’ve been looking for some kind of “rich terminal” for Oil. And I’ve been in contact with the authors of several other alternative shells (elvish, etc.) and I think some of them were interested as well.

                                                                Stuff like Notty crossed my radar, but Arcan never did, I guess because it’s a much bigger project, and TUI[1] is only part of it.

                                                                (I wrote a little bridge between the shell and browser called webpipe[2], and while people actually used it, the experience was limited. I agree about the bloat of the HTML/JS/CSS stack too.)

                                                                Another question: is there a binary interface stable or do you have to include “arcan_shmif_tui.h” ? While I understand your points about mixing control and data channels, that’s why I can use tmux over SSH an so forth, which is essential. And why bash works the same way in xterm or over SSH.

                                                                I view it as accepting limitations in exchange for flexibility. Is there an Arcan TUI protocol and could it go over SSH?

                                                                [1] https://github.com/andychu/webpipe

                                                                [2] https://github.com/letoram/arcan/wiki/TUI

                                                                1. 2

                                                                  A large reason for not running into Arcan is the marketing, or lack thereof. The little that’s out there has been in terms of technical overview/feature enumeration “this is what I’ve done” which falls flat when it comes to arguably more successful strategies like “X SUCKS! LOOK HERE THIS WILL BLOW YOUR MIND” or “HERE IS TRENDY TECH abUSED TO DO THE THING YOU LIKED”.

                                                                  Another is that the TUI angle, and its potential in the grand scheme of things, didn’t dawn on me until the experiences from writing the big DE (durden[1]) and the smaller Prio[2] project had settled.

                                                                  is there a binary interface stable or do you have to include “arcan_shmif_tui.h” ? While I understand your points about mixing control and data channels, that’s why I can use tmux over SSH an so forth, which is essential. And why bash works the same way in xterm or over SSH.

                                                                  The API in TUI itself is (pending 2-3 minor changes) stable to the extent that things will be added, but not changed or dropped. Better binary separation and a cleaner header will come when the points about how Arcan will be split up and a self-sufficient TUI shell can be built.

                                                                  I have a pretty good idea on how the network bits can be solved, and how that can get into a SSHable form but there are bigger things to fix first. For the same reason, I’m holding off on defining a line-protocol and sticking to just an API. I’m of the design philosophy that you first prototype, experiment and open up. It’s not until the technology itself has had enough time to expose and correct the non-obvious flaws that it’s time to fixate (define protocol) as an indicator of quality. The other way around so often morphs into either “the new boss, that looks strangely like the old boss” or worse, ‘design by committee’ monsters with stuff documented and required but not practically implemented, and, over time, an infinite stream of extensions.

                                                                  1. 2

                                                                    Well I think part of it is that I was looking for a “rich terminal”, which seems like a fairly separate concept from a display server/game engine/multimedia framework. So in that sense it’s not that surprising it didn’t turn up.

                                                                    I watched your video this afternoon, and while I still have only a cursory understanding of Arcan, I actually don’t see why the TUI component/protocol/API is part of the project. The display server part sounds exciting, so if you can cut the scope bit, it might be worth it to get that part out sooner.

                                                                    Once there’s something stable-ish published, the development could be parallelized. I think the blog post is trying to make this argument a little, but I don’t see why a third party couldn’t develop a TUI, and why there wouldn’t be multiple TUIs. You already need at least two – something like xterm, and whatever new abstraction you have in mind that integrates better with copy and paste, resize, and so forth.

                                                                    And someone might want a new TUI (“rich terminal”) without Arcan, running with X or Wayland.

                                                                    I can see that a display server and multimedia framework have overlap and can be combined. I’m not sure about the game engine part, but I don’t have much expertise there. But the TUI part seems mostly separate to me.

                                                                    I really like the ambition of the project, but trying to discard ONE entrenched protocol is already ambitious (the X protocol), but trying to discard TWO is insane (terminal protocols). And it sounds like the project might be even more than that!

                                                                    It’s a long story but I was working on the precursors to the Oil shell as far back as 2009-2010. I started out wanting to write a hybrid between a web server and a cluster manager (in retrospect a little like Mongrel 2), and then it grew to encompass Linux user space, e.g. containerized package management and distribution (pre-Docker and pre-systemd). But yeah obviously that was way too much work, even though I worked on it for about 5 years. I could never finish.

                                                                    Eventually I settled on a shell as the “central plumbing” that is realistic to replace, but I still have a desire to overhaul a much larger part of user space. I briefly worked on Rob Landley’s toybox project, and he also had an ambition to replace user space (text-only, so actually Arcan is complementary). However his ideas are also way too much work and will never be finished. (It’s disorienting/distracting to work on an open source project where someone talks about things 6 months away and 10+ years away with equal urgency).

                                                                    So I’m glad that I settled on the shell, since it’s an intermediate goal that naturally leads into the rest, and I could stop there without feeling bad. I haven’t really emphasized it enough on the blog, since I don’t want to talk about vaporware, but it’s really more of a programming language for user space. I not only want to replace Make and Awk (http://www.oilshell.org/blog/2016/11/13.html), but also config files, sed, regex, Python, and even stuff you would write in Java. Metaprogramming is a big theme.

                                                                    One way to think of it is to get rid of the “unhygienic ops sludge” in Unix, a problem that is only getting worse. It used to be awk+sed inside shell inside Make. Now it’s bash embedded in YAML, Docker files, Ruby, Python, etc.

                                                                    But if I fail at that then at least the shell should still work!

                                                                    And I should note the project that actually did manage to overhaul much of user space is System D. There’s a lot to say about that, but one thing they did right was to provide transition paths from all the crappy interfaces like init.d shell scripts, cron tabs, inetd, etc. The architecture seems confused but they did achieve many of their goals.

                                                                    1. 1

                                                                      I watched your video this afternoon, and while I still have only a cursory understanding of Arcan, I actually don’t see why the TUI component/protocol/API is part of the project. The display server part sounds exciting, so if you can cut the scope bit, it might be worth it to get that part out sooner.

                                                                      There’s really no need to scope much out, and depending on how much code-time I can squeeze out the coming weeks, I’m close to passing feature parity with Xorg. The real target left is network transparency and allowing for external window managers. What’s holding things back and has for a while is how broken the lowest layers really are, i.e. gbm/kms/egl/… and that’s why I’m entertaining myself with arguably more crazy stuff (VR, multi-vendor GPU load balancing, there’s a surfaceFlinger replacement experiment etc.). The situation is way worse than it ever was with ALSA when pulseAudio took some of the blame. Let RH and friends do the grunt work on this.

                                                                      Not only are they constantly changing the proper way to use the thing, some tools for proper synching etc. are still missing from most drivers, its incredibly easy to crash the GPU drivers and everything along with that from an unprivileged position. Don’t get me started on Wayland.

                                                                      I watched your video this afternoon, and while I still have only a cursory understanding of Arcan, I actually don’t see why the TUI component/protocol/API is part of the project.

                                                                      For the same reason systemd is kept together, it makes the project management overhead part so much more bearable when you don’t have dedicated CM resources etc. Arcan is the incubation chamber, the things that grow and mature in there gets split out into their own as time progress, and that’s what systemd seem to skip over :-). The older PoC applications were once part of the same code base until less and less changes were needed to the Lua API and so on. TUI is needed because SHMIF is too sharp of an instrument on its own, and if I lose the ability to do radical changes to SHMIF itself, the extension and versioning mess that ensues will make things too complicated to bother. That’s also why some of the specialized backends (on top of the political crap that’s connected) for compatibility with Qemu, Xorg, SDL2, Wine etc. won’t be upstreamed. A similar pattern is applied with senseye (api for sensors, translators and statistics - beneath that, shmif).

                                                                      I can see that a display server and multimedia framework have overlap and can be combined. I’m not sure about the game engine part, but I don’t have much expertise there. But the TUI part seems mostly separate to me.

                                                                      Both display server and game engine puts user quality of experience at the front and needs a load balancing sample - react - update - synch loop. They render with the same APIs and infrastructure and they fight for the same resources. A key difference, that the big players have known and leveraged forever, is that the display server is a position of power. The TUI part only works well if it does things in the same way the rest of the desktop stack operates, as soon as you deviate from one (user input, external process input, data composition) things get hairy. The argument here is that TUI is a simplification that stem from committing to working on fixed grids or lines and can avoid the state space explosion that comes when you unleash the incantation: “per pixel”.

                                                                      I really like the ambition of the project, but trying to discard ONE entrenched protocol is already ambitious (the X protocol), but trying to discard TWO is insane (terminal protocols). And it sounds like the project might be even more than that!

                                                                      Oh there’s more, but I don’t talk about stuff I’m not sure I can- or want- to deliver on. As hinted on in the blogpost though, the first problem is thinking that it’s easier solving either one separately. Unicode straight up tells you there’s not much fundamentally different between text and graphics. Swapping a ‘text’ version of nethack from a ‘graphics’ one is literally the character sets. The problem becomes unbearable and insane only if you start subdividing it up into “one protocol for audio”, “one protocol for video”, “one protocol for input”, “one protocol for console”, “one protocol for the display server”, “one protocol for desktop IPC” and so on as the tedious hurdles like authentication and privilege separation, avoiding buffer bloat or unnecessary copies, etc. has to be solved for every one. Then try and get synchronisation for the cross cutting issues and spend your life debugging asynch- race conditions.

                                                                      One way to think of it is to get rid of the “unhygienic ops sludge” in Unix, a problem that is only getting worse. It used to be awk+sed inside shell inside Make. Now it’s bash embedded in YAML, Docker files, Ruby, Python, etc. But if I fail at that then at least the shell should still work!

                                                                      Oh I understand completely ;-). The slowly irreversible end game and the corrosive ecosystem that emerges if you let temptations of feature creep have its way with you. The same failure mode for the desktop- level is ‘the browser’.

                                                            1. 3

                                                              Original UNIX was around printing terminals. Bell Labs Columbus, later Bellcore, did a virtual terminal interface, Berkeley did the termcap library, TERM, and process suspend because Western Electric postponed indefinitely.

                                                              It was a total screw-up that was never supposed to happen. One could detect the common cases and reflect this through libraries. Fully agree with andyc on the need to capture all of it in one go.

                                                              The problem is in proving all the interpreters - as a character based emulation of terminals mapped onto various existing terminal programs, they’re actually very small. Things like xterm grow a lot more due to extensive functionality beyond that of just the terminal. Such functionality might not map into all apps too well, so where to put it becomes an issue.

                                                              The architecture here reminds of datakit and the IBM 3270 terminal emulator - rather extensive. I’ve been thinking about a form of annotating apps/utilities that would enclose things like this as a general case.

                                                              In that case, the container for the process would match the terminal/other protocols and translate them into interactions with the container, mapping a library dynamically and modifying the window from IPC API calls instead of through the pty, putting the burden of the terminal emulators of needing to be rewritten and abstracting the terminal/protocol specific into discrete libraries, which might be abstracted from the termcap itself, which is also where you’d get (in the reverse direction) the way to autodetect which one it was …

                                                              1. 2

                                                                The comparison to datakit is indeed quite reasonable, and much more so for the state of the main project a few years ago (ok, 2007,2008 something) as everything did run over pipes. I tried pretty much every form of IPC possible before going with shared memory due to all the possible pitfalls but all in all, finding something that worked on OSX, Windows and Linux with the least amount of copies, OS specific code, few unique syscalls and being GPU- friendly didn’t leave many options (isolating ffmpeg- libraries was my biggest priority at that stage).

                                                                1. 1

                                                                  Termcap was a total screw-up? Or the whole terminal thing in Unix? And what does Western Electric have to do with any of this?

                                                                  1. 1

                                                                    A total screw up in that it took place at a time before open source when the communities didn’t interact intentionally. So the partially completed abstractions didn’t “mix” - the virtual terminal didn’t adapt to termcap/terminfo at a time when the two could have reinforced, so instead you didn’t get a proper layering in the OS/kernel, instead termcap/terminfo got embellished to serve ad-hoc need.

                                                                1. 4

                                                                  As an end user, I almost never have to worry about or deal with any of the problems this is trying to solve. When I do, it’s a minor inconvenience.

                                                                  What would be the benefit of switching to this new system?

                                                                  1. 1

                                                                    So this post was only really directed towards devs- that have gotten their hands dirty enough to have developed some intuition as to how limiting the current way is to work with. The next post (first one was ‘dream of’, this one was ‘dawn of’ so final one has to be ‘day of’) will be end user friendly with demos of some of the cool stuff.

                                                                    To help your imagination, you can look at the screenshots at upterm extraterm as some sort of idea of where the edge of what a cli shell can do while still having to work within the confines of terminal protocols. That level is possible with the infrastructure I have here, but at a bare minimum of dependencies.

                                                                    Also include the ability to reliably copy paste/drag and drop both text and media with the type intact from both the perspective of the shell and of client programs. Compliant clients gets access to uniform outsourced open/save of both user data and client state. They can announce their default keybindings tagged with a higher level description so the shell can provide rebinding or help. A program like midnight commander can go back and forth from having its current ‘look and feel’ to having the different panes and menubar being forwarded and managed by an outer window manager without doing anything. All clients can be detached and reattached without a screen- like wrapper and won’t shut down if the shell is accidentally closed or crashes. There’s more, but I’ll stop here :)