1. 45

The article covers the design and status of an ongoing attempt at getting rid of terminal protocols and their emulators as part of improving the CLI/TUI experience. [resubmitting, deleted story had the wrong url]

  1.  

  2. 17

    This one has been gathering dust in the ‘drafts’ bin for a bit too long, waiting for the opportunity to sneak it in somewhere without too many people noticing. Taking the ongoing comments in the ‘TERM is terminally broken’ post into account, I got the feeling that there might be a few in here that would become slightly provoked or at least mildly interested in the idea enough to give it some friendly pokes.

    1. 12

      I really like a lot of these ideas. I didn’t quite get the compatibility story though:

      alternative platform output target that undoes all this work and outputs into a VT100 compliant mess again – all for the sake of backwards compatibility

      Do bash, readline, and ls --color work inside your new system?

      If you want to kill something you have to reimplement it. That’s how bash killed almost all previous shells – it subsumed all their features and ran scripts targeted for them.

      That is pretty much the reasoning behind OSH in Oil (http://www.oilshell.org/). I mean part of it is that I don’t want to implement something worse than bash, so I have to understand it, and implementing it is how you understand it. I learned many things that made the Oil language better, regardless of whether it was compatible with bash.

      And the other part is that running bash scripts and auto-converting them to Oil is a pretty reliable way to get rid of bash. But it’s obviously a huge amount of work.

      I don’t know how big terminal emulators are, like xterm, or I use ROXTerm. I would guess 100K lines of code?
      There are definitely a few minimal ones that are 10K or so.

      So anyway, I think if you want to kill terminal protocols, which I think is a great idea, you have to implement them. That’s how Microsoft Excel killed all competitors too – it just opened all their file formats.

      Another thing that occurred to me: it might be better have apps dynamically switch protocols using feature detection rather than require build time options. I think you were saying that building a new shell might require #ifdef or something, to use a new Arcan API (I might have missed that part.)

      But I think being able to dynamically switch would provide a better compatibility story. I could just type “bash” or “oil” in both the old context and new context, and the binary is the same, and the same config files are read, and my user experience is kinda similar. At least for awhile.

      1. 2

        The quoted part of the compatibility story is for the other direction, running legacy things has been in place for many years. Say, for instance, that someone likes the API and uses it instead of curses to build their fancy new TUI for communicating via Matrix or managing files. The problem is that any users would be forced to run a terminal that speaks this API right? forcing people to give up their old ways like that isn’t very nice, and that’s where the compatibility layer would possibly become handy, though there are possibly some other interesting parts of shmif- that can be leveraged to avoid even that.

        Do bash, readline, and ls –color work inside your new system?

        Yes, the terminal emulator that’s in there is very much functional, with most of the normal stuff (bracketed paste, mouse, unicode, 24bit color, …). The shell itself is would be responsible for starting something in compatibility mode.

        That is pretty much the reasoning behind OSH in Oil (http://www.oilshell.org/). I mean part of it is that I don’t want to implement something worse than bash, so I have to understand it, and implementing it is how you understand it. I learned many things that made the Oil language better, regardless of whether it was compatible with bash.

        Oil looks highly interesting, I’ll be sure to remember that one when I evacuate bash from my systems :-)

        I don’t know how big terminal emulators are, like xterm, or I use ROXTerm. I would guess 100K lines of code? There are definitely a few minimal ones that are 10K or so.

        Xterm is something like 70kloc, everything built using libVTE adds at least 95kloc, ‘st’ has a decent amount of features and is in the 2k range I believe. But they are all also specialized, i.e. xterm needs an xserver, same with ‘st’, libVTE needs GTK (if I remember correctly) and so on. The solution I advocate here can work both in place of a kernel provided one (depending on the availability of low-level interfaces), or routed via a display server. It can even switch between the two at runtime.

        Another thing that occurred to me: it might be better have apps dynamically switch protocols using feature detection rather than require build time options. I think you were saying that building a new shell might require #ifdef or something, to use a new Arcan API (I might have missed that part.)

        I might have phrased that poorly, the detection is dynamic and you can switch on and off whenever you like. You can see a minimal example. The same use-/pattern is the same regardless if you’re writing a shell or some other kind of TUI.

        1. 3

          OK, it’s great to see that you are thinking about compatibility in both directions.

          I’ve been looking for some kind of “rich terminal” for Oil. And I’ve been in contact with the authors of several other alternative shells (elvish, etc.) and I think some of them were interested as well.

          Stuff like Notty crossed my radar, but Arcan never did, I guess because it’s a much bigger project, and TUI[1] is only part of it.

          (I wrote a little bridge between the shell and browser called webpipe[2], and while people actually used it, the experience was limited. I agree about the bloat of the HTML/JS/CSS stack too.)

          Another question: is there a binary interface stable or do you have to include “arcan_shmif_tui.h” ? While I understand your points about mixing control and data channels, that’s why I can use tmux over SSH an so forth, which is essential. And why bash works the same way in xterm or over SSH.

          I view it as accepting limitations in exchange for flexibility. Is there an Arcan TUI protocol and could it go over SSH?

          [1] https://github.com/andychu/webpipe

          [2] https://github.com/letoram/arcan/wiki/TUI

          1. 2

            A large reason for not running into Arcan is the marketing, or lack thereof. The little that’s out there has been in terms of technical overview/feature enumeration “this is what I’ve done” which falls flat when it comes to arguably more successful strategies like “X SUCKS! LOOK HERE THIS WILL BLOW YOUR MIND” or “HERE IS TRENDY TECH abUSED TO DO THE THING YOU LIKED”.

            Another is that the TUI angle, and its potential in the grand scheme of things, didn’t dawn on me until the experiences from writing the big DE (durden[1]) and the smaller Prio[2] project had settled.

            is there a binary interface stable or do you have to include “arcan_shmif_tui.h” ? While I understand your points about mixing control and data channels, that’s why I can use tmux over SSH an so forth, which is essential. And why bash works the same way in xterm or over SSH.

            The API in TUI itself is (pending 2-3 minor changes) stable to the extent that things will be added, but not changed or dropped. Better binary separation and a cleaner header will come when the points about how Arcan will be split up and a self-sufficient TUI shell can be built.

            I have a pretty good idea on how the network bits can be solved, and how that can get into a SSHable form but there are bigger things to fix first. For the same reason, I’m holding off on defining a line-protocol and sticking to just an API. I’m of the design philosophy that you first prototype, experiment and open up. It’s not until the technology itself has had enough time to expose and correct the non-obvious flaws that it’s time to fixate (define protocol) as an indicator of quality. The other way around so often morphs into either “the new boss, that looks strangely like the old boss” or worse, ‘design by committee’ monsters with stuff documented and required but not practically implemented, and, over time, an infinite stream of extensions.

            1. 2

              Well I think part of it is that I was looking for a “rich terminal”, which seems like a fairly separate concept from a display server/game engine/multimedia framework. So in that sense it’s not that surprising it didn’t turn up.

              I watched your video this afternoon, and while I still have only a cursory understanding of Arcan, I actually don’t see why the TUI component/protocol/API is part of the project. The display server part sounds exciting, so if you can cut the scope bit, it might be worth it to get that part out sooner.

              Once there’s something stable-ish published, the development could be parallelized. I think the blog post is trying to make this argument a little, but I don’t see why a third party couldn’t develop a TUI, and why there wouldn’t be multiple TUIs. You already need at least two – something like xterm, and whatever new abstraction you have in mind that integrates better with copy and paste, resize, and so forth.

              And someone might want a new TUI (“rich terminal”) without Arcan, running with X or Wayland.

              I can see that a display server and multimedia framework have overlap and can be combined. I’m not sure about the game engine part, but I don’t have much expertise there. But the TUI part seems mostly separate to me.

              I really like the ambition of the project, but trying to discard ONE entrenched protocol is already ambitious (the X protocol), but trying to discard TWO is insane (terminal protocols). And it sounds like the project might be even more than that!

              It’s a long story but I was working on the precursors to the Oil shell as far back as 2009-2010. I started out wanting to write a hybrid between a web server and a cluster manager (in retrospect a little like Mongrel 2), and then it grew to encompass Linux user space, e.g. containerized package management and distribution (pre-Docker and pre-systemd). But yeah obviously that was way too much work, even though I worked on it for about 5 years. I could never finish.

              Eventually I settled on a shell as the “central plumbing” that is realistic to replace, but I still have a desire to overhaul a much larger part of user space. I briefly worked on Rob Landley’s toybox project, and he also had an ambition to replace user space (text-only, so actually Arcan is complementary). However his ideas are also way too much work and will never be finished. (It’s disorienting/distracting to work on an open source project where someone talks about things 6 months away and 10+ years away with equal urgency).

              So I’m glad that I settled on the shell, since it’s an intermediate goal that naturally leads into the rest, and I could stop there without feeling bad. I haven’t really emphasized it enough on the blog, since I don’t want to talk about vaporware, but it’s really more of a programming language for user space. I not only want to replace Make and Awk (http://www.oilshell.org/blog/2016/11/13.html), but also config files, sed, regex, Python, and even stuff you would write in Java. Metaprogramming is a big theme.

              One way to think of it is to get rid of the “unhygienic ops sludge” in Unix, a problem that is only getting worse. It used to be awk+sed inside shell inside Make. Now it’s bash embedded in YAML, Docker files, Ruby, Python, etc.

              But if I fail at that then at least the shell should still work!

              And I should note the project that actually did manage to overhaul much of user space is System D. There’s a lot to say about that, but one thing they did right was to provide transition paths from all the crappy interfaces like init.d shell scripts, cron tabs, inetd, etc. The architecture seems confused but they did achieve many of their goals.

              1. 1

                I watched your video this afternoon, and while I still have only a cursory understanding of Arcan, I actually don’t see why the TUI component/protocol/API is part of the project. The display server part sounds exciting, so if you can cut the scope bit, it might be worth it to get that part out sooner.

                There’s really no need to scope much out, and depending on how much code-time I can squeeze out the coming weeks, I’m close to passing feature parity with Xorg. The real target left is network transparency and allowing for external window managers. What’s holding things back and has for a while is how broken the lowest layers really are, i.e. gbm/kms/egl/… and that’s why I’m entertaining myself with arguably more crazy stuff (VR, multi-vendor GPU load balancing, there’s a surfaceFlinger replacement experiment etc.). The situation is way worse than it ever was with ALSA when pulseAudio took some of the blame. Let RH and friends do the grunt work on this.

                Not only are they constantly changing the proper way to use the thing, some tools for proper synching etc. are still missing from most drivers, its incredibly easy to crash the GPU drivers and everything along with that from an unprivileged position. Don’t get me started on Wayland.

                I watched your video this afternoon, and while I still have only a cursory understanding of Arcan, I actually don’t see why the TUI component/protocol/API is part of the project.

                For the same reason systemd is kept together, it makes the project management overhead part so much more bearable when you don’t have dedicated CM resources etc. Arcan is the incubation chamber, the things that grow and mature in there gets split out into their own as time progress, and that’s what systemd seem to skip over :-). The older PoC applications were once part of the same code base until less and less changes were needed to the Lua API and so on. TUI is needed because SHMIF is too sharp of an instrument on its own, and if I lose the ability to do radical changes to SHMIF itself, the extension and versioning mess that ensues will make things too complicated to bother. That’s also why some of the specialized backends (on top of the political crap that’s connected) for compatibility with Qemu, Xorg, SDL2, Wine etc. won’t be upstreamed. A similar pattern is applied with senseye (api for sensors, translators and statistics - beneath that, shmif).

                I can see that a display server and multimedia framework have overlap and can be combined. I’m not sure about the game engine part, but I don’t have much expertise there. But the TUI part seems mostly separate to me.

                Both display server and game engine puts user quality of experience at the front and needs a load balancing sample - react - update - synch loop. They render with the same APIs and infrastructure and they fight for the same resources. A key difference, that the big players have known and leveraged forever, is that the display server is a position of power. The TUI part only works well if it does things in the same way the rest of the desktop stack operates, as soon as you deviate from one (user input, external process input, data composition) things get hairy. The argument here is that TUI is a simplification that stem from committing to working on fixed grids or lines and can avoid the state space explosion that comes when you unleash the incantation: “per pixel”.

                I really like the ambition of the project, but trying to discard ONE entrenched protocol is already ambitious (the X protocol), but trying to discard TWO is insane (terminal protocols). And it sounds like the project might be even more than that!

                Oh there’s more, but I don’t talk about stuff I’m not sure I can- or want- to deliver on. As hinted on in the blogpost though, the first problem is thinking that it’s easier solving either one separately. Unicode straight up tells you there’s not much fundamentally different between text and graphics. Swapping a ‘text’ version of nethack from a ‘graphics’ one is literally the character sets. The problem becomes unbearable and insane only if you start subdividing it up into “one protocol for audio”, “one protocol for video”, “one protocol for input”, “one protocol for console”, “one protocol for the display server”, “one protocol for desktop IPC” and so on as the tedious hurdles like authentication and privilege separation, avoiding buffer bloat or unnecessary copies, etc. has to be solved for every one. Then try and get synchronisation for the cross cutting issues and spend your life debugging asynch- race conditions.

                One way to think of it is to get rid of the “unhygienic ops sludge” in Unix, a problem that is only getting worse. It used to be awk+sed inside shell inside Make. Now it’s bash embedded in YAML, Docker files, Ruby, Python, etc. But if I fail at that then at least the shell should still work!

                Oh I understand completely ;-). The slowly irreversible end game and the corrosive ecosystem that emerges if you let temptations of feature creep have its way with you. The same failure mode for the desktop- level is ‘the browser’.

      2. 5

        I’m excited about this.

        It’s aimed at games, but have you looked at the design of the Glk API? I think the selection of what functionality to include and what to exclude is relevant prior art. It’s trying to create a text-centric experience which allows interesting formatting and some modest use of images and sounds, without replicating the full functionality of X11.

        1. 4

          I have not, this is great - huge thanks <3. Now I’m more than a little ashamed for overlooking the IF community when doing the initial research many many years ago, it fits so well it should’ve been obvious. I’ll add glkterm in the testing-pile for the curses- wrapper then maybe try a port.

          1. 1

            Very welcome. :)

        2. 4

          As an end user, I almost never have to worry about or deal with any of the problems this is trying to solve. When I do, it’s a minor inconvenience.

          What would be the benefit of switching to this new system?

          1. 1

            So this post was only really directed towards devs- that have gotten their hands dirty enough to have developed some intuition as to how limiting the current way is to work with. The next post (first one was ‘dream of’, this one was ‘dawn of’ so final one has to be ‘day of’) will be end user friendly with demos of some of the cool stuff.

            To help your imagination, you can look at the screenshots at upterm extraterm as some sort of idea of where the edge of what a cli shell can do while still having to work within the confines of terminal protocols. That level is possible with the infrastructure I have here, but at a bare minimum of dependencies.

            Also include the ability to reliably copy paste/drag and drop both text and media with the type intact from both the perspective of the shell and of client programs. Compliant clients gets access to uniform outsourced open/save of both user data and client state. They can announce their default keybindings tagged with a higher level description so the shell can provide rebinding or help. A program like midnight commander can go back and forth from having its current ‘look and feel’ to having the different panes and menubar being forwarded and managed by an outer window manager without doing anything. All clients can be detached and reattached without a screen- like wrapper and won’t shut down if the shell is accidentally closed or crashes. There’s more, but I’ll stop here :)

          2. 3

            Original UNIX was around printing terminals. Bell Labs Columbus, later Bellcore, did a virtual terminal interface, Berkeley did the termcap library, TERM, and process suspend because Western Electric postponed indefinitely.

            It was a total screw-up that was never supposed to happen. One could detect the common cases and reflect this through libraries. Fully agree with andyc on the need to capture all of it in one go.

            The problem is in proving all the interpreters - as a character based emulation of terminals mapped onto various existing terminal programs, they’re actually very small. Things like xterm grow a lot more due to extensive functionality beyond that of just the terminal. Such functionality might not map into all apps too well, so where to put it becomes an issue.

            The architecture here reminds of datakit and the IBM 3270 terminal emulator - rather extensive. I’ve been thinking about a form of annotating apps/utilities that would enclose things like this as a general case.

            In that case, the container for the process would match the terminal/other protocols and translate them into interactions with the container, mapping a library dynamically and modifying the window from IPC API calls instead of through the pty, putting the burden of the terminal emulators of needing to be rewritten and abstracting the terminal/protocol specific into discrete libraries, which might be abstracted from the termcap itself, which is also where you’d get (in the reverse direction) the way to autodetect which one it was …

            1. 2

              The comparison to datakit is indeed quite reasonable, and much more so for the state of the main project a few years ago (ok, 2007,2008 something) as everything did run over pipes. I tried pretty much every form of IPC possible before going with shared memory due to all the possible pitfalls but all in all, finding something that worked on OSX, Windows and Linux with the least amount of copies, OS specific code, few unique syscalls and being GPU- friendly didn’t leave many options (isolating ffmpeg- libraries was my biggest priority at that stage).

              1. 1

                Termcap was a total screw-up? Or the whole terminal thing in Unix? And what does Western Electric have to do with any of this?

                1. 1

                  A total screw up in that it took place at a time before open source when the communities didn’t interact intentionally. So the partially completed abstractions didn’t “mix” - the virtual terminal didn’t adapt to termcap/terminfo at a time when the two could have reinforced, so instead you didn’t get a proper layering in the OS/kernel, instead termcap/terminfo got embellished to serve ad-hoc need.

              2. 3

                There is arguably a rift here between those that prefer the ‘shove it in a browser’ or flashy UIs that animate and morph as you interact, and those that prefer staring into a text editor.

                I have been thinking about starting a project similar to this, but I cannot decide between those two approaches. I love and use my terminal a lot, but once you go down the multimedia path and display images, you soon want to play audio, show videos, render tables, etc. The easiest way to achieve this is to use a browser. At least a browser widget.

                The other aspect is security. If we tackle this, we should as well replace ssh and suddenly security is insanely important.

                1. 1

                  If you go with the browser you’ve already surrendered every other tangible gain in favour of “easiness” (and especially security, it’s infinite attack surface already), you might just write your shell as a plugin and live there.

                  If you take the time to study the underlying system in a bit more detail here, you’ll find that both audio and video content is actually there, but as an extra and not a necessity.

                  I don’t think SSH would need to be replaced as such, recall how it is used to tunnel X, even though there are some other benefits from doing so - and the task is somewhat simpler these days with so many good crypto primitives around.

                2. 1

                  The purpose is to use cli without terminal? Looks like it is a Rube Goldberg machine

                  1. 1

                    The point is that the terminal protocols are a Storm P machine and CLIs should integrate with the toolkits or display server protocols that are in use and not go through a ‘tower of babel’- like translation layer like VtXXX. Lots of OSes us CLIs without a terminal emulator and are much better off because of that.

                    1. 1

                      Do you mean all curses program will not work in your system? Which os has not terminal emulator and much better than unixes?

                      1. 1

                        Curses can be implemented 1:1 over the existing API, so that’s just a matter of writing the mapper, and the current implementation has a terminal emulator that can be switched on for legacy support.

                        Terminal Emulator != Command Line/Shell or the tools you use from it. The argument is against the Protocol (how all these tools communicate, the ESC, OSC, … stuff) and how it makes it impossible to write something better than what you can do with curses right now.

                        Some non-terminal emulator based CLIs: Plan9? Windows(PowerShell)? TempleOS? or OS independent ones like Notty. I assume some language problems here, but “better off” != “much better than” - only that they have the potential to be.