Important bit about composability and extensibility is that composability is extensibility on the next level. kakoune is a plug-in for unix shell/terminal, and the terminal is your extensible world.
So, you don’t “prefer composability”, you rather select an extensible layer: is your editor extensible? Your terminal? Your browser? Your kernel?
And the problem I have here is that shell/terminal combo is an engineering abomination. It gets the thing done, but 99.9 explanation for that is pure inertia, rather than a particularly good design. It makes no sense for my software to emulate hardware which was obsolete before I was born for basic programming task. Especially if parts of the emulation live in the kernel.
Curiously, this is an area which sees very little experimentation from the community: everyone is writing their terminal editor, or terminal emulator, or a shell language, building on top of XX century APIs.
Of the top of my head, I can name only two projects which think along the lines of “what if there’s something better than a terminal” and try to build platforms to build composable software on top: Emacs (which isn’t exactly XXI century either) and Arcan (for which I am still waiting when I can nix run nixpkgs#cat9https://github.com/letoram/cat9).
Had I not been so stubborn about separating durden/pipeworld/console (wm models), core-arcan (desktop server), arcan-tui (the ncurses replacement), lash (shared shell building base) and cat9 (PoC shell) as interchangable layers but rather as an integrated vertical build you could have done so by now – with the usual nixpicking when it comes to input device and gpu access. I do not nix myself though there are a handful on the discord that do (including ehrm, fully statically linked, software rendering only on m1 mac..) and could perhaps be persuaded ..
On the context of the article and Kakounes integration ambition - one arcan dev/user uses zig/arcan-tui for that, see https://github.com/cipharius/kakoune-arcan . I do not think he has completed popups / windows as discrete objects yet though.
Yeah, after writing this comment, I realized that Arcan state in nixland is better these days. I was able to install&run arcan. I couldn’t get cat9 from nixpkgs work, but cloning the repo and following “not for the faint of heart” instructions worked. However, when launching cat9 I noticed that I can’t type ~ symbol, and the key repeat was super slow (like 1 key press/second), at which point I ran out of my tinkering time quant.
Key oscillation control is up to the WM side (durden needs to toggle it on / off contextually, console does not). It is set quite low so that a jumpy keyboard that performs their own repeat events and doesn’t tag them as such (several users had that problem) still can reach the runtime config.
There is a recent patch for the keymap script that other WMs that allows offline configuration of it.
arcan_db add_appl_kv console keyrepeat n ; (n=25Hz ticks between each rising/falling event edge)
arcan_db add_appl_kv console keydelay n (n=25Hz ticks before repetition is enabled)
That’s an interesting take.
Could you expand the part about emulating obsolete hardware? What hardware is being emulated by the shell?
My understanding of the shell/terminal, is that it offers a simple and efficient interface for interacting with a computer system. It provides a text-based command line interface that allows users to execute commands, automate tasks, and manipulate data in a straightforward manner. The idea of it emulating hardware is foreign to me, so I’d like to learn more about that.
To try and offer a different explanation / mental model:
Take the ASCII table (and thus Unicode as ASCII is a subset), https://www.asciitable.com/, the predominant encoding scheme for what is often, and dangerously, referred to as “just text”. It is actually about Information Interchange – the abbreviation expanded even says as much.
Notice how a fair portion of the first half of bitpatterns do not, in fact, encode symbols to carry the written word but some rather peculiar choices, like DEL – why would a text encoding scheme have processing instructions for stripping away its own data?!.
The common use for these entries is as a variable length instruction set intended for a machine to plow through. That is what is nowadays ‘emulated’, but this exact machine hasn’t existed for a grand long time, if ever, as it is a compromise. If you would follow the instructions strictly and try to run a modern program (“text file”), the results would likely be very unpleasant unless it happens to be “just text”.
The table itself is far from complete enough to reflect the capabilities of the hardware (terminals) that were out there yet people wanted their “just text” programs to do more. Some terminals only had a humble dot matrix printer as output, others glorious vector graphics CRTs. Do you see any instructions in the table for moving a turtle around a grid, drawing lines?
So through creative use of sequences of the less relevant bitpatterns from this table, you could invent such instructions. People did. A lot of them. A few got “Standardised” as ANSI, but far from all of them and it hasn’t exactly been kept in synch with modern times. Noadays you have (several) full display server protocols worth of them. There is clipboard, program execution, synchronisation, authentication, window management, scrolling, cursor control, history control and so on. Many of them use the ESC entry as a prefix, hence the colloquial ’ESCape sequences”. All this inside humble “just text”. Here is a rough state machine just for >decoding< some of the instructions out there https://vt100.net/emu/dec_ansi_parser .
That’s just emulating the instruction part. Then we another dimension of attributes to how these instructions should be interpreted to mimic variability in hardware not captures by the instructions; not all keyboards carried luxuries like a delete key, some had funky states like “num lock”, “print screen” and other bizarre. Not all hardware could communicate with the same speed. See the manpage for ‘stty’ for how inputs should affect the generated code.
Then we have data stores with bizarre access permissions (environment variables) that are also used to pick and chose between overlays from an external capability database (that is TERMCAP/TERMINFO, why things behave differently when you set TERM=xterm vs. TERM=linux vs. TERM=doesnotexist). To this day, the often used abstraction library (n)curses has an estimator that uses your capability description to infer an instruction set, combine that with the pending drawing operations and bin against the baudrate of your virtual device to reduce the chance of partial display updates.
Then we have Interrupts and Interrupt Service Routines(!) often overlooked just like stty- as you don’t see them in STDIN/STDOUT - i.e. signals and signal handlers. SIGWINCH (Window Changed) tells you that the display magically changed properties. I am just going to leave this here: http://www.rkoucha.fr/tech_corner/sigwinch.html
This is a baseline for terminal emulation. We still haven’t gotten to the interactive shells, multiplexers or network translation (i.e. ssh).
TLDR; This is absolutely not simple. It is absolutely not efficient.
All these features were originally provided by hardware terminals, which are big boxes with keyboards and screens, without computers:
hitting Enter – i.e. you enter a line on a hardware input device, AND it’s buffered there while you type. When you hit enter, it’s sent to the actual computer !
this also predates screens – teletypes were basically “interactive printers” as far as I understand. Like you got physical paper for everything you type !!!
control codes like Ctrl-C and Ctrl-Z - send messages to the computer from the hardware terminal
ANSI color (e.g. ls --color)
ANSI sequences to go back and forth (left arrow, right arrow)
clear the screen
Also my understanding is that curses was actually extracted from Vi. These kind of apps also ran on hardware:
On the web you send HTML from the server to the browser, which formats and displays the page. Different browsers will format things slightly differently. You can choose your font.
In mainframe computing, you send ANSI escape codes to the terminal device, which formats and displays the screen. Different terminals will format things slightly differently. You can choose your font.
There is a program/database called termcap that papers over the historical differences between all the terminals manufactured by different vendors.
A big difference is that hardware terminals weren’t programmable, and software terminals today (xterm) aren’t programmable either. That is, there’s no equivalent of JavaScript for the terminal.
Also web browsers started out as software, but something like a ChromeBook makes it a little more like hardware. The Chromebook is intended to be stateless and even non-programmable, with all your apps on the server. Just like a terminal is stateless.
Most of this is the background for what you call a terminal in combination with a shell. Which is also where the original color or symbol limits stem from.
“Command Symbols” are basically just an answer to the question how you get your data together with a display-command in one stream. When all the system can do is shove bytes around and display them as they come in. From the origin of having many non-smart interfaces to one mainframe / system and multiple users and a serial connection. For example the VT100 is such an old piece of original hardware.
Then you built a ton of defacto standards which all new programs need to implement, to be actually compatible with what real programs expect, and you end up with what we have. (See also bash/sh) For example problems detecting theterminal capabilities.
Terminal emulator, the kernel, the shell, and the program running in a shell all pretend that there’s a “VT100 with more features”. Thats why we have terminal emulator / shell split, and the emulator part at all.
Of the top of my head, I can name only two projects which think along the lines of “what if there’s something better than a terminal” and try to build platforms to build composable software on top
The main reason it’s not more prominent is because I’m not “driving it” … but if you want it to happen, you should help us with it :)
I will try to publish a demo – there is actually working code on both sides, and it was recently translated to C++ (by Melvin), so I’d say the last remaining barrier is removed. (In the past, you could say Oils isn’t “production quality” because it was in Python, which was indeed slow for our use case. But now it’s pure native code.)
As mentioned in the post, Oils is different than Arcan as it doesn’t require boiling an ocean, only a large lake :) That is, ls --color, Cargo, Rust compiler, and a bazillion other utilities still work in this model.
The shell itself should not require a terminal, but most tools use some kind of terminal support, so they still need terminals.
Another comparison is the Warp terminal. The difference there is that they inject shell code to your bash and zsh, and then parse the terminal codes in order to get “blocks”. I think this probably works fine for most cases, but I’m not sure it’s correct in the presence of concurrency.
In contrast, Oils does this with terminal FD passing over Unix domain sockets, in a simple and principled way.
So I’d say that Oils is the ONLY shell that can be easily divorced from the terminal. (It will also be the only POSIX-compatible shell that also has structured JSON-like data types; Melvin is also working on translating this now, i.e. divorcing it from Python.)
The slogan is that the shell UI should HAVE a terminal, but it shouldn’t BE a terminal.
So it would be cool if someone (who knows how to program terminals and Unix domain sockets, or wants to learn) can help revive and test it against the fast native implementation of the protocol
The idea is that the GUI creates a terminal and passes through the shell to say Cargo or rustc. The shell itself does not use the terminal – it can send all its error messages and prompts and completion on a different channel, over the socket.
This gets you the “blocks” feature of Warp terminal very easily and naturally, along with shell history / completion / error messages OUTSIDE the terminal, in a GUI.
So in this way the shell is decoupled from the terminal, but the child processes can still use the terminal. Again, unlike in Arcan FE.
The same binary (afsrv_terminal) that sets up lash can be set to:
be only a terminal emulator
be only a lash runtime
wrap single commands (e.g. a legacy shell), presents a raw or vt-interpreted rendered view as well as its respective stdio with arbitrary sets of inherited other descriptors for a shell to output through.
the inherited descriptor set from 3 can includes the socket channel that arcan shmif unfolds over, bchunkhint events there comes with the same dynamic descriptor passing mechanism as in your linked.
spawn new instances of itself in any of the above configurations that hands over to the outer WM or embeds into itself.
It does look pretty awesome … The “jobs” idea looks like the Warp “blocks”, which a FANOS-based GUI would also support naturally
Personally I’m spending most of my time on the new shell language, now called YSH (And that naturally goes into self-hosted / multi-cloud distributed git-like data and polyglot computation)
And I would like some kind of GUI for that, but don’t have time to build it myself (or the expertise, really)
But Oils/YSH is now pure C++ and requires only a shell to compile, not even Make
So it should be pretty trivial to drop into any environment
Partly a backpressure problem - too many features to present already that there was no real point in going even further (about 1/4th of readers drop for each ‘section’ with a tail of ‘reads it all’ judging by the video downloads).
The non-uniform grid bits are also not developed enough to really show off, while border drawing now works as cell attributes rasterised outside the grid - there are more fancy text rendering nuances that were present when the shell process rasterised the text that had to be temporarily disabled when moving to server-side text rendering.
The interchange format (TPACK) allows for varying line properties (shaped, RTL) and other annotations. WM deferred popups, scrollbars etc. also works but better shown off in an article explicitly about the APIs rather than an example shell.
Of the top of my head, I can name only two projects which think along the lines of “what if there’s something better than a terminal” and try to build platforms to build composable software on top:
Everytime there is a new release of helix I want to try it again and every time I hit this barrier (Yes, I wish I could be using Linux at work, but I can’t)
$ echo test | hx -
Error: unable to create new application
Caused by:
Piping into helix-term is currently not supported on macOS
I pipe things into an editor all the time. Why can a super-advanced editor written in a super-advance language not support this trivial thing? I don’t get it.
The noun-verb of helix and kak seems so much better than vims verb-noun, but their “composition over extension” philosophy makes it a nonstarter for me. I extend neovim in dozens of small and large ways that make it a better editor for me. I get that most people don’t do this, and a configuration language serves them better for customization, but I really need the power of a full scripting language in my editor.
I’m kind of interested in meow, a package for Emacs that provides that noun-verb style of modal editing. The only reason I haven’t switched to using it beyond experimentation is that the provided QWERTY and Colemak bindings are very dissimilar (they preserve neither key positions nor mnemonics, or rather, some of one, some of the other). I switch between the two layouts fairly often (no way of remapping keys on the VT420, or of plugging a smart keyboard into it, for example), and two sets of meow-mode bindings is too much for me to learn right now. The idea of quickly defining selections and operating on them is great, though, and I really want to give it more attention.
I am not convinced that some 9f the advantages are actually net wins. For example, no LSP configuration. So how does it know which clangd to start? I have several versions installed on my dev machine and none of the, is both called clangd and in my default PATH. The same applies to the ‘config is not code’. For example, one of the things I have in my Vim config tells ALE to use a different clangd to the default one when I am editing files in any of the places I have clones of CHERIoT-RTOS because I want a version that knows about the extra attributes and won’t be confused by the command line flags that control this target. This is pretty common for clang-format configuration, because the output is not stable and so most projects that use it in CI require a specific version. Hopefully the no-code config can be extended with scripting?
The multi-selection feature sounds great. Faking this a little bit is why I use vim in a terminal even locally and turn off mouse mode: so mouse selections and the system clipboard don’t interfere with visual mode selections and the vim clipboard. It sounds as if Helix actually does this properly. The other examples of this looked great.
One of the reasons I stick with vim is that my muscle memory works elsewhere. For example, bash and zsh have vi modes (and even VS Code has a reasonable vim mode now). Is anyone working on a Helix mode for any shell?
There’s a workaround for this specific problem. Helix allows you to specify a language server command in the config, so having a per-project config where needed would work.
Your point stands though, the more niche your need is, the less likely it is to be supported, and there is no scripting. There are plans for a plugin interface in the future that will allow you to extend the editor without patching the source, but it’s at an early stage, and progress is slow.
I think the lack of a scripting/plugin system in these early stages of helix’s development has actually been a good thing though, as it’s pushed people to contribute to making helix support a wide array of use cases well out of the box, instead of writing a script and being done with it. My personal pet peeve right now is the lack of session history, in neovim I have a hack for getting it to behave the way I want to, but it looks like helix will soon support it in a way that doesn’t require a fragile script.
There’s a workaround for this specific problem. Helix allows you to specify a language server command in the config, so having a per-project config where needed would work.
That sounds like an exciting security hole. What happens if I put a helix config in my git repo that tells it that the LSP command is rm -rf ~ and you open a file from a git clone of my project in helix? Other editors used to have this functionality and removed it for precisely this reason.
I think the lack of a scripting/plugin system in these early stages of helix’s development has actually been a good thing though, as it’s pushed people to contribute to making helix support a wide array of use cases well out of the box, instead of writing a script and being done with it.
The flip side of this is that everyone has different requirements. You either add a lot of bloat to support features that no one will use (except the people who love the editor for having that feature), or you add an extension interface that allows everyone to implement their own bits and risk the situation where a load of people implement almost the same thing and duplicate work. Neither approach is ideal and starting with a solid core is probably the right approach early on.
I don’t know about helix, but emacs and direnv both solve the issue by prompting the user to allow the config file to be “executed”, and will do so every time it changes.
Noun-verb is much better than verb-noun. Something I’ve noticed as a Neovim user is that I tend towards visual-mode-noun-verb as a substitute for this for the most part.
What I would give my proverbial left foot for is something with this post-Vim philosophy of composability/extensibility that still played nice with the universal-ish standards (ctrl-x for copy, etc) out of the box as a first design principle? I’ve used Vim for years, poorly, because I also use a lot of other apps (e.g. Zim-wiki is my default for most everything). I guess I wish these really good configuration ideas could come without these arguably radical new ways of keyboard editing?
An advantage of using MacOS is that you use the Super key for those so the two play really well. It would be neat if I could have all my apps respect a change like that, but because of different UI toolkits on Linux and no single keyboard shortcut enforcer, this tends to not be the case.
My current middle ground is moving standard editing keys, like arrows, home, end, delete, backspace, PgUp and PgDown, to the home row of my keyboard (using QMK firmware on desktop and Kanata on laptop).
That way, I get reasonable efficient editing everywhere.
For programming, that combined with multiple cursors and syntax-aware selection gives a pretty decent code editing experience.
While I wouldn’t recommend it, i think you could bind ctrl-c/x/v to yank/cut/paste current selection to/from system clipboard in helix? (Currently space y/p - for copy paste - unsure about cut (to clipboard) - might be bound to space x?)
I use Helix as my daily driver, is awesome, super simple and the keybindings are in the beginning hard to learn if you come from vim, but you get there. Is simple, blazing fast, and no complicated plugins setup required. My config file is 5 lines. Again , extremely fast with everything I need built-in, so I can just code.
Something that I learned from suckless software is that if you are intentional about your software design where modifying the source code is the mechanism for configuration, then, for a programmer, configuration by re-compiling is around the same level of difficulty as configuration by a file / scripting language, and offers much better opportunities for integration, etc. I’d find it interesting to see a variation on Helix which had more modularity around enabling configuration through editing source code.
Note on bias, I’ve been using Kakoune full time for around 4 years now I think. I have very limited experience using Helix since getting it to work behind a firewall was a lot more work, and once I had it working, doing a key remapping was more difficult than Kakoune, and the other features weren’t love at first sight. This was at least a year ago, which is a long time at Helix development speed.
having all of those features natively integrated in the editor means that you are 100% sure that if you get the editor to start, the features will work
Since Helix has externalized the actual tree-sitter parsers, it was a moderate amount of effort to get tree-sitter integration working. Not as failure prone as my experience with early kak-lsp, but still.
Additionally, to my knowledge, Helix is exploring extensibility through wasm, so the separation of plugins and monolith is not as clear cut.
I tried out helix for a bit and immediately it felt like a much better fit than vim ever had. I’ve never used neovim but probably won’t anymore at this point anyway.
Important bit about composability and extensibility is that composability is extensibility on the next level. kakoune is a plug-in for unix shell/terminal, and the terminal is your extensible world.
So, you don’t “prefer composability”, you rather select an extensible layer: is your editor extensible? Your terminal? Your browser? Your kernel?
And the problem I have here is that shell/terminal combo is an engineering abomination. It gets the thing done, but 99.9 explanation for that is pure inertia, rather than a particularly good design. It makes no sense for my software to emulate hardware which was obsolete before I was born for basic programming task. Especially if parts of the emulation live in the kernel.
Curiously, this is an area which sees very little experimentation from the community: everyone is writing their terminal editor, or terminal emulator, or a shell language, building on top of XX century APIs.
Of the top of my head, I can name only two projects which think along the lines of “what if there’s something better than a terminal” and try to build platforms to build composable software on top: Emacs (which isn’t exactly XXI century either) and Arcan (for which I am still waiting when I can
nix run nixpkgs#cat9https://github.com/letoram/cat9).Had I not been so stubborn about separating durden/pipeworld/console (wm models), core-arcan (desktop server), arcan-tui (the ncurses replacement), lash (shared shell building base) and cat9 (PoC shell) as interchangable layers but rather as an integrated vertical build you could have done so by now – with the usual nixpicking when it comes to input device and gpu access. I do not nix myself though there are a handful on the discord that do (including ehrm, fully statically linked, software rendering only on m1 mac..) and could perhaps be persuaded ..
On the context of the article and Kakounes integration ambition - one arcan dev/user uses zig/arcan-tui for that, see https://github.com/cipharius/kakoune-arcan . I do not think he has completed popups / windows as discrete objects yet though.
Yeah, after writing this comment, I realized that Arcan state in nixland is better these days. I was able to install&run arcan. I couldn’t get cat9 from nixpkgs work, but cloning the repo and following “not for the faint of heart” instructions worked. However, when launching cat9 I noticed that I can’t type
~symbol, and the key repeat was super slow (like 1 key press/second), at which point I ran out of my tinkering time quant.Key oscillation control is up to the WM side (durden needs to toggle it on / off contextually, console does not). It is set quite low so that a jumpy keyboard that performs their own repeat events and doesn’t tag them as such (several users had that problem) still can reach the runtime config.
There is a recent patch for the keymap script that other WMs that allows offline configuration of it.
I always suspected that nixos-nix and *nix-nix was confusion waiting to happen, but I’ve never seen it in practice until now.
That’s an interesting take. Could you expand the part about emulating obsolete hardware? What hardware is being emulated by the shell? My understanding of the shell/terminal, is that it offers a simple and efficient interface for interacting with a computer system. It provides a text-based command line interface that allows users to execute commands, automate tasks, and manipulate data in a straightforward manner. The idea of it emulating hardware is foreign to me, so I’d like to learn more about that.
To try and offer a different explanation / mental model:
Take the ASCII table (and thus Unicode as ASCII is a subset), https://www.asciitable.com/, the predominant encoding scheme for what is often, and dangerously, referred to as “just text”. It is actually about Information Interchange – the abbreviation expanded even says as much.
Notice how a fair portion of the first half of bitpatterns do not, in fact, encode symbols to carry the written word but some rather peculiar choices, like DEL – why would a text encoding scheme have processing instructions for stripping away its own data?!.
The common use for these entries is as a variable length instruction set intended for a machine to plow through. That is what is nowadays ‘emulated’, but this exact machine hasn’t existed for a grand long time, if ever, as it is a compromise. If you would follow the instructions strictly and try to run a modern program (“text file”), the results would likely be very unpleasant unless it happens to be “just text”.
The table itself is far from complete enough to reflect the capabilities of the hardware (terminals) that were out there yet people wanted their “just text” programs to do more. Some terminals only had a humble dot matrix printer as output, others glorious vector graphics CRTs. Do you see any instructions in the table for moving a turtle around a grid, drawing lines?
So through creative use of sequences of the less relevant bitpatterns from this table, you could invent such instructions. People did. A lot of them. A few got “Standardised” as ANSI, but far from all of them and it hasn’t exactly been kept in synch with modern times. Noadays you have (several) full display server protocols worth of them. There is clipboard, program execution, synchronisation, authentication, window management, scrolling, cursor control, history control and so on. Many of them use the ESC entry as a prefix, hence the colloquial ’ESCape sequences”. All this inside humble “just text”. Here is a rough state machine just for >decoding< some of the instructions out there https://vt100.net/emu/dec_ansi_parser .
That’s just emulating the instruction part. Then we another dimension of attributes to how these instructions should be interpreted to mimic variability in hardware not captures by the instructions; not all keyboards carried luxuries like a delete key, some had funky states like “num lock”, “print screen” and other bizarre. Not all hardware could communicate with the same speed. See the manpage for ‘stty’ for how inputs should affect the generated code.
Then we have data stores with bizarre access permissions (environment variables) that are also used to pick and chose between overlays from an external capability database (that is TERMCAP/TERMINFO, why things behave differently when you set TERM=xterm vs. TERM=linux vs. TERM=doesnotexist). To this day, the often used abstraction library (n)curses has an estimator that uses your capability description to infer an instruction set, combine that with the pending drawing operations and bin against the baudrate of your virtual device to reduce the chance of partial display updates.
Then we have Interrupts and Interrupt Service Routines(!) often overlooked just like stty- as you don’t see them in STDIN/STDOUT - i.e. signals and signal handlers. SIGWINCH (Window Changed) tells you that the display magically changed properties. I am just going to leave this here: http://www.rkoucha.fr/tech_corner/sigwinch.html
This is a baseline for terminal emulation. We still haven’t gotten to the interactive shells, multiplexers or network translation (i.e. ssh).
TLDR; This is absolutely not simple. It is absolutely not efficient.
Here’s a good article about it - https://www.warp.dev/blog/what-happens-when-you-open-a-terminal-and-enter-ls
All these features were originally provided by hardware terminals, which are big boxes with keyboards and screens, without computers:
ls --color)Also my understanding is that curses was actually extracted from Vi. These kind of apps also ran on hardware:
https://en.wikipedia.org/wiki/Curses_(programming_library)
You can think of it as isomorphic to the web:
There is a program/database called
termcapthat papers over the historical differences between all the terminals manufactured by different vendors.A big difference is that hardware terminals weren’t programmable, and software terminals today (xterm) aren’t programmable either. That is, there’s no equivalent of JavaScript for the terminal.
Also web browsers started out as software, but something like a ChromeBook makes it a little more like hardware. The Chromebook is intended to be stateless and even non-programmable, with all your apps on the server. Just like a terminal is stateless.
I’m not as well versed in, but I can give some input:
See for example the wikipedia entry for terminal emulator or computer terminals
Most of this is the background for what you call a terminal in combination with a shell. Which is also where the original color or symbol limits stem from. “Command Symbols” are basically just an answer to the question how you get your data together with a display-command in one stream. When all the system can do is shove bytes around and display them as they come in. From the origin of having many non-smart interfaces to one mainframe / system and multiple users and a serial connection. For example the VT100 is such an old piece of original hardware.
Then you built a ton of defacto standards which all new programs need to implement, to be actually compatible with what real programs expect, and you end up with what we have. (See also bash/sh) For example problems detecting the terminal capabilities.
This bit of hardware in particular:
https://en.m.wikipedia.org/wiki/VT100
Terminal emulator, the kernel, the shell, and the program running in a shell all pretend that there’s a “VT100 with more features”. Thats why we have terminal emulator / shell split, and the emulator part at all.
https://poor.dev/blog/terminal-anatomy/ explains how all that works specifically
Helix supports the enhanced keyboard protocol, so at least there is some work happening to reform the terminal, if not replace it: https://sw.kovidgoyal.net/kitty/keyboard-protocol/
Oils too, mentioned in the last post:
https://www.oilshell.org/blog/2023/05/release-0.15.0.html#guis-and-the-headless-shell
Linking back to the genesis in 2021:
https://www.oilshell.org/blog/2021/06/hotos-shell-panel.html#oils-headless-mode-should-be-useful-for-ui-research
The main reason it’s not more prominent is because I’m not “driving it” … but if you want it to happen, you should help us with it :)
I will try to publish a demo – there is actually working code on both sides, and it was recently translated to C++ (by Melvin), so I’d say the last remaining barrier is removed. (In the past, you could say Oils isn’t “production quality” because it was in Python, which was indeed slow for our use case. But now it’s pure native code.)
As mentioned in the post, Oils is different than Arcan as it doesn’t require boiling an ocean, only a large lake :) That is,
ls --color, Cargo, Rust compiler, and a bazillion other utilities still work in this model.The shell itself should not require a terminal, but most tools use some kind of terminal support, so they still need terminals.
Another comparison is the Warp terminal. The difference there is that they inject shell code to your bash and zsh, and then parse the terminal codes in order to get “blocks”. I think this probably works fine for most cases, but I’m not sure it’s correct in the presence of concurrency.
In contrast, Oils does this with terminal FD passing over Unix domain sockets, in a simple and principled way.
So I’d say that Oils is the ONLY shell that can be easily divorced from the terminal. (It will also be the only POSIX-compatible shell that also has structured JSON-like data types; Melvin is also working on translating this now, i.e. divorcing it from Python.)
The slogan is that the shell UI should HAVE a terminal, but it shouldn’t BE a terminal.
OK the original demo screenshots are probably worth a thousands words - https://github.com/subhav/web_shell
So it would be cool if someone (who knows how to program terminals and Unix domain sockets, or wants to learn) can help revive and test it against the fast native implementation of the protocol
The Go side is here - https://github.com/subhav/web_shell/blob/master/fanos.go
Oils C++ side is here - https://github.com/oilshell/oil/blob/master/cpp/fanos_shared.c
Python too: https://github.com/oilshell/oil/blob/master/client/py_fanos.py
The idea is that the GUI creates a terminal and passes through the shell to say Cargo or rustc. The shell itself does not use the terminal – it can send all its error messages and prompts and completion on a different channel, over the socket.
This gets you the “blocks” feature of Warp terminal very easily and naturally, along with shell history / completion / error messages OUTSIDE the terminal, in a GUI.
So in this way the shell is decoupled from the terminal, but the child processes can still use the terminal. Again, unlike in Arcan FE.
Arcan covers all permutations of the concept.
The same binary (afsrv_terminal) that sets up lash can be set to:
1,3,5 had been demoed before in the dream/dawn/day and pipeworld (https://arcan-fe.com/2021/04/12/introducing-pipeworld/) presentation and was thus left out of the lash#cat9 one.
OK, I read over the most recent blog post again
https://arcan-fe.com/2022/10/15/whipping-up-a-new-shell-lashcat9/
It does look pretty awesome … The “jobs” idea looks like the Warp “blocks”, which a FANOS-based GUI would also support naturally
Personally I’m spending most of my time on the new shell language, now called YSH (And that naturally goes into self-hosted / multi-cloud distributed git-like data and polyglot computation)
And I would like some kind of GUI for that, but don’t have time to build it myself (or the expertise, really)
But Oils/YSH is now pure C++ and requires only a shell to compile, not even Make
So it should be pretty trivial to drop into any environment
Another thought I had is that the demos still look grid-based and terminal-based to me, i.e. more like tmux than a Window manager
It’s probably easy for people to gloss over the difference when presented like that
I’m thinking more like something with GUI history like browser history, and then a GUI prompt like the address bar, GUI auto-complete, etc.
I feel like that would open people up the idea a bit more
Partly a backpressure problem - too many features to present already that there was no real point in going even further (about 1/4th of readers drop for each ‘section’ with a tail of ‘reads it all’ judging by the video downloads).
The non-uniform grid bits are also not developed enough to really show off, while border drawing now works as cell attributes rasterised outside the grid - there are more fancy text rendering nuances that were present when the shell process rasterised the text that had to be temporarily disabled when moving to server-side text rendering.
The interchange format (TPACK) allows for varying line properties (shaped, RTL) and other annotations. WM deferred popups, scrollbars etc. also works but better shown off in an article explicitly about the APIs rather than an example shell.
What about ACME from plan9 (originally)?
http://acme.cat-v.org/
Personally RSI rules out mouse (and heavy trackpad) use - but it should fit the description of doing something new vis-a-vis TTYs?
Maybe worth mentioning Smalltalk (in general) - and Cuis/Pharo as open source IDEs/editing environments too. And possibly (but maybe not) Racket.
Everytime there is a new release of helix I want to try it again and every time I hit this barrier (Yes, I wish I could be using Linux at work, but I can’t)
I pipe things into an editor all the time. Why can a super-advanced editor written in a super-advance language not support this trivial thing? I don’t get it.
Does it work on Linux? If so I wonder what weird macOS specific issue is blocking it.
The problem was in a library they use. The library was fixed but the PR to use that library version is not getting merged https://github.com/helix-editor/helix/pull/5468
The noun-verb of helix and kak seems so much better than vims verb-noun, but their “composition over extension” philosophy makes it a nonstarter for me. I extend neovim in dozens of small and large ways that make it a better editor for me. I get that most people don’t do this, and a configuration language serves them better for customization, but I really need the power of a full scripting language in my editor.
I’m kind of interested in
meow, a package for Emacs that provides that noun-verb style of modal editing. The only reason I haven’t switched to using it beyond experimentation is that the provided QWERTY and Colemak bindings are very dissimilar (they preserve neither key positions nor mnemonics, or rather, some of one, some of the other). I switch between the two layouts fairly often (no way of remapping keys on the VT420, or of plugging a smart keyboard into it, for example), and two sets of meow-mode bindings is too much for me to learn right now. The idea of quickly defining selections and operating on them is great, though, and I really want to give it more attention.I am not convinced that some 9f the advantages are actually net wins. For example, no LSP configuration. So how does it know which clangd to start? I have several versions installed on my dev machine and none of the, is both called clangd and in my default PATH. The same applies to the ‘config is not code’. For example, one of the things I have in my Vim config tells ALE to use a different clangd to the default one when I am editing files in any of the places I have clones of CHERIoT-RTOS because I want a version that knows about the extra attributes and won’t be confused by the command line flags that control this target. This is pretty common for clang-format configuration, because the output is not stable and so most projects that use it in CI require a specific version. Hopefully the no-code config can be extended with scripting?
The multi-selection feature sounds great. Faking this a little bit is why I use vim in a terminal even locally and turn off mouse mode: so mouse selections and the system clipboard don’t interfere with visual mode selections and the vim clipboard. It sounds as if Helix actually does this properly. The other examples of this looked great.
One of the reasons I stick with vim is that my muscle memory works elsewhere. For example, bash and zsh have vi modes (and even VS Code has a reasonable vim mode now). Is anyone working on a Helix mode for any shell?
There’s a workaround for this specific problem. Helix allows you to specify a language server command in the config, so having a per-project config where needed would work.
Your point stands though, the more niche your need is, the less likely it is to be supported, and there is no scripting. There are plans for a plugin interface in the future that will allow you to extend the editor without patching the source, but it’s at an early stage, and progress is slow.
I think the lack of a scripting/plugin system in these early stages of helix’s development has actually been a good thing though, as it’s pushed people to contribute to making helix support a wide array of use cases well out of the box, instead of writing a script and being done with it. My personal pet peeve right now is the lack of session history, in neovim I have a hack for getting it to behave the way I want to, but it looks like helix will soon support it in a way that doesn’t require a fragile script.
That sounds like an exciting security hole. What happens if I put a helix config in my git repo that tells it that the LSP command is
rm -rf ~and you open a file from a git clone of my project in helix? Other editors used to have this functionality and removed it for precisely this reason.The flip side of this is that everyone has different requirements. You either add a lot of bloat to support features that no one will use (except the people who love the editor for having that feature), or you add an extension interface that allows everyone to implement their own bits and risk the situation where a load of people implement almost the same thing and duplicate work. Neither approach is ideal and starting with a solid core is probably the right approach early on.
I don’t know about helix, but emacs and direnv both solve the issue by prompting the user to allow the config file to be “executed”, and will do so every time it changes.
Neovim does this now as well.
Yes, I’m afraid it might very well be? I can’t find any mention of mitigation at a glance?
https://github.com/helix-editor/helix/pull/1249
Noun-verb is much better than verb-noun. Something I’ve noticed as a Neovim user is that I tend towards visual-mode-noun-verb as a substitute for this for the most part.
What I would give my proverbial left foot for is something with this post-Vim philosophy of composability/extensibility that still played nice with the universal-ish standards (ctrl-x for copy, etc) out of the box as a first design principle? I’ve used Vim for years, poorly, because I also use a lot of other apps (e.g. Zim-wiki is my default for most everything). I guess I wish these really good configuration ideas could come without these arguably radical new ways of keyboard editing?
I am not sure what you are after. Do you want a modal editor with Windows-Style keyboard short cuts or what is it that you are looking for?
An advantage of using MacOS is that you use the Super key for those so the two play really well. It would be neat if I could have all my apps respect a change like that, but because of different UI toolkits on Linux and no single keyboard shortcut enforcer, this tends to not be the case.
I bet there’s an XDG standard on this somewhere.
My current middle ground is moving standard editing keys, like arrows, home, end, delete, backspace, PgUp and PgDown, to the home row of my keyboard (using QMK firmware on desktop and Kanata on laptop).
That way, I get reasonable efficient editing everywhere.
For programming, that combined with multiple cursors and syntax-aware selection gives a pretty decent code editing experience.
While I wouldn’t recommend it, i think you could bind ctrl-c/x/v to yank/cut/paste current selection to/from system clipboard in helix? (Currently space y/p - for copy paste - unsure about cut (to clipboard) - might be bound to space x?)
I use Helix as my daily driver, is awesome, super simple and the keybindings are in the beginning hard to learn if you come from vim, but you get there. Is simple, blazing fast, and no complicated plugins setup required. My config file is 5 lines. Again , extremely fast with everything I need built-in, so I can just code.
Something that I learned from suckless software is that if you are intentional about your software design where modifying the source code is the mechanism for configuration, then, for a programmer, configuration by re-compiling is around the same level of difficulty as configuration by a file / scripting language, and offers much better opportunities for integration, etc. I’d find it interesting to see a variation on Helix which had more modularity around enabling configuration through editing source code.
Note on bias, I’ve been using Kakoune full time for around 4 years now I think. I have very limited experience using Helix since getting it to work behind a firewall was a lot more work, and once I had it working, doing a key remapping was more difficult than Kakoune, and the other features weren’t love at first sight. This was at least a year ago, which is a long time at Helix development speed.
Additionally, to my knowledge, Helix is exploring extensibility through wasm, so the separation of plugins and monolith is not as clear cut.
I tried out helix for a bit and immediately it felt like a much better fit than vim ever had. I’ve never used neovim but probably won’t anymore at this point anyway.