1. 19

I’ve never used the mouse with the terminal, but I’m not sure why…

I was just using GDB under -tui mode (text user interface) and thinking it could be improved by clicking.

Normally I’m a keyboard/shell person, but debugging is one of those cases where I feel a mouse could really help. There is a lot of state – drilling through an object graph, setting breakpoints and watch points, moving through the call stack, etc.

(This was after using Eclipse, and Eclipse crashed…)


So then I did some googling, and found zsh has some mouse support! This works!

https://unix.stackexchange.com/questions/444601/any-terminal-shell-with-mouse-support

wget http://stchaz.free.fr/mouse.zsh
. ./mouse.zsh
zle-toggle-mouse

More testing:

  • This works over ssh!
  • This works through tmux without any config changes

Does emacs have mouse support through a terminal? Do you use it? I’m a vim user, but I would expect emacs to be the terminal program with support, if any.

Ah, now I see that set mouse=a works in vim! Although I’m not sure if I would ever use this. But Iw ould use it if GDB supported it.

https://www.jacoballred.com/web-dev/enabling-mouse-support-in-vim/

Anyway I’m interested in any experience with the mouse over a terminal.

  1.  

  2. 10

    I’ve never used the mouse with the terminal, but I’m not sure why

    I don’t use it either, but I do know why: I like having the default copy/paste text behaviour work anywhere in terminals. For example, the other day I had to copy a PID from htop on a remote machine, and the mouse support prevented that. It’s like web pages where you can’t select and copy text.

    You can reimplement this, but a lot of apps don’t, or have different behaviour. In general, I find it more annoying than helpful.

    1. 11

      At least in urxvt and xterm Shift+Mouse allows you to select text even if the application tries to react the the mouse events.

      1. 4

        Ah, this works in st as well; I didn’t know this, thanks!

        1. 3

          In konsole, all gtk teminals and mintty this works as well. I would be surprised to find a terminal emulator that did not support this.

          1. 1

            At least in urxvt and xterm Shift+Mouse allows you to select text even if the application tries to react the the mouse events.

            Is it possible to invert this behavior? I would prefer if the mouse worked normally, except when shifted, then it can be captured by the application.

          2. 2

            I wonder if you can disable it on a per-operation basis? In iTerm for example, if you alt-drag, it doesn’t send anything to the program, it just highlights the displayed text.

            1. 1

              Yeah, I use option-drag on iTerm to use the terminal native selection when in e.g. vim with mouse mode on.

              Also I’d highly recommend knowing about using option-cmd-drag — it will let you do block selection of text.

              This can be very useful when you’re in e.g. tmux or vim with split windows and you only want to select a block of text not whole sets of “lines” in the terminal if that makes sense.

            2. 1

              Yes that makes sense, and is something I didn’t quite get. The author of ble.sh [1] brought that up on the Oil zulip. ble.sh contains a terminal parser in pure bash and so it could also handle the mouse :)

              [1] http://www.oilshell.org/blog/2020/03/release-0.8.pre3.html#serendipity-with-blesh

            3. 5

              I’ve written some games that use the VT220 mouse escape sequences: minesviiper and solvitaire (they also have vi-bindings, hence the names), so I think I have some experience.

              The problem I see with using the mouse in a TUI is that there is no feedback from hovering, meaning what is clickable isn’t always obvious. The only place I sometimes use it is in ncmpcpp(1): you can right-click tracks to play, or click the “playing”/“paused” text (that doesn’t look clickable) to toggle playback. Also, not everything is accessible through the mouse (in ncmpcpp, or my games), so you still need to constantly reach for the arrow keys or other hotkeys.

              For me at least, good vi-style keybindings are making me way more productive.

              1. 1

                Terminals that support reporting click events generally also support drag events and hover events, if you ask for them. On the other hand, ncurses only supports click events, so that’s all you get unless you want to roll your own input handling code.

                1. 1

                  yeah, doesn’t sound like fun doing bespoke hitbox-detection on every mouse move, since there’s no curses support for it. :( given that you can move your mouse cursor over many cells per second, a “differential drawing mode” (like ncurses does) is probably wise, too. and that requires you to design your output routines carefully (i remember it being a pain to retrofit to minesviiper).

              2. 5

                On a long command line, I click to position the cursor. It feels faster than repeating alt-b, especially when SSH lags. (In my case this is an iTerm feature: alt-click sends the appropriate number of arrow key presses.)

                1. 4

                  Does touch count?

                  My primary (personal) workstation is an iPad running blinksh. I mosh into my development boxes and work inside of tmux. I generally use my finger to scroll through terminal history when I’m working like that.

                  1. 2

                    Woah! I just tried tmux and does it bind “scroll wheel” to “up” and “down”?

                    Because outside of tmux, the scroll wheel does nothing. Inside of tmux, I get bash history. And it also works in vim.

                    That is interesting. I’ve been using tmux for like a decade, and did not know that.

                    Thanks for the info! That absolutely counts.

                    1. 2

                      Scrollback behavior depends on the terminal you use. Gnome terminal allows you to use the scroll wheel to access the scrollback, where as others like st expect you to use something like tmux as your scrollback buffer. Many also implement shift-pageup/down to navigate scrollback.

                  2. 4

                    For the opposite

                    :set mouse=

                    It should be the default, as it is irritating to try to select text to paste elsewhere and have vim switch modes just because of that.

                    1. 2

                      I remember Debian’s vim moved to mouse-by-default, but did the upstream default change too?

                      1. 2

                        Not sure if it’s from upstream, but the FreeBSD package also has mouse-by-default and I don’t think it matters. It drives me insane because I always use mouse-copy for copying between terminals and visual mode keyboard copy for copying within a vim session and so I end up having to remember to stick set mouse= in my .vimrc. Maybe soon it will motivate me to put my .vimrc in git somewhere…

                        1. 1

                          Arch’s is mouse by default, which annoys me. If the upstream is not, that’d be a bug: It’d go against Arch philosophy of not patching packages unless strictly necessary.

                      2. 3

                        I use Vim’s mouse support and Tmux’s mouse support, mostly for switching between panes and splits more easily. That being said, in a TUI, I’ve found it’s pretty important to have ways to turn the mouse suppose off and on, as when it’s turn on in a remote session (Like putty or ssh), having it turned on can gunk up sending keystrokes via paste, at least on Windows and ChromeOS.

                        1. 1

                          Hm, interesting I was able to resize a vim split with the mouse outside tmux, but not inside. I’ve never done that before, but resizing splits is one thing I don’t like doing with the keyboard! (but have for years)

                          I guess this is one reason people don’t use terminal mouse support more – it is no “zero configuration”.

                        2. 3

                          My IRC client of choice is weechat, and I find mouse support for switching between chat there very useful.

                          1. 3

                            https://fx.wtf uses mouse to open/close json parts.

                            1. 2

                              Emacs does have mouse support available in a terminal. I don’t usually use it. Generally the behavior is too surprising. I have come to expect gnome-terminal text highlighting to be distinct from emacs selections. Also if I copy/paste something with my mouse (like a link) I usually don’t want emacs involved.

                              1. 2

                                Looks like Emacs has an xterm-mouse-mode you can turn on. In Jove it defaults to on - I usually turn it off (set xt-mouse off), because otherwise I can’t paste with the middle button

                                1. 2

                                  Yes, mostly in vim. Sometimes it is the fastest way to select a range of text. Pretty sure that’s one of the first things the mouse was used for after it was invented. It’s also useful with plugins like tagbar.

                                  1. 3

                                    There’s something I use in Emacs that’s faster for selecting text than using the mouse, and that’s ace-jump-mode. (There’s probably something like it for Vim.) You hit the key combo, then the head character of the word you wish to jump to. For all words starting with that character, the character is replaced by a unique letter; you then type that letter to jump there.

                                    For example, I might want to jump to “bar” in “foo bar baz”. So I type C-c SPC to trigger ace-jump-mode, fix my eyes on the start of “bar”, then type b since I want to jump to a word starting with “b”. Now I see “foo gar haz” with the trigger characters highlighted, and press g to go to the one I want. Then the text goes back to normal.

                                    It feels really weird at first, but I can select text really quickly this way, and I want it for every damn interface now.

                                    1. 2

                                      That is pretty cool. Seems like the Vim equivalent is easymotion.

                                      1. 1
                                    2. 2

                                      I use it in VIM occasionally for scrolling, I think mainly when I’m trying to get a sense of a file, understand it’s organisation etc.

                                      1. 2

                                        i use it for resizing splits in vim, mostly - i find tasks like that easier with a mouse.

                                        1. 2

                                          Yes, I have mouse mode on for tmux, vim, and weechat when I use it. It’s very useful for scrolling large text files, selecting text, or maybe clicking a tab in tmux/vim when things are really getting crazy up in your tab bar. I use the same hotkeys with both vim and tmux so it’s the same keystrokes to remember everywhere.

                                          1. 2

                                            I use it to resize and scroll through tmux panes, because doing this by keyboard feels horrendous.

                                            And once in a while I feel lazy, and click/scroll around in Vim through NERDtree to explore some code.

                                            1. 2

                                              Yes. I use it with vim, tmux and aerc. It’s a useful extra input method even when I’m predominantly keyboard oriented.

                                              1. 2

                                                Yes! Did you try to use htop with a MacBook-like laptop? At best you have hardware function keys, but since you must keep the fn key pressed to enable them it’s awful. At worst you have a touch bar and… I don’t even want to rant about it. Hopefully there is the mouse (and htop works wonderfully with the mouse!).

                                                1. 1

                                                  I’m using Windows, so this may not apply, but…

                                                  I ended up writing quite a bit of code for mouse support just because the facilities offered by the built-in console were a bit lacking. The most interesting to me was copy-as-rich-text, allowing colors and formatting to be preserved in the copied output. At the time, a lot of people in my workplace ended up taking screenshots of their terminal and emailing bitmaps, and I really wanted to do better.

                                                  Since then I’ve kept going with a Ctrl+Click to copy + paste, and a hover to illustrate what expression is copied and pasted. This makes sense because when staring at a shell prompt the mouse isn’t really doing much, so the question became what facilities it can support to make shell input a little nicer. It also supports things like click-to-position within the current input line.

                                                  Both of these can be seen in animated form at http://www.malsmith.net/yori/ .

                                                  Right now I’m working on a TUI editor, and having mouse selection in that type of application is nice. Leaving this to the terminal means the selected text includes chrome provided by the TUI and strips out formatting that is known to the editor. I’ll probably end up with a right-click context menu at some point.

                                                  So personally I’m still in the mode of “this input facility exists, how can we make it more useful?”

                                                  1. 1

                                                    With libraries like imtui, I may start using the mouse more. I’m using it to write a small debugger/stepper for my emulator and it’s been a pleasure.

                                                    1. 2

                                                      I didn’t know about imtui, that’s cool! (I’ve heard of Dear imgui).

                                                      What motivated this question is trying to find a good front end for GDB. I’m now tempted to try to write one with a framework like imtui…

                                                      Because Eclipse crashed (and even locked up my entire computer), and GDB’s TUI mode gets confused when the debugged program prints to stdout! And I couldn’t get gdbgui to work.

                                                      Of course whatever I write would be just as unstable, although maybe the imgui model would help that? I think “inverted” control flow is harder to get right. But I’ve never used imgui so it could have different problems.


                                                      Yeah looking at imtui more, this is a very tempting project. GDB has quietly gotten super powerful over the last decade, and it needs a small and fast UI. And imtui even has mouse support, which is awesome! Debugging really benefits from mouse support.

                                                      And it runs in the browser! (though I saw some glitches). I have a hunch that gdbgui’s concurrency problems could be made simpler with imtui… or maybe it would hang the UI. I’ll have to think about it (and get familiar with GDB’s machine interface)

                                                      1. 2

                                                        so I share the same sentiment and need for a better gdb/lldb TUI. Debugger capability is in a quite interesting state factoring in RR etc. but the HCI is, to this day, not even at the level of Turbo Debugger.

                                                        If you want an OK abstraction with all the python on top of it, save the effort and go for voltron, gdb MI (and lldb-dev for that matter) leaves … much to be desired and, if python can be tolerated, voltron has done some of the heavy lifting.

                                                        When I am finished with a neovim UI, my next target for arcan-tui is gdb/lldb to finish the chain of leveraging the display server to improve debugging by dipping into LLDB/GDB. Happy to collaborate.

                                                        1. 1

                                                          Thanks for pointing out Voltron! I didn’t know about it, but that is the first TUI I’ve seen that uses GDB’s MI? Well I suppose emacs does it too.

                                                          It doesn’t seem like Voltron exposes GDB’s Python pretty-printing plugins, which I just used and found essential! [1] But I will try it and see. I’ve wanting to write a blog post called GDB quietly got awesome in the last 10 years :-) I was surprised now nicely and tightly integrated the Python plugins are with GDB.

                                                          I had some grand thoughts about shell being TUI for “outside” processes and a debugger being a TUI for “inside”. The GDB command language is of course very shell-like, and the TUI shows relevant state, although there’s a lot more room for improvement.

                                                          (I also played with rr recently [2], but then I discovered that GDB’s has its own reversible debugging, which could be efficient enough for a shell.)


                                                          And it does seem Arcan is relevant to this thread in multiple ways. For one, it appears that using a mouse with terminal apps isn’t completely normal or well-adopted. Apps support the mouse in different ways, and fight for the mouse.

                                                          I do think a better TUI interface could help with that, although I don’t know how it will get adopted.

                                                          I read the blog post and watched the video, but I only have a hazy idea of it. I agree with the general idea that terminals and TUIs are sort of an archaic and old display server. And debugging is a great use case where the state of the art needs improving. I remember reading about Arcan a few years ago and being impressed with its scope and ambition.

                                                          Personally I have been coloring outside the lines a little too, and have have this fork for over a year :-(

                                                          https://github.com/oilshell/bash-completion/

                                                          I guess that is analogous to your neovim patch – to fix apps on the other side of the wire.

                                                          I’ve also been brainstorming better UIs for shell (over on https://oilshell.zulipchat.com/ , all are welcome there). And a lot of shell is limited by terminals. But I’m still struggling trying to meet existing use cases, rather than coloring outside the lines. But brainstorming never hurts :-)

                                                          I should refresh my knowledge of Arcan. Looks like you’ve made some great progress lately!

                                                          [1] https://github.com/oilshell/oil/blob/master/devtools/oil_gdb.py

                                                          [2] https://github.com/mozilla/rr/wiki/Using-rr-in-an-IDE (it works under Ecliipse for me)

                                                          1. 1

                                                            For one, it appears that using a mouse with terminal apps isn’t completely normal or well-adopted. Apps support the mouse in different ways, and fight for the mouse. do think a better TUI interface could help with that, although I don’t know how it will get adopted.

                                                            I think there is a ‘mouse protocol’ for any possible permutation and range for each possible type of input that a ps/2 class mouse can deliver. The only universal is that the one you need won’t be implemented correctly by the terminal emulator you have, the client you are using, or both.

                                                            Problem with adoption is that if things stop at just replacing ncurses (and there are quite a few such replacements) we never get further with latency, integration and so on. If actually attacking the entire chain (emulator, shell, application) getting adoption is much, much, harder.

                                                            The tactic I am settling with is:

                                                            1. arcan + the included ‘console’ WM [1] running with the SDL backend (OSX, X11, other Wayland compositors, Old FBdev embedded etc.) works as a substitution for the ‘terminal emulator’ kind of use case, and there is a terminal emulator for the shell to use.

                                                            2. arcan-tui library can do server-side rendering these days, packing updates into a wire format.

                                                            3. (incomplete) taking shmif-server library (for the IPC setup), compositing and rendering the format from (2). allows for an application that can then composite back into ncurses/regular terminal as well as translate input, solving the other part of the legacy chain. There is a simplified arcan build that only provides the libraries for this.

                                                            4. arcan-net tool handles runtime network redirection

                                                            As for writing new shells and autocomplete, some changes has been made for that case recently.

                                                            In afsrv_terminal (the terminal emulator in arcan that renders using tui, has a ‘cli’ mode where you get the readline prompt and the setup for launching new clients from there as either legacy terminals or arcan-tui applications, but they spawn as discrete objects on the WM level, detached from the shell as such. This cli mode could outsource the analysis and querying to oil,

                                                            If launched as an arcan-tui application, it is expected to have an argument eval mode where the shell can interactively submit arguments and expect completion in return or parser feedback as to failing argument and offset. The hooks are there, some more UI magic before this is working completely. This means the ‘shell’ can be implemented on either the level of the ‘tui-shell’ or the wm level of the application arcan is running.

                                                            [1] https://arcan-fe.com/2018/10/31/walkthrough-writing-a-kmscon-console-like-window-manager-using-arcan/

                                                            1. 1

                                                              OK interesting, it sounds like you have a path, and we are thinking along the same lines.

                                                              This discussion clarified to me that Oil should have a configuration / mode that’s completely separate from a terminal. After all, a language like Python, Ruby, or JS is also separate from the terminal.

                                                              And GDB is an engine that’s separate from the terminal too.

                                                              So hopefully Oil can do its bit to get rid of some legacy, or at least it can not lock us in further to legacy :)

                                                              https://github.com/oilshell/oil/issues/738

                                                              So I think there could be an oil-tui, oil-native-gui, oil-web-gui, oil-arcan-tui, etc. If we design for the first 3, then the last one should be feasible, though I’d be interested if you think there are reasons that wouldn’t be true.


                                                              On a related note, my head spins when I think about Arcan, because it is such a big task.

                                                              For context I’ve been working on Oil for 4 years, and just recently one of the first patches to run a shell script with it landed upstream [1]. In contrast, the bash-completion fork I made over a year ago has languished. I don’t have time to upstream it, and others won’t be motivated to until they use Oil. Of the users that have the motivation, only a small fraction have the knowledge.

                                                              It sounds like you are addressing a lot of real problems so I hope that it can be adopted. My experiences with gdbgui and Eclipse wrapping GDB makes me think that the status quo could be a lot better. Then again, there are the rare good GUIs here and there that use existing tech, but core tech like debuggers are especially short on develoopers.

                                                              I think if I want to make concrete, short-term improvements to my debugging experience, I should look into patching GDB, or writing plugins. Unlike bash code, which I think is unsalvageable, it seems like GDB is pretty modular and hackable? I like the Python API, even though Python isn’t my first choice for embedding. It makes good use of Python’s object model. Crossing my fingers :)

                                                              [1] http://www.oilshell.org/blog/2020/04/release-0.8.pre4.html#the-highlights

                                                              https://github.com/dylanaraps/neofetch/commit/d2ca250856558f97c2f416747ecb6cc349179d37

                                                          2. 1

                                                            FWIW, after thinking about it, I filed this issue for Oil:

                                                            https://github.com/oilshell/oil/issues/738

                                                            Basically the idea is that you could have a shell running in something that’s not a terminal. It would have some kind of RPC / IPC interface to execute commands, get responses and error messages, and respond to completion queries, history queries, etc.

                                                            Oil is pretty modular so I think this will be possible: http://www.oilshell.org/blog/2020/04/release-0.8.pre4.html#dependency-inversion-leads-to-pure-interpreters

                                                            This also “offloads” a lot of work I’m struggling to finish… i.e. Oil will provide a basic bash-like interactive experience out of the box, but a lot of people want more than that. So I think it’s useful to have UIs as separate projects, whether TUIs or GUIs, etc.

                                                      2. 1

                                                        emacs does have mouse support. I have used it to resize windows and even to click on menubar choices sometimes. you can probably do a lot more with it than I do, for example I don’t bother using it to select text, that’s much faster with the keyboard.

                                                        IMO the real killer feature for emacs mouse support is scroll wheel support - being able to scroll file contents up and down is something I use all the time.

                                                        PS, screen (and tmux) history / copy mode also lets you use scroll wheels.

                                                        (I’m using MacOS’s Terminal.app, and you have to enable “Allow Mouse Reporting “ in the View menu to get this to work - it’s handy to know that you can disable that and get the regular MacOS mouse selection behavior when you want to e.g. scroll up in the local screen buffer instead of the remote one, or select a rectangular area with option-shift-select)

                                                        1. 1

                                                          Constantly. Buffer selection in weechat, file navigation with vim + NERDTree, cursor movement, etc. I also use focus-follows-mouse quite a lot to switch terminal windows in my tiling window manager.

                                                          There’s a popular contrast drawn between a keyboard/TUI/minimalist-window-management approach and using pointer input, with “serious” users of the former tools supposedly expected to eschew the latter. This has long seemed kind of silly to me.

                                                          1. 1

                                                            Yes, I often use vimdiff, then edit things. To switch panes (I think I should call them ‘tabs’ in vim terminology), I often switch to visual mode and click on a place where I want to use the cursor. I’m not even sure how I would do this without the mouse.

                                                            1. 1

                                                              It’s a bit meta, but if you count gnome-terminal as a “terminal application”, you can right click any number, and get it translated to hexadecimal/decimal (depending on input) and an approximate binary prefix: https://i.imgur.com/XRNBx4V.mp4

                                                              1. 1

                                                                Yes. If you google “copy paste in terminal” all the first page answers are mouse based.

                                                                1. 1

                                                                  One of the only things I intentionally use the mouse for in terminal applications is scrolling less output with the mouse wheel by specifying --mouse in my less args (LESS env var).

                                                                  1. 1

                                                                    htop is great for use of mouse with terminal, I can easily click on column headers to sort, or I can click on a process and select it to kill it or send other signals.

                                                                    I know I use the mouse with some other terminal programs, but I can’t remember!

                                                                    1. 1

                                                                      It’s interesting to me to see how many people not only use the mouse in TUIs/consoles, but actually like and prefer it for some tasks. For me, I’m happy to use the keyboard almost exclusively, using the mouse only to select, copy and paste from GUI window to GUI window. It seems a significant inconvenience to me to have to dislocate my hand from the stable “home” position over the keyboard, whence I can accurately, “look-lessly” press just about any of the 100+ keys of my keyboard. A key problem with the mouse as an interface is that you cannot do anything with it without using your eyes; leveraging visual feedback is required.

                                                                      I grant that there is a non-zero number of tasks which are more conveniently executed with a mouse, but, for me at least, that number is small, and a tiny minority (in the context of console windows).

                                                                      1. 1

                                                                        I’m using my mouse for controlling htop quite often. Not sure why not.

                                                                        1. 1

                                                                          On second thought, one place I would like to use the mouse in vim is to navigate through its help. Somehow I’ve never really liked using vim help in the editor. I’d rather just read it online with a browser. Maybe mouse support could change that.


                                                                          Yes I just tried this and it works pretty well, with cilcking over links and scroll. But “back” doesn’t seem to be bound to what I expect.

                                                                          1. 2

                                                                            Default binding is ^T for “go back” in tag navigation.

                                                                            1. 1

                                                                              I think ^O should hopefully also work in help, and IIUC is a “more generic” “jump back” shortcut, so maybe it might be worth it to learn first? Or am I wrong here? (For completion, the corresponding “jump forward” is ^I (capital i), and the list of locations is called “jumplist” I believe.) At least I seem to be using those shortcuts all the time…

                                                                              1. 1

                                                                                Yeah, that’s also fine, and might be smarter. I just learned ^T 15 years ago and it got stuck :)

                                                                          2. 1

                                                                            I’ve really been enjoying musikcube as a terminal music player. Its mouse support is quite excellent, including allowing right clicks (at least in Alacritty, it’s not working in iTerm2 for some reason) to bring up a menu for actions you can take on a given track. The only mouse support I kind of want that it doesn’t seem to have is scroll wheel support for navigating panels in the application.

                                                                            It kind of feels like a native GUI app that just happens to be all text.

                                                                            1. 2

                                                                              Yeah I’m tempted to try to write a “better” frontend for GDB that is like a native GUI app that is text… (after tryiing a whole bunch of existing front ends)

                                                                              If you make the screen big enough, it should have as much visual density as a good GUI.

                                                                              And I like that it would work over ssh, and I think fewer hooks into native GUI libraries should make it less complex / more responsive.

                                                                              But that’s probably a year+ long project so I shouldn’t get distracted. But maybe it is possible to make a quick start with imtui or something.

                                                                              fixed link: https://github.com/clangen/musikcube