1. 64
  1. 12

    Yes, cat | grep is a “useless use of cat”, but it allows me to quickly change the grep regex, which is much more annoying if I have to navigate past the filename first.

    1. 8

      I agree, and I think it’s kind of stupid to try to avoid cat outside of scripts, but that said, there is a way to solve this specific problem:

      <file grep regex
      1. 1

        That’s cool. I think I’ve read about this shell syntax before in man pages but never saw the use for it. Now I do!

        1. 1

          I’ve always liked that syntax, especially if you have a redirection at the end of a follow-on pipeline (<foo | bar >baz ) but it can break in bash with some shell constructs in the pipeline (some uses of while)

          1. 2

            I think you mean

            <foo bar | baz >qux

            I too find this pattern useful, specifically for troff pipelines.

        2. 4

          “Useless use of cat” is usually bad advice, anyhow.

          As far as I can tell, it only matters for performance-sensitive scripts. If someone’s just hacking around on the command line, cat ... | ... is extremely useful since it allows quick switching out of inputs. Let’s say I’ve got some_expensive_script.sh | ... – better to capture the output to a file, run cat output.txt | ... and build up the pipeline until it works, then swap back in the expensive script call when I’m ready.

          (Also, I very often end up accidentally writing < output.txt | grep ... because I forget to delete the pipe, and so grep gets nothing! This leads to a great deal of frustrated debugging until I notice it. Best to just keep it a pipeline while developing rather than adhering to some random internet person’s notions of terminal purity.)

        3. 11

          I really respect anyone who gets this angry at terminal emulation while also teaching me something new about terminal emulation. SIGWINCH, dang.

          1. 5

            I feel kinda jaded. Every so often somebody will say ‘look at this horrifying bit of c syntax (that we’re stuck with because some 1970s compiler supported it by accident)’; or awful ioctl archeology, horrifying vt * or … but it doesn’t really faze anymore because I’ve seen it a thousand times. (Adam and bjorn, else-thread, I’m sure can say the same.) God, what a mess computers are. I guess this is ‘acceptance’.

            1. 4

              So between in-band information (like ANSI escape codes) and out-of-band information (like signals and ioctls), GOOD LUCK WRITING A TERMINAL EMULATOR.

              Actually you don’t have to mess with the signals when writing a terminal emulator. The kernel does that. You just send the ioctl to inform the kernel that things changed (TIOCSWINSZ); Though you might send the signal yourself additionally. What’s annoying is if there’s another foreground process the signal goes to it then your other ones won’t necessarily be informed. My terminal emulator tries to hack this by resending the thing when the process changes… but that’s hard to do too since the terminal just gets pipes without necessarily having all the info.

              The linux terminal is a mess. Frankly I wish it would die. But it also does useful things and has a huge ecosystem sigh.

              1. 2

                What is “the Linux terminal”? Honest question. Are you referring to terminal emulation? In which way is that Linux specific?

                What would you do different? Do you mean that a text repl as an interface tot he system is a bad idea per se? Why? What do you suggest instead?

                1. 19

                  What is “the Linux terminal”?

                  The abstract concept of terminals as used by Linux programs. So the fact that it is random sequences mixed in to a pipe where neither side knows much about the other, with the kernel in the middle doing its own thing.

                  Of course it isn’t actually Linux specific but I thought this was an easy and convenient way to communicate the ideas used there as opposed to the ideas used in a Windows console application or an X application, etc.

                  What would you do different?

                  Almost literally everything. It is awful from top to bottom. Everyone loves to hate the Windows console subsystem (mostly because the end-user ui was questionable until kinda recently, not even allowing convenient resizes (you had to go to the properties sheet instead of dragging the window like everything else? Bah!)), but the technical foundation was much nicer, with a considerably less error-prone input API, though the output API became a bit of a pain to use, it was also at least more reliable.


                  I guess I’ll start there. Linux terminal input api is “read bytes from pipe”. The interpretation of those bytes is extremely complicated since it is technically undefined; it can change based on what terminal you have. Nowadays though most everyone emulates the xterm (which itself kinda emulates the VT100 among other things, but it is really xterm that sets the standard today), so that simplifies things a little. A little… rxvt is still pretty respectable but has a slightly different input sequence. And until recently, the Mac OS Terminal was slightly different too. And then different people’s xterm clones also can offer or not offer their own things.

                  But still, let’s say you just want to read xterm and call that good enough. The sequence is still a massively complicated state machine with very little consistency! The byte 27 means expect extra info… except when it doesn’t because the user just hit esc. Those sequences are variable length and the end character depends on what the next few characters were, so if you try to implement just a few and skip unknown ones… there be dragons. You can do it but even skipping unknown things takes a lot of preprogrammed knowledge. Once you do read a sequence successfully, their interpretation may depend on global settings in the terminal which can be left behind from the previous application. Fun.

                  Then once you decode it, various pieces of information are not even present. Want a key release event? Too bad. Want to know if it was enter or shift enter? Need an extension (and this is why most thing emulate xterm since at least then you can kinda standardize on extensions… but there’s no guarantee it is present and querying this is a pain even when it is possible, which it often isn’t).

                  Ever had a program crash then moving the mouse across the terminal outputs spam? Or find you type and nothing happens? now you know why. The terminal can’t tell the program crashed and reset itself, no, it is stuck with whatever random global state it was in then and the other application may not know how to interpret this data stream.

                  Contrast to the Windows console where input events come as structured data, creatively named an INPUT_RECORD struct. The length of these is told to you so you can skip them easily if you’re not interested. Extensions can now be easily added and applications won’t be lost. Much much nicer. Most gui things work like this too. So I’d take structured input data in my new system over the escape sequences on input. (Of course, it does burn a few extra bytes on the wire. Realistically they could have afforded this on the input even back in the physical hardware at 300 baud days but you can understand things were different then.)

                  Speaking of input events, the general terminal paradigm when programming the client application is to block for input. You know, scanf("%d", &a); printf("%d\n", a);. This isn’t necessarily bad - sure can be convenient - but it really does suck for anything more interactive. Most linux programs do some kind of event loop even with the terminal anyway. But then scanf() stops working. Now I hate scanf anyway, but again Windows indicates things could be better… on Linux, unless you work the terminal yourself, e.g. GNU readline’s implementation, input sucks. User types in that scanf prompt, makes a mistake. Backspace… just adds more spam to it, digging them in deeper. Unless you change to “raw mode”, the kernel, not the application, and usually not the terminal either (though it actually kinda can be there, this is not the standard) is doing that input sequence and it is the stupidest possible thing that can work. On Windows, by contrast, when you ask it to just read a line in cooked mode, backspace works, arrow history works (though it is janky, it is there), even tab complete is available in the standard api (though again … not great and fairly rarely used by third party applications, at least they tried).

                  I honestly don’t know why Linux cooked mode doesn’t offer at least something in terms of user experience, since it is all line buffered external to the application anyway. But like I probably would change this too.


                  Outputting data to the linux terminal is just as problematic in theory as input, but in practice it isn’t so bad since most of that is de-facto standardized….. but not all of it. Like the link mentioned here, deciding to add color or not is often done with a isatty() check or a random switch. The sequences themselves are still a state machine that can be corrupted by a program crash or whatever, and the terminal implementation to read them is still a pain.

                  The Windows way of communicating out-of-band metadata brings its own problems. You write text, then in a different function call, change color, then write more text. Bleh.

                  What I’d do is make it take structured commands. This might be length+command+data sequences (which you can imagine they didn’t do in the 300 baud days for bandwidth reasons, but nowadays we’d be generally ok on it) or heck maybe even something like XML just to trigger the haters, but in all seriousness, XML would be quite suitable for the task, given its standardized extensible markup and metadata capabilities, allowing receivers - not just terminals anymore, but any receiver including programs in a pipeline - to strip things they don’t care about. Writing <span color="red">Hello world</span> to either a pipe or a terminal would work since the receiving program can simply strip out that color data. (Of course you could in theory do that with ansi escape sequences too but you’d need to know they are there - which is communicated out of band right now - and the rules for stripping them which is again fairly complex since they come in several forms, but to be fair, parsing XML is not any easier. Anyway my real point here is to put more structure on it, the exact form of that structure isn’t terribly important.)

                  META COMMUNICATION

                  If you want to ask for the size of a terminal, or the current cursor position, or other similar things now as an application, it is a bit of a pain. You need to flush your buffers, write out the request in the same output stream as everything else, then read back - possibly buffering and/or still processing unrelated input that was already in the stream in the mean time - the answer, which comes as some of those escape sequences. If it comes at all…. maybe the recipient doesn’t understand the request or doesn’t care to answer. So you need to work in some kind of timeout, but not too quick or it will mysteriously fail on ssh connections, then when the esc sequence does come and you aren’t expecting it, you’ll perhaps read it as the user hitting other keys. What a mess.

                  If your input and output channels are structured data though, you can work it in here with less trouble. Even piping it into less could take the data and discard it for you, simplifying some application development.

                  Currently, if the terminal wants to know about the application, it might be able to ask the operating system - list open files of your pty or use other extension things. Or maybe the shell will be configured to inform you. But there’s no real hope doing it in-band, since applications wouldn’t expect such a message to come. You could, in theory, send a pseudo key sequence but the other side needs to understand it too, so unless you control the whole stack it won’t work (and now you know why I wrote my own terminal stack lol. ok not really this in particular but it is nice to have the option sometimes). Structured input and output may not necessarily give you an answer - the application can still ignore the request (like odds are the implementation of scanf would ignore anything that isn’t a “give me a line of text” input answer) - but at least you could more confidentially try without worrying about corrupting the stream.

                  I would standardize the terminal getting knowledge about what application is talking to it. Then if it crashes, it can at least reset to the previous state automatically and carry on.

                  UI PARADIGM

                  Most of the above is about the “bottom” end of the system, the implementation of the terminal and communication with it. But I said things suck top to bottom and I meant it. There’s three main kinds of linux terminal programs: full screen interactive ones (e.g. vim, mutt, irc client), interactive shells (bash, python, mysql), and batch processing pipelines (grep, awk). Some are a little bit of both, like less is full screen interactive and tends to be part of a pipeline, or you can run python interactively or part of a pipeline, etc. These generally branch on the value of isatty on either input or output explicitly, thus you might need to either openpty or --color or whatever to override its decision. Applications that fail to do this don’t work very well.

                  Moreover, the very concept of command line arguments gets dicy. touch -- -R; rm * is the classic example - rm doesn’t know if -R is a user-provided switch or a shell-provided filename. Windows, again does this differently - over there, the command line is just a string and the program is responsible for interpreting it. On both systems, pipes need special care and the interpretation of that data gets iffy. On Windows, a pipe might be UTF-8 whereas the console output is UTF-16. How do you know this? Read a couple bytes and guess. lol but not kidding.

                  Neither way is really great, but the Linux way has had pretty good real world success in being both scriptable and usable interactively, with surprisingly nice extensions offered by innovative shells, all the while being fairly easy to add your own programs to it, so we don’t want to throw all that away.

                  And that’s the real problem to making a big change here…. realistically, you’d still have to support the old ways anyway.

                  But if we’re in fantasy land, what I’d love to do is to standardize a number of flags and some of the output. For example, make --help mandatory and define it to output a certain structure so people and machines can read it. Then you can tell what the possible arguments are and what formats they take… then make the interpretation of arguments standardized for the program, so no more rm * confusion. These things together means you can script it in various forms - shell can automatically offer auto-complete by parsing --help in the background. GUI programs could even pop up a dialog box for it automatically!

                  The output, being structured for the terminal already, can be structured for other pipelines too. Metadata like color ignored by pipes while other metadata might be consumed by them. Windows Powershell does this kind of thing, though I think I’d keep some kind of primarily text output - hence my earlier suggestion of XML. XML with tags stripped is plain text again and, if done right, still readable bare in a zero-feature terminal (just like how websites can be easily read by the most basic text browser. right? RIGHT?).

                  The batch pipes and interactive shells can both be used this way. To support pipes or interactive, you can just output <user-interactive-prompt-for-line>Prompt $</user-interactive-prompt-for-line> and then read lines, whether from the prompt or from a pipe. Let the pipes strip the user-interactive part themselves.

                  So I’m trying to embrace higher level the whole way. If you want pipeline batch programs, embrace that and make them explicitly known. If you want fullscreen, that’s similar to what it is now, just easier to program since you don’t need 5000 lines of library code to parse the stream. You could end up a system that still does what Linux terminals do in practice but better.

                  But alas it is all just fantasy anyway.

                  EDIT: I wrote readf when I meant scanf so changed that over (readf is the D language version of scanf but since this isn’t for a D audience I meant to stick to the C names. oops.)

                  1. 3

                    Thanks for the fantastic write up.

                    I would standardize the terminal getting knowledge about what application is talking to it. Then if it crashes, it can at least reset to the previous state automatically and carry on.

                    Note there’s another dragon there, because the terminal supports multiple processes talking to a single terminal concurrently despite having a single global state machine. I think fixing the state issue also implies single terminal ownership. Also, in addition to resetting state on process exit, it should ensure default state on entry; currently child processes inherit the parent’s state, so we have things like processes expecting cooked mode on entry despite a shell process not using that mode.

                    over [on Windows], the command line is just a string and the program is responsible for interpreting it.

                    That’s true, but it’s become a total mess. On Linux, a file name with a space can be parsed by a shell and put in a single argv entry. On Windows, it must be quoted so the child process can parse it. But that means the parent really needs to understand the child’s parsing rules, and the parsing rules are inconsistent.

                    The Windows way of communicating out-of-band metadata brings its own problems. You write text, then in a different function call, change color, then write more text. Bleh.

                    Somewhat meta, but I’d rephrase it and say that the Linux terminal was designed for a VT100 serial line (so everything is in-band) whereas Windows was designed around an RPC model where different message types exist. If you squint a bit, the real difference is ordinary text/input/output being encapsulated in some kind of message structure - this isn’t obvious because Windows doesn’t expose a single stream of data, but it’s the obvious way to turn those function calls into a single stream of data.

                    The big problem in the Windows model is now the terminal emulator is its own process rather than the kernel, you write text (and synchronously context switch to the terminal and back before your program can continue), change color (and synchronously context switch to the terminal and back before your program can continue), write text (and…

                    It’s a shame that the Windows functions weren’t converted into a single stream (on the application side) which would have made SSH type protocols simpler, and eliminated the context switch madness.

                    Input events come as structured data, creatively named an INPUT_RECORD struct…Extensions can now be easily added and applications won’t be lost.

                    In theory, yeah. Except it’s Windows, compatibility is everything, and you don’t know what assumptions some random program made. So what actually happened is console windows became resizable but no new event is generated to indicate that. Instead the existing “buffer size changed” event is overloaded, even though the buffer size isn’t changing, just to poke client apps to know something happened. Back when resizing was via the properties dialog, applications got no notification at all. (I have code that polls to detect resizes.)

                    1. 1

                      I think fixing the state issue also implies single terminal ownership.

                      Yeah, or at least put the state per-connection if you do let output be combined.

                      But that means the parent really needs to understand the child’s parsing rules, and the parsing rules are inconsistent.

                      Indeed. What I’d actually like with the standardized metadata is that the parent DOES understand the child’s parsing rules at least to some extent, but then it is consistent. Ultimately it would be nice if the argv itself was separated a bit for options vs operands at least (could standardize on the -- thing most things already do).

                  2. 2

                    The interface you have to work with in order to implement a terminal ‘emulator’. The REPL or whatever your ‘shell’ provides doesn’t really factor into it much (more than it blocks possibilities for improvement) though it has evolved around the limitations of the forest of terminal protocols so are in many ways tainted.

                    There are subtle, special semantics for how all the infrastructure is setup to allow your command to talk to your shell to talk to your emulator to talk to this ghastly reanimated corpse of 60ies tech. There are subtle and breaking nuances between behaviours between the different UNIX derivatives. There are functions and interfaces to be used that never or rarely appear elsewhere and talked about hardly ever. Every attempt at a layer has hacks around misbehaviours in what is perceived to be above it and what is perceived to be below it.

                    For just a “fun” one out of many. Take ‘ncurses’ - the popular TUI library. It still has a ‘bandwidth estimator’ that tries to work with the ‘baudrate’ (stty(1)) of your virtual serial port talking to the pseudoterminal device deep down in the kernel to figure out the cost of ‘draw commands’ (cursor to this position, …) so that it can try and buffer just the right amount for the window to be drawn without the cursor stopping half-way through. This estimator needs to know what each draw command translates to and that is dependent on the flavour of terminal protocols available and the quirks of the emulator used, so it checks an environment variable for a terminal emulator name so that it can look those up in a database (termcap(5), terminfo(5)) of the set of hacks to enable/disable.

                    That is for moving the cursor around and writing characters with varying foreground, background and formatting (bold, italic around). There is also signal propagation, flow control, character sets, mouse cursor, clipboard, different screen modes, … There are reasons why xterm is 65+k lines of code and you don’t want to know about any of them.

                    1. 2

                      There are reasons why xterm is 65+k lines of code and you don’t want to know about any of them.

                      Well to be fair a lot of this can be simplified if you look at the de-facto standard today. My terminal emulator core is only 5,000 lines of code and my ncurses replacement is 8,800 lines of code (though a good chunk of that is my gnu readline replacement which comes in at about 2,300 lines, but like since cooked getline is so bad a terminal program without something like readline is borderline unusable, hence i integrated it there, and then there’s the Windows implementation and some integrated emulator stuff that pads it out further).

                      Though I guess xterm speaks X directly too and I put that off in a separate library… which is 20,000 lines but well that does a LOT of things and has bundled bindings and Windows and Mac support too. So it could be much smaller if it was just what the terminal needs.

                      It is still a lot of messy code on both ends though.

                      references for my versions: github.com/adamdruppe/arsd terminalemulator.d is the terminal emulator core that does the emulation based on pipes and in-memory (the front-ends are actually in a separate repo here https://github.com/adamdruppe/terminal-emulator but they’re relatively simple since the messy part is in the core lib). terminal.d is my client api. and simpledisplay.d is my window library.

                      Having done it all though on both sides and high and low level has given me a lot of knowledge that I kinda wish I didn’t have.

                      But still a de-factor simplification is < 10k lines and still works for 99% of what you’d actually do nowadays, so a fraction of all the things xterm and ncurses have implemented over the years.

                      1. 2

                        try and buffer just the right amount for the window to be drawn without the cursor stopping half-way through

                        Wow. Guess things like that is why ncurses is so well… cursed big and stuff.

                        Meanwhile good terminals are starting to support sync sequences so we can do things like Kakoune’s smart diff based rendering reliably :)

                        1. 1

                          Where is the list of clients implementing it, and what trigger should one use to probe? toggle altscreen is an obvious one, but that wouldn’t be sufficient for tmux to vim – for instance. I experimented with “vsyncing” through XON/XOFF, but as with everything else it just becomes heuristics that break in some or many edge cases.

                          From that issue: This would indicate the linux console renders every intermediate state instead of what most terminals do: render once there is no more terminal output pending.

                          Just using pending output to vsync would make find / a very unhappy command (and anything with buffer backpressure really but that one is the obvious test target).

                          Others go galaxy brain with https://github.com/cipharius/kakoune-arcan – and in return can fullscreen a window to get back to drawing single ‘racing the beam’ buffer with glyph cache shared between all clients using the same typeface, pt size and dpi.

                          1. 1

                            list of clients implementing it

                            I’m not aware of one, only a list of terminal emulators.

                            what trigger should one use to probe

                            I’m not sure what this means. I’m not aware of any test cases where sync solved a practical problem (glitch) other than the kakoune one.

                            1. 2

                              Wall-of-text warning to avoid assuming background, paragraph 2 and 3 from the end are the meat.

                              The problems are the same between terminals and other display servers. It’s one of those overarching pesky things in the same vein as ‘colour correct workflow’ - synch is quite hard; almost always the hardest one.

                              You have a producer working with a command-stream (cursor_to:dx,dy, write:abcdef) (cursor_to:dx2,dy2, write-bold:ghjkl) … and a consumer (server side) which interprets the stream and translates it to the native language of some output, possibly mixes these ‘signals’ together (composition).

                              If the output imposes a deadline (old printer outputs typically didn’t, just flow control/small-to-no buffers - avoid ‘back pressure’ from a producer sending faster than the chain can handle and you are as good as you can get. With monitors you get binned intervals to fit (vertical/horizontal blank and sync) and then stalls if you miss your timing window.

                              Terminal protocols adapted, so now you have two different screen modes - the line oriented one and the ‘altscreen’ that works just like a row-resolution tilemap, the tiles just corresponds to glyphs from fonts. As the protocol accumulated more and more features, the processing cost for each of the cells varied and you started to miss the timing window or the command buffers were even larger than what can be processed in one update. Here you get tearing, and you have no other option. It is just less noticeable with a slowly changing signal so user annoyance in terminals is less than other forms of graphics.

                              Worse still, you also get input to display latency from a typical ‘render loop’ on the producer side as a while(ev = wait_for_input()){process(ev); draw_what_changed(); } format when the ‘draw’ part impose higher costs and block (serial link and all that) while input is accumulated and released in bursts (“embarrassing pause”, “jank”, …) with possibly noticeable variations in timing between the two (jitter) attenuating the effect. This is the problem from the user perspective, it is a QoS one - if you don’t stress or test outside your ‘normal use’ you might not see the problem and cannot properly evaluate the solution.

                              Not having agency to fix clients, hacks are added to the compositor side to determine if and when it should forward. The “counting characters” trick from ncurses is such a hack, and makes somewhat more sense when you have slow/variable latency connection (so networked). The “double buffer” trick from standard graphics makes more more sense there - but both are hacks.

                              Other ones were to check which mode the terminal emulator state machine itself is in and guess how many characters you have left to work with; If there is a small burst of data coming you likely have a smaller region update, a larger one - break when you run out of time to the next synch deadline or after about as many character updates as there are cells on the screen. The you need to add a refresh deadline as well because there aren’t always natural synch points in the data stream, find / being a simple example.

                              So I that is implicit synch and first stage explicit synch – next up is fencing as you ramp up asynchronous processing. You insert command markers to indicate when it is safe to process and forward. The consumer waits for those markers. This is across a privilege boundary, so if you don’t have a timeout you have a denial of service vulnerability as the producer can always “forget” to mark the end.

                              That is what the protocol change you link suggested. Here comes legacy and feature detection. The producer asks if a mode is set (DECRQM) and gets back something that says ‘zwp_linux_explicit_synchronization_v1’ is in the ‘wl_registry’. Now comes nested composition, which is the default in the terminal stack. The terminal emulator only sees one client in a possibly deep chain or one where the end keep being swapped out. If one producer dies; is killed; shuts down; detached; backgrounded etc. after a (CSI ? 2026 h) the denial of service part kicks in and I see nothing in the suggestion to detect that has happened.

                              The follow up in heavier system graphics is ongoing now and has been chugging along for close to a decade. Fences aren’t granular enough -> timeline semaphores. Display timing keeps drifting -> cancellation of submitted buffers.

                        2. 1

                          But all those predate Linux, no?

                          Nonetheless, Interesting reply. Thank you. I wonder if there have been attempts at creating a new simpler and more robust ‘wire protocol’ for text terminals.

                          1. 3

                            But all those predate Linux, no?

                            Yes, but there are new generations of Linux users/developers that know of little to nothing else. The author of the link is either one from that group or speaking directly to it. Network effects pushing out diversity.

                            Thank you. I wonder if there have been attempts at creating a new simpler and more robust ‘wire protocol’ for text terminals.

                            It’s a 2D problem – you want better solutions for both the vertical (command to shell to desktop) and the horizontal (between commands, cat foo | grep bar). Solving for the horizontal is more intuitive and approachable (typed data exchange), so there are several of those, but you are still bound by a ton of restrictions. Eventually trying to work around those will again add ‘inband signalling’ with desktop interaction mixed in with the data you want to transfer, and anything in the pipes and filters chain can screw it up royally - much akin to what cat /dev/urandom does to your terminal emulator today. That will still not be enough for the ‘resize’ case and sideband signalling gets thrown in again in some form. Say hi to the new boss that looks strangely like the old one but with more JSON.

                            Solving for the vertical takes you into display server protocols (desktop IPC) with the added twist of special needs for text rendering and grid-of-cells/list of lines. There is even more OS cruft here and even more divergent quirks (hello systemd…). I outlined one possible solution several years ago (The dawn of a new command-line interface) – it has been working for quite a while, I just forgot to tell about it and now don’t really want to.

                            1. 2

                              Yes, and it predates Unix as well. When people say “Linux terminal” they just mean “the terminal emulator I am running on my Linux system”.

                      2. 4

                        So between in-band information (like ANSI escape codes) and out-of-band information (like signals and ioctls), GOOD LUCK WRITING A TERMINAL EMULATOR.

                        You can now get and set the terminal size without ioctl; POSIX defined SIGWINCH, tcgetwinsize, and tcsetwinsize in 2017.

                        1. 4

                          The replace for useless cat is a redirect, not reading the manpage to see if this command knows how to take a filename…

                          1. 4

                            For some utilities using the explicit file may be faster if they mmap a file instead of reading from a steam. It doesn’t matter in 99% of cases, but worth remembering when it does.

                            1. 3

                              If a program is sophisticated enough to be mmapping its input file, it should hopefully be sophisticated enough to check if it’s still able to do that when its input file is stdin (which is not necessarily a pipe and hence may be mmap-able).

                              1. 2

                                I’m not saying it won’t work. But it may have different way of operating with a pipe vs a file. For example I maintain a tool which can rely on looking back into headers at known places when given an mmapable file and can rely on system block cache, or it will build up large indexes in memory and be a bit slower if given a non-seekable pipe.

                                1. 9

                                  Sure, but that’s exactly my point: when you run prog < foo.txt, the prog process receives a stdin that is a regular file, not a pipe – so it can still mmap it (or lseek or whatever else) just as it could if run as prog foo.txt.

                                  1. 3

                                    Well I’ll be… Today I learned. Thank you! I was convinced the stdin redirection just creates a pipe.

                              2. 0

                                Yes but with a useless use of cat the receiving program isn’t getting a file argument either.

                            2. 3

                              Another fellow on the road to discovery that the terminal is at the end game of what happens when you accidentally create a windowing system vs. deliberately. Both paths leads to horror, but different kinds.

                              One of the “best” spot for evaluating such systems is ‘resize’ – it is around there where the most chaos comes, testing is hard and fixing bad designs there is really quite hard.

                              X11 got slapped in the face by the feedback loops and subsequent event-relayout storms from the lack of a conflict resolving authority. VtXXX in that clients were not written with the ‘resize’ idea in place and that there are two fundamentally different modes (well a whole lot more but the biggest) – TTY or not. ALT-screen mode attenuates it (tmux, screen, etc.). Wayland becomes really unstable (clients/compositors crashing) and jittery around resource extreme invariants (GPU memory, file descriptor slots) which is also a debugging nightmare.

                              1. 2

                                I think colors in the terminal and even syntax highlighting are overrated and often cause problems that don’t need to exist.

                                1. 4

                                  It’s an arbitrary line, and often one drawn by the aesthetics of Unix retrofetishists. Likewise,

                                  I think line drawing and even control characters are overrated and often cause problems that don’t need to exist.

                                  And this is basically the same level of complexity, if not more so than colours - having to know where the cursor is and when not to move the cursor is far more complex than just changing some colours. (But making the terminal anything other than green-on-black is less 1337, isn’t it?)

                                  The entire TTY subsystem should have been taken out back and shot decades ago due to the inane and inconsistent hacks required to make it usable, but now we’re stuck with and celebrate it instead. Even if you’re not like me who prefers bitmap/vector displays, it’s hard not to admit terminals could have been done a lot better. (The Unix Haters’ book covered this and contrasted w/ i.e. ITS and VMS’ approaches.)

                                  edit: Syntax highlighting is a total non-sequitur here too.

                                  1. 1

                                    Its not about 1337 or not.

                                    I just tried to disable most colors on the terminal and text editor and found it doesn’t make much difference. The main side effect was that things are easier to configure and have fewer problems when the color schemes don’t match my terminal.

                                    Syntax highlighting is related because more and more programs are trying to backport the text editor color schemes to the terminal output of their commands.

                                    If you like them, I don’t mind, I just think they are overrated - It is unclear what value they provide, I am unsure it actually makes scanning text output any easier.

                                    1. 1

                                      Yeah, I agree. Syntax highlighting randomly failed on one of my files a couple months ago and I just kept going without it and…. it really just makes very little difference. The file works just the same. I think highlighting is more about expectations for what a thing is supposed to look like; without it it looks unfinished if you have this expectation. But it doesn’t make a huge difference.

                                      The horrible colors of ls is actually the #2 reason why I wrote my own terminal emulator (the #1 reason being making shift+page up work consistently when nesting attached sessions, C-a [ in gnu screen drives me nuts). I made mine adjust colors to the background so it doesn’t hurt my eyes. So if ls writes yellow on white, it makes it a dark yellow so I can actually read it. Similarly if it is blue on black, ti fixes it. Independent of application or anything else.

                                      The colors now no longer make things worse. Idk if they really make it better.

                                  2. 2

                                    I think running a non-line oriented editor in a terminal is more trouble than it’s worth.

                                    1. 1

                                      I think running a non-line oriented editor in a terminal is more trouble than it’s worth.

                                      If this is your genuine opinion great, if not, I don’t see why you made this comment.

                                      I genuinely code both with and without syntax highlighting and don’t notice a difference. I suggest more people try it as it saves the burden of configuring text editors and dealing with misconfigured terminal color schemes.