1. 10

    Hello! We’re Ben, Aanand, Carl, Eva, and Mark, and we made the Command Line Interface Guidelines.

    Earlier this year, I was working on the Replicate CLI. I had previously worked on Docker so I had a bunch of accumulated knowledge about what makes a good CLI, but I wanted to make Replicate really good, so I looked for some design guides or best practices. Turns out, nothing substantial had been published since the 1980s.

    On this search I found a superb blog post by Carl about CLI naming. He must be the only person in the world who cares about CLI design and is actually a good writer, so we teamed up. We also were joined by Aanand, who did loads of work on the Docker CLIs; Eva, who is a technical writer, to turn our scrappy ideas into a real piece of writing; and Mark, who typeset it and made a super design.

    We love the CLI, but so much of it is a mess and hard to use. This is our attempt to make the CLI a better place. If you’re making a tool, we hope this is useful for you, and would love to hear your feedback.

    Some of it is a bit opinionated, so feel free to challenge our ideas here or on GitHub! We’ve also got a Discord server if you want to talk CLI design.

    1. 7

      (copy of HN comment)

      Turns out, nothing substantial had been published since the 1980s.

      The best reference I’ve found on this is The Art of Unix Programming, which was published in 2003:

      https://www.amazon.com/UNIX-Programming-Addison-Wesley-Professional-Computng/dp/0131429019

      The book is also online for free:

      Command Line Options: http://www.catb.org/~esr/writings/taoup/html/ch10s05.html

      User Interface Design Patterns: http://www.catb.org/~esr/writings/taoup/html/ch11s06.html

      Still it’s nice to see this knowledge circulating!

      Joel Spolsky’s 2003 review of this book: https://www.joelonsoftware.com/2003/12/14/biculturalism/

    1. 31

      Good advice, except the one about skipping man pages. Man pages are easier to use and almost always more useful than --help. They’re “rich text”, they automatically open in a pager and they have a pretty much standardized disposition and format.

      The only exception is when developers are too lazy to write them and instead generate them from some other, non-man documentation. These man pages usually turn out terrible. But honestly, mdoc is not that hard. If you care about user experience, you should write a man page for your program.

      They don’t work on Windows? Says who? I’m sure man is ported to Windows. And besides, why should the way Windows works dictate how I write UNIX tools? I use both Windows and UNIX, and I don’t expect them to be the same type of system – that’s why I use both.

      Also, citation needed on the claim that “not enough people” use man pages. How can you know that?

      1. 5

        I think your advice on man pages here is a bit off, or at least, it’s not the advice I would give.

        The only exception is when developers are too lazy to write them and instead generate them from some other, non-man documentation.

        You seem to be using “lazy” as a pejorative here, but I see it as a positive in this case. My main problem with writing the entire man page from scratch is that it duplicates a lot of the documentation that is shown in the --help output. So when the docs change in one place, you have to remember to make the same change in the other place too. This is annoying. So it’s not just a matter of sucking it up and writing the man page in the first place. There’s a question of continued maintenance here.

        I write a portion of my man page in asciidoc while the rest is generated straight from the argv definitions. Overall, I think the resulting man page looks pretty good. Certainly not terrible.

        They don’t work on Windows? Says who? I’m sure man is ported to Windows. And besides, why should the way Windows works dictate how I write UNIX tools? I use both Windows and UNIX, and I don’t expect them to be the same type of system – that’s why I use both.

        I don’t use Windows and I don’t know the state of man tooling on Windows. But I don’t know anyone who uses man on Windows regularly. People who use my CLI tools on Windows might not have cygwin or WSL or whatever else installed. It’s a native Windows program, so they don’t need it. It would be awfully user-hostile of me to only offer the complete docs for the tool in a way that doesn’t let them read it natively on their system. “Oh no sorry, in order to read the docs for this tool, you need to go out and download this other port of a UNIX program. Good luck setting it up.”

        That’s why I try to write man-level documentation in my argv definitions. So that when a user uses --help, they see exactly the same documentation that appears in the man page. And they don’t need a special tool to read it.

        Now obviously, man-level documentation is quite verbose, so the compromise I settled on is that the -h flag will show a much briefer set of docs.

        But the point is this. All the docs for all the flags are located in exactly one place and that place occurs right next to that flag’s definition in the code. So when you need to add a flag or change docs or whatever, everything is right there near each other. Locality is a useful feature.

        So I’d say we agree that “forget about man pages” is bad advice. It makes me sad that it’s being promoted here. But let’s not be blind to their downsides and the realities of maintaining cross platform tools.

        1. 2

          I think you have presented a very responsible way of generating and thinking about man pages here. I like your compromise between -h and --help. I agree with you for the most part!

          However, I think that your generated man page (rg.1), at least as it appears on my system (Alpine Linux), is an example of the risks of such generation. The NAME section is preceded by six empty lines, and the definition lists likewise have too much spacing in comparison to traditional man pages (example). These small details are easy to miss when generating man pages from another format.

          But this is not a fundamental problem with your method, it is probably an easy fix, nor am I against such generation in general, if only done responsibly and consciously. You have clearly put good effort into it. Other than the spacing, rg.1 looks like any other man page.

          1. 1

            Yes, to your point, I used to use Markdown and converted it to a man format using pandoc. But the results were quite terrible. I almost just decided to learn mdoc and figure out how to use that, but noticed a lot of other projects using asciidoc and getting decent results.

        2. 5

          I think man usage is closely tied to what type of programming and what environment/OS is being worked on.

          If you work mostly on JavaScript on windows almost all of the documentation (command or library-wise) you regularly interact with is going to be on the web. If you are doing C on BSD, then you probably lean on man pages a lot. Most of the devs I know at work so Java on Linux or C# on windows, and I doubt most of either group of them are used to man pages.

          And if there’s a windows implementation of man (not cygwin or WSL), I’d love to see it.

          That being said, mandoc/mdoc is delightful, and I wish it would become more popular and cause the authors to change their mind.

          PS powershell help is infuriating - it seems comprehensive, but built for a different thought process than I’ve been able to force myself into. Tips would be welcome.

          1. 7

            If you work mostly on JavaScript

            Indeed, I think this goes for web technologies in general. They don’t use man pages because they are not UNIX tools. The same goes for C++ and Java to some extent. No problem there.

            But when JavaScript developers start building UNIX tools, they should write man pages for them, because the man page is part of what a UNIX tool is. UNIX users expect their programs to have manuals.

            And if there’s a windows implementation of man

            I’m not sure to what extent the man program itself works on Windows, with how it locates man files across the system, but I know mandoc, which man uses to render man pages, is ported. You don’t need Cygwin or WSL to use it.

          2. 4

            To be fair if your only experience with man pages are Linux’s then you probably would almost always skip them.

            It was only when I started using OpenBSD that I realised the true power of the manpage.

            1. 4

              I’ve used both OpenBSD and various distributions of Linux, and I think this is somewhat true, but not entirely. OpenBSD does make a greater effort to document the system as such, which is difficult to do for a Linux distribution, because it uses a bunch of different parts written by different people. But reasonably, program documentation must be pretty much the same on both systems, at least for programs that exist on both. Linux also has a lot of good documentation for C functions.

              1. 4

                ^^ this. I literally have a dedicated vertically oriented “man screen” that synchs from word-at-cursor with a vim keybinding against the OpenBSD man pages first, with a few exceptions (ptrace, mmap, …) even if I am on a Linux machine a the moment.

              2. 3

                I’ve always got the feeling that the man pages themselfs are great but navigating them is not where it could be. Try finding something in man bash. All the information is there but you just can’t find it if you’re not already experienced.

                And ironically understanding less(1) through its man page is horrible. There are just too many not that useful esoteric options, commands, special notations for key combinations etc. pp. for a tool with such a simple and important purpose.

                Using manpages should be dead simple (like markdown for example).

                1. 2

                  My main issue with man is that I often struggle to find exactly the flag I need, as they tend to be rather lengthy. However, on a --help, I can search on my terminal and often find what I need in less steps.

                  As for your point, you should definitely discuss it with them, then the post can be improved for everyone else.

                  1. 12

                    FYI, man usually supports / to search

                    1. 4

                      Yes, but still, there is way more information, so it becomes harder to find exactly what I need. Sometimes such a simple search for a substring doesn’t cut it (if less has more fancy searches, I’m not aware of them).

                      1. 1

                        Yes, man, at least mandoc, supports tags – type ‘:t’ to jump. It also supports full text search across all manpages at once, using either whatis or apropos.

                        For writing manpages, there are also more friendly options than troff these days. For example, scdoc is reasonable. Here’s an example of scdoc documenting its own file format: https://git.sr.ht/~sircmpwn/scdoc/tree/master/scdoc.5.scd

                    2. 3

                      As was said, less (the pager man usually uses) supports / for searching and & for grep-like filtering. Much more convenient, in my opinion, than running --help output through grep.

                      That said, --help is fine for a concise specification of the tool’s options. It is not meant for more descriptive explanations. That’s where man pages are useful.

                    3. 1

                      I use app help all which prints out all help topics, and by piping that to less or your pager of choice, you’ve got essentially the same as you would have with a manpage, including formatting.

                      I could work on also providing a manpage with more or less the same text, but … what’s the point? I suppose that man app is kinda convenient, but it’s a very small convenience and comparatively a lot of work.

                      1. 4

                        The problem is that your x help all interface will always be inferior to man x because it is not a consistent interface that the user can expect every program to work with. I don’t think we appreciate that enough – being able to type man x to read a well-written documentation for every program on the system is a dream come true.

                        By not providing man pages, the developer not only annoys the user; he lowers the user’s expectation that man x will work for any x, which causes the user to look elsewhere for documentation in general, making man less popular, which in turn developers see as an excuse not to write man pages, which causes users not to expect man pages to be available, and so on forever, in a vicious circle. The more UNIX tools that are created without a man page, the less the man ecosystem will prosper.

                        In other words, I think providing man pages is responsible and commendable. For once in the UNIX world, there is a canonical and unified way of doing something. It would be a tremendous loss if we reverted to a thousand different and mutually incompatible help systems.

                        1. 2

                          Manpages aren’t consistent either; conventions differ widely. Here are four manpages from some fairly standard tools (curl, ls, top, xrandr), and the formatting of the flags is all different. I didn’t have to try many manpages to get this either: just load 4 tools from 4 different authors/ecosystems.

                          I’m not supposed to generate these manpages either according to your previous comment, so I’m supposed to endlessly muck about with this unreadable and uneditable troff nonsense for every update, which would be a duplicate of the inline CLI help and a massive waste of time IMO, just so you can type man app. Call it “lazy” if you will, but if we want to sling those sort of adjectives around then not wanting to type app help all – which gives almost identical results to man app – is much lazier, as the amount of effort for that is minimal, whereas the effort for writing non-generated manpages is pretty large.

                          1. 2

                            The formatting in those example is somewhat different, but mostly the same. The differences seem fairly irrelevant because I’ve read all of those man pages before and never noticed that they were slightly different in how they use indentation and bold text. You could say there’s a forest here to be seen beyond the trees.

                            What I said about generation in my original comment only goes for what one might call lazy and ignorant generation, characterized by these things:

                            • The end result does not read like a man page, because it is generated from a type of documentation entirely unlike a man page.
                            • The developer is largely clueless about troff/mandoc and the man page system and trusts a generation system written by somebody else, who might or might not know what they’re doing.

                            This is the type of generation I call lazy. Not generation in general. In fact, troff is a great target for generation – if you know what you’re doing. Your help x/y/z documentation, which I am not saying is bad, might be generated to troff with great success, but it depends on how it is written, and it must be done with care.

                            I realize it might sound a bit demanding: not only must you provide a man page, it must also be well-made. But naturally, if I think man pages should be provided, I also think they should be good. The way I see it, the user is always going to be lazy, but the developer should try not to be, and it is noble if one tries.

                            (Besides, developers regularly work with all kinds of arcane languages, not least the UNIX shell. Troff is just another one, and a relatively simple one at that, especially if you stick to m(an)doc, which doesn’t actually use the troff program per se AFAIK.)

                            (By the way, Git is a good example of a UNIX tool that has man pages like help x/y/z, except they’re man git-x/y/z.)

                            1. 3

                              But the result are exactly the same. Why spend a day (optimistic, will probably be more) writing some troff tool? Just because I know one “arcane” language doesn’t mean I should learn another just to fit the personal preference of some random person on the internet. Life is short. There are a lot of things of value I’d like to do. Dealing with troff is not one of them.

                              Good documentation doesn’t depend on the format; that’s just a boring implementation detail. Docs in HTML or PDF is far from my favourite format, but I’ll choose well-written HTML or PDF docs over some manpage any day, especially if that makes things easier for the author. Good documentation depends on clear writing, which is hard and time-consuming, so it’s probably best to spend time there than on technical implementation stuff. If manpages work for you: great, go for it! If something else works for you: use that.

                              1. 2

                                But the result are exactly the same.

                                What does this refer to? Maybe you can clarify.

                                Regarding format, it is not just an implementation detail. Man pages can be compared with academic papers, which also follow a more or less standardized format. People expect an abstract, an introduction, methodology, background, results and so forth. This format may sometimes feel like a burden for the authors, but ultimately, it helps them. And paradoxically, creativity tends to flourish precisely when it must work under restrictions.

                                I’ll choose well-written HTML or PDF docs over some manpage any day, especially if that makes things easier for the author

                                I think most UNIX users would be find it very annoying if some of their programs were documented in HTML, other in PDF, yet others in mdoc, etc. etc. Man is valuable because it is a universal documentation system. That doesn’t mean it has to be the only documentation system. Very complicated programs, such as Emacs, cannot and should probably not be documented entirely in man.

                                If manpages work for you: great, go for it! If something else works for you: use that.

                                Again, from the perspective I’ve represented here, I don’t think this is a proper way of thinking about it.

                                If man pages are an intrinsic part of what UNIX tools are, which I argue, then not providing a man page for a UNIX program is akin to not providing an installer for a Windows program. Some Windows programs are provided as zip files, which the user must install manually, but that is not in line with what Windows programs should be.

                                I think we should not move towards more fragmentation here. It would surely be easier for developers, because it is always easier (or at least it feels easier) to work without restrictions, but would be a great loss for users and the UNIX ecosystem in general.

                                1. 3

                                  But the result are exactly the same.

                                  What does this refer to? Maybe you can clarify.

                                  That a help command (or -h/--help flag) doesn’t need to be that different from what you get from a manpage; see e.g. this or this. You could argue this or that should be bold or underlined or whatnot, but I’d format it more or less the same in a manpage (not a huge fan of a lot of bold/underline text), and the principle is the same: short, concise plain-text documentation.

                                  The only tool that I still maintain with a manpage essentially just duplicates the --help output. It’s fairly pointless IMO, and just increases the maintenance burden for little reason (the --help output is actually better because it’s easier to align stuff better).


                                  Manpages haven’t been “the standard” for years; loads of CLI tools come without manpages, or essentially just manpage for the sake of it but without the full (or even useful) docs. GNU tools actually do this: the full documentation is usually available in info; for example yesterday I wanted to set the time to the year 2100 to test some potential 2038 bug, and date(1) has just:

                                     -s, --set=STRING
                                            set time described by STRING
                                  

                                  STRING … yeah, useful … what kind of string? It’s not really mentioned anywhere, but info date has much more detailed (actually useful) docs on this. This is common for a lot of CLI tools, not just GNU ones (although only GNU tools use that weird info thing).


                                  Your entire argument is essentially just “I prefer it like this, therefore everyone should be doing it like this”. It’s fine to prefer things and argue the advantages of this – I do this all the time – but I really don’t like it when people go around telling me what I “should” be doing with my spare time, that I’m “lazy”, or that it’s not “the proper way to think about it”. I have no doubt this was not your intention, but it comes off as fairly unpleasant and even aggressive to me. Do with that feedback what you will.

                                  1. 1
                                    1. Thanks for clarifying.

                                    2. I still maintain that man pages are a de facto standard, and I think your example proves that. Otherwise, why would the GNU tools even have man pages?

                                      Further, I would estimate that the majority of CLI tools packaged by popular Linux distributions actually do come with man pages. Sometimes, package maintainers even add them manually. Why would they do this, if they didn’t have an idea that man pages are or at least should be a standard documentation system that their users can rely on being available?

                                      I think your argument is akin to saying that there is no “standard” sense of morality or values within a culture, just because some people within that culture disagree with it. Democracy and liberalism are “standard” values in most Western countries even if some neo-fascists who live there disagree with it.

                                      A couple of exceptions don’t disprove the rule, and even large disobedience to a standard (e.g. the ancient Israelites) does not remove the standard (i.e. their ideal law).

                                    3. No, I argue for man because I think it is a valuable standard. I don’t think it’s perfect, it’s just okay. In fact, some mix between man and info would perhaps be better, in theory. But in practice man is the closest we’ve got.

                                      That’s why I think UNIX developers should put in the effort to write man pages, because I care about the preserving the man system.

                                    Sorry if I came across as arrogant. I tried as much as I could to argue as clearly and convincingly as possible and limit the claims of my own opinions by saying “I think”.

                                    I think this discussion has reached its limits, at least for now, but I’d be glad to discuss this some other time and really try to understand each other. Dank u voor de discussie!

                                    1. 1

                                      By the way, one of the things I’ve been meaning to write for years is a “unified documentation viewer”; typing doc ls will display the info page if it exists, or the man page if it exists, or the output of ls -h and/or ls --help if that works, etc. This would also include native support for rendering asciidoc and/or Markdown, as well as a pager that makes some things a bit easier.

                                      I think something along those lines is probably a more useful path forward than telling people to write manpages, and if done well it probably has the potential to outright replace manpages as the de-facto standard Unix documentation tool. I mean, there’s a reason people aren’t writing them, and addressing these reasons is more helpful (I have similar feelings to the “you should use {IRC,Email,…}” arguments that people use).

                                      1. 1

                                        doc ls

                                        That’s not a bad idea. I’d be very interested in such a tool. It would certainly be very useful, sort of along the lines of tldr.

                                        there’s a reason people aren’t writing them, and addressing these reasons is more helpful

                                        Interestingly, we both seem to take a pragmatic approach to the way we think about these things. In my case, I think that neither e-mail and man are perfect, but nonetheless I care about preserving them because they’re already so widespread, and having a standard, regardless of how imperfect that standard is, is better than fragmentation.

                                        I agree that it is ultimately ineffective to yell at people to use e-mail or man, when they’d be more persuaded if e-mail/man were actually made easier to use, which they certainly can.

                                        Sometimes, though, people just think that man is hard to use. They haven’t actually read the mandoc/mdoc manuals or really tried to find any information on how man pages are created. They’ve ruled out man(doc) beforehand, simply because a lot of people online, who in turn might not have any actual experience with man(doc) either, say that man is not modern or is hard to use or that not enough people read man pages anyway (for example, the page discussed in this thread).

                                        In these cases, while it isn’t helpful to yell at people, it might be helpful to suggest to them that their prejudices about man are incorrect and that they should look into it honestly and reassess it.

                                2. 1

                                  I wonder what @johnaj thinks of help2man or if he has another suggestion. help2man is what I’ve always suggested to cligen users.

                                  It might be “not so hard” to write in a common sub-dialect that winds up formatting well-ish in both help & man formats. Then muscle-memory man cmd folks and --help folks can all get along with minimal extra effort. Maybe sometimes a little auto-postprocessing on the output of the generated troff stuff could help.

                        2. 1

                          This has been one of our more controversial suggestions with reviewers too! This guide is intentionally meant to be a bit opinionated and to open discussion, so I’m enjoying the debate. ;)

                        1. 4

                          Text is very large and it doesn’t let me resize it. Makes it very hard for me to read. :/

                          1. 2

                            I haven’t had that problem but there’s a GitHub repo where you can log an issue about it.

                            1. 2

                              Oof, sorry. We’ve some new fancy CSS sizing things and clearly this isn’t working properly. Would you mind submitting a bug with a screenshot?

                              1. 1