1. 20
      Maybe
    
    
      Less
    
     
      Spread out?
    
    1. 1

      no ahahahha

    1. 1

      Are there any benchmarks for this?

      1. 2

        Only thing I see is the the performance graph on the author’s page here: https://kristaps.bsd.lv/kcgi/

        1. 1

          I’ve long wanted to update these with some good measurements against, say, PHP. (And on OpenBSD, too.) It’s important to have a solid measure of the performance trade-off between CGI with a compiled binary and the FastCGI clones (Python’s, PHP’s, etc.) alongside the security benefits of ephemeral processes.

          1. 1

            Wow. Thanks for that.

            15msec response sounds like an eternity. My server responds in micros over loopback, so what’s going on?

            Is there an easy way to test this?

        1. 6

          I was so happy in thinking that somebody wrote a performant tsc implementation.

          % time tsc --outFile foo.js foo.ts 
              0m01.93s real     0m02.68s user     0m00.49s system
          

          Ouch.

          % cat foo.ts
          module Shapes {
              export class Rectangle {
                  constructor (public height: number, public width: number) { }
              }
          }
          
          1. 1

            Cryptocurrencies are certainly a… textured subject of economics. But to answer the author’s question of, “How does a non-mathematician judge these things?”, well, for starters, perhaps by engaging reputable sources. Such as, say, one of those mathematicians, or perhaps an economist qualified to navigate the subject matter. The author, however, does not appear to be one of either. How did he judge?

            1. 2

              This is such a nightmare, but that’s ok, because it’s a subset of the larger CalDAV nightmare. (Inconsistent formats, inconsistent standards implementations, etc.) In kcaldav, I ended up parsing and ignoring rrules, and just fetching all entries on collection requests because computing recurrences within intervals is complicated. Let the client sort it out.

              Whenever I get back to this (unfortunately, “it just works” for my tiny calendar, which is the enemy of “make it work better”), I’ve scratched the surface (in datetime.c) with a way to bisect recurring intervals from a rule until on falls within a duration, and simply bound durations below to, say, the daily or hourly level. Either way, I’d need to pull ever rrule from the database and run this routine.

              If you’re interested in dirtying your hands with CalDAV, it’s all there, but certainly not easy.

              1. 7

                Why not just directly write man(7), which is all this tool produces? Or use the existing perlpod, pandoc, docbook, lowdown, rst2man, or any other tool doing exactly the same thing from diverse formats?

                Because I’m sure the world needs more opaque, un-indexable manpages.

                (Edit: to clarify, use mdoc(7).)

                1. 5

                  Author here. Did you even read the blog post? I answered all of these questions.

                  perlpod is built on a mountain of perl, and pandoc on a mountain of haskell. lowdown is a Markdown implementation, and Markdown and roff are mutually exclusive. RST and roff are mutually exclusive. I spoke about docbook directly in my article (via asciidoc, which is a docbook frontend). I also directly addressed mdoc.

                  Man pages are already being indexed. If you search the web for “man [anything]” you’ll find numerous websites which scrape packages and convert the roff into HTML.

                  1. 1

                    Thanks for your hack. It’s a good candidate for a port in my little os.

                    A couple of question:

                    • have you considered to avoid the bold markers around man page refs as you already have the parentheses to identify the reference?
                    • also section titles have conventional names: what about omitting the starting sharp to mark them as titles?
                    • what about definition lists? (I know they are an HTML thing, but they can be useful to describe options for example)
                    • I know tables are the most difficult format to express in a readable source form, but what alternatives did you considered and why you discarded them?

                    And btw… Thanks again!

                    1. 2

                      Glad you like it!

                      have you considered to avoid the bold markers around man page refs as you already have the parentheses to identify the reference?

                      This is an interesting thought. https://todo.sr.ht/~sircmpwn/scdoc/12

                      also section titles have conventional names: what about omitting the starting sharp to mark them as titles?

                      I’m not fond of this idea. Given that lots of man pages will need to have section titles which fall outside of the conventinoal names, and that I want all headers to look the same, this isn’t the best design imo.

                      what about definition lists? (I know they are an HTML thing, but they can be useful to describe options for example)

                      man pages do “definition lists” with borderless tables, which are possible to write with scdoc like this

                      |[ *topic*
                      :[ definition
                      |  *topic
                      :  definition
                      # etc
                      

                      I know tables are the most difficult format to express in a readable source form, but what alternatives did you considered and why you discarded them?

                      The main approach I’ve seen elsewhere is trying to use something resembling ascii art to make tables look like tables in the source document. I’ve never been fond of this because you then have to do annoying edits when updating the table to keep all of the artsy shit intact, which in addition to being just plain annoying can also bloat your diffs, lead to more frequent merge conflicts, etc.

                      An alternative some formats have used is to make aligning your columns optional, but still using an artsy-fartsy kind of style. I figure that if you’re going to make aligning the columns optional you no longer have any reason to require a verbose format like that. So I invented something more concise.

                      Also, the troff preprocessor used for tables supports column alignment specifiers and various border styles, which I wanted to expose to the user in a concise way. Other plaintext table formats often have this feature but never concise.

                      1. 1

                        man pages do “definition lists” with borderless tables

                        Do you think you could render something like this with scdoc in a source-readable way http://man7.org/linux/man-pages/man8/parted.8.html (see section OPTIONS and COMMAND)?

                        The main approach I’ve seen elsewhere is trying to use something resembling ascii art to make tables look like tables in the source document.

                        Actually it was what I was thinking about. You propose a good point, but my counter argument is that manual pages are (hopefully) read more often then they are written. But I admit that my goal is people using cat to read manual pages by default, so I can see how in a more conventional system using Troff the people most often read a rendered page, thus the annoyance is pointless. OTOH, it should be relatively easy to write a tool that take scdoc document as input and output another scdoc document where tables are automatically aligned, removing the annoyance to align the cells while writing.

                        Having said that, I find your table syntax nice.
                        I wonder if one could nest tables (I mean put a table in a cell). Also, you organize the table by rows, but given the format, some table might benefit from being organized by column.

                        1. 2

                          Do you think you could render something like this with scdoc in a source-readable way http://man7.org/linux/man-pages/man8/parted.8.html (see section OPTIONS and COMMAND)?

                          You don’t actually even need tables for this. scdoc preserves your indent. https://sr.ht/I0g7.txt

                          I wonder if one could nest tables (I mean put a table in a cell). Also, you organize the table by rows, but given the format, some table might benefit from being organized by column.

                          I think nested tables is a WONTFIX. Also not sold on column-oriented tables. IMO man pages should be careful to keep their tables fairly narrow to stay within 80 characters.

                          1. 1

                            Wow, that’s really readable!

                            Fine for nested tables. Just to be sure I explained what I meant by column-oriented (that just like nested tables might or might not be a good idea): suppose you want to create something like

                            English    Italian    Swahili
                            Hello!     Ciao!      Habari?
                            Tour       Viaggio    Safari
                            Lion       Leone      Simba
                            

                            You might prefer a syntax like

                            |[ English
                            :[ Hello!
                            :[ Tour
                            :[ Lion
                            |[ Italian
                            :[ Ciao!
                            :[ Viaggio
                            :[ Leone
                            |[ Swahili
                            :[ Habari?
                            :[ Safari
                            :[ Simba
                            

                            Or even, for such a simple table (that I don’t know if actually exists in a man page, so…), you could put each column (or row) in the same line:

                            |[ English :[ Hello! :[ Tour :[ Lion
                            |[ Italian :[ Ciao! :[ Viaggio :[ Leone
                            |[ Swahili :[ Habari? :[ Safari :[ Simba
                            

                            (that a tool could easily turn into:

                            |[ English :[ Hello!  :[ Tour    :[ Lion
                            |[ Italian :[ Ciao!   :[ Viaggio :[ Leone
                            |[ Swahili :[ Habari? :[ Safari  :[ Simba
                            

                            )

                            Ok… now I’ve really annoyed you enough for a single night… good work!

                  2. 5

                    Because you cannot have progress without research.

                    Now troff is not readable in source form.
                    This is better in this regard. You are right about indexing, but the project have a very short log. I guess we can talk about it with the author, and see what he think about that.

                    Maybe he like the idea, and add it. Or he doesn’t, and will not add it.
                    You will always be able to fork it and fine tune to you need.

                    I’m grateful to hackers who challenge the status quo.

                    1. 4

                      While mdoc(7) is great (thanks for that!) , I think your questions are answered on the page. I think lowdown is probably the closest to what u/SirCmpwn was aiming for (no dependencies, man output), maybe they hadn’t seen it?

                      Man formatting is inscrutable to the un-trained eye (most people), and we need to acknowledge the popularity of markdown is related to its ease of reading/writing.

                      1. 4

                        I think your questions are answered on the page. I think lowdown is probably the closest to what u/SirCmpwn was aiming for (no dependencies, man output), maybe they hadn’t seen it?

                        groff (as installed on every Linux distribution that uses groff for man pages, which is basically all of them, and macOS) has had native support for mdoc for at least a decade. If you install an mdoc man page and then man $thepage, you get exactly what you expect.

                    1. 3

                      I aspire to handle criticism (and praise) as well as the author. (See the comments below the article.) Well done.

                      1. 9

                        I don’t know who the author is, but what professional says things like, “We all hate writing documentation”? It sounds like a TV show representation of a “programmer”. (Badges? Readmes? Really?) Is it just in the BSD community where we’ve grown up to accept (and love) that documentation is a vital part of any tool?

                        1. 4

                          Yeah, the author seems to have a very strange model of how to develop software in his head. I browsed around the site a bit and it’s all the way down the “programmers as interchangeable idiots who should do as they’re told” path.

                        1. 1

                          Yes to both! As for the “practical” book, it just needs to be checked for correctness. Is this advice given reasonable? Are the examples true to the title? And so on.

                          1. 1

                            I note (again) that I’m looking for somebody to maintain this… (It needs a lot of love!)

                            1. 1

                              Does that include the History of UNIX Manpages, too? Some links are dead, at least.

                              And what would actually need changing? I thought the general man page format and mdoc change once in a blue moon.

                            1. 7

                              This topic resonates with me as the author of a Markdown utility. But I think it completely misses the point at what’s such a PITA about Markdown. And that’s not its annotational trespasses.

                              The problem is that Markdown’s just a human-readable subset of HTML. It’s like writing in HTML without being able to use classes, semantic markup, metadata, IDs, forms, and so on. The original Markdown accomodates for this by allowing HTML directly in the input. So basically the Markdown because a textual preprocessor. Then there’s another problem: have you ever looked at an HTML-only document? No styling? So there’s a bundled cost in needing the CSS and embedded HTML (or templating HTML) for any (not non-trivial—again, look at the basic HTML document) document formatting. Or just use a Markdown extension or implementation that does it for you… which is another problem.

                              All of the extensions and implementations balkanising the language try to sit between “presentation-ready HTML” and the original markdown. Some are reasonably standard, like metadata or tables. Then we have equations, footnotes, charts, video, etc., etc. Each of which has been implemented several ways. (Which table format would you like today?) And each one is called “Markdown”. Great! And then, some extensions clobber the original language—like metadata, which appropriates the first paragraph.

                              In my opinion, this is just going to go the way of any commodity: the largest implementation will rule, and the rest will follow. Hopefully folks will come to their senses and just use HTML, but “sense”, unfortunately, is not found in the same quantities as Markdown implementations. Mine included. :)

                              1. 1

                                Looks great.

                                Any chance you can expand the last FastCGI example with the actual error handling code? I think it would be beneficial as a discussion point of good error handling as well as provide a full production-ready example.

                                1. 2

                                  Good point. I’ll put something in—thanks!

                                1. 4

                                  One thing that puts me off from using kore is the degree of magic that seems to come with being a “framework”. No main function. An obscure documentation system. A baked-in build system. A baked-in web server. That’s a lot of duplication of mature, well-known tools—make, httpd, and so on—with none of them being trivial at all. I guess C to me is synonymous with UNIX, and frameworks like this go against what UNIX means: manpages, doing a small thing well and fitting into a larger framework, etc. It would be nice, all that being said, to split these tools apart and use them separately. (A well-written HTTP server library would be very handy—and there are a lot of questionable ones out there.)

                                  1. 5

                                    Kore author here.

                                    You must have used Kore ages ago. The build tool and web server are two separate things and have been for a while.

                                    If you don’t want to use the build tool to help you get started, automatically properly build your app or have any benefits from it, you can roll your own Makefiles. The applications are normal dso files mapped into the server its address space by dlopen() anyway. Those aren’t magic.

                                    Not having a main function sort of goes hand in hand with the fact your apps are dso’s.

                                    Yes there are more things the build tool can help you with, like injecting assets or building a single binary out of your application instead of a dso.

                                    I fully agree that the 2 year old documentation is shit, and that’s something I’m fixing for the next release :)

                                    1. 3

                                      Not to mention writing a web app in C sounds like a masochistic security mine field due to all the string processing it normally entails. C++ wouldn’t be as bad with std::string, but even then it sounds dreadful.

                                      So it’s not for me, but it still looks like a solid project, and it’s filling an interesting niche.

                                      1. 6

                                        I don’t think kristapsdz got the memo about not writing web apps in C. https://learnbchs.org/ ;-)

                                    1. 10

                                      This is true in any language, unfortunately. Choice quote that resonates with anybody who’s worked with GTK: “if you just want a few buttons and some text, using GTK is like mowing a lawn with a helicopter.” (GTK is usable at all thanks entirely to glade.)

                                      1. 13

                                        Absolutely doesn’t resonate with me. Came here to complain about this quote actually.

                                        I don’t understand this reasoning. Having one toolkit that scales from a few buttons to GIMP/Inkscape is the point of any reasonable toolkit. And it’s not like GTK always requires a complicated setup or whatever. Creating a window is really simple. Adding a button looks like, well, adding a button.

                                        By the way, the problem with OCaml is that no one made a working gobject-introspection library, and it’s stuck with GTK2 :(

                                        1. 2

                                          I get some stuff - some simple use cases are fine, but other ones are complex - for example, if you want a simple list or tree view, you have to essentially implement a bunch of MVC boilerplate to do so.

                                        2. 5

                                          Unlike GTK (at least I think) some GUI are built as a separate rendering program that communicate through a stream, so it is possible to change that program for another. But does it solve the problem?

                                          The web grew way too complex to permit a trivial implementation.

                                          Ncurses somehow feels bloated too. Though it can be bypassed: ANSI escape sequences seems to be supported by most (if not all) emulators.

                                          https://github.com/antirez/kilo

                                          1. 2

                                            The only exception I can think of is with Racket; writing cross-platform code that uses GTK is quite pleasant with that tool set. https://gitlab.com/technomancy/world-color/blob/master/world-color.rkt

                                          1. 5

                                            Those are some pretty flaky arguments regarding OpenBSD. What is “theoretical” SMP? I’m running this from a 4-core OpenBSD laptop. You know, non-theoretically. Same language snark goes with vmm: they tried to implement a hypervisor? I’ll be sure to inform mlarkin of his failure to execute. It may not be what the author wants, but that’s a different story. Anyway, if there are good comparisons between the two systems security-wise, they look like they’re in that chart from https://hardenedbsd.org/content/easy-feature-comparison. Is it up to date with the recent anti-ROP efforts?

                                            1. 2

                                              It is. OpenBSD has an SROP mitigation, whereas HardenedBSD doesn’t. HardenedBSD has non-Cross-DSO CFI (Cross-DSO CFI is actively being worked on), whereas OpenBSD doesn’t. HardenedBSD also applies SafeStack to applications in base. CFI provides forward-edge safety while SafeStack provides backward-edge safety (at least, according to llvm’s own documentation.)

                                              HardenedBSD inherits MAP_STACK from FreeBSD. The one thing about OpenBSD’s MAP_STACK implementation that HardenedBSD may lack (I need to verify) is that the stack registers (rsp/rbp) is checked during syscall enter to ensure it points to a valid MAP_STACK region. If FreeBSD’s syscall implementation doesn’t do this already, doing so would be a good addition in HardenedBSD.

                                              So, there’s room for improvement by both BSDs, as should be expected. It looks like OpenBSD is starting the migration towards an llvm toolchain, which would allow OpenBSD to catch up to HardenedBSD with regards to CFI and SafeStack.

                                              Sorry for the excessive use of commas. I enjoy them perhaps a bit too much. ;)

                                              1. 1

                                                I haven’t read the whole article, because I’m not interested in HardenedBSD.

                                                What is “theoretical” SMP? I’m running this from a 4-core OpenBSD laptop. You know, non-theoretically.

                                                The article is indeed vague about it, but I think the author meant scalability issues. Too much time spent in the kernel space.

                                                Same language snark goes with vmm: they tried to implement a hypervisor? I’ll be sure to inform mlarkin of his failure to execute.

                                                I don’t have any experience with virtualization, but the point seems to be that you can only have OpenBSD and Linux guests under an OpenBSD host which compares less than something like bhyve.

                                                1. 1

                                                  SMP

                                                  From what I have read about SMP on OpenBSD its not that it would not detect 4 or 64 cores, its that its subsystems (like FreeBSD 5.0 for example) were not entirely rewritten to fully itilize all cores, that in many places still so called GIANT LOCK is used, may have changed recently, sorry if information is not up to latest date.

                                                  vmm

                                                  Now ints very limited, can You run Windows VM on it? … or Solaris VM? Last I read about it only OpenBSD and Linux VMs worked.

                                                  Is it up to date with the recent anti-ROP efforts?

                                                  I am not sure, You may ask here - https://www.twitter.com/HardenedBSD - or on the HardenedBSD forums - https://groups.google.com/a/hardenedbsd.org/forum/#!forum/users

                                                  1. 3

                                                    or Solaris VM? Last I read about it only OpenBSD and Linux VMs worked.

                                                    It runs Illumos derivatives (eg. OpenIndiana). There’s a speicific feature missing that FreeBSD/NetBSD need which is being worked on. It doesn’t run Windows because Windows needs graphics.

                                                    1. 2

                                                      Thanks for clarification, I hope that graphics support/emulation will also came to vmm soon.

                                                      I added that information to the post.

                                                  2. 1

                                                    I’m not sure, the article seems like it makes an honest enough comparison between hardenedBSD and OpenBSD that I make OpenBSD a priority to consider the next time I need truly secure OS.

                                                    1. 3

                                                      The “One may ask…” paragraph is so slanted toward HardenedBSD over OpenBSD that I’d have immediately assumed a HardenedBSD developer or fan was writing it.

                                                      1. 1

                                                        Tried my best, I thought that it was clean enough from the article that OpenBSD is secure for sure while HardenedBSD aspires to that target with FreeBSD codebase as start …

                                                      2. 1

                                                        Tried my best, I thought that it was clean enough from the article that OpenBSD is secure for sure while HardenedBSD aspires to that target with FreeBSD codebase as start …

                                                    1. 4

                                                      If you’re interested in make-oriented systems, most of my lowdown sites use sblg for the templating part. By design Makefile-friendly. (Disclaimer: I wrote lowdown. And sblg.)

                                                      Edit: simple example being divelog.blue.

                                                      1. 2

                                                        Hi Kristaps!

                                                        Thank you for your awesome software. I’m a fan of BCHS and still learning C. I would use sblg too—it’s super fast and simple—but I needed some flexibility with rendering of my pages.

                                                      1. 6

                                                        If anybody started reading this and was very, very confused, in the following word, the word “word” is really “passage”, and “passage” means “improved word”. Unlike, of course, just “word” as you know it to mean, in which case I’m referring to a word in a “passage”.

                                                        “First of all, let me clear one important thing that pledge(1) and pledge(2) both are different things according to the man page of OpenBSD. Because parenthesis numbers indicate sections of a man page, like, (2) for system calls etc. So, here, parenthesis numbers are only to differentiate between the old pledge and new pledge or improved pledge, that’s all, nothing related to the parenthesis numbers of man page of OpenBSD.”

                                                        I think there should be a filter for medium posts. Sigh.

                                                        1. 1

                                                          I don’t think that there are lots of confusions now. I know that first pledge(1) and pledge(2) were creating confusions, but then after that, I have changed the title and also written that paragraph for clarification. I too was thinking regarding this confusion, that how should I differentiate between the old pledge and improved pledge, so, I thought it would be great if I will try to explain what is what.

                                                          But, if it is still confusing, then I apologize for my mistake. From on next time, I will keep these things in mind.

                                                          1. 2

                                                            It’s not your explanation that’s confusing—that’s just me being acerbic—it’s the notation itself. Consider using “pledge v.1” or “pledge v.2”, or better yet, “pledge 6.0” or “pledge 6.3” to refer to the OpenBSD version containing the described version. Then you can drop the explanations altogether and focus on the matter at hand.

                                                            (And don’t forget pledge 5.8: tame!)

                                                            1. 1

                                                              Oh yeah, you are right. I can also use “pledge v.1 or pledge v.2 “ or “pledge 6.2”, that’s nice. even I had asked for suggestions for this on Google+, but, no one gave.

                                                              Thank you very much for the clarification. I will update them. :)

                                                        1. 4

                                                          I absolutely agree with “developers need to pay attention to build flags”. Like the well-known -D_GLIBCXX_ASSERTIONS and -fasynchronous-unwind-tables. Only applicable, of course, when using GCC. I mean only when using GCC when this article was written. Only on Linux. With glibc. Did I mention only RedHat Enterprise? (AMD64?) (C or C++?)

                                                          For those of us C programmers writing for anything other than the intersection of the above, please, please, please “pay attention to” -W, -Wall, and if you’re feeling bold, -Wextra. (And everybody’s friend, -g.) Depending upon obscure GCC or Clang flags only muddies the waters—as if we need more difficulties keeping C software portable between Linux, BSD, and Mac OS X. (I don’t have the courage to write “Windows” here.)

                                                          Maybe the author should clarify to “HPC on Linux developers” or “kernel developers”? Or more practically, “RedHat Enterprise developers”?

                                                          1. 1

                                                            I also almost always use -pedantic, too.

                                                          1. 5

                                                            Nice timing :D - I just submitted www/kwebapp to ports@ !

                                                            1. 5

                                                              Great! It does need the newest versions of ksql+kcgi to compile resulting code… I write them all in tandem. (I recently had a bug report from somebody trying kwebapp with ksql from ports.)

                                                              1. 4

                                                                ksql is currently on 0.2.3. I ping’d MAINTAINER.

                                                            1. 7

                                                              Does anyone have pointers to background info on the issue with FreeBSD/NetBSD needing an instruction emulator, for something related to console I/O? Seems like they must be doing something fairly unique there if OpenBSD was able to emulate all these other OSs without needing an instruction emulator, but now needs one for those two.

                                                              1. 1

                                                                Wild guess: it’s the rep prefix.

                                                                1. 3

                                                                  I think it’s just the outsb part - copies data from memory to an I/O port, which is obviously not “real” in a VM so the hypervisor needs to do something with it.

                                                                  For instance, see how bhyve handles it:

                                                                  • vmm/intel/vmx.c – EXIT_REASON_INOUT saves some state and sets exit code to VM_EXITCODE_INOUT.
                                                                  • vmm/vmm.c vm_run() on VM_EXITCODE_INOUT calls vm_handle_inout():
                                                                  • vmm/vmm_ioport.c - all about emulating IO port calls.
                                                                  1. 3

                                                                    I’m sitting next to mlarkin at AsiaBSDCon and he says “uhh, no, Ted”. :) (Note this went through several edits to come to “uhh, no” so as not to offend other lobsters.) ;)