1. 3

    Hey @kristapsdz, OpenBSD node maintainer here :D

    At one point we had a TS port, but given the wide range of npm modules (and versions there within - sometimes multiple versions of the same module would be used within the same app!) we quickly hit gridlock - where we were unable to update applications and even node itself!

    It’s been a while since I looked, but npm shrinkwrap and the introduction of the package.lock file might be enough to allow for consistent distfile creation.

    If we can build consistent distfiles - ports would be fairly straight forward (basically the same process we use with Go apps and the vendor command - if you are familiar with that).

    1. 2

      You’re like a superman, @qbit, to deal with this behemoth. We’re not worthy! We’re not worthy!

    1. 2

      I think I just found my new tool for editing manpages. I just wish it I had an mdoc reference link in the help menu.

      1. 1

        There’s always

        man mdoc
        

        Plus some templates and a bit of duct tape and things look surprisingly semidecent.

        1. 1

          Is there a HTML version of that though?

            1. 1

              With a bit of muscle grease and perl-fu you can convert any man page to HTML. But that link that @nalzok posted seems to be what you want tho. I just checked it out, useful tutorial and it has a command reference at the bottom!

        1. 3

          If you fancy this, you might like sblg (disclaimer: it’s mine), which merges together HTML pages—template and content—with smarts to handle navigation, tags, and so on. You don’t even need to do the content in HTML: I do many of mine in markdown, bracketing the md->html output with the article tags sblg needs. I’ve used this for slide presentations, websites (e.g., Not Awful UW Photography, and so on. All managed by simple Makefiles.

          1. 2

            I like sblg, and I think it’s better than most static site generators (especially because it doesn’t use Jekyll’s awkward “front matter” system), but for my purposes, most of the same problems still apply.

          1. 8

            I do a lot of underwater photography with the full toolchain—from cameras to dive computers—being on OpenBSD. See my about for how it all works. The results are mostly published here, if that sort of thing interests you.

            tl;dr: shotwell/geeqie, ufraw, gimp. The first for viewing (and the part I dislike the most), the second for RAW colour values, the third for postprocessing (despeckle, sharpening).

            1. 3

              Your dive computer runs OpenBSD? Do tell …

              1. 1

                Obviously not. (Some versions of the HW OSTC have open source components apparently, but I don’t know much about that.) I meant that my dive planner (subsurface) and dive-analysis (home-grown tools interfacing via libdivecomputer) all do.

                My camera also doesn’t run OpenBSD, if that also needs clarification.

                1. 2

                  Oh. Wasn’t clear from the OP.

              2. 2

                I’d be super interested in your diving computer setup…

                1. 1

                  Thanks for the links! Very nice pics.

                  Have you submitted your writeup to Reddit? I think /r/fossphotography would love to see it.

                  1. 10

                    The website is awfully empty but I heard about this a month ago, so this might be more useful:

                    http://www.undeadly.org/cgi?action=article;sid=20190211081518

                    https://github.com/kristapsdz/openrsync

                    1. 6

                      Yes, different stages have been linked to previously:

                      Now, it looks like OpenRsync is on its way to becoming (announced as) an official OpenBSD-associated project, like all the other ones:

                      Not just a piece of software that has been imported into the tree.

                      1. 3

                        This repository is a read-only mirror of a private CVS repository. I use it for issues and pull requests.

                        They sure seem to love CVS in OpenBSD land. Why?

                        1. 1

                          The Interwebs are littered with this question - please use a web search engine of your choosing and let’s not delve into the matter deeper here.

                          Some people use OpenBSD, and not, i.e. macOS; mg(1) instead of Emacs; vi(1) instead of Vim… I guess the answer is usually the same - because it works for them.

                          1. 1

                            I did search the web. The question is common, but the only answers given are pat replies such as yours which don’t actually answer anything.

                            “It works for me” is a cop-out, and makes no sense given that this person clearly is familiar with git yet still chooses to use CVS privately. So it can’t be a case of “it isn’t broken, don’t fix it”, which only makes sense in order to save yak-shaving effort. But yaks are being shaved here in order to use CVS as the source of truth. That’s surprising.

                            “vi vs emacs” is a case of equivalent tools. CVS vs git is not such a case.

                            1. 1

                              What is it you want to know?

                              1. 1

                                I want to know why someone would actively avoid using git in favor of CVS for private use, when one is familiar with both. Even “personal fetish” is a more satisfying answer than the aw-shucks “to each his own” automated tripe.

                                Fetish is at least a conscious choice. Technical reasons would be more interesting. But I suspect the reason is actually license-based, and that’s what I would like to know.

                                1. 3

                                  Sounds like you already have an answer prepared for me, buddy! It’s just a tool that does what I need it to do. I don’t know what the fuss is about. (I can’t speak for OpenBSD—just me.)

                      1. 2

                        It would be wonderful to have a port of this on linux, but for that we really need a solid reimplementation of pledge and unveil.

                        1. 1

                          At least those can be noops without compromising function.

                          1. 4

                            No, they can’t. openrsync functions securely. Take away the security, you take away the functionality.

                            1. 2

                              No, you don’t get the benefit of those particular syscalls. Emacs on OpenBSD is pledged too, that doesn’t mean it can’t function just fine everywhere else without pledge.

                              1. -1

                                so rsync does not function?

                                1. 3

                                  That seems an unreasonably pedantic reading of “security is a feature”.

                                  1. 6

                                    Perhaps, tedu. However, rsync (and openrsync) specifically accommodate for running as root with full access to one’s file system, accepting ill-specified data coming down the wire. It’s chmoding and chowning and renaming. A lot of moving parts. And on stock rsync, without even chroot in there. (Except for, optionally, –daemon.) It’s bare-back. And especially when you’re talking to a public rsync:// daemon, you have no idea what’s running on the other end. Which is usually the case as root, I might add.

                                    I know that @rain1 amongst others have done some analysis in how to effect pledge/unveil security measures on Linux, but folks like the child derail his efforts. This is 2019 and Linux folks are stuck with seccomp-bpf and… namespaces?… it’s a mess, I know. But this attitude of “strip away the security” is harmful and irresponsible and contributes to the CVE culture. Instead of nooping, consider working together with other folks to improve the status quo.

                                    (This last bit isn’t to you, @tedu, but the nooper.)

                                    1. 2

                                      but folks like the child derail his efforts

                                      No. I will welcome a non-noop implementation with joy when it works. For now to get better, saner and safer software (just by the team and policies that created it) I can handle two syscalls nooping.

                                    2. 1

                                      it certainly would be, if that’s what their post said

                            1. 1

                              I don’t do much front end work these days but if I did, I would want a service that hosted all of your images and automatically resized them based on GET parameters, e.g. https://example.com/someimage.png?size=1024x768

                              I’m sure there are tons of products that can do this but it would also be trivial to roll one’s own. It would be easy to write some sort of step into your deploy or build process that figured out the size of the image needed and generated a URL for it. Or do what all the cool kids are doing these days and just design the whole thing in Javascript anyway.

                              1. 2

                                We actually use Cloudinary at work and it does a lot of stuff based on the URL you pass it.

                                1. 2

                                  I use cloudinary for diving photos, e.g., this one. It loads a big, medium, or small image depending upon CSS media queries. I know they have more options, but the documentation is a bit chaotic. (The page itself is generated by sblg, which creates the links for me.)

                                2. 2

                                  I work at Imgix, which does some of that: instead of uploading your images to us, we request your images then render them, with heavy caching. Also usable as a JavaScript library.

                                  1. 3

                                    I was responsible for porting some of our existing code at work to use Imgix. It worked great for us, simplifying a whole class of problems that came serving user-generate content. Reasonably powerful API (so dead simple for us to use: all query string driven). I particularly liked how it acts like a proxy, lazy loads and transforms the images.

                                  2. 1

                                    It’s trivial, unless you care about performance (for many images).

                                    There’s no mature, open source library for generalised GPU conversion (one of these days)…

                                    Formats are inconsistently supported; third party images may be too large to fit in ram once decompressed (even if the file is small); I’m sure I’ve forgotten a few other issues…

                                    1. 1

                                      In many cases (ok, not all) the site owner uploads the image. You’re not going to deliberately dos your own site with a bajillion pixel png. Caching is trivial, etc.

                                      I don’t think it’s theoretical concerns that prevent this from being more commonplace. It’s just friction and intertia. e.g. bad defaults.

                                    2. 1

                                      So, ImageMagick as a service? With caching, rate-limiting, etc? Yeah, I recall a friend of mine doing just that, like 5 years ago. Internally at his job, I mean. No idea if it’s still running.

                                      1. 1

                                        The open source community already has this and it’s called Thumbor

                                        http://www.thumbor.org/

                                      1. 4

                                        Just tested this via rsync over ssh and successfully copied a directory to a remote server. The syntax was almost the same, I used -r instead of -a, like so:

                                        $ openrsync -rv -e ssh /SRC/ example.com:/DST/

                                        I then removed one of the remote files and re-ran openrsync. It appears to have re-uploaded all the files, not just the missing one. Like the project description says, it’s “very new and very fast-moving” so surely it will continue progressing. Nice to see pledge and unveil in there also, especially as they’re not present in the rsync package/port.

                                        I’m impressed! Thank you Kristaps.

                                        1. 2

                                          You’re welcome! :)

                                        1. 3

                                          Please do not make feature requests: I will simply close out the issue.

                                          I wonder why.

                                          1. 14

                                            I’d probably pull off such a thing as well because there is a huge amount of users confusing OSS Maintainers with Commercial Support, which is really draining.

                                            1. 3

                                              Exactly.

                                              1. 2

                                                I understand but I think it would suck if everyone did that way. Sure we can always fork but one fork per feature sounds like a huge waste of resources.

                                                1. 8

                                                  I said feature requests. E.g., “can you please support –exclude”. Code is not a feature request.

                                                  1. 8

                                                    It says you’ll accept their work instead of attempts to get extra work out of you for free. That seems very reasonable. :)

                                                    1. 2

                                                      Yeah I was confusing refusing feature requests with refusing pull requests.

                                                      A feature requests first is a good way to see if the maintainer would accept a PR (and to see if other users wants it) however. Well for the way most of us use Github, I guess we could contact the maintainer instead if his email is on his profile.

                                                      1. 7

                                                        In this case, feel free to ask here! (I’m the maintainer.)

                                                  2. 2

                                                    On the other side, having feature requests open helps newcomers to see what they could help with.

                                                  3. 4

                                                    If you read the rest of the README you pretty much got the answer.

                                                    1. 3

                                                      Because that’s the only way your issue tracker won’t have 3000 open issues and 1000 open PRs in 5 years.

                                                      1. 1

                                                        Yeah because you won’t have users because you’re kind of flipping them off.

                                                      2. 2

                                                        It might be that the projects goals are purely to reach feature parity/compatibility with rsync, thus feature requests would be redundent/unnecessary.

                                                        1. 1

                                                          Probably focusing on just being rsync compatible for now I guess?

                                                          I find it interesting the dev is still using CVS too.

                                                        1. 3

                                                          It is. Thanks for sharing. Really like his static website generator and I am inspired to make something similar. First step… learn shell?

                                                          1. 6

                                                            You can write it in any language. If you’re interested in learning shell or anything web dev related, ask me anything here or on Twitter/Mastodon (DMs are open) ;)

                                                            1. 4

                                                              Kristaps’ sblg is also worth a look. My blog is using sblg and lowdown. Sites are generated on my laptop, then pushed by rdist which is also a nice tool in OpenBSD base

                                                              1. 2

                                                                Thanks! :)

                                                                1. 1

                                                                  cool, I have to check that out as well.

                                                                2. 1

                                                                  yeah, like Romans reply. Anything goes. Any language will do.

                                                                1. 3

                                                                  Author may wish to point out that reading from regular files will wait regardless O_NONBLOCK. (Because the data is technically always “available”.)

                                                                  1. 4

                                                                    And for those of us who use or want to use C/C++ for our Internet (specifically, web) systems, please let me know if https://learnbchs.org is helpful for tools and deployment strategies that aren’t limited to “use rust”. (Disclaimer: I wrote it.)

                                                                    1. 2

                                                                      Hi, yeah, personally I would be put off by that stack? OpenBSD is great and I’ve been meaning to deploying something with httpd for a minute now just to play with it, but using sqlite for a web app is a deal breaker and C as a web language…also a deal breaker. Just not what i would use personally, for a lot of different reasons. But thanks for the link and the site looks great.

                                                                    1. 2

                                                                      man abort on a Debian machine doesn’t give me any jokes. Is there a difference between the glibc html and the manpage? If so—why?

                                                                      1. 3

                                                                        The man page is not associated with Glibc and was written by other authors. It is part of the Linux man pages project. See https://www.gnu.org/software/libc/documentation.html:

                                                                        The glibc project does not maintain any man-pages; instead other projects document the glibc APIs as man-pages. The largest such collection is the Linux man-pages project. The glibc community keeps close ties with the Linux man-pages project, and many developers contribute to both sources of documentation. While the glibc manual remains the canonical source for API descriptions, the man-pages are an excellent reference. Every effort is made to keep the documentation harmonized between both projects.

                                                                        More importantly, though, the Glibc manual is generally more conversational in tone and style. There is no reason why non-technical information from the Glibc manual should be replicated in man pages.

                                                                      1. 1

                                                                        Not sure why the DB has to be on the servers (to be monitored), and not on the client/monitoring server. Personally I try to keep the monitoring workload on the servers as minimal as possible, and do all the aggregation ++ on a separate server.

                                                                        If they had done that, they also could have used snmpd from the base system, without the need for writing a collector daemon.

                                                                        1. 3

                                                                          It’s on both. The database is really just a set of circular buffers that’s transmitted in totality. But it’s on the system being monitored because hey, you might turn off the monitor. And then your history is gone. Also, even if the monitor is running, the network might fail.

                                                                          1. 3

                                                                            This is something I wish the Prometheus people had factored in to their design. I’d love to be able to restart the collectors or have network disruptions without losing data points. A locally spooled catch-up ring buffer is a great idea.

                                                                            1. 1

                                                                              A locally spooled catch-up ring buffer is a great idea.

                                                                              Would you persist it to disk? Just in case the monitored server crashes while the buffer is not empty.

                                                                              1. 2

                                                                                Persisting to disk would definitely be something I’d like to explore. I think the downside might be that it induces extra I/O load and storage pressure on what might already be a busy system.

                                                                                1. 1

                                                                                  I’m not sure you’re referring to slant, but it leaves the database on the disc. The point of slant is to be a simple tool for normative hosts: if your systems are so significantly under load, you probably want to do something like snmp.

                                                                                  That being said, I’m happy to work with folks with patches to make slant more performant. I know that ksql has lots of room for improvement!

                                                                                  1. 2

                                                                                    I’m not sure you’re referring to slant, but it leaves the database on the disc.

                                                                                    Sorry, I was talking more in the abstract – we’re using the Prometheus ecosystem pretty heavily at the moment.

                                                                                    It’s neat that slant persists to disk!

                                                                        1. 3

                                                                          One reason I think sblg isn’t as popular as this kind of tool is the auto-theme part. Like, being able to do git clone to pull down a template Makefile, blog-template.xml, and article-template.xml with associated CSS files with some kind of unifying theme. Is that something that folks use, or does one usually start from scratch when designing a blog site?

                                                                          1. 2

                                                                            I don’t know, I don’t think themes (as in layout) are important, but themes (as in example code for all the structures supported by the tool) are important. My website runs on an ancient version of hugo and probably won’t build with a current one without serious changes, but it’s a SSG with only user-supplied input by me, so I don’t care.

                                                                            I made my own theme but tbh I don’t remember if I did it from scratch or just ported it from hyde, which I was using before that. I probably did it from scratch for hyde, as I don’t have any acknowledgements in that repo. It was enormously helpful to see good examples for hugo and be able to see examples in themes, as for example good RSS support was very important for me.

                                                                            But I’ve only once run some kind of “quick, I need a website!” thing off of any SSG, and then I used jekyll because it was a team blog that should just work without any tinkering and only minimal effort to use a basic theme.

                                                                            1. 1

                                                                              Oh, Nice. I haven’t heard about this tool. I was just testing some of the static site generators. So, I did a small google search and found Hugo, Jekyll, etc. are one of the good static site generators. And, I have heard about ssg (https://www.romanzolotarev.com/ssg.html) - static site generator tool.

                                                                              I can’t compare these tools because I think I am not the right person to do so. And, as I am not a very huge fan of the website development, I was searching for something where I just need to write my content that’s all with some good minimal visibility and readability theme support. So, I have picked Hugo and started testing it.

                                                                            1. 4

                                                                              As a researcher, I worked a lot on flocking and game theory. See flocks for how beautiful these can look!

                                                                              1. 2

                                                                                Note that this was based upon OpenBSD 4.6.

                                                                                1. 10

                                                                                  If you want proper as in “canonical”, John Gruber’s original is your best choice. It is effectively the definition of markdown. But markdown has moved on beyond Gruber’s original vision, mostly without him. Still, his original vision in collaboration with Aaron Swartz is still very usable, especially for blogging.

                                                                                  Once you move beyond the original Perl code, you have numerous choices. Here are a few:

                                                                                  And there’s no shortage of online converters and other versions.

                                                                                  I’ve used pandoc for years. It doesn’t just convert markdown to html. It converts from multiple formats to multiple formats, including various slide formats as well as PDF. It supports Python and Haskell scripts, as well as Lua scripting. The Lua scripting is very interesting because it manipulates the AST in memory as opposed to Python and Haskell where pandoc marshalls the AST to a JSON representation, pipes it to the script, and then receives the output.

                                                                                  Lowdown is interesting because it’s a fork of hoedown. @kristaps went through it and added a proper AST representation internally as well as pledge and privilege separating it. He doesn’t expose the AST (yet?), but it’s fast and efficient.

                                                                                  Multimarkdown is a good choice if you want to stay in Markdown but target more than one output format. Like pandoc, mmd can export to several slide formats and pdf by way of LaTeX.

                                                                                  Lastly, the lua and awk versions are noted above because I like little languages that do so much. ;-)

                                                                                  For most of what markdown was originally designed for, text files converted to HTML, you almost can’t go wrong. Where you’re going to find your choices both expanded and somewhat complicated is when you move beyond basic markdown and start trying to write slides, documentation, books, or use add-on features like tables.

                                                                                  I’d suggest start with basic markdown as described on Gruber’s Markdown page, pick one of the simpler converters like his Perl script, and see whether you need anything beyond that. Then look at Pandoc and Multimarkdown. If you don’t need more features but want more speed, then look at Lowdown.

                                                                                  1. 2

                                                                                    lowdown does have its library exposed! pkg_add lowdown, man 3 lowdown.

                                                                                    1. 1

                                                                                      Thanks for the detailed answer!

                                                                                    1. 3

                                                                                      pkg_locate doesn’t find any of these. It does, however, find pelican and my own, sblg. I’ve also heard a lot about Roman’s ssg.