1. 10

    I’ve always liked Geany but I found the more I dug into a specific language in my career, it could’t keep up with what I expected an IDE to do. I also didn’t want to dive into writing plugins to make it feel more like an IDE that already exists.

    I’ll still prefer Geany over the Electron text editors out there like Atom and VS Code.

    1. 4

      Yeah. I used Geany for a year or two back in school and found that it was a fantastic text editor, but really didn’t offer anything beyond that. Emacs, Vim or a real heavier weight IDE wound up being my go-to tools due to superior integrations with the various languages I was working.

      I’d still reach for Geany for some quick editing if I didn’t have my full emacs instance booted.

      1. 1

        How do you feel about Sublime? It’s also part of the “hand-coded UI” camp of text editors.

        1. 1

          I never took the time to use it. It is literally never an option in my head when I think about editors. I know people seem to really like it and that’s cool!

        1. 1

          OK, thanks!

          1. 1

            Question is why it’s taking this long to just generate a new cert with the extra SAN…

            1. 4

              No one is paid to work on lobsters. If you know ansible and letsencrypt you should be able to help out.

              1. 1

                Well I don’t really know how the current Lets Encrypt cert was generated, but it’s literally just another argument. Did ask about it when it came up on IRC 3 weeks ago, but didn’t get a reply then, and figured it would probably be fixed pretty quickly then so completely forgot about it.

                1. 1

                  It was manually created with certbot but, as noted in the bug, should probably be replaced with use of acmeclient to have much fewer moving parts, if nothing else.

                  It’d be great to have someone who knows the topic well to help the issue along in any capacity, if you have the spare attention.

                  1. 1

                    I’ve done entirely too much work with acmeclient to automate certs for http://conj.io and some other properties I run. Will try and find time this weekend to take a run at this.

                    1. 1

                      That or to use dehydrated: in a text file, one certificate per line, each domain separated by a space.

            1. [Comment removed by author]

              1. 1

                Argh yeah. This essay was originally written in emacs org-mode, exported to HTML, then added to my Jekyll blog which as you can see had its way with the examples. I’ll try and get that fixed in a second :P

                Edit: Fixed, along with a typo in the map example..

              1. 5

                battlestation(s). At work I run a company-issued Macbook using spectacles to get as close as possible to a TWM interface. Most of my work is done in emacs and an ssh terminal. Keyboard is a Das in Cherry Blues with o-rings. Dual 24” outboard displays.

                At home I use a Dell XPS 15 for most of my programming. My home OS of choice is Arch Linux with Xmonad as my emacs and intellij bootloader. Nothing really fancy going on there, but it’s stable and I’ve come back to it time and time again.

                The Dell and my gaming rig share a 4k monitor, and there’s a USB3 switch to swap my mouse and keyboard back and forth. The desktop is a brand spanking new i7 + EVGA 1080 build which I haven’t put Linux on (yet).

                Synology NAS for visible on the bottom shelf. It gets used mainly for backups (borg backup is amazing), torrents and blobstore.

                1. 1

                  +1. Mechanical sympathy, algorithmic performance and tuning are some of my favorite material on here and I’d love to see a tag for them.

                  1. 11

                    Apologies to the author, but I find the comparisons to Haskell in this piece to be facile, and the practical examples of ways to solve the “nil problem” in Clojure to be glossing over the fact that nil remains a frustrating problem when nil is essentially a member of every single type (EDIT: although I should acknowledge that yes, the author does acknowledge this later in the piece–I just don’t find the rest compelling regardless). I don’t buy “nil-punning” just because it’s idiomatic (see the linked piece by Eric Normand); as far as I’m concerned talking about it as idiomatic is just putting lipstick on the pig. And fnil is a hack, the existence of which illuminates how Clojure didn’t get this right.

                    That said, I’m not a lisp guy in my heart, so maybe I’m missing the subtle elegance of nil-punning somehow–but I’d love to see a Clojure without nil in it, and I think it’s telling that these articles always end up defensively comparing Clojure to Haskell.

                    I believe that Clojure has some nice ad-hoc facilities for building composable abstractions, it’s got a lot of good stuff going for it and makes me want to shoot myself far less than any programming language I’ve used professionally to date. But as a user of the language I’ve found its type system, such as it is, to be incoherent, and I find the pervasiveness of nils to simply be a shitty part of the language that you have to suck up and deal with. Let’s not pretend that its way of handling nil is in any way comparable to Haskell (or other ML-family languages) other than being deficient.

                    P.S. Per your first if-let example: when-let is a more concise way to express if-let where the else term is always going to evaluate to nil.

                    1. 11

                      Glad I’m not the only Clojurian who had this knee jerk reaction. nil is not Maybe t. Maybe t I can fmap over, and pattern match out of. With f -> Maybe t at least I know that it’s a function which returns Maybe t because the type tells me so explicitly and the compiler barks at me when I get it wrong. This gives certainty and structure.

                      nil is the lurking uncertainty in the dark. The violation of the “predicates return booleans” contract. The creeping terror of the JVM. The unavoidable result of making everything an Object reference. I never know where nil could come from, or whether nil is the result of a masked type or key error somewhere or just missing data or… a thousand other cases. Forgot that vectors can be called as functions and passed it a keyword because your macro was wrong? Have a nil….

                      Even nil punning doesn’t actually make that sense because it mashes a while bunch of datastructures into a single ambiguous bottom/empty element. Is it an empty map? set? list? finger tree? Who knows, it’s nil!

                      Edit: wait wait but spec will save us! Well no… not unless you’re honest about the possibility space of your code. Which spec gives you no tools to infer or analyze.

                      Have a nil.

                      </rant>

                      1. 4

                        That said, I’m not a lisp guy in my heart, so maybe I’m missing the subtle elegance of nil-punning somehow

                        Minor nitpick–please don’t lump all lisps in with nil-punners! Racket and Scheme have managed to avoid that particular unpleasantness, as has LFE.

                        1. 3

                          Not to mention Shen with a type system that subsumes Haskell’s…

                          1. 3

                            Even in Common Lisp, nil isn’t an inhabitant of every type. If you add (optional) type declarations saying that a function takes or returns an integer, array, function, etc., it isn’t valid to pass/return nil in those places. I think this is more of a Java-ism than a Lisp-ism in Clojure’s case.

                            1. 1

                              Isn’t nil indistinguishable from the empty list though, in Common Lisp?

                            2. 2

                              Yeah, just take it as further evidence that I’m not “a lisp guy” in that I conflated all lisps together here–sorry!

                              1. 1

                                In that case I’ll also forgive being wrong in the ps about if vs when.

                                1. 1

                                  Oh–would you explain further then? I always use when/when-let in the way I described (EDIT: and, I should add, many Clojure developers I’ve worked with as well assume this interpretation)–am I misunderstanding that semantically, fundamentally (from a historical lisp perspective or otherwise)?

                                  1. 4

                                    The ancient lisp traditions dictate that when you have two macros and one of them has an implicit progn (aka do) then that one should be used to indicate side-effects.

                                    I know not all Clojure devs agree, but when you’re arguing that you should be less concerned than CL users about side-effects it’s probably time to reconsider your stance.

                                    1. 1

                                      Thanks!

                                      EDIT: I want to add a question/comment too–I’m a bit confused by your mention of side-effects here, as in my suggestion to use when vs. if in these cases, I’m simply taking when at face value in the sense of it returning nil when the predicate term evaluates to false. I can see an argument that having an explicit else path with a nil return value may be helpful, but I guess my take is that…well, that’s what when (and the when-let variation) is for in terms of its semantics. So I guess I’m still missing the point a bit here.

                                      …but nonetheless appreciate the historical note as I’m a reluctant lisper largely ignorant of most of this kind of ephemera, if you will forgive me perhaps rudely qualifying it as such.

                                      1. 4

                                        The typical Lisp convention is that you use when or unless only when the point is to execute some side-effecting code, not to return an expression, i.e. when means “if true, do-something” and unless means “if false, do-something”. This is partly because these two macros have implicit progn, meaning that you can include a sequence of statements in their body to execute, not just one expression, which only really makes sense in side-effectful code.

                                        So while possible to use them with just one non-side-effecting expression that’s returned, with nil implicitly returned in the other case, it’s unidiomatic to use them for that purpose, since they serve a kind of signalling purpose that the code is doing something for the effect, not for the expression’s return value.

                                        1. 3

                                          Thanks mjn, that helps me understand technomancy’s point better.

                                          Fully digressing now but: more generally I guess I’m not sure how to integrate these tidbits into my Clojure usage. I feel like I’ve ended up developing practices that are disconnected from any Lisp context, the community is a mix of folks who have more or less Lisp experience and the language itself is a mix of ML and Lisp and other influences.

                                          In any case, thank you and technomancy for this digression, I definitely learned something.

                            3. 3

                              Oh I don’t think this solves the nil problem completely, but it’s as good as you can get in Clojure. If you’re coming from Java and have no or little experience in an ML-ish language, then this conceptual jump from null to Nothing is a bit difficult, so this article was primarily written for those people (beginner/intermediate Clojurists).

                              Also you have to admit that, beyond Turing completeness, language features are an aesthetic. We can discuss the pros and cons of those features, but there is no One Right Way. A plurality of languages is a good thing - if there were a programming language monoculture we would have nothing to talk about at conferences and on internet forums :)

                              1. 6

                                Also you have to admit that, beyond Turing completeness, language features are an aesthetic. We can discuss the pros and cons of those features, but there is no One Right Way.

                                It sounds to me like you are saying that all ideas about PL are of equal worth, which I disagree with. One can have a technical discussion about PL without it ultimately boiling down to “that’s just, like, your opinion man”

                                1. 5

                                  That’s not at all what I’m saying. It’s like in art, you can say that the Renaissance period is more important than the Impressionist period in art history, but that doesn’t mean Renaissance art is the “correct” way to do art. We can also have a technical discussion of artworks, but we weigh the strengths and weaknesses of the artworks against the elements and principles of art and how skillfully they are composed, we don’t say one artwork is correct and the other is incorrect. This is the basics of any aesthetic analysis.

                                  Likewise in programming language design, we can discuss the technical merits of features, e.g. yeah Haskell has an awesome type system and I really enjoy writing Haskell, but that doesn’t mean Haskell is categorically better than Clojure. When you have a ton of legacy Java to integrate with (as I do), Clojure makes a lot of sense to use.

                                  1. 3

                                    PL criticism is done on utilitarian grounds, not aesthetic grounds. You acknowledge as much in your second paragraph when you give your reason for using Clojure. I guess you can look at PL as art if you want to but I don’t like that mode of analysis being conflated with what researchers in the field are doing.

                                    1. 2

                                      PL criticism is done on utilitarian grounds, not aesthetic grounds.

                                      Why not both?

                                      1. 1

                                        When you’re trying to figure out which language is better, it’s not a question of aesthetics.

                                        Though to be fair, “better” is really difficult to nail down, especially when people end up arguing from aesthetics.

                            1. 2

                              Wow cmake is taking a lot of heat here. Admittedly I’m not doing anything particularly interesting with it, but CMake has served just fine as the build system for one of my toy projects - dirt - dirt/CMakeLists.txt. Admittedly I mostly interact with it via a bunch of bash - dirt/dirt and I’m only targeting a single platform but compared to figuring out the pattern language to use core Make or figuring out Auto-tools which…. didn’t seem particularly user friendly when I looked at it briefly.

                              1. 5

                                Maintaining a stack by yourself in code seems rather unfortunate (look at how much longer the code is than the generator version), and very much points to a language deficiency. There’s no excuse for a function call being slower than a list access, and tail call support would remove the stack depth issue.

                                1. 6

                                  Tail call support wouldn’t get rid of the need for a stack here because this function calls itself more than once aka “isn’t tail recursive”.

                                  I would try to add the code from the article with comments showing where it is not tail recursive with comments, but pre-formatted text is not working for me at the moment.

                                  1. 2

                                    You’re right. I find it hard to believe that stack depth would be an issue at least for a balanced tree, but again if it is Python really should offer a way to have deeper stacks in the language rather than have users end up writing their own stack emulation.

                                    1. 5

                                      So this precise issue was the root of a performance problem in the clojure.tools.emitter.jvm project which I used to work on. Essentially the code generator worked by emitting OpcodeList = List[Either[OpcodeList, Opcode]]. The resulting structure was then traversed one opcode at a time, which meant that you had to recursively call next() on arbitrarily many lazy generator terms in order to get the next opcode you wanted to emit.

                                      Using the stack of iterators pattern described here becomes an optimization then, because you elide n-1 calls to next() which have to walk back down the stack of iterators to the bottom-most iterator in order to get the next actual opcode you want. Because you’re traversing back up and down many many nested stateful iterators, there isn’t a way to optimize this recursion ala with TCO because you still have to go down n-1 iterator structures to figure out what buffer you’re actually taking the next() of.

                                      In comparison with the stack of iterators pattern, your next() just needs to peek the top of the stack, take the next from that, and recurs only if it’s another iterator structure. This means you traverse down the entire depth of the tree precisely once, rather than doing so once for every leaf of the tree.

                                      1. 5

                                        there isn’t a way to optimize this recursion ala with TCO because you still have to go down n-1 iterator structures to figure out what buffer you’re actually taking the next() of

                                        The mutable variable in the example stack-of-iterators code, or the one you’d get from a clojure loop/recur, is what provides that benefit. In fact, Clojure performs this optimization for nested LazySeq objects: https://github.com/clojure/clojure/blob/e547d35bb796051bb2cbf07bbca6ee67c5bc022f/src/jvm/clojure/lang/LazySeq.java#L58 - Note that it flattens lazy seq thunks recursively and elides that work on future .seq() calls via the synchronized mutable field.

                                        Sadly, the optimization is lost if you’re making your own nested structures because you have the vector -> seq -> vector loop going on. You could recover it a mutable variable for the top element on the stack. That’s precisely what TCE would provide as long as you could ensure that the recursive call was actually in tail position. Sadly, the ‘next’ interface is not ideal for that. You need something that returns both the next item and the continuation. So instead of (next seq) -> seq, you’d want (uncons seq) -> [head tail]

                                        In the case of tools emitter, you could probably also have achieved this effect without mutability via an automatic splice-on-construction collection type. Instead of [:blah … [:foo …] …. :bar] you’d have (spliced :blah … (spliced :foo …) …. :bar) and pay that cost on construction.

                                        1. 1

                                          Is there a standard name for a list uncons that returns an option/maybe? It’s a function I’ve often found myself wanting.

                                          1. 1

                                            Pattern matching and related constructs generally eliminate the need to name it. However, I’d just call it uncons.

                                            In Clojure, you can write (if-some [[head & tail] seq] …), but if I recall correctly, the underlying implementation still always uses first/next. Haskell fails better here with the colon/cons syntax in patterns, but trades a sequence abstraction for a concrete list type (at least without various extensions).

                                            1. 1

                                              I’m thinking Scala here, and I’d prefer to avoid the match/case syntax entirely if possible since it’s very easy to use unsafely.

                                1. 15

                                  Okay so this is a bit of a rant but hear me out.

                                  I’ve been writing primarily Clojure for several years now. In the Clojure universe there are some de-facto conventions for the significance of very brief variable names. For instance m denotes a map, and col or seq a list type or other sequence source. This works out pretty well for all the reasons that John outlines here - my programs in Clojure are written against an interface which is conventionally named by a truncated name, not a concrete type like clojure.lang.AFunction or java.util.Map. The single letter variable names wind up being binding for existential types… that is m is the sign by which I reference <T instanceof Associative, Iterable>.

                                  At my day job, I work on an almost all Python team and have now repeatedly gotten into code review scuffles with my coworkers over my choice of variable names. Because I approach programming from this interface oriented perspective, the most meaningful names are short ones much in the same sense that John is after here which convey only the interfaces or other features which I need, rather than the longer names preferred by my team which seek to convey the entire context of the original use of the code I write.

                                  The starkest contrast is, I think, in the use of classes and named class fields as opposed to functions. Functions are composable, single purpose things which you can reason about in isolation because all the relevant context is in the call stack and application. Class members especially in a context without strong static types are forced to use naming to fully distinguish the context and meaning of the value(s) they contain, which leads to long and task specific names in order to convey all the relevant context.

                                  So yes I absolutely agree that long variable names are primarily used as a crutch in lieu of types, or due to high cyclomaitc complexity and consequent low reusability.

                                  1. 9

                                    IBM model M, dvorak. Nothing special.

                                    1. 7

                                      IBM model M, dvorak.

                                      The most hipster of combination.

                                      1. 3

                                        I have a Unicomp model M-like keyboard, but with a dvorak layout. Love it.

                                        But… My favorite has to be a TypeMatrix 2030 USB, with a black US dvorak sleeve (the image below is a UK layout, but it doesn’t differ that much) Took less time to get used to than I thought, even with the Enter key in the center of the keyboard, and for me this thing just wins wins hands down.

                                        http://www.typematrix.com/shop/images/products/2030-skin-045-b-uk-dvorak-860x360.png

                                        1. 2

                                          Pfft–not even Colemak or Norman? Everyone has heard of Dvorak these days.

                                        2. 3

                                          Well at least @God has good taste in keyboards too.

                                          I use Das Keyboard (a model M derivative) at work and at home. Cherry blue switches, O-rings to limit bottom out impact and noise. I’m now on my 4th Das and have no intent to switch it up anytime soon. On the go I have a Leopold II (also cherry blues and o-rings). At home and mobile I use a gifiti wrist rest, workspace allowing. Only downside is that the wrist rests get seriously chewed up by watch clasps.

                                          1. 4

                                            You’re on your 4th Das? What are you doing to the poor things? :-)

                                            1. 4

                                              Fighting zombies and coworkers who break the build ;)

                                              Edit: a conjugation

                                          2. 3

                                            I used a HHKB Pro2 for a while, as well as the Lite version, and now I’m back to my Model M. Still my favourite keyboard of all time. And at this point, I’ve lost count of how many I’ve tried.

                                            Oh, and the main reason I’m back to using it is my office is now sufficiently far away from the bedroom so my wife can actually sleep when I’m typing.

                                            1. 3

                                              IBM model M, ps2->usb adapter

                                              Arguably best keyboard ever made, still available for dirt cheap

                                            1. 4

                                              macOS Sierra’s native full screen apps/Spaces & Spectacle for basic window management on non-fullscreen spaces.

                                              1. 1

                                                Yeah +1 to Spectacle. On my linux machines at home I run xmonad + xmobar, after switching from Awesome and when I got my work macbook the lack of tiling support drove me absolutely nuts. Spectacle isn’t a full replacement by any means, but it beats the heck out of the built in window manager.

                                              1. 4

                                                Now to port Genera to it?

                                                1. 6

                                                  I posted PreScheme link in page’s, comment section in case that helps author out. It was a C replacement used to build verified Scheme48. I figure eventually a LISPer in systems programming or embedded will find something about it useful for them, too. :)

                                                  1. 2

                                                    Ooh nice! I hadn’t heard of this before!

                                                    1. 3

                                                      Link here too might as well:

                                                      https://en.wikipedia.org/wiki/PreScheme

                                                      The papers are pretty detailed each covering interesting or useful things. A consistency of usefulness that’s rare in academia. ;) Another you might like to look up is Carp LISP which has Rust-like, memory safety instead of GC. IIRC that is.

                                                  2. 4

                                                    Cool as that would be, there’s a major snag here. I haven’t looked at the uLisp implementation but the AVRs are a Harvard architecture. That is, they have two memory banks - one for program instructions and one for data. Projects like uLisp and the various forth interpreters targeting Arduinos are forced to forego compilation or JIT and any real hope of self-hosting ala Genera.

                                                    I was looking at writing a lisp OS for my Arduino fleet a while back and came to the conclusion that you pretty much just wanted to use a different chip both for memory model and performance reasons.

                                                  1. 5

                                                    I wanted to learn Lisp, so I picked up Clojure. Really enjoying it so far!

                                                    I started with Learn Clojure in 15 minutes which gives nice taste of Clojure syntax. Now I am currently doing Clojure for the Brave and True, which just a brilliant book imo. Its lot of fun reading it and really engaging.

                                                    1. 4

                                                      Best of luck digging into Clojure, it’s something I really got a lot of value out of for a long time and hope you do too.

                                                      I would be fascinated to see a blog post or just a gist of notes on things you found to be stumbling blocks both with the language and tooling.

                                                      1. 2

                                                        Yes, I am hoping to learn lots of new things (always been programming in imperative languages).

                                                        As of now I haven’t had any issues, because mostly I am learning and haven’t really written anything big. I have a side project to build a small job board and I have decided to use Clojure for it. Then I will know the real stumbling blocks and difficulties. I will surely write a blog post or a self post here in Lobsters.

                                                    1. 11

                                                      So basically, you already run code you personally never reviewed or tested, HTTPS is enough, our script is good, we will continue to recommend this install method.

                                                      Gotcha.

                                                      compared to maintaining (and testing) half a dozen package formats for different distros.

                                                      Again with that bullshit. Let. Packagers. Do. Their. Jobs.

                                                      1. 10

                                                        compared to maintaining (and testing) half a dozen package formats for different distros.

                                                        Again with that bullshit. Let. Packagers. Do. Their. Jobs.

                                                        I’m torn on this. On the one hand, yes. Having 3rd party packagers is great for pretty much everyone and ideally f/oss software organizations should maintain and license their software so that repackaging is possible.

                                                        The problem with this view is that user maintained packages lag upstream, often by quite a lot, and as an engineering org having lagging user packages means having to deal with innocent users who are stuck with fixed bugs. See a classic rant on this from jwz.

                                                        So yes. By all means I want package managers (or /opt monodir installs) so I can uninstall your software eventually and update it sanely along with everything else, but really that means that the developing org does have to take on packaging responsibilities or else pay the costs associated with not taking them on. For all that I dislike curl | bash, it definitely seems to be a local optimum.

                                                        Disclaimer: this is my view as a user, I’ve never participated in the community as a (re)packager or distributing an original package except in publishing Maven artifacts which don’t require repackaging and thus don’t count.

                                                        1. 5

                                                          Again with that bullshit. Let. Packagers. Do. Their. Jobs.

                                                          I certainly agree with letting packagers do their jobs. However, it seems many users see this as the project’s responsibility, rather than the distribution’s. I read this part of the post as being about that sentiment.

                                                          1. 7

                                                            From the perspective of a young project trying to gain traction, taking on this responsibility can noticeably help with uptake. Unfortunately, individual projects - even very large ones, like languages - are not in a great position to make things work smoothly at the scale of an entire distribution. (I’d count Linux distributions and also things like Homebrew as basically the same sort of thing, here.)

                                                            I think the ideal is for projects to keep offering these “fast start” setups, but recognize that they are not going to be the long-term solution, nor the solution for everybody.

                                                            I wish they would at least encourage users to even think about the security implications, but focusing on that aspect isn’t the heart of why this keeps happening.

                                                            1. 3

                                                              As an (occasional, part time) packager, the ideal would be to get some frozen version of the uptream release (or a reference to same) that I can transform into the package. A magic button I can press whenever I like to get the latest version at that time does not meet that need.

                                                              If a young project wants to get into distros (I don’t know whether that’s the kind of traction you’re thinking of, if not then obviously disregard) I’d suggest that’s what they should be thinking about doing, and the curl|sh should be a wrapper over that

                                                              1. 1

                                                                I was mostly thinking in terms of mindshare - users and contributors. I think a lot of developers don’t necessarily think of distros as part of their plan at all. That’s exactly what I wish were different. :)

                                                                It makes a lot of sense, now that you say it, that getting a frozen version is the biggest need there.

                                                          2. 4

                                                            This has come up before - https://lobste.rs/s/ejptah/curl_sh

                                                            I still haven’t seen a strong argument against this mode of installation, but I still hear a lot of “you’re doing it wrong” anger. I’d be very interested in a cogent argument (comment or link pointer) about why this is bad, as it feels to me like the culture has been shifting in the direction of curl | sh

                                                            Shell-based installations for people who want to run the “arbitrary” code from the developers doesn’t prevent you from using packages and dependency management. What’s the problem here?

                                                            1. 4

                                                              Often times these scripts are targeted at give OSs (often the big 3, which excludes things like NixOS, OpenBSD, FreeBSD… etc), or more commonly, send flags to commands that aren’t available on all systems (sed on OpenBSD for example, was missing -i up until recently).

                                                              These missing flags can be disastrous. No one ever writes scripts to handle errors that pop up from missing flags. The result of which can be clobbered files, removed /’s.. or really anything..

                                                              1. 1

                                                                Agreed. Forgetting the use of curl, install.sh itself is the problem.

                                                              2. 3

                                                                If the connection fails during the middle of the script download, whatever has been downloaded will already have been executed. Imagine if you’re on a system that doesn’t utilize the –no-preserve-root option to rm, and the script called for rm -rf /some/directory/here, but the connection terminated at rm -rf /, your system would be hosed.

                                                                There’s no way to audit that what was performed during one curl | bash instance will be the same thing performed in another instance, even if done only seconds later. There’s no way to audit what exactly was done without taking a forensic image of the system beforehand.

                                                                Simply relying on HTTPS for security doesn’t protect against all threat actors. Certain actors can, and have in the past, performed man-in-the-middle attacks against SSL/TLS encrypted connections. There are companies like Blue Coat, who provide firewall and IPS appliances, and who also are CAs and can perform man-in-the-middle attacks of every SSL/TLS connection for those sitting behind its appliances. This can also be done in any enterprise setting where the client has a corporate CA cert installed and the corporate firewall can do SSL/TLS inspection. Often times, the private key material for the corporate CA certificates are stored insecurely.

                                                                The same holds true, and is especially grievous, when the installation instructions say to use curl | sudo bash.

                                                                No, thank you. I’ll stick to my package manager that keeps track of not only every single file that is placed on the filesystem, but the corresponding hashes.

                                                                edit[0]: Fix typo

                                                                1. 4

                                                                  connection fails

                                                                  TFA addresses this.

                                                                  audit

                                                                  Download the script and look at it. If you have reason to believe that the upstream is gonna serve you something malicious on subsequent installs, then you should audit the entire source you are installing, not just the installer.

                                                                  HTTPS

                                                                  If you don’t already have the package in your distro’s repositories, then you will need to use HTTPS or a similar mechanism to download it. There is no way to verify it against a hash either, because you will need to use HTTPS or a similar mechanism to download the hash. I’m sure there are more reliable (and exotic) ways of bootstrapping trust but in practice nobody will use them.

                                                                  This also has nothing to do with curl | bash in particular; this attack applies to, say, downloading a tarball of the source and ./configure && make && make install.

                                                                  1. 2

                                                                    This is what I love about FreeBSD’s ports tree: it solves all of what you just brought up. Each port entry (like www/chromium) already contains a list of hashes for all files it needs to download. Additionally, when using the binary package repo, the repo is cryptographically signed and the public key material is already on my system. No need for HTTPS in the slightest.

                                                                    1. 3

                                                                      I don’t disagree with you here, using packages with your distro is preferable to curl | sh when the option is available. I see curl | sh as a convenient way of performing an installation when that option is not available. There is a lot of paranoia over curl | sh though that would lead one to believe that is more insecure than other forms of non-package installation, and I think having an article that counters these misconceptions is valuable.

                                                                  2. 3

                                                                    If the connection fails during the middle of the script download, whatever has been downloaded will already have been executed. Imagine if you’re on a system that doesn’t utilize the –no-preserve-root option to rm, and the script called for rm -rf /some/directory/here, but the connection terminated at rm -rf /, your system would be hosed.

                                                                    The sandstorm script is specifically designed to avoid that failure case:

                                                                    # We wrap the entire script in a big function which we only call at the very end, in order to
                                                                    # protect against the possibility of the connection dying mid-script. This protects us against
                                                                    # the problem described in this blog post:
                                                                    #   http://blog.existentialize.com/dont-pipe-to-your-shell.html
                                                                    _() {
                                                                    
                                                                    set -euo pipefail
                                                                    
                                                                  3. 1

                                                                    I take issue with recommending and even defending it as good practice. If you want to use it, no one can stop you.

                                                                    1. 3

                                                                      Yes, it’s clear you take issue with that. My question is why?

                                                                      1. 5

                                                                        Oh wow, let’s see:

                                                                        • Running arbitrary code that is probably in no way tested with your distro, unless you’re running something very popular. So it can pretty easily mess something up if used on a system where there is something the authors of the script did not expect.
                                                                        • If it does fail in some unexpected way, cleanup can become a nasty issue.
                                                                        • There is no clear uninstall procedure. You can’t just pacman -R the thing.

                                                                        Basically it’s the same reasons why you shouldn’t just blindly do make install from source, only there are no DESTDIR and PREFIX.

                                                                        1. 2

                                                                          I see where you’re coming from now.

                                                                          I think one of the drivers for people not considering those reasons (with server-side software, anyway) is that while package management tries to solve the issues you’ve identified, it hasn’t been particularly successful or reliable. A common solution which does work is to use installers and shell scripts to build an image and replace the whole system when you need to upgrade/downgrade/cleanup/uninstall – this is perfectly compatible with sandstorm’s position.

                                                                          In this case the end user is responsible for their system and its administration (which is exactly what every license says anyway). The idea that people should only install things provided by their distro feels a bit paternalistic/outmoded.

                                                                          1. 3

                                                                            You seem to think that my position is “install from packages or else”. No. There is a plethora of valid approaches to administering your system. My problem is that Sandstorm are recommending end users do something that can pretty easily shoot their foot off and then defend it with hand-waving and and arguing with paranoid strawmen.

                                                                            I’d really have no problem if they’d even mention at some point in their install instructions something like “or look in your distro repos, it might be packaged already, yay!”, but no. They specifically ignore distro repositories altogether.

                                                                            That and they recommend building from HEAD despite even having regular tagged releases, which is also a small red flag for me.

                                                                            UPD: To be clear, this is very relevant:

                                                                            In this case the end user is responsible for their system and its administration

                                                                            They deliberately recommend an installation method that requires the user to really know what they are doing. That can be valid, just not as the default option. Recommending such a volatile approach as the default install method for end users is at the very least irresponsible.

                                                                          2. 1

                                                                            Got it, you take issue with the very concept of install scripts, not the practice of piping one to sh from curl.

                                                                      2. 1

                                                                        The only real argument I read against curl|sh is the one regarding network issues. let’s say the script you’re curling include a line as:

                                                                        CONFDIR=".coolsoft"
                                                                        rm -fr $HOME/$CONFDIR/tmp
                                                                        

                                                                        And curl get a connection reset right after the first ‘/’, you’ll loose your whole $HOME. I do agree that there are way to prevent this kind of things to happen, by using “guideline”, “best practice”, “defensive programming” or whatever, but if you don’t read the script you’re pipingi.to sh, this is something that can happen to you.

                                                                        Another grief against this method is that sometimes, the devs ask you to run

                                                                        curl | sudo sh
                                                                        

                                                                        But that’s a different topic

                                                                        1. 1

                                                                          I’d be very interested in a cogent argument (comment or link pointer) about why this is bad

                                                                          This reddit link and discussion cover some of the evil possibilities.

                                                                        2. 2

                                                                          Again with that bullshit. Let. Packagers. Do. Their. Jobs.

                                                                          “Software should only be distributed through official distro channels” (the only consistent interpretation I can find of your statement) is far from universally held idea, so I’d expect an opinion stated this forcefully to come with some reasoning.

                                                                          1. 1

                                                                            Look. If someone wants to build packages for every distro ever — I can’t and don’t want to stop them. But don’t use that argument like someone’s forcing you to build those packages. It was your own damn choice.

                                                                            1. 2

                                                                              Their “own damn choice” was to sidestep the distro packages thing and create a shell script installer. You appear to have an issue with that, since you called it “bullshit.” I doubt Sandstorm cares if others package their software differently (packagers can Do. Their. Jobs!), since it is Apache licensed. What exactly is your objection?

                                                                              1. 1

                                                                                Wait what. It’s a simple case of a false dichotomy. Look. They are saying that they only have two choices, and two choices only: an installer script that you pipie into your shell or building and testing half a dozen packages. Like someone is forcing them to. It’s pretty obvious hand-waving.

                                                                                Those are not the only two possible choices for the project. They know it, you know it, I know it. Don’t get caught on a simple fallacy.

                                                                                Sure, no one stops packages from packaging the thing. But they are telling users to bypass the repos. That’s not helping.

                                                                                1. 2

                                                                                  Where in the article are they telling people to bypass the repos? In fact they even say

                                                                                  However, once Sandstorm reaches a stable release, we fully intend to offer more traditional packaging choices as well.

                                                                                  Also, I am confused to what any of this has to do with whether curl | bash is secure or not.

                                                                                  1. 1

                                                                                    Also, I am confused to what any of this has to do with whether curl | bash is secure or not.

                                                                                    That’s the hand-waving part. They’ve mentioned the packaging issue for no reason other than confuse you even more, which is why I initially said it was bullshit. No one is forcing them to build packages for all or even all major distros, but they go out of their way to use it as an argument for… what exactly?

                                                                                    1. 2

                                                                                      What irks me about this discussion is that if sandstorm.io had provided a URL for a Debian/RPM repository no one would bat an eye. And yet those packages would be just as “arbitrary” as this shell script, and certainly harder for to read for those inclined to do so.

                                                                                      I’m all for using distro-provided packages - but let’s acknowledge that installing third-party software depends on some combination of trust and technical know-how, and curl doesn’t measurably change the quantities.

                                                                                      1. 2

                                                                                        And yet those packages would be just as “arbitrary” as this shell script, and certainly harder for to read for those inclined to do so.

                                                                                        I actually agree. A package built by the distro for the distro is at the very least tested and signed by people with at least some track record. I’m almost as much against devs providing their own packages as the main way of installing their shit because my distro’s maintainer would almost always do a better job of making sure the package works and is consistent with the distro’s guidelines. In short: a package from the repos has a much smaller chance of surprising me.

                                                                                        It’s just that a script is even worse because packages you build for distros, and even if you don’t know the distro as well, you will probably at least superficially test the package on the target distro. A script is supposed to be run by whoever on whatever system out there, which has so many ways to fail.

                                                                          2. 1

                                                                            Unfortunately I would imagine if most packagers were just doing their jobs (as in employment that pays their bills) they would have no time to update packages.

                                                                            1. 2

                                                                              Yeah my company has paid for a substantial number of hours I’ve spent on AUR packages over the years :)

                                                                          1. 4

                                                                            Apartment hunting! Made it safely to SF (hit me up!) now I just gotta find long term housing before I start with Twitter.

                                                                            On a totally unrelated note I’m learning a whole lot about HTTP in the process of building a craigslist screen scraper. Cache-Control, If-Modified-Since, css/js resources and a bunch of other nuts and bolts I haven’t had to mess with before have all suddenly become very very important. Craigslist does some interesting bot detection stuff, and it’s been super interesting so far trying to understand what they’re doing and learn enough about browsers to provide the right behavior. If there’s interest in this I may put together a blog post on the process.

                                                                            Once I get settled, I’m looking forwards to continuing working on Haskellbook and mucking with Rust.

                                                                            1. 16

                                                                              The good:

                                                                              • This elegantly solves the problem of macros such as ns which are pretty complex and require a great deal of effort to parse. There should be another announcement later this week about applying this validation structure to macros. Noises have been made about all of clojure.core getting specs, and users being able to turn on specs “everywhere” for better errors at some performance hit.
                                                                              • It’s nice that this provides an “out of the box” standard for talking about the structure of data produced and consumed without having to take on a dependency such as prismatic/schema or worse roll an internal equivalent to any of the above.
                                                                              • That spec provides some destructuring/parsing “for free” is really handy, and will play nicely with other things.
                                                                              • That spec can hook into clojure.test and has data generators is really neat. EDIT: @tbaldridge informs me that test.check is out of the box. I was wrong..

                                                                              The bad:

                                                                              • Really. You can’t write about a dynamic contracts system without making digs at static type systems. Sigh.
                                                                              • This doesn’t do anything to deal with Clojure’s existing woes around hinting (using and making explicit JVM types).
                                                                              • Not sold on having keys specified separately from their values. That I’ll have to play with.
                                                                              • No attempt is made to statically verify that specs “flow” (as they would if they were types), instrumentation/validation seems like it’s gonna be entirely user specified with all the pain that can bring.
                                                                              • We’re getting a new reader macro
                                                                              • Applying specs to fns isn’t the greatest thing ever. From @puredanger on HN:
                                                                              (spec/fdef clojure.core/range
                                                                                :args (spec/alt
                                                                                       :infinite (spec/cat)
                                                                                       :range-0  (spec/cat :end number?)
                                                                                       :range    (spec/cat :start number? :end number?)
                                                                                       :step     (spec/cat :start number? :end number? :step number?))
                                                                                :ret #(instance? clojure.lang.Seqable %))
                                                                              
                                                                              1. 2

                                                                                We’re getting a new reader macro

                                                                                Where?

                                                                                1. 2

                                                                                  http://dev.clojure.org/jira/browse/CLJ-1910

                                                                                  As I understand it, this exists pretty exclusively to simplify the spec stuff, and whatever the final application of the spec stuff to macros is scheduled for later this week.

                                                                                  1. 1

                                                                                    Namespaced keywords have always been a little clunky to me. There’s some kernel of something good in there, but I’m not quite sure I understand all the nuances or tradeoffs to it. Curious to see if spec and/or this reader macro causes a spike in namespaced keyword usage for things other than spec. Double curious to see what that actually means for Clojure programs.

                                                                                    1. 1

                                                                                      I don’t think it’s very nuanced - namespaced keywords give you a global naming scheme, which is good as a key for other things. CLJ-1910 (and CLJ-1919 for namespaced key destructure) will remove most of the duplication of namespaces that make them more annoying to type (in the keyboard sense).

                                                                                      Additionally, we are evaluating a change to allow namespace aliases to refer to non-existent namespaces so that you could declare aliases there for keywords or symbols, purely for naming purposes. This would also ease another pain point around pervasive use of namepace aliases.

                                                                              1. 2

                                                                                Moving time! Graduated from UT after 5yrs this past Saturday, working on finding an apartment in the Bay Area before I start with Twitter on the 6th. When I get some downtime from apartment hunting, I want to mess with Clojure 1.9’s new spec stuff, but am trying to focus my efforts on learning and migrating to haskell.

                                                                                1. 1

                                                                                  Congratulations on graduating :) Good look house hunting!

                                                                                1. 4

                                                                                  $SCHOOL: writing my last term paper ever right now, last exam Monday next week then I get to help gentrify the Bay Area.

                                                                                  $PROJECTS: Whole lot of nothing until the above is cleared. I recently “discovered” (reinvented from ignorance) a tactic for writing fluent APIs in Java. When I have some free time, I want to dig into the generated classfiles and have a look at how (or if) this style impacts checkcast emission. If it’s a win, I may take a swing at refactoring Jaunt/Clojure’s various interfaces so that this sort of information is preserved in the type system rather than being continuously re-asserted by inline casting. The problem with doing so, and the reason why there’s bytecode inspection required is it’s not clear how this change would impact Jaunt/Clojure bytecode generation.

                                                                                  But I have to graduate first.

                                                                                  1. 3

                                                                                    School:

                                                                                    • Two weeks of classes left before graduation. After five years the end is in sight.
                                                                                    • Catching up on reading missed due to a fraternity outing eating the whole weekend. Got to drive $REALLY_FAST in $NOWHERE East TX tho which was fun.

                                                                                    Projects:

                                                                                    • Jaunt is on hold for the time being. I don’t have any immediate friction points, and am happy with the present 0.3.0-SNAPSHOT for my day to day use working on other hobby stuff. It’s not Haskell, but it’s better than stock Clojure. The last thing I want to implement before I drop 0.3.0 is an efficient reached-by analysis for the compiler. Changes in argument lists (read: type) are a common pain point for me when exploring/refactoring. It’d be awesome if the compiler could issue warnings that previously callsites have become erroneous.
                                                                                    • Hyperwave is my experiment in building a really crappy Twitter clone atop redis and is probably gonna get the most love this week. I haven’t done a whole bunch of CRUD development, so it’s interesting and educational to make myself work with a real datastore and think about implementing streams/websockets efficiently etc.
                                                                                    1. [Comment removed by author]

                                                                                      1. 4

                                                                                        If you haven’t already seen them, there are some minimal distros at there that you could use for reference/inspiration:

                                                                                        As for C compilers, it’s kinda hard to find any that have “modern” support and compile fast code. Here are a few interesting ones that I’ve seen:

                                                                                        1. 2

                                                                                          voidlinux.eu has a musl version as well and a slightly different take on package managers.

                                                                                        2. 2

                                                                                          Alternatives to which compilers and why?

                                                                                          1. 1

                                                                                            TCC is p cool