1. 5

    If you’ve read this far and are thinking “If Redo is so great, how come it hasn’t taken over?”

    I want a build system that can help me with the following:

    • build variants – e.g. don’t do “make clean” between debug / release / ASAN builds. GNU make has some tortured support for this, but it requires you to use it correctly, and I haven’t seen many people do it (I think it’s VPATH or something). AFAIK redo doesn’t help you with this any more than GNU make does – probably less.
    • Writing correct incremental builds, e.g. specifying all your dependencies. I don’t want a tradeoff between fast builds and correct builds. GNU make doesn’t help you with this enough, and neither does redo AFAIK.
    • Correct parallel builds. I think apenwarr’s redo does better here, but the original redo doesn’t. But I haven’t seen a real convincing demonstration of that in any case. I think the declarative graph model helps you analyze bottlenecks in your build (critical paths), but redo’s non-declarative model may thwart that to some extent.

    In other words the whole point of a build system is to let you write something that’s fast and correct (which is not easy), and neither GNU make or redo helps enough with that.


    FWIW I sort of made fun of the Android build system circa 2015 here for using GNU Make (they no longer use it):

    https://lobste.rs/s/znrsap/tech_notes_success_failure_ninja#c_l04v6d

    ButI didn’t mention that it was a very parallelizable build. It was much more parallelizable than Firefox’s build as I recall. Android was building much more but it could saturate 32 cores easily. Other build systems do not do this – you have to try to get a really parallelizable build, and that’s what I want a build system to help me with.

    As I mentioned there, I’m probably going to switch the build to Ninja + a shell or Python generator. This recent post was very cool

    https://lobste.rs/s/952gdv/merkle_trees_build_systems

    I think that architecture has a lot of potential for hard build problems in the future. Some context on Oil’s problem here: https://github.com/oilshell/oil/issues/756


    In other words I think redo is fine and cool but it doesn’t help enough with hard build problems. You could make an analogy to Forth. Sure it’s more elegant for certain problems, but if it falls down for harder problems, then it’s not a net win. It has its place, but it’s not surprising that it’s not more popular.

    1. 3

      build variants – e.g. don’t do “make clean” between debug / release / ASAN builds. GNU make has some tortured support for this, but it requires you to use it correctly, and I haven’t seen many people do it (I think it’s VPATH or something). AFAIK redo doesn’t help you with this any more than GNU make does – probably less.

      The easiest way to do this is to have something like

      .PHONY: %.var
      .PRECIOUS: %.var
      %.var:
      	@echo "$($*)" | cmp -s - "$@" || echo "$($*)" > $@
      
      %.o: %.c CC.var CFLAGS.var
      	$(CC) $(CFLAGS) -c -o $@ $<
      

      This will 100% work, but it takes some discipline to specify all used variables as dependencies.

      Writing correct incremental builds, e.g. specifying all your dependencies. I don’t want a tradeoff between fast builds and correct builds. GNU make doesn’t help you with this enough, and neither does redo AFAIK.

      Yeah, to do something like this in make you have to do something like

      $(OBJS) := # ...
      $(DEPS) := $(OBJS:.o=.d)
      
      %.o: %.c
      	$(CC) $(CFLAGS) -MMD -c -o $@ $<
      
      -include $(DEPS)
      

      but of course this only works for C files. For a more comprehensive treatment of this problem, have a look at tup. It puts an overlay FUSE filesystem on your source directory to track what objects use what files. One downside may be that its limited syntax makes it less useful as an all-in-one config/build tool than make is. If you use tup, you may end up needing to use something like autotools/cmake/meson to generate your tupfiles.

      Correct parallel builds. I think apenwarr’s redo does better here, but the original redo doesn’t. But I haven’t seen a real convincing demonstration of that in any case. I think the declarative graph model helps you analyze bottlenecks in your build (critical paths), but redo’s non-declarative model may thwart that to some extent.

      I believe if you have proper dependency tracking, then this becomes trivial to do. Well-written makefiles tend to parallelize extremely well, but only if the author has included all the dependency info.

      1. 2

        Yeah, the short way to say it is that the build should fail if the Makefile is broken. Bazel has that property, and the linked Merkle tree post describes a system that appears to have it as well.

        In other words, the build tool should help you write a correct build config. GNU make doesn’t help you at all – instead you are left with a bunch of obscure rules that you’re not sure if you followed.

        You can state the rules, and maybe follow them yourself, but if I give you 5,000 lines of someone else’s Make, then good luck.

        On top of that, make can’t be statically parsed, so you can’t write a lint-type tool for it. I don’t think redo can be either. You can statically parse shell (which is what Oil does) but redo has semantics on top of shell, that occur after runtime substitutions.

        1. 2

          I tried tup, and liked it initially, but was ultimately put off by it. I don’t like that it requires a fuse module (heavy dependency, doesn’t work well on platforms that aren’t linux, kind of hacky). It also broke its promise to never screw up a dirty build.

          I don’t know what the solution is for projects that require complicated builds. I try to keep mine simple—list of source files converted to objects—so plain make can handle it, but sometimes that’s just not practical. Meson is decent, but not great. Cmake and autotools are hell. Language-specific build systems (cargo, dub, …) tend to be good, but inflexible.

          1. 1

            I tried tup, and liked it initially, but was ultimately put off by it. I don’t like that it requires a fuse module (heavy dependency, doesn’t work well on platforms that aren’t linux, kind of hacky). It also broke its promise to never screw up a dirty build.

            Yeah, I primarily mention it because it’s sort of the natural end-point of something like redo. I wonder if one could do something like this with ptrace. Does windows have an api for that? I wonder what the performance is like when compared to fuse.

            Cmake and autotools are hell.

            I agree. CMake does make windows builds better, but at the cost of its awful configuration language. I think there’s an urge to reach for a tool like them whenever one starts to have significant configuration for a project. I think Kconfig does a pretty good job, now if only it could get recursive dependencies :)

        2. 2

          but redo’s non-declarative model may thwart that to some extent

          It is non declarative, but after a full build, you get .dep.$name files that declare the build tree, which redo-dot or others implementation’s tools can make use of.

          Writing correct incremental builds, e.g. specifying all your dependencies. I don’t want a tradeoff between fast builds and correct builds. GNU make doesn’t help you with this enough, and neither does redo AFAIK.

          I fail to see how either fails at doing so: as soon as the full dependency graph is there, it is a problem that is solved by both?

          1. 2

            It’s easy to introduce bugs in Makefiles simply by leaving off a dependency. You add a dependency to the code but forget to update the Makefile. (the gcc -M stuff helps for C/C++ code only, though it’s not exactly a great mechanism)

            So the build still works in the sense that you’ll get some output file and will exit status 0, but the incremental build is subtly broken, and may remain that way for a long time.

            What I’m thinking of is more along the lines of what’s described here:

            https://lobste.rs/s/952gdv/merkle_trees_build_systems

            • Using something like bubblewrap to run build steps in a deterministic environment / lightweight container. If you leave off a dependency, you get a build error – file not found. Bazel does this and it’s very effective at enforcing the correctness of the build.
              • For speed, there could be an option to run without containers / chroots. But for changing the build description, the enforcement is very valuable.
            • Directory trees, not files, as dependencies. For many problems I suspect this coarse-grained solution may work better. Checking every file is tedious and causes a lot of stat() traffic on the file system.
            1. 1

              So not only about reproducing related files according to logical rules, but the entire directory tree with hashing to reproduce a filesystem image reproducibly (which might be a software project working directory), then provide a hash of the whole build.

              Sure, then Make and redo might be used for doing this, but it is not their doing nor default behavior.

              On a side note, redo tracks changes in dependencies with hashes put in .dep.$filename, but not of a whole directory hierarchy and just of the individual files.

          2. 1

            I’ve been using CMake for the past year and a half exactly because it checks all the boxes. I recommend using the Ninja generator alongside it - it’s parallel by default and significantly faster than Make.

            The one drawback of CMake is that you essentially have to treat the build system as its own part of the software that needs maintenance. I believe this tradeoff is well worth for larger projects.

          1. 15

            This is the complete list of domains where a user’s next submission could be blocked for having a majority of submissions from them. It’s a little close to being a worst-of list I don’t like to share queries of (especially where the submitters column is 1…) but I wanted to include it as a list of examples of content marketing.

            MariaDB [lobsters]> select domain, count(*) as submitted, count(distinct stories.user_id) as submitters, (select count(*) from stories s where s.domain_id = domains.id group by s.user_id order by 1 desc limit 1) as from_one_submitter from domains join stories on domains.id = stories.domain_id group by domain having count(*) > 5 and (from_one_submitter + 1) * 2 > count(*) order by 2 desc;
            +-------------------------------------+-----------+------------+--------------------+
            | domain                              | submitted | submitters | from_one_submitter |
            +-------------------------------------+-----------+------------+--------------------+
            | blogs.msdn.microsoft.com            |       206 |         36 |                132 |
            | jvns.ca                             |       159 |         36 |                 80 |
            | blog.netbsd.org                     |       126 |         13 |                 85 |
            | css-tricks.com                      |       116 |         15 |                 86 |
            | developer.telerik.com               |       113 |          5 |                101 |
            | codewithoutrules.com                |        93 |          2 |                 92 |
            | righto.com                          |        91 |         19 |                 64 |
            | flippinawesome.org                  |        90 |          2 |                 86 |
            | phoronix.com                        |        85 |         31 |                 44 |
            | oilshell.org                        |        83 |          4 |                 80 |
            | devblogs.microsoft.com              |        75 |         20 |                 42 |
            | spin.atomicobject.com               |        74 |         20 |                 39 |
            | blog.softwaremill.com               |        73 |          4 |                 55 |
            | lemire.me                           |        72 |         22 |                 45 |
            | ponyfoo.com                         |        68 |          5 |                 64 |
            | mail-index.netbsd.org               |        61 |          5 |                 44 |
            | citusdata.com                       |        60 |         10 |                 41 |
            | alistapart.com                      |        58 |         14 |                 45 |
            | hillelwayne.com                     |        58 |          3 |                 56 |
            | smashingmagazine.com                |        57 |         17 |                 33 |
            | dailydrip.com                       |        55 |          4 |                 39 |
            | vermaden.wordpress.com              |        55 |          2 |                 54 |
            | vuejsdevelopers.com                 |        52 |          1 |                 52 |
            | schneems.com                        |        50 |          6 |                 45 |
            | eev.ee                              |        49 |         17 |                 31 |
            | crate.io                            |        48 |          2 |                 47 |
            | driftingruby.com                    |        48 |          1 |                 48 |
            | bendyworks.com                      |        46 |          8 |                 25 |
            | petecorey.com                       |        44 |          2 |                 43 |
            | codepen.io                          |        42 |         14 |                 21 |
            | kev.inburke.com                     |        42 |          5 |                 38 |
            | imaginarycloud.com                  |        41 |          3 |                 39 |
            | zwischenzugs.com                    |        41 |          5 |                 36 |
            | intoli.com                          |        40 |          3 |                 28 |
            | pythonspeed.com                     |        40 |          1 |                 40 |
            | ma.ttias.be                         |        39 |          7 |                 30 |
            | pdfs.semanticscholar.org            |        39 |         11 |                 24 |
            | fsf.org                             |        38 |         15 |                 20 |
            | tboox.org                           |        38 |          1 |                 38 |
            | mattwarren.org                      |        37 |          5 |                 31 |
            | nanxiao.me                          |        37 |          2 |                 36 |
            | daringfireball.net                  |        36 |         11 |                 24 |
            | christophermeiklejohn.com           |        35 |          4 |                 32 |
            | dolphin-emu.org                     |        35 |          6 |                 24 |
            | filfre.net                          |        35 |          9 |                 25 |
            | 250bpm.com                          |        34 |         12 |                 22 |
            | blogs.gnome.org                     |        34 |         16 |                 17 |
            | christine.website                   |        34 |          2 |                 32 |
            | newrustacean.com                    |        34 |          1 |                 34 |
            | tech.marksblogg.com                 |        34 |          3 |                 32 |
            | flak.tedunangst.com                 |        33 |          6 |                 28 |
            | kevq.uk                             |        33 |          2 |                 32 |
            | os2museum.com                       |        33 |          7 |                 26 |
            | pixelstech.net                      |        33 |          2 |                 30 |
            | stackbuilders.com                   |        33 |          5 |                 17 |
            | blog.ikura.co                       |        32 |          2 |                 30 |
            | blog.logrocket.com                  |        32 |          6 |                 22 |
            | blog.runnable.com                   |        32 |          4 |                 22 |
            | craigkerstiens.com                  |        32 |          8 |                 22 |
            | deliberate-software.com             |        31 |          4 |                 27 |
            | promptworks.com                     |        31 |          4 |                 27 |
            | bravenewgeek.com                    |        30 |         10 |                 21 |
            | drmaciver.com                       |        30 |         14 |                 16 |
            | beza1e1.tuxen.de                    |        29 |          4 |                 26 |
            | bluishcoder.co.nz                   |        29 |          6 |                 24 |
            | neo4j.com                           |        29 |          6 |                 16 |
            | solipsys.co.uk                      |        29 |          5 |                 24 |
            | blinkingcaret.com                   |        28 |          2 |                 27 |
            | joezimjs.com                        |        28 |          1 |                 28 |
            | geeklan.co.uk                       |        27 |          1 |                 27 |
            | interrupt.memfault.com              |        27 |          2 |                 26 |
            | victorzhou.com                      |        27 |          2 |                 26 |
            | blog.mariusschulz.com               |        26 |          3 |                 21 |
            | davidgerard.co.uk                   |        26 |          1 |                 26 |
            | gigasquidsoftware.com               |        26 |          2 |                 21 |
            | gkbrk.com                           |        26 |          2 |                 25 |
            | jeremymorgan.com                    |        26 |          2 |                 25 |
            | saturnflyer.com                     |        26 |          2 |                 25 |
            | sicpers.info                        |        26 |          5 |                 21 |
            | blog.higg.im                        |        25 |          1 |                 25 |
            | pragtob.wordpress.com               |        25 |          1 |                 25 |
            | raganwald.com                       |        25 |         12 |                 13 |
            | telerik.com                         |        25 |          2 |                 21 |
            | hoelz.ro                            |        24 |          1 |                 24 |
            | metaredux.com                       |        24 |          3 |                 22 |
            | nedbatchelder.com                   |        24 |          7 |                 17 |
            | netbsd.org                          |        24 |          8 |                 13 |
            | osnews.com                          |        24 |          6 |                 18 |
            | raymii.org                          |        24 |          3 |                 22 |
            | verisimilitudes.net                 |        24 |          2 |                 23 |
            | about.sourcegraph.com               |        23 |          5 |                 15 |
            | arp242.net                          |        23 |          9 |                 14 |
            | blog.jle.im                         |        23 |          7 |                 13 |
            | blog.pragmaticengineer.com          |        23 |          4 |                 20 |
            | blog.sqreen.io                      |        23 |          3 |                 21 |
            | dragan.rocks                        |        23 |          3 |                 18 |
            | ecc-comp.blogspot.com               |        23 |          2 |                 22 |
            | parsonsmatt.org                     |        23 |          5 |                 19 |
            | bitemyapp.com                       |        22 |          3 |                 19 |
            | cambus.net                          |        22 |          3 |                 20 |
            | tedium.co                           |        22 |          8 |                 13 |
            | blog.takipi.com                     |        21 |          6 |                 10 |
            | brooker.co.za                       |        21 |          9 |                 11 |
            | carlchenet.com                      |        21 |          2 |                 20 |
            | beastie.pl                          |        20 |          1 |                 20 |
            | kanoki.org                          |        20 |          2 |                 19 |
            | lethain.com                         |        20 |          8 |                 11 |
            | modernweb.com                       |        20 |          2 |                 18 |
            | wilfred.me.uk                       |        20 |          4 |                 15 |
            | blog.ably.io                        |        19 |          4 |                  9 |
            | blog.codeship.com                   |        19 |          8 |                  9 |
            | blog.dantup.com                     |        19 |          1 |                 19 |
            | blog.floydhub.com                   |        19 |          6 |                 14 |
            | blog.graphqleditor.com              |        19 |          2 |                 13 |
            | blog.testdouble.com                 |        19 |          7 |                 10 |
            | ops.tips                            |        19 |          2 |                 17 |
            | people.eecs.berkeley.edu            |        19 |          9 |                  9 |
            | shape-of-code.coding-guidelines.com |        19 |          8 |                 12 |
            | sigusr2.net                         |        19 |          2 |                 18 |
            | sourcegraph.com                     |        19 |          6 |                 10 |
            | testdroid.com                       |        19 |          2 |                 17 |
            | wyeworks.com                        |        19 |          4 |                 12 |
            | blog.bitsrc.io                      |        18 |          6 |                 13 |
            | michaelochurch.wordpress.com        |        18 |         10 |                  9 |
            | monades.roperzh.com                 |        18 |          1 |                 18 |
            | patshaughnessy.net                  |        18 |          7 |                 11 |
            | raphlinus.github.io                 |        18 |          5 |                 13 |
            | stratus3d.com                       |        18 |          2 |                 17 |
            | yegor256.com                        |        18 |          8 |                 10 |
            | amir.rachum.com                     |        17 |          2 |                 16 |
            | blog.hboeck.de                      |        17 |          5 |                 13 |
            | blog.packagecloud.io                |        17 |          5 |                 12 |
            | doxsey.net                          |        17 |          2 |                 16 |
            | fluentcpp.com                       |        17 |          4 |                  8 |
            | jackhiston.com                      |        17 |          2 |                 16 |
            | jezenthomas.com                     |        17 |          2 |                 16 |
            | jlongster.com                       |        17 |         10 |                  8 |
            | oshug.org                           |        17 |          1 |                 17 |
            | ponylang.org                        |        17 |          4 |                 14 |
            | prathamesh.tech                     |        17 |          1 |                 17 |
            | silvestarbistrovic.from.hr          |        17 |          1 |                 17 |
            | talospace.com                       |        17 |          2 |                 12 |
            | virtuallyfun.com                    |        17 |          6 |                 10 |
            | wordsandbuttons.online              |        17 |          5 |                 13 |
            | 256kilobytes.com                    |        16 |          3 |                 14 |
            | bfilipek.com                        |        16 |          4 |                 10 |
            | blog.carlosgaldino.com              |        16 |          2 |                 15 |
            | blog.jessitron.com                  |        16 |          6 |                 10 |
            | blog.librato.com                    |        16 |          3 |                 11 |
            | brianmckenna.org                    |        16 |          3 |                 14 |
            | coolcoder.in                        |        16 |          2 |                 13 |
            | ds9a.nl                             |        16 |          5 |                  9 |
            | fitzgeraldnick.com                  |        16 |          4 |                 13 |
            | fsharpforfunandprofit.com           |        16 |          8 |                  9 |
            | itsfoss.com                         |        16 |          5 |                  9 |
            | lauradhamilton.com                  |        16 |          1 |                 16 |
            | microservicesweekly.com             |        16 |          1 |                 16 |
            | notamonadtutorial.com               |        16 |          1 |                 16 |
            | objective-see.com                   |        16 |          6 |                 11 |
            | omgubuntu.co.uk                     |        16 |          7 |                 10 |
            | aiprobook.com                       |        15 |          1 |                 15 |
            | blog.asrpo.com                      |        15 |          1 |                 15 |
            | blog.cubehero.com                   |        15 |          1 |                 15 |
            | blog.drewolson.org                  |        15 |          6 |                  9 |
            | blog.garage-coding.com              |        15 |          1 |                 15 |
            | blog.jakubarnold.cz                 |        15 |          2 |                 14 |
            | blog.jonlu.ca                       |        15 |          2 |                 13 |
            | blog.scottnonnenberg.com            |        15 |          2 |                 14 |
            | boxbase.org                         |        15 |          3 |                 12 |
            | cmcenroe.me                         |        15 |          1 |                 15 |
            | degoes.net                          |        15 |          7 |                  7 |
            | dev.theladders.com                  |        15 |          1 |                 15 |
            | geshan.com.np                       |        15 |          3 |                 13 |
            | jozefg.bitbucket.org                |        15 |          8 |                  7 |
            | oshogbo.vexillium.org               |        15 |          3 |                 12 |
            | thefullstack.xyz                    |        15 |          1 |                 15 |
            | vaibhavsagar.com                    |        15 |          6 |                  9 |
            | well-typed.com                      |        15 |          6 |                  9 |
            | zendev.com                          |        15 |          1 |                 15 |
            | alexkyte.me                         |        14 |          2 |                 11 |
            | bitquabit.com                       |        14 |          4 |                 10 |
            | blog.jcoglan.com                    |        14 |          8 |                  7 |
            | blogs.technet.microsoft.com         |        14 |          6 |                  8 |
            | chrisshort.net                      |        14 |          2 |                 13 |
            | clever-cloud.com                    |        14 |          4 |                  8 |
            | csswizardry.com                     |        14 |          4 |                  9 |
            | dustycloud.org                      |        14 |          4 |                 11 |
            | haiku-os.org                        |        14 |          5 |                 10 |
            | hardenedbsd.org                     |        14 |          2 |                 13 |
            | hypothesis.works                    |        14 |          5 |                  9 |
            | ithare.com                          |        14 |          5 |                 10 |
            | jepsen.io                           |        14 |          3 |                 12 |
            | jugad2.blogspot.com                 |        14 |          1 |                 14 |
            | juxt.pro                            |        14 |          4 |                  7 |
            | meyerweb.com                        |        14 |          4 |                 11 |
            | orbifold.xyz                        |        14 |          2 |                 13 |
            | paperswelove.org                    |        14 |          6 |                  7 |
            | paragonie.com                       |        14 |          4 |                 10 |
            | penguindreams.org                   |        14 |          1 |                 14 |
            | sethvargo.com                       |        14 |          2 |                 13 |
            | tiny-giant-books.com                |        14 |          1 |                 14 |
            | videlalvaro.github.io               |        14 |          3 |                 11 |
            | yodaiken.com                        |        14 |          3 |                 12 |
            | alexgaynor.net                      |        13 |          4 |                 10 |
            | blog.learngoprogramming.com         |        13 |          1 |                 13 |
            | blog.ploeh.dk                       |        13 |          7 |                  7 |
            | boston.conman.org                   |        13 |          2 |                 12 |
            | defn.io                             |        13 |          1 |                 13 |
            | dirk.to                             |        13 |          1 |                 13 |
            | dmitryfrank.com                     |        13 |          3 |                 11 |
            | dspace.mit.edu                      |        13 |          7 |                  6 |
            | engineering.appfolio.com            |        13 |          5 |                  6 |
            | getstream.io                        |        13 |          2 |                 12 |
            | hookrace.net                        |        13 |          8 |                  6 |
            | jvt.me                              |        13 |          1 |                 13 |
            | loige.co                            |        13 |          1 |                 13 |
            | longren.io                          |        13 |          1 |                 13 |
            | maxhallinan.com                     |        13 |          2 |                 12 |
            | mempko.wordpress.com                |        13 |          2 |                 12 |
            | onebigfluke.com                     |        13 |          5 |                  9 |
            | tedinski.com                        |        13 |          5 |                  8 |
            | tel.github.io                       |        13 |          3 |                 11 |
            | benlakey.com                        |        12 |          2 |                  8 |
            | blog.demofox.org                    |        12 |          4 |                  8 |
            | chrismm.com                         |        12 |          4 |                  9 |
            | codon.com                           |        12 |          5 |                  7 |
            | daverupert.com                      |        12 |          4 |                  8 |
            | dylanfoundry.org                    |        12 |          2 |                 11 |
            | inaka.net                           |        12 |          4 |                  6 |
            | readtext.org                        |        12 |          1 |                 12 |
            | robustperception.io                 |        12 |          2 |                 11 |
            | snowplowanalytics.com               |        12 |          1 |                 12 |
            | transposit.com                      |        12 |          1 |                 12 |
            | ubuntu.com                          |        12 |          5 |                  8 |
            | ably.io                             |        11 |          5 |                  6 |
            | autodidacts.io                      |        11 |          1 |                 11 |
            | blog.arkency.com                    |        11 |          4 |                  6 |
            | blog.ponyfoo.com                    |        11 |          1 |                 11 |
            | bloomca-me.github.io                |        11 |          2 |                 10 |
            | csrc.nist.gov                       |        11 |          7 |                  5 |
            | cstheory.stackexchange.com          |        11 |          6 |                  6 |
            | designpepper.com                    |        11 |          1 |                 11 |
            | flaviocopes.com                     |        11 |          1 |                 11 |
            | gavinmiller.io                      |        11 |          1 |                 11 |
            | howistart.org                       |        11 |          4 |                  7 |
            | icyphox.sh                          |        11 |          4 |                  8 |
            | iridakos.com                        |        11 |          2 |                 10 |
            | itnext.io                           |        11 |          5 |                  5 |
            | javiercasas.com                     |        11 |          1 |                 11 |
            | kaushikghose.wordpress.com          |        11 |          2 |                 10 |
            | kristerw.blogspot.com               |        11 |          5 |                  5 |
            | lauris.github.io                    |        11 |          1 |                 11 |
            | letterstoanewdeveloper.com          |        11 |          3 |                  9 |
            | longren.org                         |        11 |          1 |                 11 |
            | michaelburge.us                     |        11 |          3 |                  9 |
            | notes.eatonphil.com                 |        11 |          5 |                  7 |
            | nrempel.com                         |        11 |          1 |                 11 |
            | pagetable.com                       |        11 |          7 |                  5 |
            | productchart.com                    |        11 |          1 |                 11 |
            | romanzolotarev.com                  |        11 |          2 |                 10 |
            | runnable.com                        |        11 |          1 |                 11 |
            | shipilev.net                        |        11 |          7 |                  5 |
            | spootnik.org                        |        11 |          2 |                 10 |
            | tenderlovemaking.com                |        11 |          6 |                  5 |
            | theprogrammersparadox.blogspot.com  |        11 |          1 |                 11 |
            | thorstenball.com                    |        11 |          5 |                  5 |
            | traininjs.com                       |        11 |          1 |                 11 |
            | urbit.org                           |        11 |          6 |                  5 |
            | v8.dev                              |        11 |          3 |                  9 |
            | vmssoftware.com                     |        11 |          4 |                  8 |
            | additiveanalytics.com               |        10 |          1 |                 10 |
            | arcan-fe.com                        |        10 |          2 |                  9 |
            | arne-mertz.de                       |        10 |          4 |                  6 |
            | arrdem.com                          |        10 |          1 |                 10 |
            | blog.codacy.com                     |        10 |          2 |                  9 |
            | blog.okturtles.com                  |        10 |          3 |                  8 |
            | bradfrost.com                       |        10 |          2 |                  9 |
            | centurylinklabs.com                 |        10 |          1 |                 10 |
            | chargen.one                         |        10 |          3 |                  8 |
            | devup.co                            |        10 |          1 |                 10 |
            | eigenstate.org                      |        10 |          1 |                 10 |
            | grsecurity.net                      |        10 |          5 |                  6 |
            | kevinmahoney.co.uk                  |        10 |          2 |                  9 |
            | kmjn.org                            |        10 |          2 |                  9 |
            | learnbchs.org                       |        10 |          4 |                  7 |
            | manifest.fm                         |        10 |          1 |                 10 |
            | mapzen.com                          |        10 |          1 |                 10 |
            | marianoguerra.org                   |        10 |          3 |                  8 |
            | naildrivin5.com                     |        10 |          4 |                  5 |
            | pages.cs.wisc.edu                   |        10 |          6 |                  5 |
            | pathsensitive.com                   |        10 |          3 |                  8 |
            | pheelicks.com                       |        10 |          1 |                 10 |
            | pothibo.com                         |        10 |          1 |                 10 |
            | precompile.com                      |        10 |          1 |                 10 |
            | quickleft.com                       |        10 |          1 |                 10 |
            | redblobgames.com                    |        10 |          5 |                  6 |
            | singularityhacker.com               |        10 |          1 |                 10 |
            | skerritt.blog                       |        10 |          2 |                  8 |
            | snowsuit.io                         |        10 |          2 |                  9 |
            | sobolevn.me                         |        10 |          1 |                 10 |
            | superjavascript.com                 |        10 |          1 |                 10 |
            | svnweb.freebsd.org                  |        10 |          4 |                  6 |
            | tenfourfox.blogspot.com             |        10 |          2 |                  7 |
            | theory.stanford.edu                 |        10 |          5 |                  6 |
            | wozniak.ca                          |        10 |          3 |                  8 |
            | yellerapp.com                       |        10 |          4 |                  5 |
            | 24ways.org                          |         9 |          2 |                  8 |
            | ai.googleblog.com                   |         9 |          4 |                  6 |
            | akkartik.name                       |         9 |          6 |                  4 |
            | alchemistowl.org                    |         9 |          4 |                  5 |
            | backtrace.io                        |         9 |          5 |                  5 |
            | badcode.rocks                       |         9 |          1 |                  9 |
            | blog.appliedcompscilab.com          |         9 |          1 |                  9 |
            | blog.bigbinary.com                  |         9 |          4 |                  4 |
            | blog.getambassador.io               |         9 |          2 |                  8 |
            | blog.grandstack.io                  |         9 |          1 |                  9 |
            | blog.jenkster.com                   |         9 |          5 |                  4 |
            | blog.joinmastodon.org               |         9 |          5 |                  5 |
            | blog.ret2.io                        |         9 |          6 |                  4 |
            | blog.tintagel.pl                    |         9 |          2 |                  8 |
            | blogs.telerik.com                   |         9 |          1 |                  9 |
            | cambium.consulting                  |         9 |          2 |                  7 |
            | cnn.com                             |         9 |          6 |                  4 |
            | conal.net                           |         9 |          5 |                  5 |
            | crawshaw.io                         |         9 |          6 |                  4 |
            | cs.berkeley.edu                     |         9 |          5 |                  5 |
            | danielcompton.net                   |         9 |          2 |                  8 |
            | fedoramagazine.org                  |         9 |          5 |                  4 |
            | fusion.net                          |         9 |          6 |                  4 |
            | goto.ucsd.edu                       |         9 |          5 |                  5 |
            | hackazach.net                       |         9 |          1 |                  9 |
            | hakibenita.com                      |         9 |          2 |                  8 |
            | infoscience.epfl.ch                 |         9 |          3 |                  7 |
            | joachim-breitner.de                 |         9 |          4 |                  5 |
            | jonlennartaasenden.wordpress.com    |         9 |          1 |                  9 |
            | kamalmarhubi.com                    |         9 |          5 |                  5 |
            | kennethreitz.org                    |         9 |          6 |                  4 |
            | learnk8s.io                         |         9 |          2 |                  5 |
            | lists.zx2c4.com                     |         9 |          3 |                  7 |
            | lyonwj.com                          |         9 |          2 |                  8 |
            | monkeysnatchbanana.com              |         9 |          2 |                  8 |
            | naughtycomputer.uk                  |         9 |          2 |                  8 |
            | pharr.org                           |         9 |          5 |                  4 |
            | philcalcado.com                     |         9 |          2 |                  8 |
            | pluralsight.com                     |         9 |          3 |                  4 |
            | ponylang.io                         |         9 |          1 |                  9 |
            | prl.ccs.neu.edu                     |         9 |          3 |                  5 |
            | qfpl.io                             |         9 |          3 |                  7 |
            | rachelandrew.co.uk                  |         9 |          2 |                  8 |
            | rain-1.github.io                    |         9 |          3 |                  7 |
            | ro-che.info                         |         9 |          5 |                  4 |
            | robertwpearce.com                   |         9 |          1 |                  9 |
            | rubypigeon.com                      |         9 |          4 |                  5 |
            | ryanbrink.com                       |         9 |          1 |                  9 |
            | stormpath.com                       |         9 |          3 |                  4 |
            | taylor.fausak.me                    |         9 |          4 |                  4 |
            | tech.adroll.com                     |         9 |          5 |                  5 |
            | vfoley.xyz                          |         9 |          1 |                  9 |
            | abe-winter.github.io                |         8 |          4 |                  5 |
            | anupshinde.com                      |         8 |          1 |                  8 |
            | azeria-labs.com                     |         8 |          5 |                  4 |
            | beepb00p.xyz                        |         8 |          3 |                  6 |
            | blog.bloomca.me                     |         8 |          1 |                  8 |
            | blog.jfo.click                      |         8 |          4 |                  5 |
            | blog.obligd.com                     |         8 |          2 |                  7 |
            | blog.scottlogic.com                 |         8 |          5 |                  4 |
            | blogs.technet.com                   |         8 |          4 |                  4 |
            | cbloomrants.blogspot.com            |         8 |          4 |                  5 |
            | cis.upenn.edu                       |         8 |          5 |                  4 |
            | cloudbootup.com                     |         8 |          1 |                  8 |
            | codeblog.jonskeet.uk                |         8 |          4 |                  5 |
            | community.rapid7.com                |         8 |          3 |                  6 |
            | composition.al                      |         8 |          5 |                  4 |
            | crondev.blog                        |         8 |          1 |                  8 |
            | dave.autonoma.ca                    |         8 |          1 |                  8 |
            | daveceddia.com                      |         8 |          2 |                  7 |
            | doc.ic.ac.uk                        |         8 |          5 |                  4 |
            | dockyard.com                        |         8 |          4 |                  4 |
            | fivethirtyeight.com                 |         8 |          5 |                  4 |
            | fixingtao.com                       |         8 |          1 |                  8 |
            | franciskim.co                       |         8 |          1 |                  8 |
            | gazerlog.com                        |         8 |          1 |                  8 |
            | gilesbowkett.blogspot.com           |         8 |          4 |                  4 |
            | ieee-security.org                   |         8 |          4 |                  4 |
            | inko-lang.org                       |         8 |          3 |                  6 |
            | jaspervdj.be                        |         8 |          3 |                  6 |
            | jocellyn.cz                         |         8 |          1 |                  8 |
            | kyleisom.net                        |         8 |          2 |                  7 |
            | learntemail.sam.today               |         8 |          3 |                  4 |
            | m50d.github.io                      |         8 |          2 |                  7 |
            | mako.cc                             |         8 |          4 |                  5 |
            | michaelboeke.com                    |         8 |          2 |                  7 |
            | osmocom.org                         |         8 |          2 |                  7 |
            | pointieststick.com                  |         8 |          2 |                  7 |
            | probablydance.com                   |         8 |          5 |                  4 |
            | push.cx                             |         8 |          2 |                  7 |
            | pythonsweetness.tumblr.com          |         8 |          3 |                  6 |
            | randomhacks.net                     |         8 |          4 |                  5 |
            | redditblog.com                      |         8 |          3 |                  4 |
            | remotesynthesis.com                 |         8 |          4 |                  5 |
            | rystsov.info                        |         8 |          2 |                  7 |
            | sgoel.org                           |         8 |          1 |                  8 |
            | sourcesort.com                      |         8 |          1 |                  8 |
            | storj.io                            |         8 |          4 |                  5 |
            | tbf-rnd.life                        |         8 |          1 |                  8 |
            | thoughtcrime.org                    |         8 |          5 |                  4 |
            | trackchanges.postlight.com          |         8 |          5 |                  4 |
            | travisdowns.github.io               |         8 |          4 |                  4 |
            | vincent.bernat.im                   |         8 |          4 |                  5 |
            | weblog.jamisbuck.org                |         8 |          1 |                  8 |
            | wezm.net                            |         8 |          2 |                  7 |
            | wikimatze.de                        |         8 |          1 |                  8 |
            | 200ok.ch                            |         7 |          3 |                  5 |
            | alpinelinux.org                     |         7 |          4 |                  3 |
            | andregarzia.com                     |         7 |          1 |                  7 |
            | angersock.com                       |         7 |          1 |                  7 |
            | atilanevesoncode.wordpress.com      |         7 |          3 |                  5 |
            | ben-evans.com                       |         7 |          3 |                  4 |
            | bitsofco.de                         |         7 |          3 |                  5 |
            | blather.michaelwlucas.com           |         7 |          4 |                  4 |
            | blog.0day.rocks                     |         7 |          3 |                  5 |
            | blog.2ndquadrant.com                |         7 |          4 |                  4 |
            | blog.bejarano.io                    |         7 |          2 |                  6 |
            | blog.bitrise.io                     |         7 |          1 |                  7 |
            | blog.dereferenced.org               |         7 |          1 |                  7 |
            | blog.hackership.org                 |         7 |          2 |                  6 |
            | blog.honeybadger.io                 |         7 |          4 |                  4 |
            | blog.keen.io                        |         7 |          1 |                  7 |
            | blog.meetupfeed.com                 |         7 |          1 |                  7 |
            | blog.mirabellette.eu                |         7 |          1 |                  7 |
            | blog.mirabellette.netlib.re         |         7 |          1 |                  7 |
            | blog.particle.io                    |         7 |          3 |                  3 |
            | blog.plan99.net                     |         7 |          5 |                  3 |
            | blog.powerdns.com                   |         7 |          2 |                  6 |
            | blog.saeloun.com                    |         7 |          5 |                  3 |
            | blog.soykaf.com                     |         7 |          3 |                  5 |
            | blog.steveklabnik.com               |         7 |          4 |                  4 |
            | blog.superfeedr.com                 |         7 |          1 |                  7 |
            | blog.thecodewhisperer.com           |         7 |          1 |                  7 |
            | blog.threatstack.com                |         7 |          2 |                  6 |
            | boinkor.net                         |         7 |          1 |                  7 |
            | bristolcrypto.blogspot.com          |         7 |          3 |                  3 |
            | brycv.com                           |         7 |          1 |                  7 |
            | calnewport.com                      |         7 |          4 |                  4 |
            | cfenollosa.com                      |         7 |          2 |                  6 |
            | chrispenner.ca                      |         7 |          5 |                  3 |
            | chrisseaton.com                     |         7 |          5 |                  3 |
            | codearcana.com                      |         7 |          4 |                  3 |
            | corecursive.com                     |         7 |          5 |                  3 |
            | cryptome.org                        |         7 |          5 |                  3 |
            | cs.mcgill.ca                        |         7 |          4 |                  3 |
            | cseweb.ucsd.edu                     |         7 |          5 |                  3 |
            | dbader.org                          |         7 |          3 |                  5 |
            | duartes.org                         |         7 |          4 |                  3 |
            | engineering.freeagent.com           |         7 |          1 |                  7 |
            | f-droid.org                         |         7 |          2 |                  6 |
            | freebsdfoundation.org               |         7 |          5 |                  3 |
            | functional.works-hub.com            |         7 |          3 |                  5 |
            | gamedev.net                         |         7 |          5 |                  3 |
            | gpfault.net                         |         7 |          2 |                  6 |
            | gregnavis.com                       |         7 |          1 |                  7 |
            | grimoire.ca                         |         7 |          5 |                  3 |
            | gwern.net                           |         7 |          2 |                  6 |
            | hackflow.com                        |         7 |          2 |                  6 |
            | henrikwarne.com                     |         7 |          5 |                  3 |
            | hosted.ap.org                       |         7 |          2 |                  6 |
            | junglecoder.com                     |         7 |          1 |                  7 |
            | keen.io                             |         7 |          3 |                  5 |
            | keith-mifsud.me                     |         7 |          1 |                  7 |
            | kristaps.bsd.lv                     |         7 |          5 |                  3 |
            | libreboot.org                       |         7 |          4 |                  3 |
            | lukew.com                           |         7 |          2 |                  6 |
            | meiert.com                          |         7 |          2 |                  6 |
            | mgba.io                             |         7 |          4 |                  3 |
            | modern-sql.com                      |         7 |          3 |                  5 |
            | mollyrocket.com                     |         7 |          5 |                  3 |
            | mondaynote.com                      |         7 |          3 |                  5 |
            | mtlynch.io                          |         7 |          5 |                  3 |
            | natpryce.com                        |         7 |          2 |                  6 |
            | nibblestew.blogspot.com             |         7 |          4 |                  3 |
            | nikita-volkov.github.io             |         7 |          4 |                  4 |
            | opendylan.org                       |         7 |          3 |                  5 |
            | paulox.net                          |         7 |          2 |                  6 |
            | perspectives.mvdirona.com           |         7 |          4 |                  4 |
            | pkgsrc.org                          |         7 |          3 |                  5 |
            | quuxplusone.github.io               |         7 |          3 |                  5 |
            | racket-news.com                     |         7 |          2 |                  4 |
            | rcoh.me                             |         7 |          3 |                  5 |
            | rdegges.com                         |         7 |          3 |                  4 |
            | redox-os.org                        |         7 |          5 |                  3 |
            | reverse.put.as                      |         7 |          3 |                  4 |
            | rickcarlino.com                     |         7 |          1 |                  7 |
            | robey.lag.net                       |         7 |          4 |                  4 |
            | sandimetz.com                       |         7 |          3 |                  5 |
            | scarybeastsecurity.blogspot.com     |         7 |          5 |                  3 |
            | setosa.io                           |         7 |          5 |                  3 |
            | sgt.hootr.club                      |         7 |          1 |                  7 |
            | soc.github.io                       |         7 |          1 |                  7 |
            | stressgrid.com                      |         7 |          1 |                  7 |
            | strugee.net                         |         7 |          2 |                  5 |
            | sulami.github.io                    |         7 |          1 |                  7 |
            | sysadvent.blogspot.com              |         7 |          5 |                  3 |
            | taniarascia.com                     |         7 |          1 |                  7 |
            | tante.cc                            |         7 |          2 |                  6 |
            | teh.id.au                           |         7 |          2 |                  6 |
            | theartofmachinery.com               |         7 |          4 |                  4 |
            | thebuild.com                        |         7 |          3 |                  5 |
            | twolfson.com                        |         7 |          1 |                  7 |
            | ultimaratioregum.co.uk              |         7 |          2 |                  6 |
            | unterwaditzer.net                   |         7 |          1 |                  7 |
            | usrsb.in                            |         7 |          3 |                  5 |
            | vladocar.github.io                  |         7 |          1 |                  7 |
            | vmcall.blog                         |         7 |          2 |                  6 |
            | wccftech.com                        |         7 |          2 |                  6 |
            | whitane.com                         |         7 |          1 |                  7 |
            | yogthos.net                         |         7 |          4 |                  4 |
            | 0xcc.re                             |         6 |          1 |                  6 |
            | 9front.org                          |         6 |          4 |                  3 |
            | acha.ninja                          |         6 |          1 |                  6 |
            | ai.google                           |         6 |          4 |                  3 |
            | alediaferia.com                     |         6 |          1 |                  6 |
            | alex.dzyoba.com                     |         6 |          3 |                  4 |
            | alexrs.me                           |         6 |          1 |                  6 |
            | anishathalye.com                    |         6 |          1 |                  6 |
            | balaskas.gr                         |         6 |          3 |                  4 |
            | benedikt-bitterli.me                |         6 |          3 |                  3 |
            | benwilber.github.io                 |         6 |          1 |                  6 |
            | bernsteinbear.com                   |         6 |          2 |                  5 |
            | bitcannon.net                       |         6 |          2 |                  5 |
            | blog.cesanta.com                    |         6 |          2 |                  5 |
            | blog.ericgoldman.org                |         6 |          2 |                  5 |
            | blog.esciencecenter.nl              |         6 |          2 |                  4 |
            | blog.felixangell.com                |         6 |          3 |                  4 |
            | blog.frankel.ch                     |         6 |          2 |                  5 |
            | blog.fuzzing-project.org            |         6 |          2 |                  5 |
            | blog.minio.io                       |         6 |          1 |                  6 |
            | blog.park.io                        |         6 |          1 |                  6 |
            | blog.parsely.com                    |         6 |          2 |                  5 |
            | blog.prakashvenkat.com              |         6 |          1 |                  6 |
            | blog.talosintel.com                 |         6 |          4 |                  3 |
            | blog.zdsmith.com                    |         6 |          1 |                  6 |
            | blogs.apache.org                    |         6 |          2 |                  5 |
            | bowero.nl                           |         6 |          2 |                  5 |
            | branchandbound.net                  |         6 |          1 |                  6 |
            | bsdcan.org                          |         6 |          4 |                  3 |
            | bubbl.in                            |         6 |          2 |                  5 |
            | chr4.org                            |         6 |          1 |                  6 |
            | cipht.net                           |         6 |          4 |                  3 |
            | commonsware.com                     |         6 |          1 |                  6 |
            | cookieplmonster.github.io           |         6 |          2 |                  5 |
            | creativedeletion.com                |         6 |          3 |                  3 |
            | cs.nyu.edu                          |         6 |          4 |                  3 |
            | cybertec.at                         |         6 |          2 |                  5 |
            | dacav.roundhousecode.com            |         6 |          1 |                  6 |
            | dbp.io                              |         6 |          4 |                  3 |
            | developer.squareup.com              |         6 |          3 |                  3 |
            | discuss.ocaml.org                   |         6 |          3 |                  3 |
            | dmathieu.com                        |         6 |          2 |                  5 |
            | domm.plix.at                        |         6 |          2 |                  5 |
            | dotlayer.com                        |         6 |          1 |                  6 |
            | dragonflydigest.com                 |         6 |          4 |                  3 |
            | ecalamia.com                        |         6 |          1 |                  6 |
            | eclecticlight.co                    |         6 |          4 |                  3 |
            | embeddeduse.com                     |         6 |          1 |                  6 |
            | enotagain.com                       |         6 |          1 |                  6 |
            | eurogamer.net                       |         6 |          3 |                  4 |
            | ftp.openbsd.org                     |         6 |          4 |                  3 |
            | gafferongames.com                   |         6 |          4 |                  3 |
            | gamozolabs.github.io                |         6 |          3 |                  4 |
            | garbagecollected.org                |         6 |          1 |                  6 |
            | golem.de                            |         6 |          1 |                  6 |
            | gynvael.coldwind.pl                 |         6 |          2 |                  5 |
            | herbertograca.com                   |         6 |          4 |                  3 |
            | holger-peters.de                    |         6 |          2 |                  5 |
            | homakov.blogspot.com                |         6 |          4 |                  3 |
            | hydraz.semi.works                   |         6 |          2 |                  5 |
            | inessential.com                     |         6 |          4 |                  3 |
            | io.pellucid.com                     |         6 |          1 |                  6 |
            | isc.sans.edu                        |         6 |          3 |                  4 |
            | joeellis.la                         |         6 |          1 |                  6 |
            | jpadilla.com                        |         6 |          1 |                  6 |
            | julien.danjou.info                  |         6 |          3 |                  4 |
            | julienblanchard.com                 |         6 |          1 |                  6 |
            | kateheddleston.com                  |         6 |          3 |                  4 |
            | kitchensoap.com                     |         6 |          4 |                  3 |
            | korban.net                          |         6 |          1 |                  6 |
            | kyleconroy.com                      |         6 |          2 |                  5 |
            | limelight.link                      |         6 |          1 |                  6 |
            | loper-os.org                        |         6 |          4 |                  3 |
            | lord.io                             |         6 |          3 |                  3 |
            | lowrisc.org                         |         6 |          3 |                  4 |
            | markkarpov.com                      |         6 |          3 |                  4 |
            | matthias-endler.de                  |         6 |          4 |                  3 |
            | michaelnygard.com                   |         6 |          4 |                  3 |
            | mikekohn.net                        |         6 |          3 |                  4 |
            | mooreds.com                         |         6 |          2 |                  5 |
            | nelenkov.blogspot.com               |         6 |          1 |                  6 |
            | nextjournal.com                     |         6 |          3 |                  4 |
            | ngoldbaum.github.io                 |         6 |          1 |                  6 |
            | noidea.dog                          |         6 |          2 |                  4 |
            | norswap.com                         |         6 |          1 |                  6 |
            | number-none.com                     |         6 |          4 |                  3 |
            | openmirage.org                      |         6 |          4 |                  3 |
            | opensourceconnections.com           |         6 |          3 |                  3 |
            | openstreetmap.org                   |         6 |          4 |                  3 |
            | paleotronic.com                     |         6 |          4 |                  3 |
            | pattern-match.com                   |         6 |          3 |                  3 |
            | pcmag.com                           |         6 |          3 |                  4 |
            | piechowski.io                       |         6 |          1 |                  6 |
            | probablyfine.co.uk                  |         6 |          1 |                  6 |
            | purelyfunctional.tv                 |         6 |          2 |                  5 |
            | pythonforengineers.com              |         6 |          1 |                  6 |
            | realtimeapi.io                      |         6 |          2 |                  4 |
            | rkn.io                              |         6 |          1 |                  6 |
            | roy.marples.name                    |         6 |          1 |                  6 |
            | rubygems.org                        |         6 |          2 |                  5 |
            | ryanbigg.com                        |         6 |          3 |                  4 |
            | silvestar.codes                     |         6 |          1 |                  6 |
            | sizovs.net                          |         6 |          2 |                  5 |
            | streaming.media.ccc.de              |         6 |          3 |                  4 |
            | supertcp.com                        |         6 |          1 |                  6 |
            | sweetness.hmmz.org                  |         6 |          1 |                  6 |
            | talkoverflow.com                    |         6 |          1 |                  6 |
            | techblog.shutl.com                  |         6 |          1 |                  6 |
            | terathon.com                        |         6 |          2 |                  4 |
            | tomassetti.me                       |         6 |          3 |                  3 |
            | triplebyte.com                      |         6 |          3 |                  4 |
            | triplefault.io                      |         6 |          2 |                  5 |
            | typeclasses.com                     |         6 |          1 |                  6 |
            | ucare.cs.uchicago.edu               |         6 |          3 |                  4 |
            | ungleich.ch                         |         6 |          2 |                  5 |
            | venam.nixers.net                    |         6 |          3 |                  4 |
            | vidarholen.net                      |         6 |          3 |                  4 |
            | vvvvalvalval.github.io              |         6 |          3 |                  3 |
            | yggdrasil-network.github.io         |         6 |          3 |                  4 |
            | zdziarski.com                       |         6 |          3 |                  3 |
            | zerotosingularity.com               |         6 |          1 |                  6 |
            | zge.us.to                           |         6 |          1 |                  6 |
            | ziglang.org                         |         6 |          4 |                  3 |
            | zoetrope.io                         |         6 |          2 |                  5 |
            | zork.net                            |         6 |          3 |                  4 |
            +-------------------------------------+-----------+------------+--------------------+
            641 rows in set (3.42 sec)
            
            
            
            1. 37

              I am 100% on board with mitigating content marketing (especially the egregious examples you provide) but yeah… I’m uneasy about this as an active member of the community whose own blog appears on this list because it’s one of the only places I submit links to. I visit lobste.rs almost every day and appreciate that so many of the links are relevant to my interests and especially that the front page is slow moving enough that I can actually keep up with it, but I don’t submit many stories other than my own posts, and this list makes me feel like that’s not a welcome pattern. I appreciate that it is pretty close to the behavior of a self promoter (though I guess it is in a way, but I am not selling anything, I just happen not to have much else to show). This leaves me with four options, if and when I start actively blogging more (which is something I would like to do soon, and also the reason I’m commenting):

              1. start submitting more posts I didn’t write to offset those that I did.
              2. only submit my own posts when they’re extra special by some metric or another
              3. get other people to submit my posts
              4. don’t submit my posts

              1 seems spammy af, 2 is a little weird but maybe the right answer (I would prefer just to submit everything and let the algorithm sort through them, where the algorithm is people’s eyes… tbh this is sometimes an HN strength, as you’re not guaranteed a front page land there and even if you get it, it fades quickly.)

              3 is just gross, and 4 bums me out.

              also it just makes me feel bad that I fall in this bucket I guess. Is “lurker who submits their own work from time to time” just doomed to be bucketed with “content marketing garbage spam”?

              1. 14

                I’m sorry I made you feel bad, and I agree these are all bad choices. Your posts are certainly welcome, and folks up and down this thread are already brainstorming ways to fix it.

                1. 2

                  Maybe there should be a separate page on Lobsters for bloggers to submit an RSS/Atom feed! That way regularly blog content can be labeled separately, and vetted for quality of the blog itself, rather than the person/people submitting it to Lobsters.

                  1. 4

                    I certainly don’t want to have a whole RSS feed autoposted here, but I’m not saying your idea is bad - something along the line of a moderation queue for whole RSS feeds for stuff that >50% gets posted anyway is something to think about.

                    1. 3

                      I think jcs experimented with adding Planet-style aggregation years ago.

                      FWIW, this assumes 100% of the blog’s posts are in scope, and someone needs to clean up titles/tags, sometimes add context.

                  2. 16

                    So three questions:

                    1. What’s the average upvote score for articles from these sites? What about average upvote score for the articles submitted by the majority submitter?
                    2. How much content marketing stuff from dev.to, hackernoon, and medium is not on here?
                    3. How many of the majority submissions are from authors? Like I’d guess a lot of stuff from blogs.msdn.microsoft.com is from people who specifically are looking for interesting articles from there while everybody else isn’t.

                    (Disclaimer I’d really like to keep posting stuff from my site, but if the rest of the majority-things are by content marketers then it’s still worth it overall. Also, I think I’m well-known enough that other people would probably post stuff from my site anyway.)

                    1. 9

                      What’s the average upvote score for articles from these sites? What about average upvote score for the articles submitted by the majority submitter?

                      I think this is a key detail. I don’t self-submit often, but the 12 posts I have submitted over 2 years have generally been well received. I’d say the same goes for you. Well received self-submissions are something we don’t want to discourage. If someone is self-posting and the posts are not getting many votes then it seems more likely they should be subject to the post limit.

                      1. 9

                        I agree with corporate-operated blogs in this list, but I don’t understand the presence of private blogs such as jvns.ca or Daring Fireball. Those blogs usually have high-quality content (in my humble opinion) and I’ve never seen any form of content marketing on those websites (I could be wrong though).

                        1. 7

                          I think Daring Fireball is such an outlier, isn’t it like one the top 3 most-read sites with a 100% Apple focus anyway? I’m not using any Apple stuff, but I can’t help but getting the news anyway - and I personally don’t like reading it.. but I think it’s very often news-heavy and we don’t so many “New product by X” posts here, for a good reason.

                          Yes, Apple is a bad topic for me to comment, but Daring Fireball is not a personal blog anymore. He’s one of the major Apple-ecosystem influencers and pundits. This is 100% a business website by now.

                        2. 2

                          I also think this would be a useful component to include. My domain falls on the list, and I’m a bit sad to see my domain on the list. It’s technically content marketing, but I put a massive amount of effort into each post and the content is generally very well received both here and on Hacker News. I would understand the decision to block it, but I think that taking the average submission rating into account would still address the problem without eliminating content that is a good fit for the site. This also incentivizes people to only submit their best content.

                          1. 1
                            1. Averages are misleading because of site growth over time. (Related)
                            2. A significant amount. (Related)
                            3. Feels like most, but this a trickier query than I have spare brainpower to write. I worry that it would incentivize not checking that box because it’d be subtle and deniable.
                          2. 14

                            Is there anything we could do in terms of vetting a user? For example, as far as I am concerned, @andyc, @akkartik and @hwayne all submit rather high quality content from their own blogs, I’d rather not miss them.

                            1. 11

                              This just struck me when I was trying to submit my Nix rebuttal post.

                              1. 2

                                That’s a shame… I’ve always enjoyed your posts on christine.website and I think it would be a real loss if your website were to be blocked.

                                1. 2

                                  Apparently it’s going to be fixed and this was just an extreme over-reaction. I’m still gonna post though :)

                              2. 10

                                15 of the top 30 in this list are sites that I probably actively visit when they’re linked from Lobste.rs and otherwise don’t visit. Excluding them may surface less popular content but it would also seem to reduce the visibility of content that I’ve come to trust and want to get from Lobste.rs. 7 of the next 20 are in that same bucket, so penalizing by this metric would inhibit submissions 44% of the top 50 domains considered content marketing.

                                1. 8

                                  @pushcx, thanks for all the hard work on the site. I really appreciate this site a lot. Now, I’m a bit offended to be included in this list. I’m a web developer and volunteer for some FOSS communities, I have a low traffic blog and every now and then I share some of my blog posts here. Not all of them, just the ones I think have value for users here. I’m not selling anything on this site, and my blog posts are not related to anything I sell.

                                  If this website is going to be hostile to blogs with such low traffic as mine with very few posts per month, about 500 users per month, then I think this site loses a ton of value. Not because they will be missing me but because the criteria that includes me and my site here, will also include lots of other minor bloggers. No one is posting my posts here because I don’t have a large number of readers, I’m probably faster to share content than some hypothetical reader.

                                  Seriously, scanning this list I can see many blogs I actually subscribe and that provide me with good content, not upselling anything. I don’t think this metric is a good way to filter who you want to filter. What this metrics leads to is that only content from domains that are so popular that a ton of people repost content from them will end up in this site. This is not only an echo chamber that makes whatever is popular, more popular. But also gatekeeping small people who just want share their own story every now and then, and will prove that the blogosphere is dead since we can’t share blog posts.

                                  1. 6

                                    As Lobster’s is my primary article discovery site, I’m a bit worried that this change will cause me to miss great articles because the author couldn’t post it, and they don’t have other avenues / connections to get it noticed (read: they suck at marketing, don’t have an RSS / atom feed, etc).

                                    I typically self submit (I’m on the list here) and sometimes it gets upvoted, sometimes it doesn’t. I kind of thought that was the purpose of votes, and flags, and such, but ¯\_(ツ)_/¯.

                                    At least this change creates a new “market.” We can create a voting ring of sorts that keeps track of their submissions to certain domains and nominates someone to submit queued up links. The list of domains above is a good seed list of potential members, too….

                                    I should probably just go back to over subscribing to RSS / Atom

                                    1. 6

                                      Could you run this query? I’m not entirely sure it’s correct, but it should sort by the percentage of single user submissions.

                                      select domain, count(*) as submitted, count(distinct stories.user_id) as submitters, (select count(*) from stories s where s.domain_id = domains.id group by s.user_id order by 1 desc limit 1) as from_one_submitter, (select count(*) from stories s where s.domain_id = domains.id group by s.user_id order by 1 desc limit 1)/count(*)*100 as submited_by_one_percentage from domains join stories on domains.id = stories.domain_id group by domain having count(*) > 5 and (from_one_submitter + 1) * 2 > count(*) order by 5 desc;
                                      
                                      1. 6

                                        Looks correct to me. Results.

                                        (I put the original list in a comment because I saw it as central to the discussion, so I didn’t want it on free hosting even though I don’t have a convenient place to indefinitely host a small static file attached to Lobsters.)

                                      2. 5

                                        I was a little worried to see my name On A List, and also a little flattered to know that my blog was getting so many posts from here. I had to look into who was doing all the posting, and, well, maybe you ought to watch out yourself ;)

                                        1. 4

                                          TIL I’m a content marketer, alongside such evil spammers as the Free Software Foundation and Hillel Wayne! And the tell was that I used federated blogging technology—my own article feed at my own domain, integrated into the fediverse via RSS—rather than high-quality centralised platforms like Facebook notes or Medium.

                                          Unfortunately, I’m really bad at content marketing. I tend to publish articles that explore ideas about software that I (and others, mercifully) find interesting, rather than SEO-first eyeball scrapers. I forgot to monetise with affiliate content, adverts, store links, or anything. Nobody even clicks the tip jar button on the site.

                                          I guess I need to find a community where us “Badly-marketed Low-value Output Generators”, or “BLOGgers”, can share, discover, and comment on each other’s posts without disrupting those people who are focused on consuming Medium-rate content. Preferably with a focus on programming, just because that is the niche vertical I have chosen to exploit with my rational-minded acquisitive process. Does anyone know where that community hangs out?

                                          1. 4

                                            Maybe copy query output to a gist or nopaste and link it?
                                            I found it hard to read due to line wrapping. :/

                                            1. 3

                                              Wow… Okay. Did this site just become hostile to its users? My site is on the list.

                                              1. 4

                                                No.

                                              2. 1

                                                First of all, thanks for the good work. Now, since you invite:

                                                kibitzing about particulars

                                                and pointing out false positives. I expected @ahu’s site to be on there and it is[1], since one of his stories got moderated in a way I disagree with. (And I see other sites that I think are high-quality, but I’m happy to also see a lot of low-quality sites on the list.)

                                                Having said that, if you look at his submissions you’ll see that it is mostly geeky stuff so I would rather classify him as an author than a content marketeer. He is a geek and happens to be a pretty good writer. His last article on Huawei, 5G and Europe has been doing some good rounds on Twitter (I won’t link to it directly but search for “5G: The outsourced elephant in the room”).

                                                The only thing he perhaps went overboard with is the DoH centralization by Firefox on Cloudflare, which is an item we (PowerDNS) are pretty concerned about, but not for business reasons as implied by its proponents. Hell, we live in The Netherlands and couldn’t care less if we get fired or not since we got actual social security.

                                                [1] ds9a.nl, but I didn’t expect blog.powerdns.com to be there as well

                                                disclaimer: I’m a PowerDNS employee and ahu was my ‘boss’

                                              1. 8

                                                Ah, here is a topic I am somewhat well-versed in! Uplink is a classic – I spent plenty of time with it in my adolescent years. There are definitely more than 2 of these games!

                                                • Slavehack - an MMO of sorts. You have your own box & IP address and spend time hacking into both NPC machines and other players’ machines. It has a non-linear “quest line” of sorts which allows you to upgrade your software (viruses, firewalls, password crackers, etc.). A big part of the game is clearing access logs when you access a remote machine because other players might find your IP address and try to make you part of their botnet ;)
                                                • Blue Sky - Made by exosyphen studios, who have made a lot of hacking games. At the time I played it, I remember it being described as an uplink clone, but “Blue Sky” was definitely almost as well-known as Uplink.
                                                • Hacker Evolution - Also made by exosyphen. This was the first hacking game I ever played as a kid, so it holds a soft spot in my heart. At the time I seem to remember people describing it as “worse uplink”, so I don’t know if it still holds up. They seem to have spun a whole series out of this, with some quite recent entries (released in 2015 and 2016).
                                                • Hacker Experience - This was announced on HN and reddit a few years ago. I remember playing it briefly and enjoying the interface. It reminded me of a more modern take on Slavehack. Here’s an article about what happened to it and here’s a video of someone playing it if you want to see what it was like. It looks like someone took the source code for Hacker Experience (it was open source if I remember correctly?) and put up Hacker Wars.
                                                • hackmud - I’ve not personally played this, but it looks quite interesting. I’ll probably give it a try sometime.

                                                Lastly, Digital: A Love Story is really more of a visual novel, but it has some hacking themes, and I quite enjoy the narrative. I highly recommend trying this one, even if it isn’t a “hacking game” like the ones above.

                                                1. 2

                                                  I have tried both uplink and hacker evolution, and I can see the characterization of “worse uplink”, however its important to remember that uplink is a really good game. I think hacker evolution is still fun, but if you’ve never played uplink it’s probably worth playing through first.

                                                  1. 1

                                                    This is all very interesting, thank you for sharing! Do you think these games use total immersion, or is it something specific to Uplink and Hacknet?

                                                    1. 2

                                                      Hmm, I don’t recall any of these games being “totally immersive” as you describe. I seem to remember Slavehack had a radio station and corresponding live chat, but I might be mis-remembering. The forums were also a bit like game cheating forums, with people posting their “wins” and “tutorials” on how to hack–it wasn’t role play, but it had a similar atmosphere to me. Anyway, this has all gone away since it was many years ago and I’m probably mistaken on some details :)

                                                  1. 21

                                                    My friends and I are the type who love Zachtronics games and things like Factorio and Kerbal Space Program.

                                                    These games are great and can teach a lot - and yet, once played for a few hundred hours, they start to feel empty. After a long conversation probing this problem, one of my group’s most elite players said something I find very profound.

                                                    “Nothing simulates engineering as well as doing engineering.”

                                                    “Hacking”, as you use the term, is basically a very specific kind of software engineering. Unlike aerospace engineering, it’s very accessible and rewarding in a short time - unlike game development. So maybe the real reason there are so few games like this is that the people who would be making them are off hacking things instead.

                                                    1. 6

                                                      Yeah, I think you’re right. After a while, these sort of games start feeling like work, at which point you might as well do the actual work instead, and gain something from it!

                                                    1. 1

                                                      Nice one! A few months ago I set up my own streaming server, but the latency was way too big for my usecase. What’s the latency like on your setup?

                                                      1. 2

                                                        If you want low latency streaming of your desktop to a small, private audience (eg watching a movie together or playing a game with a friend) checkout Parsec. The latency is so low it’s scary.

                                                        1. 2

                                                          5-10 seconds, it’s about what I get with Twitch/Youtube so I think it’s good enough :)

                                                          1. 1

                                                            I see, that’s pretty much what I was seeing too. Thank you for your response!

                                                        1. 1

                                                          Reading this makes me very sad I had ro drop Godot. The main reason for it being that everyone I’ve heard of uses Spine by Esoteric Software Ltd for animations. There’s a dead fork of a fork of a repository adding support, but although some things have been easy to maintain, it’s broken on Godot 3.1.

                                                          Esoteric has been promising support for something like two years and recently went on record as not having started the work yet(!)

                                                          I’m sure Godot’s cut-outs would suffice if animators knew how to work them.

                                                          It would also be nice to be able to add proper 3D elements into a game made in 2D mode, but with the focus on figuring out OpenGL and Vulkan for another half-year, I don’t really see this is a fit for a long time.

                                                          If you don’t need those things, I recommend Godot. The community is sweet and the editor is joyful.

                                                          1. 1

                                                            Godot recently added the ability to transform 2D sprites using meshes & bones - with a bit of work that should be able to do most of what Spine can do.

                                                            As for proper 3D elements, there is the option of using a render target texture to show a 3D scene in the 2D viewport. However, I expect you’d be on your own when it comes to interacting with the 3D objects - you’d have to do the coorditante transformation maths yourself.

                                                            1. 2

                                                              Outsourcing animations to someone used to dealing with separate Spine files does unfortunately not match very well with anything in the proper engine. This is probably the reason a third-party product is popular :/

                                                              There were other small glitches as well, which may have been fixed now, like parallax layers and camera zooms not working together too well.

                                                              Regardless, I wish Godot the best!

                                                              1. 1

                                                                Oh yeah, from that perspective the Godot animation story is far from ideal - any animator would have to work in the likely unfamiliar Godot editor, instead of being able to specialise in the one tech that works across engines. And, while it is constantly getting better, Godot does come with a certain amount of ‘open source’ clunk that would be especially painful to people used to dealing with polished proprietary tools.

                                                                If you don’t mind me asking, what engine are you switching to?

                                                                1. 2

                                                                  UE4 would be overkill, and its Spine support seems to lack normal maps, though I find it very interesting.

                                                                  We conducted a feasibility study of porting our scripts to a Unity asset, by making an interp or compiler of sorts. It looks very promising and should save us money in the long run.

                                                                  The sad part is that I got very familiar with Godot, nearly fell in love with it, and know very little of Unity. Now I’m making enough in billables from Python work that I can outsource a bunch of the compiler work and hope this doesn’t backfire, and after this gig see where we’re at.

                                                                  OTOH we’re needing a meta-ish script system anyway, so if the asset fails us, all we need is another “backend” for Unity.

                                                                  If Godot worked, and if I hadn’t spent time and money making stuff on it only to learn a design concept for a Unity-based architecture, it would be more affordable. Unity on Linux was also pretty much a broken hack when we started and I prefer working through Linux; now it’s way better.

                                                          1. 7

                                                            Working on a Rust REPL for exploring music theory, with an associated library that lets you describe music using code. Right now, it’s essentially Google for chords! The target audience is musicians who find it difficult to work with harmony on their main instrument (drummers, bassists, guitarists to an extent, bowed instruments, woodwind & brass…)

                                                            I’m hoping to open-source the code this weekend. Let me know if this sounds interesting :)

                                                            1. 1

                                                              Sounds interesting!

                                                            1. 4

                                                              Page doesn’t open without javascript enabled. Can’t imagine why it would be required for a blog post.

                                                              1. 1

                                                                Flag the post as broken link. That’s what I regularly do with Medium.com links.

                                                                1. 1

                                                                  Thank you for your comment, I’ve flagged this up with our web developer :)

                                                                  I’ve rehosted the post on GitHub in the meanwhile.

                                                                  EDIT: It should work without JavaScript now!

                                                                1. 2

                                                                  Anybody who has tried to set up any sort of non-trivial IoT network has run into this issue. It is very difficult to create a network which covers the entire house without relying on a plethora of gateways and routers, which significantly increases complexity and cost

                                                                  I think a layer 3 switch supporting VLANs is enough. Bonus if the switch can provide power through PoE fore wire-connected devices.

                                                                  1. 1

                                                                    You are correct, that would make setting up a network easier. And you can certainly do a lot of IoT work that way!

                                                                    However, the point I was trying to make was more about interoperability - right now a lot of IoT protocols make it difficult to talk to the WWW, usually only the gateway can do that. This means that the gateway will likely need custom software that translates between web standards and its proprietary protocol. Thread simplifies things by natively supporting IPv6 routing right down to the end device.

                                                                    There are also applications where connecting an Ethernet cable is impractical. However, these mostly pop up in industry. For instance, using IoT tags to determine the location of free range cattle, or any other application where a radio device has to move a lot.

                                                                  1. 3

                                                                    How does Tcl compare to something like Bash, which also seems to be optimised for typeability?

                                                                    Hm, I suppose that Bash (the language) doesn’t really do much for you. What’s really useful for typing things quickly is the Unix CLI interface paradigm, but even that can be inconsistent. I wonder what a scripting language that brings the concept of CLI tools and arguments to the language level would feel like (e.g. let the user define their own functions, used like foo -a baz --bar baf output.txt

                                                                    1. 11

                                                                      One thing Tcl does better than bash: if you use a variable containing spaces, that won’t get exploded into multiple words. (unless you ask for it)

                                                                      $ tclsh
                                                                      % proc dashit {first rest} {puts "$first -- $rest"}
                                                                      % set x [list one two done]
                                                                      one two done
                                                                      % dashit $x "more text"
                                                                      one two done -- more stuff
                                                                      

                                                                      Contrast that with bash:

                                                                      $ bash
                                                                      $ x="one two done"
                                                                      $ function dashit() { echo $1 -- $2; }
                                                                      $ dashit $x "more text"
                                                                      one -- two
                                                                      

                                                                      Hm, I suppose that Bash (the language) doesn’t really do much for you. What’s really useful for typing things quickly is the Unix CLI interface paradigm.

                                                                      If we subtract the Unix CLI interface paradigm from bash/dash/sh/fish/zsh/ksh/csh, is there a language left over? If bash didn’t conform to that paradigm for you, it wouldn’t be bash but a completely different language, less suitable for shell use.

                                                                      cp my/file $myarchive  # Bash
                                                                      
                                                                      run(['cp', 'my/file', myarchive]  # Python, after `from subprocess import run`
                                                                      

                                                                      (By the Unix CLI paradigm, I suppose you mean mrules like

                                                                      • first word is a command (or look for an executable of the same name)
                                                                      • subsequent words are command arguments
                                                                      • words are space-separated, so quoting arguments is unneccesary
                                                                      • refer to variables with $varname.

                                                                      I wonder what a scripting language that brings the concept of CLI tools and arguments to the language level would feel like (e.g. let the user define their own functions, used like foo -a baz --bar baf output.txt

                                                                      That’s pretty much exactly what Tcl is. You’re even allowed to redefine built-in names! Have a look at “Tcl the misunderstood”, especially the headers “Eval and uplevel” and the following bit from “Radical language modifications = DSL”:

                                                                      Tcl looks like a configuration file by default:

                                                                      disable ssl
                                                                      validUsers jim barbara carmelo
                                                                      hostname foobar {
                                                                          allow from 2:00 to 8:00
                                                                      }
                                                                      

                                                                      The above is a valid Tcl program, once you define the commands used, disable, validUsers and hostname.

                                                                      1. 2

                                                                        Wow this was a very thorough comment, you’re making me want to learn Tcl now!

                                                                      2. 7

                                                                        FWIW I started to morph shell/bash into a sane language with Oil, while still retaining its typeability.

                                                                        • You can opt into a mode that removes the need for double quotes in almost all cases. In other words, there will be less of a difference between the “interactive session pasted into a file” and the “right thing”.
                                                                        • I improved the semantics of arrays and associative arrays. The lack of any real data types besides strings is something that makes shell seem like “not a programming language”. It has loops, functions, and conditionals, but it doesn’t have good data types like Python. (Or even Tcl, which doesn’t have data types per se but has predictable parsing of strings into data types!)
                                                                          • A nice typeability improvement is that you can opt into @a as an alias substitute for "${a[@]}" (which is the correct way to use arrays). The latter is really hard to type correctly and I didn’t even know it before I started implementing a shell! I find that most people don’t use arrays in shell (because they’re not POSIX, and they have bad semantics.)
                                                                        • I’m adding a JS-like var to which you can think of as a better and builtin version of Tcl and shell’s expr
                                                                        • He says that Matlab arrays [1 2 3] are better than lists [1, 2, 3]. Oil will have lists for JSON compatibility, but it also has arrays like this @[1 2 3] or @[bare words], rather than [1, 2, 3] and ['quoted', 'words'].
                                                                          • Some people suggest using Python for shell-like tasks, but I do think it would be silly if you had to type ['ls', '-l'] instead of ls -l.

                                                                        I linked to some details here: https://lobste.rs/s/pk64xp/p_idea_faq#c_lkpane

                                                                        I am looking for feedback on these new features! Right now the best way to provide feedback is to login to Zulip, which supports a few login services like Github so you don’t have to create a new password.

                                                                        BTW shell functions work great and have a decent syntax. They let you define something that behaves just like an external tool or builtin. They can be transparently put in pipelines, which no other language provides (you can think of shell functions as having “proper” stdin, stdout, stderr, unlike Perl, Ruby, or Python functions).

                                                                        However one funny problem is that the shell builtin getopts is difficult to use and inflexible. It has both “hidden state” and implicit variables which are hard to get used to (e.g. if you’re coming from Python, Ruby, JS, etc.)

                                                                        So fixing/enhancing getopts is also on my TODO list. There is a bunch more and I’m looking for people to try rewriting some shell snippets with them and give feedback. I rewrote a couple small programs, shown in the linked Zulip posts.

                                                                        1. 2

                                                                          One confusing thing about shell functions is that you can define global variables inside them but that doesn’t work if you call the function in a pipeline or an expansion, so given

                                                                          foo() { x=1; echo foo; }
                                                                          

                                                                          you can do

                                                                          foo; echo $x
                                                                          

                                                                          but not

                                                                          foo | grep o; echo $x
                                                                          

                                                                          or

                                                                          printf "%s bar\n" $(foo); echo $x
                                                                          

                                                                          The explanation is that running the function in a pipeline or substitution implies running it in a subshell which can’t set variables in the outer shell, but this semantics can cause weird problems when you’re refactoring…

                                                                          1. 1

                                                                            Yes, one idea I had a long time ago to address that was to force a prefix on shell functions in pipelines, to make the subshell explicit, sort like:

                                                                            ls | subshell myfunc | wc -l
                                                                            
                                                                            instead of 
                                                                            
                                                                            ls | myfunc | wc -l
                                                                            
                                                                            Or more conventionally
                                                                            
                                                                            ls | (myfunc) | wc -l
                                                                            

                                                                            But I’m not sure if it’s a good idea now, and there are a bunch of other pitfalls I think are more common to address first.

                                                                            Other things that Oil does better that can mitigate it:

                                                                            • variables aren’t global by default. Usually in functions you want to mutate locals.
                                                                            • variable names are statically known, e.g. something like local $1 where $1 is var=val isn’t idiomatic. So you could warn about this in a lint tool. I think that might end up being the best option.
                                                                        2. 5

                                                                          How does Tcl compare to something like Bash, which also seems to be optimised for typeability?

                                                                          I’d say that it’s about the same, maybe better. Most simple shell commands are also legitimate TCL commands. The substitution rules are cleaner though, function definition is nicer &c.:

                                                                          echo [ls -l /etc]
                                                                          proc foo {bar baz} {exec echo $baz $bar}
                                                                          

                                                                          versus:

                                                                          echo $(ls -l /etc)
                                                                          function foo () { echo $2 $1 }
                                                                          

                                                                          (Typed on my phone, not tested, might not be right, I haven’t used Tcl in a year or two)

                                                                          1. 2

                                                                            It’s simplicity also makes it easy for 3rd parties to implement on their own. That makes it more popular in appliances and embedded, esp for administration. Cisco is probably the most famous use case. Here’s a tiny implementation that runs in 10KB on a microcontroller. In the 1990’s, it was also used by ArsDigita for web apps.

                                                                          1. 2

                                                                            Thanks for sharing. I don’t know how anyone in good conscience could contribute to this as an open source maintainer though.

                                                                            1. 3

                                                                              Looks like most of the contributions come from accounts made purely for working at GCHQ: a lot of them follow a naming convention of letter + 6 digits, and only commit on working days. https://github.com/gchq/Gaffer/graphs/contributors

                                                                            1. 6

                                                                              A WWW developer not knowing HTML, while amusing, is merely a symptom of popularity damaging the WWW. The rest of it is that the WWW was fundamentally poorly designed from the beginning.

                                                                              The maximum that a format permits is rather important, as it dictates exactly how much the format can be abused. The WWW has already killed itself with JavaScript, which ties into this very article.

                                                                              In any case, HTML is awful and weak, CSS is complicated, and if you need any further evidence the WWW was broken from the beginning, then considered that HTTP is a textual protocol with misspelled words. It’s garbage from the start.

                                                                              Still, the old garbage that was the WWW was better than this new garbage, sure.

                                                                              1. 1

                                                                                Is popularity the biggest source of damage, or is it orthogonal to it? I feel like the excessive commercialisation of the web is a lot more harmful: there are ads and tracking almost everywhere you go, laser-targeted SEO is bringing less-than-useful articles to the forefront (and I am being polite here, try finding any sort of cooking recipe nowadays), and probably the worst offender is ads that disguise themselves as content. I’m sure a bunch of the latter are regularly shared here!

                                                                                You could say that the reason people are trying to make money off of the web is because it’s popular, but I don’t believe that there is something innate about popularity that makes its commercialisation necessary.

                                                                                1. 4

                                                                                  I find a physics metaphor helpful: popularity exerts a cultural force in the direction of commercialisation, which requires a continual opposing force to hold back, in the same way that water will flow downhill unless stopped.

                                                                              1. 1

                                                                                For me it depends on the sort of work. There’s only about 5-6 hours of creative and thoughtful programming in me, the kind that is truly productive and leaves you satisfied at the end of the day. Those hours are interleaved with breaks where I do shallower work, chat with colleagues or keep up with tech news.

                                                                                However, if I’m debugging code and staring at GDB, I can see myself doing that for 7 hours straight without really noticing time go by, which is probably not healthy…

                                                                                1. 7

                                                                                  Is the problem OP complains about (professional front-end devs who don’t know HTML) real? When I was a kid, there was nary a middle-school student who didn’t know HTML…

                                                                                  My biases make me want to believe him, but if I want to believe something, that’s a red flag that I should be extra careful to make sure it’s really true.

                                                                                  1. 10

                                                                                    It is absolutely real. Many devs learn a framework and only speak and think within that framework, not realizing it’s a pile of code that sits on top of the ‘native’ html/css/js stack. I’ve worked with several. It’s a dangerous mindset to take but it isn’t all bad. Those frameworks exist to increase productivity, so some amazing stuff gets built quickly. But sometimes, without that lower-level knowledge (and I kind of smirk at the idea that html/css/js is ‘low level’), weird things happen.

                                                                                    1. 6

                                                                                      Wow. I’ve run into some folks who are afraid to write vanilla javascript, but I’m lucky enough to have never run into somebody who can’t at least generate HTML.

                                                                                      1. 2

                                                                                        I was reading this and waiting for the point where it would say something that wasn’t blindingly obvious (to me), but it never happened. But I suppose there must exist people to whom this is news, otherwise how would you explain the existence of so many bad web apps?

                                                                                      2. 2

                                                                                        I’ve been learning webdev recently (I have experience with embedded & IoT programming, but not much actual web stuff) and I find it sort of unfathomable to not learn anything about such a fundamental technology…

                                                                                        1. 3

                                                                                          Right? Though, to be fair, when I started coding it was also pretty normal for people to learn assembly – the expectation to understand history & the system as a whole has gone down as entry-level wages have increased.

                                                                                          1. 2

                                                                                            But then again assembly is one or two levels of abstraction above literally thinking about voltages and transistors, which are incredibly complex and basically require abstraction to do any sort of productive work!

                                                                                            Is the same thing true about HTML and CSS? Maybe, but definitely not to the same degree. I think you could draw the line at Single-Page web applications, which is where I believe relying only on vanilla JS makes things difficult. But a lot of websites don’t need to be SPAs, and creating something simpler would make the web significantly more accessible.

                                                                                            1. 5

                                                                                              I wouldn’t say that assembly is one or two levels of abstraction above voltages. Assembly is one level of abstraction above microcoded machine language. Beneath that, you’ve generally got people thinking in terms of gates. In college, as part of a sophomore-level class, we were supposed to design a 4-bit CPU from gate level to opcodes & then write assembly for it – something that wasn’t terribly difficult & gave some insight into how CPUs work, but also didn’t teach me a damned thing about electrical engineering.

                                                                                              HTML & CSS are document-formatting systems, and they require workarounds and abstractions in order to be borderline usable as application GUI libraries. Even then, there’s enough abstraction leakage that I would expect everybody who does web app development, even if they primarily use a big framework, to occasionally need to write some plain javascript that injects plain HTML into the DOM.

                                                                                      1. 2

                                                                                        I have been wrestling with these exact same concepts for two weeks now! Very useful blogpost and I wish I had it earlier.

                                                                                        1. 8

                                                                                          It’s nice to know I’m not the only one that’s grumped about this.

                                                                                          There is a very real war for space and audience on places like Lobsters. We have to be observant.

                                                                                          1. 2

                                                                                            I feel like Lobsters in particular is a lot better about this. HackerNews is pretty much all hype, but Lobsters feels like a place where you can almost always have genuine technical discussions.

                                                                                            I hope it stays this way!

                                                                                            1. 14

                                                                                              The price of good content is eternal vigilance and a vigorous use of flags.

                                                                                              1. 3

                                                                                                …but it’s far from enough. The crowd that gathers around a medium makes the quality.

                                                                                              2. 4

                                                                                                Given my contentious history on HN, I find it to be about 1/5 to 1/3 shills and otherwise people trying to set the narrative. Of course, underlying it all is a VC firm. And what dang and such write, is you don’t do something for free in business. Unsurprisingly, it’s used as a media manipulation tool.

                                                                                                With lobsters, I don’t feel like I’m a product. I don’t feel sold to, nor do I feel like a piece of meat to be slabbed and weighed. Sure, we have our spats here, but they feel like the old usenet style holy war battles.

                                                                                            1. 4

                                                                                              I’ve been following Rust for a while and it’s shaping up to be a very interesting language. Focusing on polish also seems like a sign of maturity, which is welcome.

                                                                                              Is anyone familiar with embedded & bare metal applications of Rust? How does it compare with C++?

                                                                                              1. 12

                                                                                                My company, Ferrous Systems, actually does it’s main business in embedded. We didn’t expect that, expecting Rust on microcontrollers to be something to carry for a while, making our money with database development and stuff. We run trainings, which sell extremely well. Also, we have a couple of clients where we prototype new platforms.

                                                                                                The current state of Rust on embedded is: If you have no problem writing some drivers on your own, or linking C libraries for that, we’re there on ARM and some other microcontrollers. RISC-V and ESP support is coming. There’s patterns, even higher-level libraries like RTFM and small RTOS available and in use. Cargo and the whole stack supports cross compilation and is really clean and nice to work with. There’s an professional ecosystem, even if it is young.

                                                                                                Language-wise, we’re done and stable, though.

                                                                                                This year, the goal is to make Rust embedded more featureful, by supporting more devices. Next year, the goal is to make it as convenient as desktop Rust.

                                                                                                Adding to what @iswrong is writing, I can also highly recommend the the Decawave DW1000 boards, which we have great support for, if radio applications are interesting to you.

                                                                                                Rust lands somewhere between C and C++: It has abstractions like C++ and uses them to full effect, but it is fundamentally a “functions and data” language like C, without classes and (automatic) dynamic dispatch.

                                                                                                I’m happy to answer any specific questions!

                                                                                                1. 2

                                                                                                  That is actually very encouraging, thank you for your answer! My workplace has been using IAR for their bare metal work, and it’s left a bad taste in my mouth - we stumbled into a compiler bug that could only be fixed by disabling a specific optimisation, or by spending thousands on a new version of the compiler. I feel like Rust’s development model would’ve given us a lot more agency in solving that particular problem.

                                                                                                  If we end up going for an ARM micro for a future project, I will definitely start a discussion on Rust.

                                                                                                  Concerning C++ vs. Rust, I was mostly curious about how they compare in terms of cross-platform capability. For instance, IAR doesn’t have the best support for modern C++ features, so we have mostly been using GCC whenever we can get away with it. In your experience, how does Rust adoption compare to modern C++ (C++11 or 14)?

                                                                                                2. 6

                                                                                                  I did some embedded Rust on STM32 (Blue Pill) and the Micro:Bit for fun recently. The basic stuff is in place: you can compile for embedded ARM targets, the binaries (with no_std) are lean enough to put on flash. You can debug programs on-device with e.g. STM-Link v2 or a Black Magic Probe. Library-wise there is also quite a lot of plumbing, such as the heapless library for basic containers that do not do dynamic allocations. Obviously, you also get a lot of the safety benefits from Rust, this goes deeper than one might expect. The device peripherals are types such that it is hard to use them incorrectly. E.g., if you use a pin as an input pin, it gets an input-specific type:

                                                                                                  https://docs.rs/stm32f1xx-hal/0.2.1/stm32f1xx_hal/gpio/gpioc/struct.PC13.html#method.into_pull_up_input

                                                                                                  Where it is still quite lacking is support for drivers: there are drivers out there, most of which are listed on the page that @nickpsecurity links to. But they vary from full-fledged (e.g. the SSD1306 OLED display driver worked pretty well) to very rudimentary. Networking is also limited, there is japaric’s stack, but japaric states that it is mostly a personal experimental thing. smoltcp looks promising, but it’s still quite a bit of work to hook it up with drivers, e.g. I tried to use smoltcp with an ENC28J60, which entails writing some low-level code yourself. The same is true for e.g. Bluetooth support, what is out there is still rudimentary.

                                                                                                  So, it very much depends on what you want to do. If you have some application that primarily uses the microcontroller or some basic peripherals (LEDs, LDRs, or whatever), you are fine. If you want to do more, do some investigation before buying specific I2C devices.

                                                                                                  If your goal is just to play a little with Rust on embedded, you can do it now. Just get an STM discovery board, Blue Pill (with the right resistors and a good regulator), or a Micro:Bit. Blue Pills can be had for a few dollars. So, you can play around with embedded Rust without breaking the bank. If you go for a Blue Pill, also get an STMLink v2 (clone) device, it makes your life much easier for flashing and debugging than the STM32Duino bootloader that some devices come pre-flashed with.

                                                                                                  1. 6

                                                                                                    Check this out.

                                                                                                  1. 3

                                                                                                    I’m not entirely sure who this article is written for or what it’s trying to achieve. It heavily uses FP jargon but then goes in and insults the very people who are most likely to understand it.

                                                                                                    Language interop is a noble goal to strive for, but I’m not sure whether a new operating system is how you would fix that. Different languages have very different calling conventions - C and Python could not be more different, and I think that even C++ and Rust use CPU registers in different ways whenever you call a function.

                                                                                                    1. 9

                                                                                                      I’m not entirely sure who this article is written for or what it’s trying to achieve. It heavily uses FP jargon but then goes in and insults the very people who are most likely to understand it.

                                                                                                      I felt the same way scanning it for a time, but by the time I got to the end it had won me over. So maybe it’s written for me?

                                                                                                      I think a lot of people in the FP crowd tend to say very simple things in a very complex way – but I admit even though I think these things are simple, I don’t know how to explain them in a less complex way that preserves the accuracy of the FP explanation!

                                                                                                      Obviously analogies can be made “simple” explanations, but there’s always some subtle gap that if you nit too hard you get shit on.

                                                                                                      I keep in the back of my mind that FP has that going for it, and as we struggle to understand how to best tell computers what we want them to do, FP offers a much better jumping off point than starting from register models or NAND gates (or the universal turing machine). But can we do better?

                                                                                                      I think so, and since I don’t see many research efforts in this space, this type of article provided a certain about of reassurance to that.

                                                                                                      I’m not sure whether a new operating system is how you would fix that. Different languages have very different calling conventions

                                                                                                      Sure, generally this kind of glue is called FFI, and it’s either implemented with some kind of trampoline code assembled at runtime, or an IPC. Most of what you’re thinking about is in the former, but SQL and databases typically fall into the latter (and yet aren’t required to! see sqlite).

                                                                                                      However in multics and VMS the typical strategy for a module wasn’t to have a main() function, but to have several main() functions exposed. It’s a little like having every function marked int f(int argc,char *argv[],char**envp) and simply being able to call any of them, but none of these main() functions parse commandline switches (or roll an IPC, except for interop. with other systems), because your data is also tagged (or boxed) whenever you cross one of these boundaries.

                                                                                                      Languages with concrete data types (C, q, maybe Java?) this is easy, but algebraic types throw a massive wrench into it because now the algebra needs to have equivalence across the boundary, and it doesn’t: Consider a json document that you’ve “parsed” with Haskell– there’s no way to hand over that (parsed) document type to OCaml without the same boxing required to speak to C. It’s bonkers! Having introspection like JavaScript and Python and Erlang feel like they get you further because the middleware can poke around at the types (does it have a .then method? it’s a promise!), but this too is a lie: how can you pass that closure over the network, and what happens if when it’s there it tries to edit the scope in the original process? (q avoids this by simply not having closures, but it’s a bit unsatisfying…)

                                                                                                      Dot-Net and Powershell look like a good move in this direction, but “everything needs to be dot-net”, and as great as F# is, talking to the rest of the dot-net world brings back the boxes…

                                                                                                      … maybe a new operating system is needed?

                                                                                                      This isn’t so strange: Dot-net is basically it’s own operating system; Erlang’s VM is basically it’s own operating system. With a broad-enough definition of “operating system”, this line of thinking definitely makes sense, so maybe for the traditional definition of an “operating system” this can make sense too.

                                                                                                      I’m not yet convinced of this, but I’m definitely going to keep my eyes open.

                                                                                                      1. 3

                                                                                                        As someone who understands FP jargon, I’m confident the author does not.

                                                                                                      1. 2

                                                                                                        I’ve always found it interesting to read articles about GPU performance. They seem so alien and strange when compared to (mostly) sequential CPUs. It’s especially fascinating to discover a counter-intuitive aspect of shader performance, such as how expensive branches are, or dependent texture reads.

                                                                                                        1. 7

                                                                                                          The creator of RexPaint also made Cogmind, a fantastic sci-fi roguelike which I feel pushes the boundaries the genre can achieve. The tool was actually created for this game, and was used in designing the game’s distinctive UI, as well as artstyle and animation.

                                                                                                          The development blog is absolutely filled with great information about designing a commercially successful roguelike.