1. 4

    Interesting takeaways from the APNIC Labs blog post:

    In setting up this joint research program, APNIC is acutely aware of the sensitivity of DNS query data. We are committed to treat all data with due care and attention to personal privacy and wish to minimise the potential problems of data leaks. We will be destroying all “raw” DNS data as soon as we have performed statistical analysis on the data flow. We will not be compiling any form of profiles of activity that could be used to identify individuals, and we will ensure that any retained processed data is sufficiently generic that it will not be susceptible to efforts to reconstruct individual profiles. Furthermore, the access to the primary data feed will be strictly limited to the researchers in APNIC Labs, and we will naturally abide by APNIC’s non-disclosure policies.

    This joint project has an initial period of five years and may be renewed. Upon the expiration of the initial period, or at any time thereafter, APNIC shall consider a request by Cloudflare for a permanent allocation of these IPv4 addresses to Cloudflare. APNIC undertakes to refer any such request to the regional Address Policy Special Interest Group as a matter of a change to the current research use designation of these IPv4 addresses, and APNIC shall be bound to the outcomes of this policy group.

    https://labs.apnic.net/?p=1127

    1. 1

      Off-topic: I signed up for the newsletter on queue.acm.org a few weeks back and haven’t seen an email from them. Do they do monthly digests? Do you find yourself checking the website frequently for new articles?

      1. 2

        It should be every week, have you checked your spam folder?

        1. 1

          Unfortunately, I can’t answer those questions. I don’t regularly use ACM. I got that article off Hacker News.

        1. 1

          I’m not fully understanding what issue is being described here. Is it that the archive URLs are unreliable, i.e. the “Source code (zip / tar.gz)” URL?

          1. 2

            The hash of the auto-generated tar files is not stable. I assume the compression level changes or the tar implementation to create them.

            1. 1

              And what about the zip files?

              1. 3

                Same problem with zip files.

                The OpenBSD ports tree stores checksums of release artifacts to ensure authenticity of code that is being compiled into packages.

                Github’s source code links create a new artifact on demand (using git-archive, I believe). When they upgrade the tooling which creates these artifacts the output for existing artifacts can change, e.g. because the order of paths inside the tarball or zip changes, or compression level settings have changed, etc.

                Which means that trying to verify the authenticity of a github source link download against a known hash is no better than downloading a tarball and comparing its hash against the hash of another distinct tarball created from the same set of input files. Hashes of two distinct tarballs or zip files are not guaranteed to match even if the set of input files used to create them is the same.

                1. 1

                  Thank you for the detailed response! I understand the issue now.

                  There are likely tradeoffs from GitHub’s perspective on this issue, which is why they create a new artifact on demand. They maintain a massive number of repositories on their website, so they probably can’t just store all those artifacts for long periods of time as one repository could potentially be gigantic. There are a number of other reasons I can think of off the top of my head.

                  Why not have the checksum run against the file contents rather than the tarball or zip?

                  1. 3

                    Why not have the checksum run against the file contents rather than the tarball or zip?

                    One reason is that this approach couldn’t scale. It would be insane to store and check potentially thousands of checksums for large projects.

                    It is also harder to keep secure because an untrusted archive would need to be unpacked before verification, see https://lobste.rs/s/jdm7vy/github_auto_generated_tarballs_vs#c_4px8id

                    I’d rather turn your argument around and ask why software projects hosted on github have stopped doing releases properly. The answer seems to be that github features a button on the web site and these projects have misunderstood the purpose of this button. While some other projects which understand the issue actively try to steer people away from the generated links by creating marker files in large friendly letters: https://github.com/irssi/irssi/releases

                    I’d rather blame the problem on a UI design flaw on github’s part than blaming best practices software integrators in the Linux and BSD ecosystems have followed for ages.

                    1. 2

                      Some more specifics on non-reproducible archives: https://reproducible-builds.org/docs/archives/.

                      Why not have the checksum run against the file contents rather than the tarball or zip?

                      Guix can do something like that. While it’s preferred to make packages out of whatever is considered a “release” by upstream, it is also possible to make a package based directly on source by checking it out of git. Here’s what that looks like.

            1. 11

              They are just following the trend. Support XMPP to get people using it and then drop it when they have enough power.

              1. 5

                Wasn’t this the reason Google Talk shut down and got replaced with Hangouts?

              1. 13

                The trick here is that the bar will disappear when you fill it up. My bar disappeared when I donated the remaining balance. If you’re still seeing your bar, it means you need to donate the remaining balance.

                1. 2

                  Huh? I donated before the bar was put there, I still see it. What do you mean by “remaining balance”?

                  1. 10

                    I think zg was going for humor, that if someone wants to donate all of the remaining amount to reach the goal, the fundraiser will end and the bar will be removed.

                    1. 3

                      what’s “humor”?

                1. 3

                  I’m so glad I moved to my own domain, own email, own calendar, own contacts, own backup, own you-name-it server. I replaced every conceivable cloud provider that I was consuming and to this day I am very glad that I took the time to do it because it’s shit like this that I get to chuckle at.

                  I highly encourage anyone who depends on any cloud provider to ask yourself this: do I like the service I’m being provided? Are there alternatives I could run myself? It’s questions like these that led me to obtain the experience I needed to land a job.

                  1. 1

                    Could you expand?

                    Where do you host your services? How much time did it take for you to set it up? How much maintenance does it need? Also did you have any problems with mobile?

                    Lately outside of mail I’m thinking about photos hosting. I would like to tag photos and a camera icons dedicated to certain tags.

                    1. 2

                      Sorry, let me clarify one point: I do rely on one cloud provider: DigitalOcean. I run my email, contacts, and calendar services within a droplet on DigitalOcean. I routinely have backups scpd from the VPS to my local machine which has a 8 TB RAID setup, which is where I backup other things as well. I also run my own webdav service which allows me to sync up documents between my laptop and iPhone.

                      I could technically avoid the reliance on DigitalOcean if I purchased my own hardware and placed it into a colocated datacenter, but that would be costly, and it kind of goes a bit beyond the idea of no cloud provider dependency. I’m fine with relying on DigitalOcean because I know that I have backups in case I need to switch to a different provider.

                      I also purchase CDs and import them to iTunes then sync them onto my iPhone. The frustrations of dealing with bad LTE coverage led me to make this choice a long time ago, and I’ve been happy ever since.

                      1. 1

                        I am on fastmail and it works just fine. Comes with mail, contacts, calendar and cloud storage.

                        1. 1

                          But that’s not what OP meant, is it? Replacing one provider with another is not “my own domain, own email, own calendar, own contacts, own backup, own you-name-it server”. Also he said: “I replaced every conceivable cloud provider”.

                          1. 1

                            I think there is a big difference between fastmail and google, namely that fastmail is not an ad company and you pay for your mail/calendar etc.

                    1. 1

                      There clearly is a spectrum of possible ways we could think about how to program the problems we’re trying to solve. As we have seen in the course of the last 100 years or so, some paradigms have stood out more than others, while others have had their good parts taken from them.

                      I would argue that the very early paradigms stood out because they were easy to understand and iterate on. The last 20 years or so has shown that paradigms had to shift to accommodate to scale, i.e. when the Internet started to take off, developers had to go from handling a dozen users to handling upwards of a few billion users. I think that the “scale paradigm shift” is coming to an end since we’ve got many services on the Internet which can accommodate to massive scale.

                      1. 3

                        Be aware of an efficient markets fallacy or purposeful evolution fallacy here. Our current paradigm makes it possible to build services at internet scale, and there are a small handful of successful examples. This is not the same as having converged on a paradigm for building at scale, nor is it the same as having found the best or even a good paradigm for building at scale.

                      1. 4

                        Work:

                        • I can’t talk about it.

                        Personal:

                        1. 2

                          Awesome ! I’m also reading this learn Haskell from Chris, I’ve started the CIS194.

                          Thanks for H99, that should be a nice complement if I want more exercises.

                          Are your sharing your learning experience anywhere?

                          1. 1

                            I’m not sure what I’d share in my learning experience, to be honest. Most of what I’ve learned is tricks that are succinctly shown in the challenge solutions.

                        1. 22

                          In November of last year, RC announced that they’d be experimenting with “mini” one-week batches. Being away from work for just one week felt very reasonable - it’s about as disruptive as going to a conference, but much more educational! It seemed like my time had come!

                          Well, this suddenly makes attending the Recurse Center much more appealing.

                          1. 6

                            You should attend! It was a great experience.

                            1. 4

                              I would apply if I could find a project I thought I could focus on! As a mere self-taught web developer, I feel like I’m not ready yet. I have a long way to go in terms of basic knowledge (mostly low level languages, some algorithms) before being able to focus on something meaty for a week. But I would like to do something compiler related.

                              1. 7

                                As a mere self-taught web developer, I feel like I’m not ready yet.

                                If you knew you were ready then there wouldn’t be anything to learn. :)

                                1. 6

                                  NAND2Tetris is a pretty popular choice for people wanting to learn more low-level computing. And I don’t think “mere” is a good word to describe someone learning enough of a complex topic with several different paradigms at work like full-stack web dev up to a level that they can be gainfully employed at it.

                                  Edit: Also, I think people see the applying similarly to tech job interviewing. The shape is there, but Recurse doesn’t strongly filter on technical ability. If someone can write programs and want to learn more, that’s enough.

                                  1. 1

                                    NAND2Tetris is a pretty popular choice for people wanting to learn more low-level computing.

                                    That is a neat book. It would definitely be a fun project for RC given it covers several areas simultaneously… CPU, compilers, low-level software… without overwhelming reader.

                                    Only thing skimming it made me wonder was what would be next thing to read on that topic to build digital design skills. Something that was an incremental step giving useful skills instead of a huge leap. Did you or anyone else here get a solid recommendation about what to read next?

                                  2. 3

                                    I would apply if I could find a project I thought I could focus on!

                                    That’s me. The write-up’s people have done about the RC experience make it seem pretty incredible. I’d love to go there to just chill, focus on some projects, and listen to all those other people are doing. I’m just not sure what one or two things I’d focus on with a whole week of free time and good environment. I’d kind of want to make that time really count with the right projects. Staying focused is also a personal weakness of mine, though, as many have probably noticed.

                                    1. 4

                                      Then find the time for it. Take an hour out of your free time to organize your free time, i.e. find what you can cut out of your schedule to start working on “the right project.”

                                      1. 1

                                        That’s good advice. I guess it’s a discipline thing I gotta work out. I’m too easily distracted esp by good learning opportunities. :)

                              1. 1

                                What is the current state of Plan 9 development?

                                1. 7

                                  9front is actively developed.

                                  1. 1

                                    Is 9front usable on desktop? By usable, I mean that there’s some mail client (I don’t mind CLI, I use Mutt anyway), some audio / video player (mpv is just fine) and some browser that understands modern websites (yeah, I hate JS too, but it’s inavoidable). I guess the last part is the worst :)

                                    1. 9

                                      The last part is indeed the worst. For web browsing, there’s mothra and that’s about it. Mothra does not support JavaScript. Here is the relevant bit of the FQA.

                                      Russ Cox described his motivation for creating Plan 9 from User Space like this:

                                      I ran Plan 9 from Bell Labs as my day to day work environment until around 2002. By then two facts were painfully clear. First, the Internet was here to stay; and second, Plan 9 had no hope of keeping up with web browsers. Porting Mozilla to Plan 9 was far too much work, so instead I ported almost all the Plan 9 user level software to FreeBSD, Linux, and OS X.

                                      1. 2

                                        Yes there’s a mail client, playing videos depends on the format, modern browser…no, by design mostly.

                                        1. 3

                                          there is no support for video playback at all.

                                          1. 2

                                            What can you use 9front for? I don’t mean playing in VirtualBox or whatever VM software you use, but for serious usage. I’ve always wanted to play with it more, but playing just for the sake of playing with it makes me isn’t interesting for me :)

                                            1. 11

                                              The system excels at manipulating text. It can playback most popular audio formats, and it can display many popular image and document formats. It does not (currently) have any support for video playback. There is no modern web browser (the native browser, mothra(1), ignores CSS, js, and many HTML tags). The system includes a PC emulator called vmx(1) that is capable of hosting Linux or OpenBSD, but currently the guest’s framebuffer is emulated entirely in software, so performance is pretty awful, and programs like web browsers are barely usable.

                                              1. 1

                                                Now, that is something, thanks!

                                                What about use as a server? Since this is Plan9-derivative, I assume all Plan9 servers (CPU, Auth, 9P etc. are available). I can also see the included HTTP server. Can it use TLS? What about others protocols (like XMPP, DNS authoritative server etc.)?

                                                I see there’s a port of OpenSSH, but it’s at version 4.7, which can’t do ED25510 :/ Is there any other SSH client (I mean, one written for 9front)?

                                                I hope you don’t get angry by my questions, I just want to know what I can use 9front for. You kind of made me again interested in it, so I’ll install 9front on a spare PC.

                                                1. 6

                                                  I’m the admin for basically all of the 9front official websites, and the cat-v.org sites, all hosted on 9front for several years. TLS is supported, but there is no support for SNI, so the end result is most current mobile browsers will refuse the self-signed/wrong-domain-name certificate. I also host all my DNS on 9front, pushing updates automatically to slaves at dns.he.net.

                                                  You didn’t ask about mail, but all the 9front mailing lists are also hosted on 9front, with upas(1) and a rather primitive mailing list manager called ml(1). I also host my personal e-mail with upas(1).

                                                  The system includes a native SSH2 client called ssh(1).

                                                  http://fqa.9front.org is probably the best overall resource for information about the system. It includes links and pointers to most other relevant sources. Unfortunately it tends to lag behind the current state of the system at times, mainly because of time comstraints.

                                              2. 3

                                                The Introduction To Plan 9 from the 9front FQA might interest you.

                                                1. 1

                                                  I read it, I used 9front for a few hours some time ago, so I’m not a complete newcomer.

                                                  What I miss is some overview of available software. I can see that there is https://bitbucket.org/mveety/9front-ports, but it doesn’t seem official.

                                                  EDIT: Nvm, just found https://code.9front.org/hg/ports/

                                                    1. 1

                                                      Thanks, that’s what I was asking for.

                                                2. 1

                                                  I’d really like to get around to porting emacs to Plan 9. That might be the sort of work I could actually do. I’d love to port Firefox to Plan 9, but … that just isn’t going to happen.

                                                  It’s a pity, because emacs & a web browser are the only things that Plan 9 is really missing.

                                                  1. 3

                                                    I think it really needs a hardware accelerated graphics stack. Things would improve dramatically after that.

                                                    I would love it if the plumber can talk to my phone. An Android/iOS app that reads a web link from plumb and display it on the phone would solve the browser problem.

                                                    As to the editor… just use acme.

                                                    1. 3

                                                      it’s trivial to plumb a link to a script that opens ssh to a remote host and runs a command.

                                                      1. 1

                                                        I would love it if the plumber can talk to my phone. An Android/iOS app that reads a web link from plumb and display it on the phone would solve the browser problem.

                                                        I’d think that could easily be doable with a small Android app to listen for GCM messages.

                                                        As to the editor… just use acme.

                                                        But that wouldn’t be emacs, and emacs is what I want to use:-)

                                                      2. 2

                                                        emacs has been ported to plan 9 more than once.

                                                        1. 1

                                                          Really? I did a quick googling, but no joy. Is it in the main emacs tree?

                                                          1. 2

                                                            looks like i’m not able to reply from mothra.

                                                            there were a couple of (old) ports on sources, which i think is permanently down. there exists a mirror at http://9p.io.

                                            1. 2

                                              Yay, I’m a hat doffing guinea pig! 😆

                                              1. 3

                                                And let us hope that @journeysquid never doffs the banana king hat.

                                                1. 1

                                                  That was hilarious. Thanks for sharing it.

                                              1. 3

                                                RackSpace still uses it, and they’re a relatively large service provider similar to AWS (in their offerings, not userbase).

                                                1. 1

                                                  Rackspace is now also an AWS/Azure/GoogleCloud consulting company. They seem to have given up on the direct competition part.

                                                1. 1

                                                  I’d like to volunteer as a moderator. I’m here: https://linkedin.com/in/zzatkin

                                                  1. 1

                                                    This looks horrendously unreadable.

                                                    1. 2

                                                      It’s actually interactive. You can click a button like ‘m’ (or hit m on the keyboard) and see shortcuts available when you’re composing a new message. If you are actually using mutt this makes more sense than a huge list of modes & hotkeys in groupings.

                                                      1. 1

                                                        Aha, that explains it. I was viewing the page via my iPhone.

                                                      2. 1

                                                        I agree - different colours could have been chosen.

                                                      1. 2

                                                        My site won’t allow this due to Content Security Policy. 😏 Incidentally, I’m also able to happily open suspicious emails in Squirrelmail because of the same fact, which is an interesting defense against spam.

                                                        1. 2
                                                          1. 2

                                                            If I understand the problem correctly, I’m surprised nobody has mentioned Consul + consul-template.

                                                            You’ll first want to set up a consul cluster of 3 or 5 hosts (for redundancy) then configure consul as an agent as well as consul-template templates on the hosts that need to get their configuration dynamically updated. The configuration that you want can be pulled from the Consul key-value store. This key-value store can also be used for “any number of purposes, including dynamic configuration, feature flagging, coordination, leader election, and more.”

                                                            1. 6

                                                              I love ncdu.

                                                              1. 1

                                                                du just doesn’t cut it for me. Unless I have time and remember how to use sort correctly.

                                                              1. 2

                                                                Think of something you’re terrible at and tackle it head on. Look for the awesome list for it on GitHub (search for “awesome ”). Look up YouTube videos. Read research papers. Study. Study. Study. Ask questions. Stupid questions. Seek a community to participate in for it.

                                                                Once you’ve mastered your weaknesses, your baseline will be much higher.

                                                                1. 5

                                                                  I’m setting up my base station for amateur radio! I just got a power supply and antenna for my IC-7100, and I’ll be listening on the air soon.

                                                                  1. 1

                                                                    Nice, I always wanted to get into ham radio but never managed to make enough free time.

                                                                    1. 1

                                                                      It’s never too late! http://www.eham.net/newham/

                                                                      1. 1

                                                                        What discouraged me was the sheer amount of real estate a setup would take on my desk. I don’t have the luxury of having a free room to dedicate to the hobby, so my study would have to house all the extra kit, which I simply don’t think it would!

                                                                        1. 1

                                                                          A handheld radio is not much bigger than a cell phone and it’s a good first radio to purchase. No need to buy a huge base station until you’ve gotten more familiar with the hobby!