1. 2

    Something I’m not clear on after reading that: does this work because http-fetch resumes at the start of each pack file, or is http-fetch actually able to resume in the middle of a pack file? Hence does this work pretty much any time a repo is available via http or does it require the remote side to do some extra work to break up the pack files into chunks small enough that resuming works?

    1. 2

      I’m sorry but I didn’t test that case, I assumed that the “resume incomplete packfile” case would variate from git server implementations, and because of that focused in assumption that it would start download the packfile from the beginning since I wanted a universal solution that should work with any server implementation.

      1. 2

        Ah, thanks. I wasn’t quite sure whether the splitting the packfiles into 1MB blocks was something you did to make it easier for resumption to work, or whether it was only done to make testing that the method works easier.

        FWIW, all the commonly used httpds that I know of (e.g. Apache, Microsoft IIS, nginx, I’m almost sure lighttpd does too) support HTTP range requests for static files out the box with no configuration. I wouldn’t be surprised if resuming individual files via HTTP turned out to work on every single implementation that you find in the wild.

    1. 10

      Neat surprise, I maintain this (and similar) pages. Originally started because I wanted to show de-facto support for modern crypto but now I refer back to the pages as a starting point in choosing software that does one particular task or another. I think modern crypto has basically “won” even if there are still gains to be made. One thing I’ve noticed is that the de-facto support goes way beyond de-jure support to a degree that looks suspicious for standards bodies. It’s been the main factor in convincing me that standards bodies need to be taken down a notch – let’s have more public competition and less “design by committee.” I think the community is moving more in that direction and it’s good to see.

      There are over 1700 unique outbound links on these pages. I scan their http status codes with automated scripts several times a week and make other efforts to prevent link rot and outdated info. It takes a surprising amount of time. For instance some people say they support a cryptographic primitive but you look at the code and it’s something else, so you have to check, and it takes time. Good example of the importance of useful, correct documentation.

      Minor thing I noticed: a handful of github pages were deleted after the Microsoft purchase, but not as many as one might have expected based on public discussion. Now that things have settled I’ll check each one manually to look for “page moved to gitlab” type messages where the github repos remain with http 200 status codes, but emptied out. It takes a lot of time to maintain these pages but it’s helped some people so it feels good.

      1. 3

        Yea it was a quite awesome link, and seeing i2p in it warmed my heart<3 (disclamer; I’m a i2p dev)

        1. 2

          Was pleasantly surprised to see Yggdrasil, a project I work on, listed there too!

        2. 2

          Not sure if you’re aware but the font size on that site is absolutely huge on Chrome on Linux. I have to dial it down to 67% the original size to make it readable. Here’s a screenshot to illustrate, with lobsters for comparison: http://i.imgur.com/4I8eegV.png

          1. 1

            Thank you for the feedback! Yeah, font size is an issue these days and it’s something I’ve wrestled with. A while back there was an article about how font sizes haven’t kept pace with monitor / screen resolution increases, which I think is hard to disagree with. The situation is compounded by the large variety of “monitors” from phones to what amount to widescreen TVs. If you have a simple suggestion for HTML/CSS that doesn’t use any JS and makes everyone happy I’d be very interested in hearing it.

          2. 1

            Thank you for all your work! I’m pleased to have two projects in the list. I wonder if there will be an equivalent for post-quantum crypto once the algorithm advice stabilises.

            1. 1

              Yes, there already is a pqcrypto list but it’s kinda shabby IMO (check the links under the homepage). The pqcrypto situation is very fluid at the moment, even chaotic. As one example of many, the front-runner library is libpqcrypto which contains 77 cryptographic systems (50 signature systems and 27 encryption systems). There are more post-quantum algorithms than apps using those algorithms. Also libpqcrypto doesn’t even compile on OpenBSD, a bummer for me personally. IMO for certain things like VPNs, combining an ephemeral X25519 key exchange with a pre-shared key, like WireGuard can optionally do, is a sensible thing to do in 2018 until we get real pqcrypto off the ground.

          1. 4

            However, iBoot is closed source, but

            … wasn’t the source leaked too?

            1. 6

              Yea it was, seems apple got it removed however ( https://github.com/github/dmca/blob/master/2018/2018-02-07-Apple.md )

              However this might be the real thing, https://github.com/ShapManasick/iBoot - if not I know I cloned it to a private server somewhere :)

            1. 5

              This is not a great way to analyze cryptographic algorithms. The grains of sand are a good analogy to use for brute force search, but cryptographic attacks are rarely equivalent to pure brute force search on the key space. There is more analysis that has to be done to find the effective key space you would have to search over to have an equivalent level of difficulty.

              You don’t need to be a math expert to know this, all the analysis has been done. According to NIST (the relevant paper is here), a 2048-bit RSA key is equivalent to a 112-bit search space, and according to ANSSI it’s equivalent to 100 bits.

              1. 1

                Thanks for the feedback, it’s my first post on the subject. But yes, I understand your point that it’s not the best way to explain it because of algorithms with sub-exponential running time for factoring integers and so on. But I’ve yet to be more familiarised with the details in it, and just wanted to try explain for myself and whoever wanted to read it how big numbers we’re talking about.

              1. 4

                Honestly I wish that people talking about crypto topics would stop using the “grains of sand” and such visualizations. I think it doesn’t necessarily reflect the concepts of scale very well as it doesn’t take into account how much of that “sand” we can process. It doesn’t really matter if there are more “pieces of sand than in the whole world” if I have buckets that can move all that “sand” in a month. I see no mention of Shor’s in here either, and ignoring the quantum situation seems like a mistake, and I highly suggest reading up on Post-quantum RSA.

                1. 3

                  Post-Quantum RSA is more like an elaborate joke. If you really care about quantum-safe crypto you should look into things like ntru, rlwe, mceliece or hash-based signatures.

                  1. 1

                    Well, I agree actually, but do you got a better way to explain it for the general public?