I’d like to note that Go dependencies are at least verifiable, but they are not reproducible if you depend on GitHub repos. In fact, there was a big issue a couple of years ago akin to the leftpad incident where go-bindata, a tool and library used to bundle data into Go programs (before go embed was a thing) just vanished from the earth.
Perhaps surprisingly (?), I’d say the Maven repository is a better alternative for reproducible builds if you’re able to avoid version ranges. They aren’t as easily removed, from what I gather that only happens if sensitive or copyrighted material is discovered. Though, while I haven’t been in the Java ecosystem for quite some time, it seems like there have been a lot of breaking changes in the language itself since I last looked.
I’d like to note that Go dependencies are at least verifiable, but they are not reproducible if you depend on GitHub repos.
I thought that this was no longer correct, but it seems proxy.golang.org may remove cache entries for various reasons.
proxy.golang.org does not save all modules forever. There are a number of reasons for this, but one reason is if proxy.golang.org is not able to detect a suitable license.
This was a little shocking, the license constraint seems reasonable, but I am concerned about reproducibility.
The go-bindata example predates Go modules, so it might not be applicable now. A module can still get erased from Github and then dropped from the Go module proxy cache, but if it’s popular, it’s more likely to linger in the proxy cache long enough to clone somewhere. You couldn’t pull a leftpad and then cause problems overnight. It would be more like a slow burn of problems as people realize the original is gone and the cache version is all that’s left.
You can also take the work from postmarketOS and port it over to mobile-nixos, they work pretty similarly. I’ve now ported mobile-nixos to a few old devices and I love it. I now have an ebook reader, a games console and various phones running NixOS.
The user interfaces are never quite right, but it’s great to have a bunch of powerful ARM-based devices with full access to NixOS.
I’m a bit surprised no one has brought up the pinephone yet (or the pro, since that’s the one that’s got usable hardware specs). It has a keyboard case, is relatively modern ARM, and moderately good firmware/software support (YMMV).
Nice to finally see this written down. I experienced this when we had the idea to put a sync-to-customers server on a micro instance, because of course the service needs basically no CPU. Had to upgrade to a large instance just because we could not get data out. 60mbit/s explains everything, this is about the same upload speed as my home DSL.
The host’s tap interface is one ip address below the address of the guest on a /31 subnet. We need to increment the op address returned from ifconfig by 1 to get the guest’s ip.
One thing that has bugged me about the FAQ guidance for disk encryption is that the swap volume is contained within the SOFTRAID volume. I presume it isn’t smart enough to know not to double encrypt swap pages. I’ve taken to setting vm.swapencrypt.enable=0, but in the past I’ve also created the swap volume outside of the SOFTRAID.
The pseudocode (which looks like Go) in the article is explicitly not what the Go time package does. In fact, the section on monotonic clocks has the exact same example:
For example, this code always computes a positive elapsed time of approximately 20 milliseconds, even if the wall clock is changed during the operation being timed:
Because of the requirement that a client should be easy to code, the entire PKI infrastructure undergirding TLS on the web was ditched. Instead, the culture is to use long-lived self-signed certs that are trusted on first use.
Maybe DANE could have solved these issues but considering that the instructions on how to set up a self-signed cert are already fragmented and obscure I don’t see DANE being a huge step forward for server operators.
It should be possible for the same protocol-agnostic browsers that work with carl for HTTPS-over-HTTP to set up Gemini-over-HTTP. You’d “just” have to write the Gemini client, and either send Gemtext as text/plain or translate it to HTML. Then you teach the browser about a new protocol proxy gemini on localhost:port.
Yes! It’s funny, part of the impetus for writing my post was that there didn’t seem to be anything I could point people to that discussed these issues. I think Adam was in the same boat, and then we ended up posting on the same day. :-) I actually highly recommend his post as well, because despite the overlap his is more focused on rate-limiting, and mine is more focused on background on how to think about XFF and a variety of use-cases. I have it linked from the bottom of my post.
Difftastic seems to use bleeding-edge rust features, so it was easier for me to package for Guix than Debian. If you can get guix on your system I could send you a recipe (or I’ve submitted it for inclusion in the guixrus channel also)
I don’t have a good sense of what rust versions are available on different distros. I’m trying to be conservative, and only increase the rust version when there’s a benefit, but I’m not sure what threshold to use.
How old are the default rust versions on your distro?
A good side effect of using a password manager in the browser, is that it won’t be fooled by this. The user may of course override it by pasting in their password regardless – it is therefore necessary to train the users to always be extremely suspicious if the username and password isn’t autofilled/detected by the password manager.
I’ve noticed a number of legitimate (Shopify?) e-commerce websites that prompt the user to enter their PayPal credentials directly into elements on the merchant’s website. It’s crazy that they’re encouraging this kind of user behavior.
I’ve noticed a number of legitimate (Shopify?) e-commerce websites that prompt the user to enter their PayPal credentials directly into elements on the merchant’s website. It’s crazy that they’re encouraging this kind of user behavior.
Crazy or not crazy, it depends on how willing you are to even entertain the idea of the current web as something sane.
I have looked at some of their projects, and I think it is astonishing what they were able to achieve with plain terminals! However, I have the feeling that the “visuals” are perhaps “too glamorous”… This is in fact a trend I see appearing, in which many CLI application use too much “eye candy”, especially emoticons where for example a simple [x] would be enough…
On the other side, I love their SSH-based applications idea, i.e. wish, as an alternative to web-based applications… (In fact, at the startup I’m involved in Console9, I was thinking of using such a SSH-based UI as an alternative to the web-based UI.)
For nearly all users they will only notice timestamps in a zip archive if they break. Since the intent is for zip files to not break I think the intention is perfectly obvious to them. For those who do notice it’s a harmless easter egg.
This is not just in a zip file; these times will be written to disk. Looking around a random node_modules on my disk reveals a ton of files with this modification date. 1980-1-1 or 1970-1-1 is a more obvious zero value.
Go uses its own memory allocator. In fact, Go doesn’t use any C library at all (except sometimes libc, but even then, it only uses libc to call system calls and things like getpwuid_r, not for malloc).
They are currently not using Subresource Integrity but working around that with a combination of fetch(), TextEncoder() and crypto.subtle.digest(). That’s really surprising.
I would have assumed that they register a ServiceWorker to handle all fetch events and then replace the existing request with a fetch(sameURL, { integrity: expectedHash})… The variables have names like workaround, so maybe they are dealing with some browser inconsistencies here?
It adds a further (and “independent” from the web app provider) audit point. Suppose an attacker compromises whatsapp web server/CDN, she would be able to change as well subresource integrity hashes in the HTML source of the web page. With Code Verify she would have also to compromise CloudFlare verification endpoint (with the compromised hashes).
why go1.19, when 1.21 is available and 1.19 isn’t supported anymore?
And why not fasthttp if they’re going to put their finger on the scales with httpbeast?
fasthttp isn’t spec-compliant and doesn’t support HTTP/2. Is httpbeast doing the same kind of shortcuts?
httpbeast seems to only implement HTTP 1.1.
From their GitHub page:
I’d like to note that Go dependencies are at least verifiable, but they are not reproducible if you depend on GitHub repos. In fact, there was a big issue a couple of years ago akin to the leftpad incident where go-bindata, a tool and library used to bundle data into Go programs (before go embed was a thing) just vanished from the earth.
Perhaps surprisingly (?), I’d say the Maven repository is a better alternative for reproducible builds if you’re able to avoid version ranges. They aren’t as easily removed, from what I gather that only happens if sensitive or copyrighted material is discovered. Though, while I haven’t been in the Java ecosystem for quite some time, it seems like there have been a lot of breaking changes in the language itself since I last looked.
I thought that this was no longer correct, but it seems proxy.golang.org may remove cache entries for various reasons.
This was a little shocking, the license constraint seems reasonable, but I am concerned about reproducibility.
The go-bindata example predates Go modules, so it might not be applicable now. A module can still get erased from Github and then dropped from the Go module proxy cache, but if it’s popular, it’s more likely to linger in the proxy cache long enough to clone somewhere. You couldn’t pull a leftpad and then cause problems overnight. It would be more like a slow burn of problems as people realize the original is gone and the cache version is all that’s left.
Article is from 2014
Does anyone know of any platforms like this available now?
Yes, the MNT Pocket is pretty much a successor of the N900: https://mntre.com/media/reform_md/2022-06-20-introducing-mnt-pocket-reform.html
It’s open hardware, and the indie lab / company that produces them has a track record of delivering their promises.
PinePhone with the keyboard case is not too far off.
N900 with postmarketOS can do all of the “things your iphone can’t”, and it also has functional wifi, a modem, etc.
postmarketOS supports a bunch of devices, you might even have one laying around!
https://wiki.postmarketos.org/wiki/Devices
You can also take the work from postmarketOS and port it over to mobile-nixos, they work pretty similarly. I’ve now ported mobile-nixos to a few old devices and I love it. I now have an ebook reader, a games console and various phones running NixOS.
The user interfaces are never quite right, but it’s great to have a bunch of powerful ARM-based devices with full access to NixOS.
I’m a bit surprised no one has brought up the pinephone yet (or the pro, since that’s the one that’s got usable hardware specs). It has a keyboard case, is relatively modern ARM, and moderately good firmware/software support (YMMV).
modem is 3G, so it won’t work with U.S. carriers at least. there may be a few countries which still have 3G carriers but I don’t know which.
still a great suggestion though.
I love my N900, but it’s also really starved for RAM, even back in the day. I can’t imagine how well a modern Linux stack would run on it.
it runs alright as long as you don’t try to use the “modern” web :D
remiss not to mention the DragonBox Pyra, though “available” is a stretch:
https://pyra-handheld.com/boards/pages/pyra/
This is handy too https://www.ec2throughput.info/
Well this would have been handy a few months back when we migrated our ingestion layer to aws.
Kept running into odd auto scaling issues because we were using t3a.large instances which without burst has a very low actual max throughout.
Nice to finally see this written down. I experienced this when we had the idea to put a sync-to-customers server on a micro instance, because of course the service needs basically no CPU. Had to upgrade to a large instance just because we could not get data out. 60mbit/s explains everything, this is about the same upload speed as my home DSL.
This is wicked cool! What version of linux is compatible with vmd?
I think I’m running Ubuntu 20.04.
Just having a look at the script. Why do the IP addresses need converting at all? Why the need for
INET_NTOAandINET_ATONfunctions?The host’s tap interface is one ip address below the address of the guest on a /31 subnet. We need to increment the op address returned from ifconfig by 1 to get the guest’s ip.
One thing that has bugged me about the FAQ guidance for disk encryption is that the swap volume is contained within the SOFTRAID volume. I presume it isn’t smart enough to know not to double encrypt swap pages. I’ve taken to setting
vm.swapencrypt.enable=0, but in the past I’ve also created the swap volume outside of the SOFTRAID.Previously: https://lobste.rs/s/5hsu3a/help_pick_syntax_for_css_nesting
It’s not the same. It’s a continuation of the story. There are other options to chose from this time.
Twitter thread with illustration of usage
The pseudocode (which looks like Go) in the article is explicitly not what the Go
timepackage does. In fact, the section on monotonic clocks has the exact same example:This is great! Does anyone have Gemini stuff for retro platforms yet? The lack of TLS on those platforms was keeping Gemini off.
TLS for Gemini never made much sense to me.
Because of the requirement that a client should be easy to code, the entire PKI infrastructure undergirding TLS on the web was ditched. Instead, the culture is to use long-lived self-signed certs that are trusted on first use.
I’ve written more about this here: gemini://gerikson.com/gemlog/gemini-sux/TLS-and-its-discontents.gmi
Maybe DANE could have solved these issues but considering that the instructions on how to set up a self-signed cert are already fragmented and obscure I don’t see DANE being a huge step forward for server operators.
It should be possible for the same protocol-agnostic browsers that work with carl for HTTPS-over-HTTP to set up Gemini-over-HTTP. You’d “just” have to write the Gemini client, and either send Gemtext as
text/plainor translate it to HTML. Then you teach the browser about a new protocol proxygeminionlocalhost:port.See also https://lobste.rs/s/udvcbq/perils_real_client_ip
Yes! It’s funny, part of the impetus for writing my post was that there didn’t seem to be anything I could point people to that discussed these issues. I think Adam was in the same boat, and then we ended up posting on the same day. :-) I actually highly recommend his post as well, because despite the overlap his is more focused on rate-limiting, and mine is more focused on background on how to think about XFF and a variety of use-cases. I have it linked from the bottom of my post.
He also has a good post about the special challenges of IPv6 in rate-limiting, which apparently (as with XFF itself) almost nobody is handling well: https://adam-p.ca/blog/2022/02/ipv6-rate-limiting/
This is useful to me as well because my legacy YouTube account won’t connect with the Roku or mobile apps.
Feel free to reach out if you need any help with these scripts :-) I’m really happy that they’re helpful to other people.
Rust doesn’t seem to have first class support on Ubuntu… What is the recommended way of installing something like this on Ubuntu LTS?
I’ve had good luck building with Rust via asdf
https://rustup.rs/ ?
Difftastic seems to use bleeding-edge rust features, so it was easier for me to package for Guix than Debian. If you can get guix on your system I could send you a recipe (or I’ve submitted it for inclusion in the guixrus channel also)
I don’t have a good sense of what rust versions are available on different distros. I’m trying to be conservative, and only increase the rust version when there’s a benefit, but I’m not sure what threshold to use.
How old are the default rust versions on your distro?
I have rustc 1.48.0
A good side effect of using a password manager in the browser, is that it won’t be fooled by this. The user may of course override it by pasting in their password regardless – it is therefore necessary to train the users to always be extremely suspicious if the username and password isn’t autofilled/detected by the password manager.
I’ve noticed a number of legitimate (Shopify?) e-commerce websites that prompt the user to enter their PayPal credentials directly into elements on the merchant’s website. It’s crazy that they’re encouraging this kind of user behavior.
Or there’s Plaid, which has you enter the credentials for your bank and then the 2FA code into whatever app or website you are connecting.
Crazy or not crazy, it depends on how willing you are to even entertain the idea of the current web as something sane.
I have looked at some of their projects, and I think it is astonishing what they were able to achieve with plain terminals! However, I have the feeling that the “visuals” are perhaps “too glamorous”… This is in fact a trend I see appearing, in which many CLI application use too much “eye candy”, especially emoticons where for example a simple
[x]would be enough…On the other side, I love their SSH-based applications idea, i.e. wish, as an alternative to web-based applications… (In fact, at the startup I’m involved in Console9, I was thinking of using such a SSH-based UI as an alternative to the web-based UI.)
Perhaps it’s time to add a version of
NO_COLORfor emojis?This is fine but setting the time to the zip epoch boundary would make the intention more immediately obvious to users.
For nearly all users they will only notice timestamps in a zip archive if they break. Since the intent is for zip files to not break I think the intention is perfectly obvious to them. For those who do notice it’s a harmless easter egg.
This is not just in a zip file; these times will be written to disk. Looking around a random
node_moduleson my disk reveals a ton of files with this modification date. 1980-1-1 or 1970-1-1 is a more obvious zero value.Sure but I think the whimsy is worth the cost here. Whimsy has value culturally and shouldn’t be discounted.
Amusingly, literal zero mtime breaks things: https://github.com/alexcrichton/tar-rs/blob/master/src/header.rs#L751L760
Oh dear, it’s even worse than that:
https://github.com/alexcrichton/tar-rs/commit/60c6bd81d73fd0e340cfb0e147aae13ce23e18c6
Well that’s quite exciting. Anyone used it with golang?
Go uses its own memory allocator. In fact, Go doesn’t use any C library at all (except sometimes libc, but even then, it only uses libc to call system calls and things like getpwuid_r, not for malloc).
I think they may have been thinking of something like this: Manual Memory Management in Go using jemalloc
Thanks for that. Tbh I was hoping to get “free” performance improvements because I know the codebase I work on allocates quite freely.
Right you are. https://github.com/golang/go/blob/master/src/runtime/malloc.go
The source of the extension is on GitHub if someone wants to figure out how this actually is supposed to work - https://github.com/facebookincubator/meta-code-verify/
They are currently not using Subresource Integrity but working around that with a combination of
fetch(),TextEncoder()andcrypto.subtle.digest(). That’s really surprising.I would have assumed that they register a ServiceWorker to handle all
fetchevents and then replace the existing request with afetch(sameURL, { integrity: expectedHash})… The variables have names likeworkaround, so maybe they are dealing with some browser inconsistencies here?(Using TextEncoder is also a bit error-prone. I wrote it up for them in https://github.com/facebookincubator/meta-code-verify/issues/128).
Does this offer anything over just using subresource integrity?
It adds a further (and “independent” from the web app provider) audit point. Suppose an attacker compromises whatsapp web server/CDN, she would be able to change as well subresource integrity hashes in the HTML source of the web page. With Code Verify she would have also to compromise CloudFlare verification endpoint (with the compromised hashes).