Threads for nolanl
-
I use unison to sync most files nightly. I also have a script https://github.com/nolanl/rp to sync things in a given git repo while I’m working, so I can use my slow laptop to edit files locally, and then run builds and tests on my much faster desktop. Works pretty well.
-
The history of QEMU and virtualisation from this article is strange. Prior to KVM, Fabrice Bellard wrote his own kernel module as a QEMU accelerator, which allowed QEMU to use hardware virtualisation features to run faster. KVM was, to my knowledge, the third approach to this and was designed to support some abstractions that were usable by QEMU and by other things (and things like Firecracker have shown that this separation was valuable). As far as I know, the KVM parts were upstreamed to QEMU, they didn’t maintain a long-term fork.
The problem with maintaining a fork is that you have to merge from upstream and upstream has no incentive to make that easy unless it’s a very friendly fork. That’s typically a losing proposition unless you have a small patch set or your rate of changes is significantly higher than upstream.
-
Prior to KVM, Fabrice Bellard wrote his own kernel module as a QEMU accelerator, which allowed QEMU to use hardware virtualisation features to run faster.
It sounds like you’re talking about KQEMU, which predates hardware virtualization.
For guest ring3 code, it runs the code directly in ring3, with shadowed page tables. For guest ring0 code, it also runs the code in ring3, trapping out and letting the host qemu do its normal binary translation if the guest kernel does something that a) doesn’t work, and b) doesn’t trap when run in ring3.
The big one is turning off interrupts, which kqemu obviously can’t actually do (or the guest could DoS the host trivially), but is visible to code running in ring3. Loading the address of the IDT, GDT or LDT also fall into that category, though kqemu doesn’t (and can’t) handle that case.
Since the code to directly execute guest ring0 code is a hack and doesn’t work in all cases (though it is secure in all cases), there is a mode to only use kqemu for guest ring3 code, and rely on qemu’s binary translation for all guest ring0 code.
It is an interesting hack, radically simpler (though slower and less compatible) than VMware’s wild hacks that made their BT VMM work. Looks like the technical page is down, but archive still has it: https://web.archive.org/web/20071225070235/http://fabrice.bellard.free.fr/qemu/kqemu-tech.html
KVM was, to my knowledge, the third approach to this
I don’t recall the exact timing of Xen and KVM shipping hardware virtualization support, but they were pretty close to contemporaneous. So KVM was either first or second.
-
-
I would never have imagined that the term ‘air-gapped’ even includes systems that have had a usb device plugged into them that was ever plugged into a non-air-gapped system. If someone sent me confidential documents on a usb drive my first thought would be ‘damn there is probably all sorts of malware on this’. Also people forgetting to disable autorun on a windows machine with sensitive material on it… When I saw the headline I was expecting something much more sophisticated.
-
-
no data-diodes for inbound? (or if so, what COTS products have you found for that).
-
I’ve always been fascinated with air-gapped systems/networks so thanks for the post/background. Is there a reason why y’all don’t just ship systems without USB ports altogether, or is it cheaper to grab an off-the-shelf system and then airgap it rather than build a custom PC for that purpose?
-
In a lot of cases, they don’t actually epoxy the USB ports, they fit removable covers on them. It turns out that they’re actually useful for debugging purposes, as long as you can restrict their use to authorised people.
Back in the early Xen days, a certain three-letter agency looked at using direct device pass through of one NIC to an untrusted VM to avoid the need for separate computers for their secure and insecure networks. The idea was to have a separate NIC for the insecure network and run insecure machine as a VM. Even without VM escapes, they were concerned that you could establish a covert channel based on the interrupt latency of the insecure NIC and exfiltrate data.
My favourite air-gap side channel attack was a Tor node decloaking attack from Stephen Murdoch and friends. He discovered that the temperature change from a Tor node processing a lot of traffic was detectable from the thermal sensors in other machines in the same rack (and probably from other machines in nearby racks with more samples). If a nation-state adversary can buy space in the same co-located datacenter as your Tor node, then they can start to unpeel the onion.
-
Back in the early Xen days, a certain three-letter agency looked at using direct device pass through of one NIC to an untrusted VM to avoid the need for separate computers for their secure and insecure networks.
That’s really funny to learn. In the early VMware days (late 2000 or so) that same three-letter agency wanted us to partner with HP to deliver “NetTop”. The idea is to run SELinux on the host, and then use its restrictions to run multiple Windows VMs at different classification levels.
It didn’t pan out, since the NSA security team was able to send a half a bit per second or so of data from a cooperating process in the high security VM to a cooperating process in the unclassified VM by attaching and detaching virtual USB devices, and measuring the timing perturbations in the other VM. No doubt there were many other timing side channels they could have used.
There were running jokes for years internally along the lines of “Out for vacation all next week. If you need to reach me, simply plug and unplug virtual USB devices in the agreed upon pattern…”.
-
Not really at the same level, but when I did my thesis at a large Swedish telecommunications manufacturer in the late 90s I heard of someone bypassing the firewall by accessing a machine that wrote data from the WAN onto a hard drive, which was then read by the LAN interface, and vice-versa. Slow but if you want to access Slashdot, needs must.
-
-
-
-
Seems to me that it is also expedient to reinstall the OS, preferably with non-windows. I understand it depends what you are doing but if I would think about using raspberry pi or better yet, arduino. Then desolder any communication hardware from the board. You would have to use only self built or self programmed software so if you are doing anything more complicated than reading documents it could be an issue. This was another thing that surprised me about the article. If one is doing something this sensitive why would one trust a vendor installed OS? Or one with as commonly compromised as windows?
-
-
-
-
Fortunately for Israel, the folks working at the Iranian nuclear facility did not have the same mindset and plugged in a USB stick containing malware that infected the control systems of their centrifuges and destroyed them.
-
-
It’s probably worth noting that Windows actually does support all of the OCI infrastructure and so a desire to use Kubernetes is not really a reason to avoid Windows. I don’t know about GCE, but with Azure Container Instances or Azure Kubernetes Service you can spin up Windows or Linux containers with the same orchestration framework. The cost part is still relevant though, you need to pay for Windows licenses.
Chrome is a large C++ project, but you might well be using more C++ features. In particular, to reduce binary size, Chrome compiles without exceptions or RTTI, which means that these features on Windows are less well tested than elsewhere. LLVM doesn’t support non-call exceptions at all, so if your codebase relies on the ability to unwind with an SEH exception from an invalid memory access or a floating point trap, it will not work correctly if compiled with LLVM.
The process of downloading proprietary components that the author says only Microsoft can distribute, packaging them, and deploying them elsewhere, makes me really nervous. I think the Visual Studio EULA allows this kind of thing, I’m 99% sure the XCode EULA explicitly prohibits it. It’s the sort of thing that I’d avoid on the principle that it forces me to talk to a lawyer.
It would be nice to understand why the author didn’t want to create a case-insensitive FS to store all of the files that expect to be on a case-insensitive FS. I’m not really up to speed with all of the latest in Linux filesystems, but on FreeBSD it’s trivial to create a ZFS dataset that’s case-insensitive.
-
It would be nice to understand why the author didn’t want to create a case-insensitive FS to store all of the files that expect to be on a case-insensitive FS.
On Linux, an easy way to do this is create vfat FS in a file and then loop mount it. I assume the author didn’t use loop mounting or ZFS (or even ext4, it supports case-insensitivity on a sufficiently recent kernel) because they wanted to keep the build self-contained in a container.
-
-
Back in 2000, AMD included cmov in its 64-bit x86 ISA extensions. Then, Intel had to adopt them when Itanium flopped.
The first sentence is technically true, in the sense that AMD64 did include cmov, but the instruction was originally introduced in 1996 by Intel with the Pentium Pro, and became widespread with the Pentium II.
-
The Alpha also had a conditional move as well. RISC-V doesn’t have one. It’s a very interesting microarchitectural trade-off.
On a low-end core, a conditional move can require an extra port on the register file (this was why it was such a large overhead on the Alpha): it requires you to read three values: the condition (cheap if you have condition codes, a full register if you don’t), the value the source register to be conditionally moved, the value of the destination register that may need to be written back to if the condition is false.
On a high-end core, you can fold a lot of the behaviour into register rename and you already have a lot of read ports on rename registers so conditional move isn’t much overhead.
In both cases, conditional move has a huge effect on the total amount of branch predictor state that you need. You can get away with significantly less branch predictor state with a conditional move than without and get the same performance - a modern compiler can transform a phenomenal number of branches into conditional moves. The total amount varies between pipeline designs (it’s been years since I measured this, but I think it was about 25% less on a simple in-order pipeline, more on an out-of-order one).
Once you have sufficient speculative execution that branch predictor performance becomes important to performance, conditional move becomes incredibly important for whole-system performance. For x86 chips, it would probably start to make sense around the Pentium, given a modern compiler. Compilers in the ’90s were much less good at the if conversion than they are now and so it may not have made very much difference when it was introduced in the Pentium Pro that it would do on an equivalent pipeline today.
Arm went all-in on predication from the first processor and so managed without any branch prediction for much longer than other processors. It’s much easier for a compiler to do if conversion if every instruction is conditional than if you just have a conditional move. AArch64 dialled this back because compilers are now much better at taking advantage of conditional move and microarchitects really hate predicated loads (compiler writers, in contrast, really love them, especially for dynamic languages).
-
-
-
1m30.6s on a 32 core Threadripper. Not a fair comparision by any metric, but pretty amusing.
-
With that type of speed, you can just compile the kernel at boot… https://bellard.org/tcc/tccboot.html
-
-
-
And, most recently, QuickJS. So many impressive projects from one person.
-
-
-
-
-
I would love to see some examples where a company changed the license of their codebase to comply with a dependency. I have never seen this happen in my entire career, even though they are obligated to. I have seen countless examples of people rewriting code to rid of a dependency with an incompatible license, I have even done it myself.
I understand and respect the pure ideology of strong copyleft licenses (that in a perfect world, everything would be open source), but I don’t believe that it actually yields more open source in the world. In practice, people simply avoid it and it yields more duplicated work that ends up either proprietary (more common worst case) or more permissive (best case).
It is difficult to prove, but I feel that the “leading by example” ideology of permissive licenses is responsible for far more code being made open source in practice (though I acknowledge this is not everyone’s goal).
-
I think OpenWRT exists because linksys had to do this.
I was just looking into this question myself today and have this history of openwrt on my reading list if that helps.
-
Linksys did that, and then stopped maintaining the code, and switched to another OS for most of its hardware. Was that VMX, perhaps? I don’t remember. Some commercial OS. They stated that the reason was that linux needed too much RAM, which I find difficult to believe. Slimming down a linux kernel is IMO likely to be simpler than porting a code base to a new OS, so I tend to believe that Linksys’ stated reason was a polite lie. They did release a single model that ran linux, and I bought that model.
-
-
All the vendors do a fantastically bad job.
-
-
-
-
-
I have seen countless examples of people rewriting code to rid of a dependency with an incompatible license
This is a very good case, IMO, and is my primary motivator for using (A)GPL on most of my newer work. I would much rather force the big bads to rewrite my code than simply profit off my work, and we have some evidence that they will do that to avoid (A)GPL sometimes.
I would love to see some examples where a company changed the license of their codebase to comply with a dependency.
I think to be fair on this one you have to also include all the code that started or stayed freedomware because of the requirement. This would include the example from the OP of the same downstream hosting the other project in compliance.
-
-
You can use an (A)GPL’d dependency on an MIT’d project, just the resulting “combined work” is effectively (A)GPL’d. Some projects even have build flags to choose to build in “GPL mode” or not depending on which dependencies you use. It’s all about goals.
If you want your software to be used to build nonfree products so badly that you reimplement something under the GPL to use as a dependency for your otherwise MIT’d project… I mean, more power to you, right? It’s your choice.
We have examples of Apple, Google, VMWare, Linksys, and others doing rewrites to avoid using GPL’d code, and I would say that is the point, for me.
-
I wrote before:
It is difficult to prove, but I feel that the “leading by example” ideology of permissive licenses is responsible for far more code being made open source in practice (though I acknowledge this is not everyone’s goal).
My goal is to provide as much value to the society as possible through each unit of my effort. I want people to use my code, I want people to profit from my code even if I get nothing for it, I want people to build things they otherwise wouldn’t have built because my code enables it. I am going to make the effort to make my code as accessible and permissive as possible, this often means avoiding (A)GPL dependencies and often duplicating effort in the process.
I recognize that your goal is not the same, and that’s fine. I just hope that you also recognize the reality that Apple/Google/VMWare/Linksys etc don’t care at all, they’ll simply not even look at AGPL code and move on. If they find AGPL code in their stack by accident, they will purge it. If they’re caught in a situation where they are in legal trouble, they will do the absolute minimum to comply with that version and purge it moving forward.
Overall, my bet is that choosing strong copyleft licenses has more of a net-negative effect on people who share my goal than any measurable effect on “the big bads”.
-
Apple/Google/VMWare/Linksys etc don’t care at all, they’ll simply not even look at AGPL code and move on
again, I consider that a win for me
-
It sounds as if your primary aim is to prevent some some people from using your code, without blocking access for too many other people. As opposed to the BSD/MIT/Apache licenes, whose primary aim is to make software available for all to use, without any attempt at dividing the world into us and them.
-
Close. The goal is to prevent some uses which in this world tends to leave out some users.
-
The goal is to prevent some uses
It is obligatory at this point to remind everyone that the AGPL should not be considered a Free Software license, as it does not grant Freedom 0. In fact, its entire purpose is to withhold Freedom 0 from recipients of the software in order to try to gain leverage over them.
-
The AGPL only triggers if you modify the software (since otherwise no copyright is in play and no license would be relevant). So if you just run unmodified software (freedom 0) the AGPL does not apply or restrict you.
-
-
The FSF’s stance on freedom is that you shouldn’t be allowed to have too much of it, lest you use it to do things the FSF disapproves of.
The AGPL was simply a reaction to the discovery that there were more things of which the FSF disapproved and which had not been foreclosed by prior licenses, so a new license was concocted to ensure that dangerous freedom wouldn’t get around too much.
-
-
I would flip that around and suggest that the rhetorical gymnastics the FSF uses to try to trick people into thinking their positions are coherent are something to see.
Essentially, they want to bludgeon everyone else with an absolutist position, while never being held to that same absolutism in their own actions. Or, more succinctly, they want to be able to compromise “freedom” when they think doing so will achieve a higher/larger goal. But woe to anyone else who tries doing that – then they’ll tell you that compromising freedom is never acceptable, no matter how good or great the thing you’d achieve by doing it!
Their adoption of the AGPL, which does not conform to their own original definition of Free Software and on those grounds never should have been accepted as a Free Software license, is just one especially obvious proof of that.
-
-
-
-
-
-
-
-
My goal is to provide as much value to the society as possible through each unit of my effort.
I want freedom for users to educate themselves and contribute as opposed to becoming mindless consumers.
That’s why I believe AGPL is a good license for applications.
I also believe that for libraries and frameworks MIT, APL or MPL work better to achieve that goal.
Having more educated people - in my opinion - is better than having more usable code in the long-term.
-
Overall, my bet is that choosing strong copyleft licenses has more of a net-negative effect on people who share my goal than any measurable effect on “the big bads”.
As someone who also prefers to release under permissive licenses: this, a million times. Big companies will always have a way to work around copylefted software, so it literally is not cutting them off from being able to do things the FSF disapproves of. Like, it’s not causing them to angrily shake their fists and yell “I would have gotten away with it, if not for you meddling Free Software kids!” It’s just causing them to use alternatives that aren’t under the FSF’s licensing regime.
Meanwhile, as the FSF gets ever more paranoid about ever more arcane “loopholes” in its licenses, the worries of small-time open-source developers go up as we juggle increasingly large transitive dependency trees that might have anything lurking in them. Not to mention whatever “loophole closure” the FSF might roll out next with brand-new extensions of what is or isn’t a derivative work.
-
-
-
-
-
I think the NcFtp client famously changed its licence to the GPL so it could use Readline… then in 1999 or so it switched licences again. The
copyright.hfile included in ncftp 1.9.5 says:static char copyright[] = "@(#) Copyright (c) 1992, 1993, 1994, 1995 by NCEMRSoft and Copyright (c) 1985, 1989 Regents of the University of California.\n All rights reserved.\n";…but the comment at the top of that file says “All rights reserved” and:
Redistribution and use in source and binary forms are permitted provided that: (1) source distributions retain this entire copyright notice and comment, and (2) distributions may not be sold for profit on physical media such as disks, tapes, and CD-ROMS, without expressed written permission.
…which is granting some rights so clearly they’re not all reserved.
Meanwhile, Wikipedia cites a Common Lisp implementation named “CLISP” as having switched to the GPL but I’m not sure what licence it switched from.
As perhaps a more famous example, the Objective C system that Mac OS X used at least during the PPC era was GPL’d because back in the day NeXT wanted to use GCC as their compiler, and the FSF said they couldn’t use GCC and keep the Objective C bits proprietary. Of course, as soon as Mac OS X got serious momentum behind it, Apple poured resources into LLVM and Clang…
-
That is a fascinating journey, thank you for sharing!!
Wonder if there’s anything more recent? Mid-90s certainly predates my career. I feel I am more in tune with modern open source culture, also I remember reading somewhere that more permissive licenses like MIT really took off in the era of Github.
-
-
At a previous employer we wanted to use an AGPL-licensed library as part of our SaaS offering. We wrote the extensions that directly linked to it into its own microservice and licensed that as AGPL and put it on GitHub. Rest of the SaaS product stayed proprietary since calling the AGPL parts over HTTP does not trigger the AGPL. Well, the legalities on that are very unclear, since “intimate enough” on the GPL FAQ. Not sure if we did the right thing legally, and morally I’m even less sure.
Last I heard the library in question was relicensed as BSD, so the issue is moot and nobody is using the old one anymore.
-
-
The place I work at builds tools that help other companies stay compliant with open source licenses. A lot of our bigger and most risk-averse customers (e.g. hardware manufacturers) actually take the stance that once GPL is brought into their first-party code, that code is “tainted” (i.e. you can’t make it compliant again just by removing the GPL dependency, because the commits where the GPL dependency were integrated are forever tainted by GPL and are forever in the commit history of any subsequent commits). Their default action is actually to publish “tainted” parts of their code base as open source to stay compliant - they feel that they’d rather publish some maybe-not-super-important parts of their IP rather than risk the trouble of a lawsuit.
-
Place I used to work had a codebase under GPLv2 (containing lots and lots of GPLv2 source by other people), decided it would be convenient if their stuff was AGPL instead, got told “no that’s impermissible” (I can’t remember if they actually tried it out they got told no before actually trying it) and went with GPLv2 instead of making a huge mess out of it. Dunno if that’s close enough to count.
Replacing all the GPLv2 code in there would’ve cost about the company’s yearly turnover times two, prolly, so doing anything other than just complying with the license as written was a non starter.
-
I know of several cases where the licensing was changed from foo to “either foo or gpl, your choice”, but I don’t think that’s what you really had in mind, right? You had in mind a change that grants users substantial additional rights?
So I agree with your intuition that the permissive licenses have achieved more, even if not quite the same.
-
-
Yes, and I think that if that were a significant effect, then I would have noticed it by now.
FWIW I worked at Trolltech until 2001; the team members’ reactions to the mail we got from GNU fans from 1994 until I left weren’t in the least favourable. At the time I thought I was special, we were special, but maybe we weren’t. Maybe most people who are, uhm, educated by GNU fans react negatively to the experience.
-
-
What stack being GPL? Libc and libX11 wasn’t, etc.
GNU fans sent us a lot of mail that might be described, somewhat uncharitably, as walls of text written by people who had much spare time and little salesmanship talent. For someone who has code to write and customers to help, dealing with yet another clueless wall of text is unappealing or worse.
-
-
-
-
-
I would love to see some examples where a company changed the license of their codebase to comply with a dependency.
I think this is a weird standard. Alternatively: examples where existing reciprocally licensed codebases were built upon instead of started from scratch?
- GCC and its myriad of backends including …
- Objective-C
- Linux
- Webkit / Blink
- MySQL
- Heaps of emulators
- Git
- ffmpeg
- Blender
- VLC
I feel like this is a target rich environment. What domains do you care about?
(* why is it always a company?)
-
Consider it a focus group.
The viral clause affects two groups: People who want to the viral clause to bind others, and people who are bound by the clause and wouldn’t have chosen the GPL otherwise. If you want to know about the viral clause of the GPL, then it makes sense to look at the reactions of a focus group inside each group. GP’s question is a nice way to find some in the latter group.
-
The viral clause
The use of “viral” makes me worry this isn’t a good faith response…
The viral clause affects two groups: People who want to the viral clause to bind others, and people who are bound by the clause and wouldn’t have chosen the GPL otherwise.
GPL code has no agency. That latter group chose to use GPL code. I see no complaints of “we pirated Oracle and now we have to pay a licensing fee” or “we pirated Oracle to get traction, and now we’re forced to rewrite.”
And I think there are more than two groups. e.g. people who specifically choose to work on the GPL projects.
-
“Viral” was common parlance when I learned about the GPL, in the early nineties. I agree that it has acquired more negative connotations since then.
More unaffected groups don’t matter. Unless you want to argue that the pool of people who’ll work on, say, GPL’d code but not APL’d code or closed-source code is so large that it will affect companies’ strategy for theiir implementation work?
-
I think most license selection is driven more by the authors and less by their lawyers, yes.
-
-
-
- GCC and its myriad of backends including …
-
-
Category 5 is cheaper, but can only support speeds of 100 Mbps. Category 6 is slightly more expensive but you will need it to get full gigabit speeds.
This isn’t entirely correct: cat5 is only rated for 100Mbps, but cat5e will do 1Gbps just fine and is significantly cheaper than cat6a and more flexible.
This is a pretty good read on the differences between cat5e/cat6.
-
While YMMV, I agree here and offer my experience. My house was built and wired in 1998 just before the change over from cat 5 to cat 5e. I did an addition in 2003 with cat 5e. I run gigabit on a mix of both of those cable types with no problem. I figure that there are two reasons for this. First, as the wikipedia article says, most cat 5 cable actually meets cat 5e specifications even though it wasn’t tested that way. Second, with regard to bandwidth, drop length matters at least as much as the cable you use. My longest drop might be 35 meters. My average drop is probably just under 10 meters. At those lengths, it was a good bet to replace my 100Mb/s switches with Gigabit swtiches and cross my fingers/keep the receipts.
-
My house was built in the early 90s, probably just after they switched from installing 4-wire copper phone cables to installing Category 3 cables instead. However, these Category 3 cables still support gigabit speeds without issue (I use it every day with our symmetric 300 Mbps Internet connection), despite being stapled to the struts.
I’m not saying all Cat 3 will do this, just that some cables do indeed meet higher specifications, per above.
-
-
-
-
-
Building a debian image deployment situation for the ever increasing number of raspberry pi and related boards in my house. Would like to have a git repo build an image that can be either written to an SD card for initial use, and then pushed over ssh for updates.
-
Hasn’t been updated at all since ~2010, and hasn’t been meaningfully changed since the early 2000’s.
-
It’d be sad to see wire go. Their UI and some features were better than signal for everday users, also they did stuff like opensource their server software and experiment with a rust client. (And I know people were verifying their crypto core.) Really wasted potential, and I’ll have to move part of my family to another client again.
-
-
-
Signal is effectively yet another walled garden, and moxie insists on keeping it that way. I would avoid it at this point.
I recently set up my own XMPP server and successfully migrated friends and family to using it. Some even registered with other servers! We are all using OMEMO. There were a few hiccups but it has gone pretty well, and now we are using a federated protocol that has been around for ~20 years.
-
I’ve seen some stuff regarding that, but, I’ll be honest: I’ve never tried XMPP and Matrix, I’ve only watched it from the outside. There still seem to be many hiccups, and I’m really keen on having a solution that works and where I don’t have to do the support from long distance when something breaks. In my experience you’ll otherwise loose people over to stuff where it’s “just working” and then you don’t have any saying in that decision anymore. (Friends moving to telegram/whatsapp, or people only being reachable there because all other systems just have a broken push notification system for them and/or won’t get re-installed next time the phone is wiped.)
So I’m ok with signals walled garden if it’s working.
Because the sad truth is that most people don’t have the time or nerves to get things working again, even though it may seem simple, but you’re also just used to that. It’s why google wins on android, everything else is subtle but annoyingly harder to maintain or set up.-
-
-
-
I like signal, but I’ve been hesitant to hand out my phone number to “unvetted acquaintances” online, mostly due to the fact that many companies (like banks!) wrongly rely on sms based auth mechanisms. If signal allowed the use of email as a “contact” instead of a phone number, I would find it usable for many additional use-cases.
-
-
-
-
-
-
Depends on what you need.
If you can live with the security of Telegram that is an excellent option ux-wise. Excellent clients for Android, iOS, Linux, Windows and I guess Mac as well (I haven’t tested it but AFAIK it is not just another Electron wrapper.)
It is however not e2e-encrypted though. Client to server is encrypted like gmail and my bank and they seem to care a lot even if some of the things they say or do doesn’t make sense to me.
I use it for a lot of my messaging.
-
It is however not e2e-encrypted though
Secret chats are.
The paranoid crowd likes to dunk on Telegram for not making secret the default/only way to chat, but I like the convenience of non-e2e chats (search all the history on any device without any problems and so on)
-
-
-
I’m still rooting for Matrix because they’re completely open source, federated (I think this is a really important point many overlook) and end-to-end-encryption. The company behind Matrix has a good reputation and even created the Matrix foundation to prevent this kind of corporate takeover of the communication platform. Just be aware that while e2e-encryption was implemented from the start and you can use it today, there are finishing touches being done to the UI right now so that it gets more usable/practical (cross-signing of new devices you log in to). The encryption stuff is a little harder to implement because Matrix is federated but there is a clear plan of how to solve all the issues and I hope they are finished soon :)
-
-
They couldn’t fix a 100% CPU bug on iOS that has been plaguing anyone who uses it for VOIP calls over the last 2+ years. I lost faith in their talent a long time ago.
-
There are similar outstanding problems on (older?) android devices too. On my old Samsung S4 your UI would hang for up to three seconds upon sending an image or loading high-through chats. Signal catches up in some aspects but still feels much more MVP. (I’ve had huge issues with random message-timers going on/off in bigger groups and messages never getting delivered or only partially, as well as people getting personal notifications about failed message receiving, which never got existed. Stuff that never happened on wire.)
Edit: Also signal really needs a progress-indicator which shows how many more messages to load, I sometime wait for ~800 upon opening the desktop app.
-
-
-
andyc
Oil Shell Developer
edited
3 years ago
|
link
| on:
apt-tool: an alternative to debootstrap and multistrap for deterministic and predictable root filesystems. - Paul Knopf
I’ve had this problem before, but don’t you think the .NET dependency re-introduces the predictability / stability problem? It’s a circular build dependency.
Since your operating system is now considered your application, you’d need the same predictability in your outputs.
It uses .NET, so install using the following command. sudo dotnet tool install AptTool –tool-path /usr/local/bin/
It’s analogous to tools like Chef trying to solve versioning problems on your base image, but then creating the same problem by depending on specific versions of Ruby and specific versions of Ruby packages.
Whether this is a problem in practice depends on your requirements, but it’s never gone as long as the circular dependency exists.
I’m sort of “starting at the bottom” with Oil, and this problem is why distros use shell so much in the first place! Shell has no runtime dependencies except libc and no build dependencies except the C compiler.
But it looks like Guix is doing good work to eliminate this circular dependency in another way. On the face of it, it seems like it’s going “overboard” a bit. But I think it’s good work and I don’t see any other solution.
https://guix.gnu.org/blog/2019/guix-reduces-bootstrap-seed-by-50/ (comments )
https://lists.gnu.org/archive/html/info-gnu/2018-08/msg00006.html comments
I’m still looking for PR to Oil to kick off the use of Guix or Nix. I got a couple PRs but they trailed off and didn’t appear to solve the problem (I welcome follow-up on those.)
https://github.com/oilshell/oil/issues/513
Under Help wanted I say I’ll accept any PR that makes
build/dev.sh minimalwork in Guix or Nix. All it does is build a Python extension module (under Python 2) and run several Python programs to generate code, so it’s dirt simple with zero weird dependencies.On Ubuntu it’s just the
python-devpackage, and the C compiler. Once that works, I imagine it should be to make other dependencies reproducible.The concrete problem I want to solve is that I’m on Ubuntu and other devs are on say Arch Linux, so we have slightly different versions of tools and libraries. My understanding is that Guix or Nix can solve that problem without creating a lot of circularity too. That is, I assume we won’t run into the problem that you need version X.0 of Guix on Ubuntu and version X.1 of Guix on Arch, etc.
-
It’s a circular build dependency.
Can you elaborate on this? I’m not entirely sure what you are saying here.
creating the same problem by depending on specific versions of Ruby and specific versions of Ruby packages.
Are you saying .NET is unpredictable? That just isn’t true. Sure,
dotnet tool install AptToolwill install the latest version, introducing “unpredictability”, but you can solve this easily by specifying--versionin your build CI/CD (which is what I would absolutely recommend). If you install the same version of a .NET global tool, you will have the same binaries/versions/dependencies at all times, giving you the same predictable outputs. This problem isn’t even solvable when using any of the tools out there to build Debian/Ubuntu rootfs (debootstrap,multistrap, etc).edit: Also,
apt-toolis a pure .NET package with no dependencies. It simply shells out todpkgandapt.Again, I’m not sure what you mean by “circular build dependency”, as I’m not using the generated rootfs to build another
apt-tool.And yes, predictable rootfs file systems are achievable via other distros (Gentoo, NixOS, Guix, etc), but
apt-toolis the only solution forapt-based systems.-
-
Interesting, I didn’t see that one!
Here is the repo.
It looks like it still lacks some features I need, such as multi-repos, pinning versions, and it is intended to be used with only the official Debian repos.
Thanks for sharing!
-
-
for deterministic and predictable root filesystems
This tool is helpful to create rootfs for debian-based distributions with reproducable outputs
Well I’m interested in clarification on what exactly you mean by deterministic, predictable, reproducible. What’s the claim? In what sense is debootstrap / Debian NOT reproducible / deterministic?
I can think of a few reasons, but I’m curious what problems you encountered. I have worked with it on the server side, not for embedded systems. When I worked on it, package builds were not deterministic, but I think they fixed that problem: https://wiki.debian.org/ReproducibleBuilds
However I think it’s probably true that debootstrap isn’t reproducible. But I haven’t used it in awhile and I’m curious what state it’s in. BTW I ran debootstrap with Oil shell: Success with Aboriginal, Alpine, and Debian Linux
So do you mean it in the mathematical sense? Like these binaries are a deterministic function of these sources and tools?
Now, I’m not saying you need to solve this problem for it to be useful. Because if you do you’ll probably be led down a long rathole like Guix has done.
But I want to know what the claim is. I don’t know anything about the .NET ecosystem. Do they make any claims about reproducibility?
Most language package managers like ecosystems like Python’s pypi, Node’s npm, don’t make any claims about reproducibility. It takes a lot of “extra” work to make builds reproducible. So if they don’t say it, then it’s almost certainly not. The burden of proof is on them.
For example, some package systems let you upload binaries you built on your desktop (like R’s, and I think Python’s). And nobody knows what tools are on there. And even if they don’t – even if they enforce a CI/CD system – you don’t know what’s in there either.
The problem is “recursive” – specifing
--versionjust punts it to another layer. Are .NET packages reproducible? In a practical sense they may be stable, but that’s a different claim than “reproducible” or “deterministic”. When you say something’s a deterministic function, you have to specify what it’s a function of. Some package ecosystems are “stable” practically speaking but not reproducible.Like I said, it’s a big rathole. If you haven’t heard of it already you can google for “trusting trust attack”. The Guix authors are explicitly addressing it.
edit: To be a bit more specific: What’s the complete set of transitive build dependencies of
apt-tool? sources and tools?-
I think the idea of a deterministic system in this case is you end up with all the same versions of software much like a npm
packages-lock.jsonfile would let you do. I don’t believe he’s referring to the reproducibility of theapt-toolsoftware itself -
I think we are talking past each other a bit.
I’m not going for “reproducible builds” in the sense you are speaking of, I’m going for “reproducible root file systems with specific versions installed”.
Consider that
apt-get update && apt-get install network-managerwill always install the latest version, introducing unpredictability. Theapt-toolcommand does the same thing (installing a package), but pins the version and locks it down via source control.-
Well I’m saying the post doesn’t make it clear what problem this is solving. You’re making a bunch claims with the words reproducible, deterministic, predictable, some of which already mean something specific in the Debian world.
There are limits to what you’re saying but it doesn’t acknowledge any of those limits.
You’re saying that the versions of the debs is locked? In the past I’ve included a
file://URL in/etc/sources.listand that can lock the versions too. It won’t retrieve anything from the network – only from the file system.In other words, you need to be able to recompile older versions years later and have some confidence that the output is the same.
If this is your goal, then I would want to know what claim you’re making about the reproducibility of
apt-tool.If I have one of your JSON files years later, I can’t build it again unless I have
apt-tool. This isn’t a theoretical problem – many I have tried to build software years later on a different OS, and the build doesn’t work. It’s actually a surprise when it does work.You should link to any claims that .NET repo makes, and Debian makes. As you said elsewhere in the thread, Debian doesn’t necessarily keep old packages. In that case, I would consider the
file://solution to be better (and much simpler). What about the .NET repo?-
Well I’m saying the post doesn’t make it clear what problem this is solving.
The “Why?” section of my post is pretty clear. In short,
aptis a rolling package manager, meaningapt-get update && apt-get install xxxwill get you the latest version. Because of this, if I install a rootfs with debootstrap, I automatically get updates. I don’t want these updates automatically. I want to generate a rootfs with the same exact package versions for years on end, using the sameimage-lock.jsonfile.I understand that the Debian repos only have the latest version of the packages available, but for commercial linux appliances, you will typically would use a proxied apt repo that caches all upsteam debs locally, forever. Therefore, my
image-lock.jsonwill be valid for as long as the proxied apt repo stays around.You’re making a bunch claims with the words reproducible, deterministic, predictable, some of which already mean something specific in the Debian world.
You are right, but these are the best words to describe what I’m trying to do, and the context of my post makes it clear. If you pick out one word from the post, of course it could mean something entirely different.
I would want to know what claim you’re making about the reproducibility of apt-tool.
apt-toolwill useimage-lock.jsonto generate the same exact rootfs, forever. That’s it. That means thatapt-tool generate-rootfswill give you the same outputs, regardless of what the upstream apt repo is doing (provided the versions are still at least available).You should link to any claims that .NET repo makes, and Debian makes.
This is getting pedantic. I think things are clear at this point.
I would consider the file:// solution to be better (and much simpler).
Not really. You moved the apt repo offline, but it’s still an apt repo that is rolling. If someone pushes to the folder, then your previous builds will all be different. Same problem.
Using
apt-tool, you can sit on top of a normal rolling apt-repo (network or file), and let the tool manage fixing the versions.What about the .NET repo?
What about it? Are you asking “what happens if the GitHub repository for apt-tool disappears?”
-
-
-
-
-
-
oracle2025
edited
3 years ago
|
link
| on:
apt-tool: an alternative to debootstrap and multistrap for deterministic and predictable root filesystems. - Paul Knopf
Does Debian keep an Archive of all Packages and Versions ever created? I remember trying to install a “historic” version of Debian, and I had to spend some time finding an installation medium.
-
Debian does not keep an archive of older packages, which makes this tool problematic when using standard debian repos.
However, in embedded systems, it’s typical to use an apt proxy that caches all upstream versions as they become available.
I should make this known in the post.
-
snapshot.debian.org keeps all versions released. To do reproducible builds, you end up with something like: deb http://snapshot.debian.org/archive/debian/20190126T211315Z/ buster main in your sources.list.
-
-
nolanl
3 years ago
|
link
| on:
The Year 2019 in Review: This Was, Once Again, Weirder Than the Last One
Those failures in the “grep(1)-centric script” are pretty alarming; hopefully they’re fixed now, and no one tried to log in with a username of `exploit_written_in_bashcode_here`
-
Interesting thought. The scriptery that finds usernames from ssh login failures as it stands now (and has for a number of years) is more like a two-step process.
The first extracts the likely user names and the IP addresses from the failures from the auth log. This part is naive enough that if what is where a username should be in the log line contains any kind of whitespace, the IP address output file will have at least one non-IP-address in it (typically the word “from”).
The next is a one-liner that compares each line in the usernames output file (the namestotry.txt file referenced in the article) to the list of existing spamtraps.
So I suppose there is a chance that if they managed to get a still valid exploit into the auth log with no whitespace that would lead to bits being just chopped off at the end, that attack vector could succeed, if it is structured so grep would somehow feed the thing to a subshell.
I’ve been pondering rewriting those things as something not-shellscript for a while, but there’s always the issue of sitting down to actually do that.
-
I am currently living in Spain and working on a 6 year old Thinkpad X1 Carbon, and an LLVM build takes a good bit over an hour. I also have a 32core Threadripper back in San Francisco, which does the same build in less than 2 minutes.
I wrote a script to continuously sync my edits as I work from my laptop back to my desktop. It supports running commands on the remote host (make, ./runtests, etc) ensuring that the latest changes are fully synced before the command starts.
If you’d like to try it (Linux only for now), it is available here: https://github.com/nolanl/rp