Thanks!

Federation seems like a fast path to ossification.

I have been thinking about this. There are certainly many protocols that are unchangeable at this point but I don’t think it has to be this way.

Web standards like HTML/CSS/JS and HTTP are still constantly improving despite having thousands of implementations and different programs using them.

From what I can see, the key to stopping ossification of a protocol is to have a single authority and source of truth for the protocol. They have to be dedicated to making changes to the protocol and they have to change often.

Federation seems like a fast path to ossification. It is much harder to change things without disrupting people if there are tons of random servers and clients out there. Also, remember how great federation worked out for xmpp/jabber when google embraced and then extinguished it? I sure do.

It may seem so, but that doesn’t mean it will happen. It has happened with xmpp, but xmpp had other problems, too:

• Not good for mobile use (some years back when messenger apps went big, but mobile connections were bad)
• A “kind-of-XML”, which was hard to parse (I may be wrong here)
• Reinventing of the wheel, I’m not sure how many crypto standards there are for xmpp

Matrix does some things better:

• Reference server and clients for multiple platforms (electron/web, but at least there is a client for many platforms)
• Reference crypto library in C (so bindings are easier and no one tries to re-implement it)
• Relatively simple client protocol (less prone to implementation errors than the streams of xmpp, imho)

The google problem you described isn’t inherent to federation. It’s more of a people problem: Too many people being too lazy to setup their own instances, just using googles, forming essentially an centralized network again.

It’s not nonsense, it’s just not an assurance. Nothing is. Open source, decentralization, and federation are the best we can get. However, I sense you think we can do better, and I’m curious as to what ideas you might have.

I find it really hard to believe the kernel would keep that userspace pointer around so …

https://github.com/torvalds/linux/blob/e978de7a6d382ec378830ca2cf38e902df0b6d84/net/socket.c#L1665

https://github.com/torvalds/linux/blob/e978de7a6d382ec378830ca2cf38e902df0b6d84/net/socket.c#L188

Bind is there too: https://github.com/torvalds/linux/blob/e978de7a6d382ec378830ca2cf38e902df0b6d84/net/socket.c#L1487

I work with 8-bit AVR MCUs. I often found myself having to cut corners and avoid certain abstractions, because that would have resulted either in larger or slower binaries, or would have used significantly more RAM. On an Atmega32U4, resources are very limited.

They seemed to have done pretty well for a long time without having one though.

I made an assumption, but reading your reply and that of @skade you are right that there are lots of uses for the data from a criminal perspective, especially for a site the size of reddit.

I think it’s rather difficult to write rust in a C like manner. This contrasts with go, where you can basically write C code and move the type declarations around and end up with somewhat unidiomatic but working go.

I think C++ as a better C works because you still have libc besides the STL, etc. The Rust standard library uses generics, traits, etc. quite heavily and type parameters and lifetime parameters tend to percolate to downstream users.

Though I think a lot of value in Rust is in concepts that may initially add some complexity, such the borrow checker rules.

You could argue the same for C++ (start with C and add extra features). Complexity comes with the whole ecosystem from platform support (OS, arch), compiler complexity (and hence subtle difference in feature implementations) to the language itself (C++ templates, rust macros). It’s challenging to limit oneself to a very specific subset on a single person project, it’s exponentially harder for larger teams to agree on a subset and adhere to it. I guess I just want a safer C not a new C++ replacement which seems to be the target for newer languages (like D & Rust).

Junior and senior developers also enjoy language features such as map, reduce, filter, and generics.

Those are great!

deterministic memory allocation, soft realtime, forced error checking, zero-cost abstractions, and (of course) memory safety.

Where are you finding juniors who care about this stuff? (no, really - I would like to know what kind of education got them there).

Map, reduce and filter are easily implemented in Go. Managing memory manually, while keeping the GC running, is fully possible. Turning off the GC is also possible. Soft realtime is achievable, depending on your definition of soft realtime.

Compiling a very large go project with a cold cache might take a minute (sub-second once the cache is warm).

Compiling a fairly small rust app with a warm cache has taken me over a minute (I think it’s a little better than that now).

Yes, and superior to Rust in that regard. Also the strict requirement to not have unused dependencies contributes to counteract dependency rot, for larger projects.

Both https://github.com/janestreet/async and https://github.com/ocsigen/lwt allow concurrent programming in OCaml. Parallelism is what you’re talking about, and I think there are plenty of domains where single process parallelism is not very important.

You are right. There is Multicore OCaml, though: https://github.com/ocamllabs/ocaml-multicore

This is how the OpenBSD ports tree works. Anyone can send a new port or an update to the ports@ mailing list. It then gets tested & committed by developers.

In this specific instance, I think what hurt Arch here is too good tooling. The community developed a lot of automation tools that boil down third party package installs to pressing enter a bunch of times - even with all the warnings present people stopped reviewing the packages. If I recall correctly, the main point of AUR was to gather community packages then promote the popular ones (by votes) to trusted repositories - essentially the promotion to trusted repos lost meaning as everyone can install yaourt/pacaur or $pkgmgr du jour and just go on with their life.

Also, while you shouldn’t rely on others to spot malicious code, the fact that the malicious modifications were spotted and reverted after about 9 hours shows that the AUR is subject to at least slightly more scrutiny than random scripts from github or elsewhere are.

Admittedly, it doesn’t sound like this particular attack was very sophisticated or well hidden.

re vulnerability brokers in general

You’re giving me a lot of examples but missing or disagreeing with a fundamental point. I’m going straight for it instead. It’s been bugging me a lot in past few years. It’s that most users and developers want their product to be vulnerable to achieve other goals. They willingly choose against security in a lot of ways. Users will go with a product that has lots of failures or hacks even with safer ones are available because it has X, Y, or Z traits that they think is worth that. The companies usually go with profit and/or feature maximization even when they can afford to boost QA or simplify. Both commercial and FOSS developers often use unsafe languages (or runtimes), limited security tooling, or small amounts of code reviews. These behaviors damn-near guarantee a lot of this software is going to be hacked. They do it anyway.

So, the market is pro-getting hacked to the point they almost exclusively use things with lots of prior CVE’s. The network effects and oligopolistic tactics of companies mean there’s usually just a few things in each category. Black hats and exploit vendors are putting lots of time and money into the bug hunting that those suppliers aren’t doing and customers are voting for with their wallet. There’s going to be 0-days found in them. If there’s damage to be done, it will be done just as each party decided with their priorities. With that backdrop, will your bug in a Linux or BSD make a difference to whether folks buying from Zerodium will hack that platform? Probably not. Will it make a difference as to who gets paid and how much if you choose responsible disclosure over them? Probably so.

To drive that home, Microsoft, IBM, Google, and Apple all have both the brains and money to make their TCB’s about as bug-proof as they can get. If they care about security, then that’s a good thing to do. If their paying users care, then it’s even more a good thing to do. They spend almost nothing on preventative security compared to what they make on their products and services. They don’t care. They’ll put the vulnerabilities in themselves just to squeeze more profit out of customers. Letting a broker have them before someone else isn’t making much difference. That’s at least on damage assessment angle.

I think about it differently if the customer is paying a lot extra for what’s supposed to be good security. I think the supplier should be punished in courts or something for lying with the cost high enough that they start doing security or stop lying about what they’re not doing. Also, I think suppliers who have put good effort in shouldn’t be punished over a little slip or a new class of attack. I’d rather people finding those get paid so well by the companies and/or a government fund that they don’t go to vulnerability brokers most of the time. I’m just not having much sympathy for either users or suppliers griping about vulnerability brokers if they both favor products they know will get hacked because they accepted the tradeoffs. Whereas, projects that focus on balance of features and security with strong review often languish with low revenues or (for FOSS) hardly any financial contributions.

re suspicious builds

All software is insecure and suspicious until proven otherwise by strong review. That’s straight-up what security takes. Since you mentioned it, the guy (Paul Karger) that invented the compiler-compiler attack that Thompson demod laid out some requirements for dealing with threats like that. Reproducible builds don’t begin to cover it, esp malicious developers. For the app, you need precise requirements, design, security policy, and proof that they’re all consistent with nothing bad added or good subtracted. Then, a secure repo like described here. Object code validation like in DO-178C regulations if worried about compilers. Manual per app or use a certifying compiler like CompCert after it is validated. Then, Karger et al recommended all of that be sent via protected channel to customers so they can re-run the analyses/tests and build from source locally. All of that was what would be required to stop people like him from doing a subversion attack. Those were 1970’s to early 1990’s era requirements they used in military and commercial products.

re someone introduces vulnerability

I’d correct the vulnerability. I’d ask them if it was a slip up or they’d like to learn more about preventing that. I’d give them some resources. I’d review their submissions more carefully throwing some extra tooling at them, too. Anyone that keeps screwing up will be out of that project. People who improve will get a bit less review. However, as you saw above, my standard for secure software would already include some strong review plus techniques for blocking root causes of code injection and (if needed) covert channels. Half-ass code passing such a standard should usually not lead to big problems. If it is, they or their backers are so clever you aren’t going to beat them by ejecting them anyway. Ejection is symbolic. Just fix the problem. Add preventative measures for it if possible.

Notice what I’m doing focuses on the project deliverables and their traits instead of the person. That’s intentional. If I have to trust them, my process is doing it wrong. At the least, I need more peer review and/or machine checks in it. As Roger Schell used to say, software built with the right methods is trustworthy enough that you can “buy it from your worst enemy.” He oversold it but it seems mostly true on low-to-mid-hanging fruit.

Yes, or you can set up a paranoid firewall that way…

Here’s a longer excerpt from the second edition’s preface.

Prior to the publication of the first edition, the manuscript was reviewed by a professional C programmer hired by the publisher. This individual expressed a firm opinion that the book should not be published because “it offers nothing new—nothing the C programmer cannot obtain from the documentation provided with C compilers by the software companies.”

This review was not surprising. The reviewer was of an opinion that was shared by, perhaps, the majority of professional programmers who have little knowledge of or empathy for the rigors a beginning programmer must overcome to achieve a professional-level knowledge base of a highly technical subject.

Fortunately, that reviewer’s objections were disregarded, and “Mastering C Pointers” was released in 1990. It was an immediate success, as are most books that have absolutely no competition in the marketplace. This was and still is the only book dedicated solely to the subject of pointers and pointer operations using the C programming language.

To answer your question, then, all we can conclude is that a “professional C programmer” reviewed the first edition before it was published, recommended against publishing it, but the book was published anyway. If the quoted portion were the reviewer’s only objection, then we could surmise that the reviewer didn’t know much either, or didn’t actually read it.

I recommend a scene from Hal Harley’s film “Fay Grim” (the sequel to “Henry Fool”) here. At a point, Fay questions the publishers decision to publish a work (‘The Confessions’) of her husband - she only read “the dirty parts” but still recognized the work as “really, really bad”.

Excerpted from a PopMatters review: “One proposal, from Simon’s publisher Angus (Chuck Montgomery), will lead to publication of Henry’s (admittedly bad) writing and increased sales of Simon’s poetry (on which royalties Fay and Ned depend to live). (Though the writing is, Fay and Angus agree, “bad,” he asserts they must press on, if only for the basest of reasons: “We can’t be too hard-line about these things, Fay. Anything capable of being sold can be worth publishing.”)”

I’m not hiding much past what’s private or activates distracting biases. I’ve been clear when asked on Schneier’s blog, HN, maybe here that I don’t work in the security industry: I’m an independent researcher who did occasional gigs if people wanted me to. I mostly engineered prototypes to test my ideas. Did plenty of programming and hacking when younger for the common reasons and pleasures of it. I stayed in jobs that let me interact with lots of people. Goal was social research and outreach on big problems of the time like a police state forming post-9/11 which I used to write about online under aliases even more than tech. I suspected tech couldn’t solve the problems created by laws and media. Had to understand how many people thought, testing different messages. Plus, jobs allowing lots of networking mean you meet business folks, fun folks, you name it. A few other motivations, too.

Simultaneously, I was amassing as much knowledge as I could about security, programming, and such trying to solve the hardest problems in those fields. I gave up hacking since its methods were mostly repetitive and boring compared to designing methods to make hacking “impossible.” Originally a mix of public benefit and ego, I’d try to build on work by folks like Paul Karger to beat the worlds’ brightest people at their game one root cause at a time until a toolbox of methods and proven designs would solve the whole problem. I have a natural, savant-like talent for absorbing and integrating tons of information but a weakness for focusing on doing one thing over time to mature implementation. One is exciting, one is draining after a while. So, I just shared what I learned with builders as I figured it out with lots of meta-research. My studies of work of master researchers and engineers aimed to solve both individual solutions in security/programming (eg secure kernels or high-productivity) on top of looking for ways to integrate them like a unified, field theory of sorts. Wise friends kept telling me to just build one or more of these to completion (“focus Nick!”). Probably right but I’d have never learned all I have if I did. What you see me post is what I learned during all the time I wasn’t doing security consulting, building FOSS, or something else people pushed.

Unfortunately, right before I started to go for production stuff beyond prototypes, I took a brain injury in an accident years back that cost me most of my memory, muscle memory, hand-eye coordination, reflexes, etc. Gave me severe PTSD, too. I can’t remember most of my life. It was my second, great tragedy after a triple HD failure in a month or two that cost me my data. All I have past my online writings are mental fragments of what I learned and did. Sometimes I don’t know where they came from. One of the local hackers said I was the Jason Bourne of INFOSEC: didn’t know shit about my identity or methods but what’s left in there just fires in some contexts for some ass-kicking stuff. I also randomly retain new stuff that builds on it. Long as it’s tied to strong memories, I’ll remember it for some period of time. The stuff I write-up helps, too, which mostly went on Schneier’s blog and other spaces since some talented engineers from high-security were there delivering great peer review. Made a habit out of what worked. I put some on HN and Lobsters (including authored by’s). They’re just text files on my computer right now that are copies of what I told people or posted. I send them to people on request.

Now, a lot of people just get depressed, stop participating in life as a whole, and/or occasionally kill themselves. I had a house to keep in a shitty job that went from a research curiosity to a necessity since I didn’t remember admining, coding, etc. I tried to learn C# in a few weeks for a job once like I could’ve before. Just gave me massive headaches. It was clear I’d have to learn a piece at a time like I guess is normal for most folks. I wasn’t ready to accept it plus had a job to re-learn already. So, I had to re-learn the skills of my existing job (thank goodness for docs!), some people stuff, and so on to survive while others were trying to take my job. Fearing discrimination for disability, I didn’t even tell my coworkers about the accident. I just let them assume I was mentally off due to stress many of us were feeling as Recession led to layoffs in and around our households. I still don’t tell people until after I’m clearly a high-performer in the new context. Pointless since there’s no cure they could give but plenty of downsides to sharing it.

I transitioned out of that to other situations. Kind of floated around keeping the steady job for its research value. Drank a lot since I can’t choose what memories I keep and what I have goes away fast. A lot of motivation to learn stuff if I can’t keep it, eh? What you see are stuff I repeated the most for years on end teaching people fundamentals of INFOSEC and stuff. It sticks mostly. Now, I could’ve just piece by piece relearned some tech in a focused area, got a job in that, built up gradually, transitioned positions, etc… basically what non-savants do is what I’d have to do. Friends kept encouraging that. Still had things to learn talking to people especially where politics were going in lots of places. Still had R&D to do on trying to find the right set of assurance techniques for right components that could let people crank out high-security solutions quickly and market competitive. All the damage in media indicated that. Snowden leaks confirmed most of my ideas would’ve worked while most of security community’s recommendations not addressing root causes were being regularly compromised as those taught me predicted. So, I stayed on that out of perceived necessity that not enough people were doing it.

The old job and situation are more a burden now than useful. Sticking with it to do the research cost me a ton. I don’t think there’s much more to learn there. So, I plan to move on. One, social project failed in unexpected way late last year that was pretty depressing in its implications. I might take it up again since a lot of people might benefit. I’m also considering how I might pivot into a research position where I have time and energy to turn prior work into something useful. That might be Brute-Force Assurance, a secure (thing here), a better version of something like LISP/Smalltalk addressing reasons for low uptake, and so on. Each project idea has totally different prerequisites that would strain my damaged brain to learn or relearn. Given prior work and where tech is at, I’m leaning most toward a combo of BFA with a C variant done more like live coding, maybe embedded in something like Racket. One could rapidly iterate on code that extracted to C with about every method and tool available thrown at it for safety/security checks.

So, it’s a mix of indecision and my work/life leaving me feeling exhausted all the time. Writing up stuff on HN, Lobsters, etc about what’s still clear in my memory is easy and rejuvenating in comparison. I also see people use it on occasion with some set to maybe make waves. People also send me emails or private messages in gratitude. So, probably not doing what I need to be doing but folks were benefiting from me sharing pieces of my research results. So, there it is all laid out for you. A person outside security industry going Ramanujan on INFOSEC and programming looking for its UFT of getting shit done fast, correct, and secure (“have it all!”) while having day job(s) about meeting, understanding, and influencing people for protecting or improving democracy. Plus, just the life experiences of all that. It was fun while it lasted. Occasionally so now but more rare.

I have to admit similar misgivings (unsurprisingly, I came here via @apg and know @apg IRL). For someone so prolific and opinionated you have very little presence beyond commenting on the internet. To me, that feels suspicious, but who knows. I’m actually kind of hoping you’re some epic AI model and we’re the test subjects.

yeah this is a very real possibility. I’m a strong believer in the gradient of security but the idea of just walking down and being able to unlock all the doors is v scary

(also: why does this even need to be on the internet?? We made electronic devices before bluetooth low energy, it really feels like we should be able to make a lot of this stuff in an offline way)

I remember when someone in the D community proposed to include a basic web server in the standard library. Paraphrased:

“Hell no, are you crazy? A web server is a huge complex thing.”

“Why not? Python has one and it is useful.”

I think just that there are two implementations. One is just called “Lua”, is an interpreter written in C, supposedly runs pretty fast for a bytecode interpreter. The other is LuaJIT and runs much faster (and is the one benchmarked here).

Extra context: LuaJIT isn’t up to date with the latest Lia either, so they’re almost different things, sorta.

LuaJIT is extremely impressive.