Threads for hlandau

  1. 3

    This has some weird bugs. The precedence of a(b) >> c &= d is definitely not (a(b)) >> (c &= d), it is (((a(b)) >> c) &= d) (which is invalid in C as you can’t assign to an expression). The precedence of a *= b != c ? d : e is not ((a *= (b != c)) ? d : e), it is (a *= ((b != c) ? d : e)). There seems to be some issues with this game’s understanding of the precedence of assignment operators and also the associativity of comparison operators.

    1. 1

      Perhaps this is a bug in Tree-sitter? You can see the AST it makes here.

      Edit: Reported:

    1. 1

      I wonder why the author doesn’t want to talk about ColdFusion :-) I remember having a great time with it.

      1. 1

        and also with Userland Frontier. Those were the days.

        1. 1

          Author here. Honestly, I never used it. It’s of course proprietary, but also certainly the path less travelled. It was always the mysterious exotic evil twin technology to PHP from my perspective, similarly to how people remember Macromedia Shockwave much less than Macromedia Flash. From what I saw of it, I didn’t see much to make it compelling over PHP. When I saw sites using it with the telltale .cfm extension, I always wondered how they ended up using it and why.

          I’d certainly be interested to read about Coldfusion and what things make it interesting or better if you’d be interested in writing about it…

          1. 1

            I used it for just a brief period, not enough to write a good post. I had a good time doing it, but not enough time to make it an informed opinion. I might try to write about running Userland Frontier in 2022 rsrsrs, that should be fun. At the moment I’m swamped with the book about decentralisation that I’m writing, but these small pieces about the road not taken are very dear to my heart and I might pursue them as a palate cleanser between chapters.

            I really liked your post and as someone using a SSG to generate my blog, I miss being able to do mild dynamic stuff. I kinda can at generation time because my blog is built with Racket and I can simply add Racket code mid-post and hope for the best. It is not the same as generating the content at request time, which means that some forms of workflow are not possible (such as a commenting system), but it is better than none.

            I wonder what alternatives to PHP exist these days that allows one to drop in and out of HTML easily with a similar deployment story. I can’t think of a single one (oh, there is a FOSS clone of ColdFusion, I never used it but maybe it fits this).

            1. 1

              Someone on HN mentioned something called “Lua Server Pages”, which sound similar, and are implemented in a library on top of e.g. mod_lua:


              I imagine you could come up with something very PHP like built on top of Scheme with <?scm ... ?> or similar. I’m very fond of Scheme and often use Guile for XML processing using SXML. In fact I’ve been experimenting with using it as a technical writing platform for generating books… I should do a writeup about it sometime.

              1. 3

                There used to be support for “SSP” (Scheme Server Pages) in the Spiffy web server which I maintain, that used <?scheme .. ?> and <? .. ?>. Because it’s pretty ugly and rife with security problems (just as plain PHP templating is), I eventually deprecated it and moved it out to a separate egg which has not been ported to CHICKEN 5 due to lack of interest.

                The canonical way to serve HTML from Scheme is to use SXML (XML as s-expressions), which is not vulnerable to (most) injection problems and also allows you to effortlessly transfer between code and data (by using Scheme’s quasiquote) in a way that can’t leave you with mismatched closing tags as an added benefit!

                1. 3

                  Indeed, you’re preaching to the choir in my case.

                  Though the article I wrote is in part motivated by the fun I’ve had with PHP in the “hackish state” in the past, I’ve honestly come to the view that templating is one thing PHP should (ironically, given its origins) never be used for. The reason for this is that the lack of autoescaping makes its use as a templating language unconscionable from a security perspective nowadays, in my view.

                  Fundamentally one of the reasons the software industry has so many security issues is its failure to practice problem class elimination. The aviation industry has been good at practicing problem class elimination: when an accident happens, we try to figure out how to prevent that entire class of accidents from happening again. Terrain collisions keep happening, so someone invented GPWS, etc.

                  We have comparable things now with memory-safe languages, prepared statements, and autoescaping, etc. Yet embarrassingly SQL injection, XSS, etc. are still an absurdly common vulnerabilities.

                  I’ve actually come to the conclusion that generating (X)HTML using string templating systems is fundamentally a hack and always has been. The rejection of XHTML by web developers and the notion of having to generate well-formed XML as being unreasonably burdensome, as represented by the HTML5 movement, was very telling. In any other case this wouldn’t be taken seriously — if someone claimed that having to write (or generate) grammatically correct C code is unreasonably burdensome, we’d be laughing.

                  This is as opposed to serializing an AST, as in the case of SXML or DOM serialization — an approach which makes outputting malformed XML literally impossible. Problem class elimination. XSS is eliminated but not even by autoescaping — it would be a mistake to call it autoescaping, it’s simply correct serialization of a string as a string in the target format.

                  But string templating systems with autoescaping tacked on remain dominant. Autoescaping isn’t perfect either as it isn’t context-sensitive and just assumes escaping a given set of characters will suffice. I suspect the reason string templating for HTML remains dominant is because of how cumbersome (and historically, less performant) AST-based templating languages have historically been (see XSLT).

                  We have seen some progress with the rise of JSX, though. But you’re correct, there really is no nicer environment for writing and generating XML than Scheme. I use it as a typesetting environment for some yet-unpublished projects of mine, with an executable Scheme program that generates SXML transformable ultimately to XHTML or TeX. Aside from how useful quasiquoting is in general, I like to define terminology for technical writing as Scheme defines, which has the interesting attribute that references to terms defined earlier in a document (which should be turned into hyperlinks, etc.) are actually checked at build time:

                  (dt bipipe "bipipe" "A combination of two "unipipes", one in each direction.")
                  ( ...
                    (p "A "bipipe" is realized as two realized "unipipes"."))

                  Now that I’ve written this comment, I guess this does raise the question of why not apply this to a PHP-like application, where you write websites in SXML for mildly dynamic applications. Honestly, that’s a really nice idea, and would lead to the levels of creativity enabled both by Scheme and by a PHP-style hackish environment (while being actually conscionable to use, unlike PHP-as-a-templating-language). I might seriously look into that…

                  1. 1

                    Fundamentally one of the reasons the software industry has so many security issues is its failure to practice problem class elimination.

                    Amen, brother!

                    1. 1

                      I recommend you look at Scribble or Pollen. Neither of them are really geared towards websites, which makes them less than dynamic than what you’d want for some things, but they do let you write XML pretty easily, and make it easy to layer on abstractions and computations. I’m working on a system called Colophon to try and make Pollen friendlier for websites.

                  2. 1

                    I was playing with cgilua in early 2000s. I was a student at PUC Rio at the time (where Lua was invented) and we were all aware of it. It was fun, but it was not as straightforward as PHP to jump into and out of HTML. I don’t recall using mod_lua back then, I remember cgilua being run from cgi-bin like any other cgi.

                    I think there is room for niche engines like that. I’d definitely read about someone using Lua or Scheme in a similar manner. Some years ago, I built a backend for a large personal project using Lua and the Sailor framework, it felt very refreshing.

            1. 6

              I’ve been experimenting a little with using cron jobs to bring back some of this “mildly dynamic” content within a static site generator framework. For example, it’s fairly easy to integrate things like the current weather into a page by just regenerating it once an hour; you don’t need it to be looked up on literally every page load. Other kinds of mildly dynamic content, like anything customized to a specific reader, is admittedly not a good fit, but a decent number of my personal uses of PHP-style stuff would work fine at once-an-hour update speeds.

              I’m not sure any of the popular SSG frameworks have real first-class support for this though. As the article notes, the low-friction part of being able to just insert a little code into the template is important.

              1. 2

                Author here. I like this idea. Here’s another random example of mildly dynamic functionality: long ago as a child, I remember customising a PHP web forum so that the theme it used would vary based on the time of day. Not something that would work well internationally, but if most users of a forum were in the same time zone it worked.

                The script I use to generate my site actually supports embedded JS, which is evaluated at generation time, PHP style, even if the output is ultimately static:


                The output is generated by manipulating a DOM imported from the original XHTML source file of each page to add headers, footers, etc. (Markdown files are automatically converted to XHTML first, then treated the same way.) These scripts can programmatically generate new DOM nodes and edit existing ones. The index on the front page is generated by such an embedded script.

                1. 2

                  The output is generated by manipulating a DOM imported from the original XHTML source file of each page to add headers, footers, etc.

                  That sounds like the sort of thing that is very hard to understand and change after it’s written. You can do the same thing with JavaScript and have it actually use the user’s real timezone. Ideally it would just be a matter of toggling a class on the body like .morning-theme, etc. and having CSS do the rest.

                  It’s sort of incredible the stuff we did back in the day. It definitely had its advantages, but I think the biggest advantage was just the amount of free time I had as a young person. :-)

              1. 7

                As I recall, CGI was present very early on, definitely by 1995, and early websites definitely made use of it — obviously for form submission, but it was also sometimes used for serving pages.

                There were also early servers, like Netscape’s, that ran their own custom server-side app code — I don’t know for sure but I suspect they had their own C-level plugin system for running handlers in-process to avoid the high overhead of CGI.

                I’m still wondering why only PHP became available as an easy in-process scripting language. It’s not like you couldn’t build a similar system based on Python or Ruby or JS. Maybe it was the ubiquity of Apache, and the Apache developers not wanting to add another interpreter when “we already have PHP?”

                1. 14

                  As mentioned in the article, there were other Apache modules providing similar functionality, such as mod_python. There were also CGI approaches to the same template-forward bent, such as Mason (which was perl). If there was anyone saying “why support another since we already have PHP?” it was admins on shared hosting services. Each additional module was yet another security threat vector and a customer service training.

                  1. 6

                    I was at a talk given by Rasmus Lerdorf (creator of PHP) once and he claimed it was because the PHP implementation was the most basic, limited version possible and it therefore it was very simple to isolate different users from each other. This made PHP very popular with cheap shared hosters. Whereas the Perl implementation was much more thorough and hooked (not sure what the correct terms are) into the whole of Apache and therefore it needed a dedicated server. Much more expensive.

                    1. 2

                      Yeah. Even though mod_php is a single module loaded into a single Apache instance, it was designed with some sandboxing options like safe_mode. Or you could use PHP CGI and isolate things even better (running as the user’s UID).

                      Other language hosting modules for Apache like mod_perl didn’t offer the same semantics. I also recall mod_perl being pretty oriented towards having access to the web server’s configuration file to set it up. People did use Perl before the rise of PHP, but most often via CGI (remember iKonboard?)

                      1. 3

                        mod_perl was more oriented toward exposing the apache extension API so that you could build apache modules in perl, as I remember it. It got used to write some cool web applications (slashcode springs to mind) that’d have been hard to write (at that scale) any other way at the time. But mod_php was a very different beast, just aiming to be a quick way to get PHP to work well without the overhead of CGI.

                        I agree with the article… there’s nothing now (other than PHP, which I still use now for the kind of pages you mention, the same way I did in the early ‘00s) that’s nearly as low-friction as PHP was back then to just add a couple of dynamic elements to your static pages.

                        1. 2

                          Yeah, I was at a small web hosting company in the late ’90s, early 2000s, and we used PHP CGI with our shared hosting.

                    2. 10

                      It’s not like you couldn’t build a similar system based on Python or Ruby or JS.

                      Not quite. The article touches this, although not explicitly, you have to read a bit between the lines.

                      PHP allowed for easy jump in and out static and dynamic context like no other alternative. It still does this better than anything else. This was in the core of the language no need to install third party libraries. It also included a MySQL client library in its core with work out if the box. Essentially, it shipped with everything necessary in the typical setup. No need to fiddle with server set up.

                      The language was also arguably more approachable for beginners than perl with a multitude of simple data structures easily accessible through the infamous array() constructor. It also retained familiarity for C programmers, which were a big audience back then. While python for example, didn’t.

                      One thing I don’t agree with is the simplicity nor the deployment model. It’s only simple in the context of the old shared hosting reality. If you include setting up the server yourself like we do nowadays, it is actually more cumbersome than a language that just allows you to fire up a socket listening on port 80 and serve text responses.

             it was marketed and packages that made all the difference.

                      1. 9

                        Yes, but it was “better” in the sense of “making it easy to do things that are ultimately a lousy idea”. It’s a bit better now, but I used it back then and I remember what it was like.

                        Convenience feature: register_globals was on by default. No thinking about nasty arrays, your query params are just variables. Too bad it let anyone destroy the security of all but the most defensively coded apps using nothing more than the address bar.

                        Convenience feature: MySQL client out of the box. Arguably the biggest contributor to MySQL’s success. Too bad it was a clumsy direct port of the C API that made it far easier to write insecure code than secure. A halfway decent DB layer came much, much later.

                        Convenience feature: fopen makes URLs look just like files. Free DoS amplification!

                        Convenience feature: “template-forward”, aka “my pages are full of business logic, my functions are full of echo, and if I move anything around all the HTML breaks”. Well, I guess you weren’t going to be doing much refactoring in the first place but now you’ve got another reason not to.

                        The deployment story was the thing back then. The idea that you signed up with your provider, you FTP’d a couple files to the server, and… look ma, I’m on the internet! No configuration, no restarting, no addr.sin_port = htons(80). It was the “serverless” of its day.

                        1. 21

                          Yes, but it was “better” in the sense of “making it easy to do things that are ultimately a lousy idea”. It’s a bit better now, but I used it back then and I remember what it was like.

                          It was better, in the sense of democratizing web development. I wouldn’t be here, a couple decades later, if not for PHP making it easy when I was starting out. The fact that we can critique what beginners produced with it, or the lack of grand unified design behind it, does not diminish that fact. PHP was the Geocities of dynamic web apps, and the fact that people now recognize how important and influential Geocities was in making “play around with building a web site” easy should naturally lead into recognizing how important and influential PHP was in making “play around with building a dynamic web app” easy.

                          1. 3

                            Author here, I couldn’t have put it better. “PHP was the Geocities of dynamic web apps” — this is a brilliant way to put it. In fact I’m now peeved I didn’t think of putting it like this in the article. I’m stealing this phrase for future use. :)

                          2. 2

                            Absolutely. And indeed, I saw those things totally widespread to their full extent in plenty of code bases. To add a bit of [dark] humor to the conversation, I even whiteness code that would use PHP templating capabilities to assemble PHP code that was fed to eval() on demand.

                            But I am really not sure you can do anything about bad programmers. No matter how much safety you put in place. It.s a similar situation with C. People complaining of all the footguns.

                            Can you really blame a language for people doing things like throwing a string in an SQL query without escaping it? Or a number without asserting its type? I really don’t have a clear opinion here. Such things are really stupid. I .not sure it is very productive to design technology driven by a constant mitigation of such things.

                            EDIT: re-reading your post. So much nostalgia. The crazy things that we had. Makes me giggle. Register globals or magic quotes were indeed… punk, for lack of a better word. Ubernostrum put it really well in a sister comment.

                            1. 4

                              But I am really not sure you can do anything about bad programmers. No matter how much safety you put in place. […] Can you really blame a language for people doing things like throwing a string in an SQL query without escaping it?

                              Since you mention magic quotes … there’s a terrible feature that could have been a good feature! There are systems that make good use of types and knowledge of the target language to do auto-escaping with reasonable usability and static guarantees, where just dropping the thing into the query does the secure thing 98% of the time and throws an “I couldn’t figure this out, please hint me or use a lower-level function” compile error the other 2%. PHP could have given developers that. Instead it gave developers an automatic data destroyer masquerading as a security feature, again, enabled by default. That’s the kind of thing that pisses me off.

                          3. 3

                            I definitely had a lot of fun making mildly dynamic websites in PHP as a teen, but I wouldn’t want to get back to that model.

                            They might have a style selector at the top of each page, causing a cookie to be set, and the server to serve a different stylesheet on every subsequent page load. Perhaps there is a random quote of the day at the bottom of each payload.

                            JS in modern browsers allows that kind of dynamicity very nicely, and it’s easy to make it degrade gracefully to just a static page. It will even continue to work if you save the page to your own computer. :)

                          4. 6

                            I’m still wondering why only PHP became available as an easy in-process scripting language. It’s not like you couldn’t build a similar system based on Python or Ruby or JS. Maybe it was the ubiquity of Apache, and the Apache developers not wanting to add another interpreter when “we already have PHP?”

                            I am someone who is, these days, primarily known for doing Python stuff. But back in the early 2000s I did everything I could in PHP and only dabbled in Perl a bit because I had some regular business from clients who were using it.

                            And I can say beyond doubt that PHP won, in that era, because of the ease it offered. Ease of writing — just mix little bits of logic in your HTML! — and ease of deployment via mod_php, which for the developer was far easier than messing around with CGI or CGI-ish-but-resident things people were messing with back then. There are other commenters in this thread who disagree because they don’t like the results that came of making things so easy (especially for beginning programmers who didn’t yet know “the right way” to organize code, etc.) or don’t like the way PHP sort of organically grew from its roots as one guy’s pile of helper scripts, but none of that invalidates the ease PHP offered back then or the eagerness of many people, myself included, to enjoy that easiness.

                            1. 4

                              mod_php was always externally developed from Apache and lived in PHP’s source tree.

                              1. 3

                                The other options did exist. There were mod_perl and mod_python for in-process (JS wasn’t really a sensible server-side option at the time we’re talking about), mod_fastcgi and mod_lisp for better-than-CGI out-of-process (akin to uwsgi today), and various specialized mod_whatevers (like virgule) used by individual projects or companies. mod_perl probably ran a sizeable fraction of the commercial web at one point. But they didn’t take PHP’s niche for various reasons, but largely because they weren’t trying to.

                                1. 2

                                  There was also the AOL webserver, which was scriptable with TCL. It looks like this was around in the early nineties, but perhaps it wasn’t open sourced yet at that point? That would definitely make it harder to gain momentum. Of course TCL was also a bit of an odd language. PHP still had the benefit of being a seamless “upgrade” from HTML - just add some logic here and there to your existing HTML files. That’s such a nice transition for people who never programmed before (and hell, even for people who had programmed before!).

                                  Later on, when Ruby on Rails became prominent (ca 2006), it was still not “easy” to run it. It could run with CGI, but that was way too slow. So you basically had to use FastCGI, but that was a bit of a pain to set up. Then, a company named Phusion realised mod_passenger which supposedly made running Ruby (and later, other languages like Python) as easy as mod_php. The company I worked for never ran it because we were already using fastcgi with lighttpd and didn’t want to go back to Apache with its baroque XML-like config syntax.

                                  1. 2

                                    I worked at at shared hosting at the time of the PHP boom. It all boiled down to the safe mode. No other popular competitor (Perl / Python) had it.

                                    Looking back, it would have been fairly cheap to create a decent language for the back-end development that would have worked way better. PHP language developers were notoriously inept at the time. Everyone competent was busy using C, Java, Python and/or sneering at the PHP crowd, though.

                                    1. 1

                                      It’s not like you couldn’t build a similar system based on Python or Ruby or JS.

                                      There’s ERuby which was exactly this. But by then PHP was entrenched.

                                      I did a side project recently in ERuby and it was a pleasure to return to it after >10 years away.

                                    1. 28

                                      There’s a simple solution to this:

                                      1. Create a Google account
                                      2. Send a PR
                                      3. Agree to the CLA
                                      4. Have the PR merged
                                      5. Send a GDPR notice to Google requiring that they delete all PII associated with the Google account and close it.

                                      Repeat this process for every single patch that you submit. Eventually, Google’s compliance team will either bankrupt the company or come up with a better process.

                                      There’s also a plan B solution that works well for me:

                                      1. Don’t contribute to Google-run open source projects until they learn how to work with the community.
                                      1. 8

                                        You say “the community” as though there is just one, or that it is a well-defined term.

                                        We have a large community of Go contributors from outside Google that we do work well with. It so happens that these people all have created Google accounts to log in to Google-run web sites - including our code review site - much the same way I have to create a GitHub account to post on Go’s issue tracker. We may be losing out on contributions from a few people, perhaps yourself included, who for one reason or another cannot create such an account. That’s unfortunate but hardly the common case.

                                        1. 5

                                          How can you measure the number of folks who would contribute if there wasn’t a silly requirement to make a google account vs the number of folks who did in order to contribute? Sounds an awful lot like a survivorship bias.

                                          1. 1

                                            Even Apple is able to interact with the open source community better than this

                                          2. 6

                                            Becoming a contributor […] Step 0: Decide on a single Google Account you will be using to contribute to Go. Use that account for all the following steps and make sure that git is configured to create commits with that account’s e-mail address.


                                            I guess you’re not supposed create multiple accounts. But I do think your suggestion is clever.

                                            1. 3

                                              I guess you’re not supposed create multiple accounts. But I do think your suggestion is clever.

                                              The solution does not require multiple accounts, assuming each is deleted after use.

                                            2. 6

                                              I don’t think this really works. It also requires you to have a phone number to create a Google account in the first place. So people without phone numbers are effectively banned from contributing.

                                              1. 7

                                                Indeed, new Google accounts requiring a phone number is the worst aspect. Virtual phone numbers may not work.

                                                1. 3

                                                  Wait until they ask you for a scan of your ID when you send them a GDPR request.

                                                  1. 1

                                                    Which means contributing to Go (and presumably other google projects?) requires giving Google your phone number as well? In addition to the various “you give up your right to ever sue us if we break the law” contracts?

                                                    1. 1

                                                      If I remember correctly, it only needs the number to register, and you may use an anonymous burner SIM, if you can buy one in your country (more and more countries are banning this). It is a nuisance in any case.

                                                      1. 1

                                                        “It only needs a number to register”: I don’t want thinks it’s reasonable that I should have to give Google - the advertising and surveillance company - my phone number when I already have an email address, a couple of alias addresses, and now things like iCloud that provides automatic alias addresses whenever you need them, etc.

                                                        There is no justification for requiring your email be from a specific provider, unless you want to do more than simply email the account. I feel a little conspiracy nutter saying stuff like that, but we’re talking about a company that has been caught, and sued, and lost, for intentionally circumventing privacy measures. That has repeatedly attempted to tied a browser’s state their platform’s identity mechanisms, and then automatically share that information with sites, periodically “forgetting” a user had opted out of that, and relinking the browser’s identity.

                                                        I respect many of the engineers working at Google. But I do not, and will not ever trust them. They’ve demonstrated that they are not trustworthy on far too many occasions for it to not be a systemic problem.

                                                2. 3

                                                  They’ve almost certainly streamlined the account deletion process to the point where the handful of developers doing this would add almost no appreciable burden to Google.

                                                  1. 1

                                                    Let’s automate this process~

                                                  1. 11

                                                    Some other criteria this misses are:

                                                    • Privilege separation: If one is forced to use SSO (e.g. a Google account) to access service X, anyone who gets access to the account can access service X. Users should have the option of separating services into as many privilege separation contexts as they like. Forcing users to use SSO prevents this. (For an instructive example, when Microsoft bought GitHub I was waiting to see if they were going to try and force GitHub accounts to be merged with Microsoft accounts — because it’s the sort of thing big tech is obsessed with doing when they acquire a company (all accounts must be unified!) — and something which would absolutely have caused me to flee GitHub.)
                                                    • Non-tracking: Related to privilege separation, forced use of SSO means the identity to use service A can be correlated with the identity to use service B. This shouldn’t be a requirement, and users should be able to have arbitrarily many identities.

                                                    The PAKE idea suffers from the issue that it treats the server as untrusted, but the server has to be trusted to send the pake attribute in the first place. This is also the issue suffered by all browser-based crypto, and why all browser-based crypto schemes are ultimately futile.

                                                    You do mention having a different UI, but most users won’t notice the difference. I don’t think it would be possible to have browsers warn on absence of this attribute as it would impose requirements on existing websites as to how they store password hashes, etc. which may be infeasible to comply with. (e.g., what if authentication is done against an AD domain?)

                                                    1. 2

                                                      but the server has to be trusted to send the pake attribute in the first place.

                                                      Not necessarily. The assumption is that you are using a password manager. This password manager would record that the password was using a PAKE and refuse to auto-fill it into anywhere that the site can access. So even if you were MITMed there would be two scenarios:

                                                      1. The MITM sends a PAKE input field and can’t do anything with the password.
                                                      2. The MITM strips the pake attribute and the password manager refuses to input the password.

                                                      Of course the UX for this will be hard, but the browsers are already dealing with wrong certificates and other UX that has this level of scariness.

                                                      1. 3

                                                        The assumption is that you are using a password manager

                                                        But why not just have a authentication manager in combination with TLS client auth or http auth?

                                                        This would prevent from getting a password near the website front-end and with a good separation in the back-end also away from server implementation of the business logic.

                                                        The big problem with TLS client auth (and http auth) is the beautiful UI in most browsers. In order to have PAKE implemented secure against server takeover and XSS the browser must have a similar UI. Require the use of a password manager would remove the ability to easy type the password on a complete new device. Also the browser need to handle the private key for the session. To complete the authentication there must some protocol based on the private key. The browser must also protect this key and prevent to sign arbitrary data. Otherwise a XSS-attack could use the key to sign another authentication session.

                                                        So please keep the authentication as far away from the website as possible and make the UI for secure auth better. A better UI for TLS client auth would do this. I don’t know WebAuthn good enough to say, if it does this, but it looks like a good start.

                                                    1. 5

                                                      Macromedia Fireworks also did this. The native file format for Fireworks, a vector graphics editor, was in fact… PNG. You could use it as a normal PNG, but it had additional data attached to it that let you open it and edit it in Fireworks. It also had an option to export a normal, non-Fireworks PNG without this data attached.

                                                      1. 6

                                                        Is Lobsters a document or an app?

                                                        1. 4

                                                          Presentation wise I would lean more towards a set of documents.

                                                          1. 2

                                                            The article talks about client side and server side processing, but its logic seems very client focused, and uses a very lean client.

                                                            On the server, there’s always going to be some code that translates an incoming HTTP request to obtain data from a static file. It doesn’t make a huge amount of difference if that code is getting the data from a database, or a zip file. There’s going to be code running either way, the difference is the maturity of that code.

                                                            1. 5

                                                              It does make a difference because http is designed with full support for downloading files. With headers reserved for filename hints, and even for things like if it should be displayed in the brow as we or downloaded to the local filesystem. The first webservers did this and still to do this day. It’s fisrt class functionality and if you keep things limited to that, it is trivial to move the site. Heck, it is even browsable directly without http involved at all.

                                                              I agree with the author, but @carlmjohnson question is not to be disregarded so quickly. From the point you exposed a database view, you introduce expectation on the dynamic nature of the content. And from there, arises demand for interactivity. And at that point you are fiddling with the DOM in the client. Then you ask yourself… Ok… Does it really make sense to assemble html in the server and in the browser? What about the server having a well defined http API that spits whatever data I need? We have arrived to SPAs.

                                                              By all means, I’m all for sites like lobsters with full page form submissions, simple forms and buttons. But the pressure for shiny looking websites from the general public is just enormous.

                                                              1. 3

                                                                But the pressure for shiny looking websites from the general public is just enormous.

                                                                I wonder if that pressure comes from the the general public?

                                                                1. 4

                                                                  It comes from the boss who saw something shiny on a competitor’s site.

                                                                  1. 3

                                                                    I think it does. While there are many non tech savvy people that instinctively would prefer their tried and true software that works… I believe they are still the minority. Sexy screenshots trump everything. If you are in the industry as a worker, you get left behind if you don’t embrace it. Engineers that put together shiny things with flashy colors and lots of padding will be promoted.

                                                                    1. 1

                                                                      You can have “sexy” “documents”, is pretty sexy design wise, but it still and acts and looks like a “document”. Content is niche but I don’t believe its design would be rejected by the general public.

                                                                      1. 2

                                                                        I don’t think so at all. People would reject it in an eye blink given an alternative with huge title text, lots of padding, large round avatars and all links looking like buttons with large rounded corners and flat design.

                                                                        Look at how discord completely took over all existing forum software. What other reason are there besides flashy looks?

                                                                        1. 1

                                                                          Did Discord take over all forum software? I recall the old web forum model becoming unpopular well before Discord became a thing; it seems like Facebook replaced it as much as anything. Since Discord is a chat program, it doesn’t seem to me to be comparing like with like.

                                                                          As for why these proprietary platforms won, I see there as being two reasons. The first is that these platforms realised they could use graph data (or in Discord’s case, multiple “servers”) to create a platform which scales to an infinitely large number of people and infinitely large number of communities, enabling a network effect which leads to a network effect monopoly. In short on Facebook you’re “bubbled” according to your position in the graph (the people you’ve friended). Compare this with a web forum in which everyone sees the same thing (and in which new subforums can only be created by administrators). This model naturally scales only so far and a traditional forum will always have some specific subject of focus for this reason. Moreover, if you were involved in web forums, you might recall that smaller forums (in which everyone knew each other) had a very different feeling to larger ones; and as smaller ones grew to be larger ones, their feeling changed in this way. By using graph data the modern social network can allow one to have a more “local” community while also being able to communicate with a much larger global network of people. Of course, this requires people to provide this graph data to them (which they do by adding people); the value of this graph data to commercial and state surveillance interests is a very convenient coincidental benefit to these platforms.

                                                                          A second likely reason might easily be “dopamine engineering”. That’s not quite the same thing as “people want flashy UI”.

                                                                          1. 2

                                                                            I meant discourse. Sorry.

                                                                            It is essentially the same functionality of phpbb and the like, with a flashier design.

                                                                            1. 4

                                                                              I’d argue Discourse is a lot less flashy than phpBB; phpBB style forums have a lot of extraneous chrome (unless it’s a buy/sell forum, why do I care about the poster’s location?) that Discourse ditches in favour of content and widgets focused on navigating content. (Of course, Discourse isn’t the first; it feels like a spiritual successor to Vanilla for me.)

                                                                              1. 1

                                                                       doesn’t like an app to me.

                                                                                Edit: But it is.

                                                                                Could very well be a progressively enhanced SSR web site. Design would be mostly the same.

                                                                    2. 1

                                                                      I think @calmjohnson’s question is interesting because is a document, it’s just not the same kind of document as a static web page and that serves to highlight the underlying problem: web browsers have evolved from a mechanism for displaying a document to being a framework for providing document viewers. This isn’t a new development. Netscape 2.0 was the first web browser to support a mechanism for providing custom viewers for other kinds of documents (Mosaic / Netscape 1 provided a mechanism for opening other kinds of document in a different application).

                                                                      Perhaps the real questions that need asking are:

                                                                      • To what degree is this document different from a static HTML page?
                                                                      • What is the smallest possible viewer for a document of this kind?
                                                                1. 4

                                                                  I worked with Saar and Nico on this, happy to answer any CHERI-related questions.

                                                                  1. 1

                                                                    IBM has been using ECC syndrome to sneak in a memory tag bit for each 16 bytes of RAM for decades (see my article here) with no overhead. Have you given any thought to using ECC in this way?

                                                                    Outstanding work by the way.

                                                                    1. 2

                                                                      We have considered it. There are three downsides:

                                                                      • There aren’t enough ECC bits spare unless you want to compromise ECC integrity (one vendor tried this recently and it was a very bad idea).
                                                                      • Many other things also want to use the non-existent spare ECC bits.
                                                                      • The tags have some very useful locality properties that are useful in a high-end pipeline design.

                                                                      The last is my main reason for not liking that approach. With the temporal safety work, for example, it’s useful to be able to quickly skim past cache lines that don’t contain tags. You can do this if the lines are not in cache yet and you have a hierarchical tag cache design that stores the tags off to one side and can pull in an entire page (or at least half a page) in a single DRAM read. You can then prefetch only the lines that have capabilities. Similarly, in the Morello mode that we use for a read-pointer barrier in concurrent revocation, if the tag cache can quickly reply with ‘no tag here’ even while waiting for the data from DRAM then you can potentially move a load further along the pipeline (you don’t know what the data is, but you know it won’t trap [unless you have ECC with precise traps turned on]).

                                                                      ECC bits are also not free: you’re still consuming die area and power for them. We’ve found that roughly 80% of pages in a pure-capability system have no tag bits. That’s 32 bytes of ECC bits that you’d need to power, even for pages that aren’t using tags. With a hierarchical tag cache, you can avoid allocating tag storage space entirely for pages (or some other granule) that don’t store capabilities (or MTE colours, or anything else that wants to use physically indexed metadata).

                                                                      1. 1

                                                                        Very interesting.

                                                                        As an aside, IBM’s POWER9 user manual states that their memory controller, which uses standard ECC DIMMs, does “64-byte memory ECC” and supports “correction of up to one symbol in a known location plus up to two unknown symbol errors” (page 186 of this). Reading between the lines, I interpret this (“in a known location”) as a reference to erasure coding as opposed to error correction coding. The idea seems to be that from an information theory perspective, recovering a bit that you know you don’t know the value of (i.e. a tag bit) is less of an ask than correcting bits when you don’t know which bit might have been corrupted. Though my understanding of information theory here is nonexistent and I could be wrong.

                                                                        I can’t really see IBM compromising on RAS since it’s a specific emphasis of their platform, so it seems like they have some way to do it without compromising what ECC offers. I could see it involving larger read sizes though.

                                                                        What you write about tag scanning is very interesting though. The idea of being able to grab a whole page’s worth of tag bits for “GC” purposes certainly sounds like a worthwhile tradeoff — interesting stuff.

                                                                        1. 1

                                                                          The ECC scheme is very tightly coupled with the memory. Memory ECC schemes are biased towards the failure modes that they expect. In the simple case, bit flips from charged to discharged are more likely than the opposite but now that memory cells are so small there’s a lot more subtlety in the specifics of individual fabrication techniques. It quite possible to design an ECC scheme that is incredibly robust in the presence of random errors and happens to do incredibly badly in the specific case of one vendor’s memory technology, whose most common failure mode hits the weakest point in the ECC scheme’s space.

                                                                          I don’t know anything specifically about IBM, but given their mainframe background, I suspect that they tightly couple their memory controller design to a specific memory technology for any given system generation and so can bias their ECC scheme aggressively. I also wouldn’t be surprised if ECC at the 64-byte granularity is just the first tier in their memory integrity scheme. I know that on some systems they do RAID-5-like striping for memory, they may also keep some coarser-grained error correction metadata that they can hit on a slow path if ECC reports uncorrectable errors.

                                                                          Note that revocation isn’t quite the same as GC, it’s the logical dual. GC guarantees that deallocation doesn’t happen until all pointers have gone away. Revocation ensures that all pointers have gone away as a result of deallocation. We can do this accurately with CHERI because the tag bit lets us accurately identify pointers.

                                                                  1. 1

                                                                    Name seems very bait.

                                                                    Article doesn’t actually dive into K-Line’s history.

                                                                    1. 2

                                                                      What information are you looking for?

                                                                      1. 1

                                                                        Specifically, “How the K-line got its name”.

                                                                        The title matters.

                                                                        1. 7

                                                                          Except I did explain this.

                                                                          In short, banning someone from a server was facilitated by adding a K: line to the configuration file; the K stands for “kill”.

                                                                          The “kill line” terminology could conceivably originate from Usenet’s “kill file” terminology.

                                                                          1. 1

                                                                            Sure, but that’s almost shorter than the title, making the title bait.

                                                                            1. 3

                                                                              What would have been a better title, in your opinion?

                                                                              1. 1

                                                                                Just “History of IRC daemon configuration” -i.e. the same but without the bait- would have been alright.

                                                                                1. 4

                                                                                  but the title isn’t bait, it says how the k-line got its name. were you expecting an entire article only about the k-line specifically?

                                                                                  the title is also a reference to a class of titles that go “how the x got its y”. I think this comes from a bunch of children’s stories by Kipling but I’m not sure, it likely goes back further than that.

                                                                                  1. 1

                                                                                    were you expecting an entire article only about the k-line specifically?

                                                                                    I was expecting a focus on that which the article lacks.

                                                                                    the title is also a reference to a class of titles that go “how the x got its y”. I think this comes from a bunch of children’s stories by Kipling but I’m not sure, it likely goes back further than that.

                                                                                    I see.

                                                                      2. 1

                                                                        I disagree, I learned a lot.

                                                                        1. 1

                                                                          I learned a lot, too. But I didn’t learn how the K-line got its name.

                                                                      1. 9

                                                                        Asking before a website could set a cookie is actually how browsers from the 90s worked. Lynx still works like that by default.

                                                                        The problem with asking the browser is that … every website will just ask this. Even for something as pointless and intrusive as notifications every damn fucking website will ask you to send those horrible things. I have the notification permissions set to just “always deny” in Firefox.

                                                                        And if every website (including Lobsters, for example) would ask for cookie permissions people will just click “yes”. I would just click “yes”; life is short, I have better things to do than review 200 cookies every day. Besides, there are many more tracking techniques than just “cookies”, and the focus on just that is rather outdated.

                                                                        I’ve been trying to come up with a better alternative ever since the EPrivacy directive was introduced, and thus far I haven’t really managed to think of something better. I think the GDPR is a step in the right direction as it focuses less on “information stored in the browser” and more on “identifiable information”.

                                                                        Enforcement is an issue, but this is a fixable issue.

                                                                        1. 9

                                                                          Asking before a website could set a cookie is actually how browsers from the 90s worked.

                                                                          But that’s not what the law demands. Lobsters has no cookie popup. Neither does GitHub. Even though both sites use cookies.

                                                                          And it’s not because either of them are flouting the law, but because they’re not using the cookies for tracking. The browser can’t possibly know if a cookie is used for tracking, or for authentication, or even potentially for both. That’s one thing that makes legal solutions different from technical ones; the police have permission to check what the server side is doing, while your browser does not.

                                                                          1. 7

                                                                            I get your point here, but can we please not further spread the myth that “the police” go about enforcing laws like this? A better phrase may be “the courts” or more simply “the state”

                                                                            1. 1

                                                                              Well, sure; but the article was talking about asking for permission to set any cookie, as I understood it anyway. I’m not sure it’s realistic to ask notifications only for “bad” cookies, that will only work if it’s enforced, and if the (current) law is enforced by the regulatory bodies then this entire proposal is a bit of a moot point as regular “cookie popups” will work pretty much identical.

                                                                            2. 2

                                                                     is certainly a step in the right direction!

                                                                              1. 1

                                                                                A saner default would just be to limit cookies to session duration and auto-delete them when all tabs from that origin are closed. I have the Firefox extension “Cookie AutoDelete” set to do this. If you visit a website for 30 seconds, you get cookies for 30 seconds.

                                                                                The EU cookie law was insane from the beginning because browsers give people the power to control this in the first place. It would have made sense for something like, for example, facial recognition in a shopping mall, because that’s not something you have the power to prevent. It treats “setting cookies” as though it’s something done that bypasses browser controls, when literally no cookie can be set without the browser agreeing to it. The article above even suggests something resembling a browser permission request, but this misses the point that this should always have (and always has) been the role of the browser, and not some website-implemented website-specific UI.

                                                                                1. 5

                                                                                  Most users don’t want their login and settings cookies to be deleted when they close a window; they just never want to have Google Analytics enabled, regardless of whether they keep their session open or not.

                                                                                  1. 3

                                                                                    I use Cookie AutoDelete as well, but I don’t think it’s really an option “for the masses”, at least not with the current implementation/UI. An improved version with a friendlier non-technical UI could perhaps be an option though.

                                                                                    But this still won’t prevent other types of fingerprinting/tracking, so it’s a very limited solution anyway. The more prevalent cookie blocking becomes, the more incentive there is to circumvent it and use other methods. This is why I don’t think these kind of technical means are really the road forward, unless all fingerprinting/tracking becomes impossible/hard, and that’s a lot easier said than done because a lot of these things rely on pretty essential features.

                                                                                1. 1

                                                                                  This is a nice approach. Another good approach is to patch the binary to load a DLL and then have that DLL overwrite function pointers or arbitrary bytes in the parent binary at runtime. Compared to just editing the binary directly, this has several benefits: it makes it easy to reimplement functions in C; you can document your assembly changes much better; patches by different people can be composed fairly easily.

                                                                                  Compared to your approach: this can be legally better (you can distribute a patcher and a DLL rather than a full binary); you don’t need to do the fixing up of the disassembly.

                                                                                  The “thinker” mod for Sid Meier’s Alpha Centauri is a good implementation of this.

                                                                                  There’s also the PRACX and OpenSMACX projects doing similar things.

                                                                                  1. 2

                                                                                    Indeed — in this case I mainly just wanted to be able to understand and instrument the binary better for RE purposes, but if I were planning on distributing some kind of augmentation then this is the way to go. I’ve used runtime hooking-based enhancements for certain games before, and it’s sometimes amazing the extent to which things can be enhanced. Entire ecosystems have upon occasion emerged from such tools.

                                                                                    A typical approach would be to have a “launcher” which spawns the original with CreateProcess using the SUSPENDED flag, then uses Read/WriteProcessMemory (and/or CreateRemoteThread) to inject some kind of shellcode — which, as you say, might choose to load the rest of itself by simply calling LoadLibrary. Nowadays, libraries like EasyHook make this sort of thing easier than ever.

                                                                                  1. 12

                                                                                    I think a lot of people will miss the point of this, but don’t let it get to you. I miss websites like this, and value them — and the people who would take the time to craft them. A web of small communities, each unique, now gone, replaced with baleful “social networks”. The more people building ‘indieweb’, the better.

                                                                                    1. 2

                                                                                      Totally agreed. At the same time, a lot of people actually might get it, so it’s good it was posted here.

                                                                                      The (rather shitty) name aside, it feels sort of like a web BBS.

                                                                                      1. 2

                                                                                        Or like usenet :)

                                                                                    1. 7

                                                                                      For people who are lost between all these data storage layers and connectors, from SCSI, to iSCSI, to SAS, to ATA, to PATA, SAT, ATAPI, how it links to the VFS, file systems, and the device mapping, etc. I’ve written an article, a while back, summarizing simply the link between all of these layers. It might not be as deep and precise as the link posted here but I think it does a good job giving a simple overview.

                                                                                      1. 1

                                                                                        Added a link. Cheers.

                                                                                      1. 2

                                                                                        Only 15 cores, huge SMT — I guess they’re going all in on extracting maximum performance from each core, but that’s the opposite direction of everyone else.. (Ampere is going to have a 128-core next year!)

                                                                                        1. 6

                                                                                          POWER9 was also sold in SMT8 configuration with half the number of cores. It’s just a trick in which two cores are fused together for software licencing reasons; some proprietary enterprise software is licenced per “core”. SMT4 or SMT8 is selected by different fusing at chip packaging time; it’s not a different mask.

                                                                                          If/when POWER10 is eventually shipped by Raptor, it’ll most likely be in an SMT4 fusing; the number of threads will be the same, with double the number of cores and half the number of threads per core, just like POWER9. (Compare Raptor’s POWER9 offerings with the SMT8 offered by most of the POWER9 servers on IBM’s website. All powered by the same mask.)

                                                                                        1. 2

                                                                                          I had a Noppoo Choc mini with nkro, but the implementation was buggy and I’d get double letters in macos (unusable) and occasional double letters in Linux. I used a blue cube adapter to force it into the boot protocol.

                                                                                          Also, isn’t it also a limitation on how you wire your keyboard?

                                                                                          1. 2

                                                                                            I had a Noppoo Choc mini with nkro, but the implementation was buggy and I’d get double letters in macos (unusable) and occasional double letters in Linux. I used a blue cube adapter to force it into the boot protocol.

                                                                                            Unfortunately, buggy firmware in USB devices is ridiculously common.

                                                                                            HID stacks in OSes/windowing systems also don’t necessarily treat edge cases or rarely used report descriptor patterns equally, so you can end up with macOS, Linux/X11, and Windows doing slightly different things.

                                                                                            It’s likely your issue could have been worked around software side too, I assume it worked “correctly” in Windows? I’m not aware of a generic HID driver for macOS which lets you arbitrarily rewrite report descriptors and reports into a format that WindowServer/Core Graphics deals with as intended. I’m guessing there might be some kind of built-in system for this in Linux or Xorg though.

                                                                                            Also, isn’t it also a limitation on how you wire your keyboard?

                                                                                            Yes, definitely, though that’s not as simple as supporting a hard limit of N simultaneous key presses, but rather that certain combinations of key presses become ambiguous, depending on which keys are wired to the same matrix rows and columns.

                                                                                            1. 2

                                                                                              I hear some old USB NKRO keyboards used ridiculous hacks like enumerating as multiple keyboards behind a hub, with the first keyboard reporting the first six scancodes, the second reporting the second, etc., or something. Of course, this is a completely ridiculous and unnecessary hack which implies that the people designing the keyboard don’t understand HID (or that the HID stacks of major OSes were too buggy at the time to work properly, perhaps?)

                                                                                              As for keyboard wiring, that’s a separate matter. My post discusses the limitations of the USB protocol. What the keyboard microcontroller does to ascertain which keys are pressed is entirely up to it. In practice, to save cost keyboards use a key matrix, which creates key rollover limitations. More expensive NKRO keyboards tend to still use key matrices, as I understand it, but add some diodes to the matrix which facilitates NKRO if and only if the assumption that only one key will change between key scans is not violated (a fair assumption if the scan rate is high enough, due to the infeasibility of pressing two keys at exactly the same time.)

                                                                                              FWIW, I also seem to recall that it’s common for modern “NKRO” keyboards to actually only be 10-key rollover, on the premise that humans only have 10 fingers (feels like dubious marketing to me.) I’m unsure as to whether this is to do with the key matrix, or whether they just decided to use a 10-element array as their reporting format rather than a bitfield.

                                                                                              However, nothing stops you from making a keyboard which, for example, wires every key individually up to a microcontroller with hundreds of pins (and thus has the truest possible NKRO). It would simply be prohibitively expensive to do so, less because of the MCU, more because of the PCB layers it would require; I worked this out some time ago and suspect it would take about an 8-layer PCB.

                                                                                              The Model F keyboard is known for supporting NKRO as an inherent benefit of its capacitative sensing, unlike its successor the Model M. Someone made an open hardware controller for existing Model F keyboards, enabling them to be retrofitted with USB, with full NKRO support.

                                                                                              1. 1

                                                                                                Can you explain why a hundred traces would require multiple PCB layers? In my mind, the MCU goes in the middle, with traces spidering out to each of the keys, and a ground belt surrounding the board. A second layer would be used to get the data and power into the MCU.

                                                                                                1. 1

                                                                                                  Maaaaaybe this would be feasible with a large QFP/QFN package? The chip I was looking at was only available as BGA with the necessary pin count; the escape routing seemed infeasible with a low number of layers, and the manufacturer recommended 6-8, IIRC.

                                                                                                  1. 1

                                                                                                    Oh yeah, pin arrays are dark magic as far as I’m concerned.

                                                                                            1. 7

                                                                                              LCC in the list was famously used by Quake 3 to generate bytecode for a scripting VM. It was chosen because it was easily retargetable.

                                                                                              Some more FOSS compilers which are missing from this list:

                                                                                              • The Plan 9 compilers (which famously used to be used by Go as well), 8c, 9c, etc.
                                                                                              • romcc, a C compiler written specifically for the Coreboot project which generates code which doesn’t require RAM and uses only CPU registers. IIRC it’s no longer used, though, probably since all modern CPUs support cache-as-RAM.
                                                                                              • More assorted random compilers: cproc, 8cc, andrewchambers/c, lacc, scc
                                                                                              1. 2

                                                                                                Ads have been an important source of revenue for the publishing industry since forever; well before the internet. User tracking for ads on the internet is indeed problematic, but simply going “literally every single ad is bad” is not helping solve any problems.

                                                                                                Either way, the entire article is little more than a “literally every single ad is bad” rant, so I just flagged it as off-topic.

                                                                                                1. 24

                                                                                                  I prefer this one:

                                                                                                  What I find remarkable is the way both sides of this debate seem to simply assume the large-scale capture and exploitation of human attention to be ethical and/or inevitable in the first place.

                                                                                                  1. 11

                                                                                                    Could you explain further how this is off-topic? It would be on-topic on HN, and I’d presume that it’s on-topic here too.

                                                                                                    I feel that your argument is depressingly corporatist. “Toxic resource X has been an important source of revenue for the X industry since forever; well before modern mass media. People suffering from exposure to X is indeed problematic, but simply going ‘literally every single application of X is bad’ is not helping solve any problems,” right? This could be applied just as well to:

                                                                                                    • Unpasteurized fruits and the food preparation industry
                                                                                                    • Amazon packages and the logistics industry
                                                                                                    • Conflict minerals and the mining industry
                                                                                                    • Slavery and the cotton, pineapple, and sugar industries
                                                                                                    • Tetraethyl lead and the oil industry
                                                                                                    • Chloroflurocarbons and the aerosol industry

                                                                                                    Maybe it is unthinkable for you to imagine that all modern advertising and marketing techniques are psychologically damaging, but not everybody agrees with you. Famously, over a decade ago, São Paulo banned billboards and other outdoor advertisements; they did this in part because they believed that it would improve the health of people.

                                                                                                    1. 10

                                                                                                      I agree with your points, but the issue with this kind of “hot take” that the author presents is that it’s more of a rant/bragging piece than anything remotely rewarding attention. The author doesn’t like ads. So what? What do I as a reader get out of this post? There are no solutions presented. Only a few talking points being rehashed and a rant about Google at the bottom.

                                                                                                      I disagree that it’s off-topic, but I don’t find the post to be valuable, so I am leaning towards spam myself.

                                                                                                      1. 7

                                                                                                        Author here. I thought the solutions were apparent; using a browser which, in my experience, can actually block ads effectively, and, in rare cases where that fails (e.g. burnt-in sponsorship segments in videos), doing whatever possible to prevent the content of the ad reaching one’s senses (muting, averting eyes, etc.).

                                                                                                        1. 1

                                                                                                          Author said their solution was Firefox + Ublock Origin + NoScript. Said they don’t see ads. Also said first two were really good without NoScript.

                                                                                                        2. 4

                                                                                                          It would be on-topic on HN, and I’d presume that it’s on-topic here too.

                                                                                                          There’s a lot of stuff that’s on-topic on HN that’s off-topic here.

                                                                                                          That said, I see this particular submission as on topic.

                                                                                                          As to whether advertising is harmful, either to individual’s mental health, or to political entities, it’s extremely debatable. In the very least, lumping everything into “advertising”, instead of focussing on stuff like corporate surveillance and the concentration of media power in companies that rely on advertising revenue, is not constructive.

                                                                                                          Banning advertising in general would require very thorough reworking of the concepts of free speech, and of commerce. There’s no constituency for it, nor is there, as far as I can see, any ideological theory for it.

                                                                                                          1. 2

                                                                                                            I find your comparisons to things like slavery and conflict minerals distasteful and insulting. Sorry, but I have little interest to hold any kind of discourse on these terms.

                                                                                                            1. 2

                                                                                                              That’s too bad, then, because those are the examples I picked. I could have picked more nuanced cases, like the breeding of plutonium isotopes in enriched-uranium nuclear power plants, but I decided to go with examples that were unambiguously corporatist and harmful.

                                                                                                              For what it’s worth, I’m glad that you felt insulted; it helps me understand what is important to you. It sounds like advertising technology is important either to your salary or your mental health. I wonder whether you can muster the empathy to understand that the actions of big businesses around us are not necessarily healthy for us, and in fact might be harmful.

                                                                                                              On a meta-note, you seem more interested in explaining how to act than how you reason. Your first post told us that you are one of the half-dozen people who added off-topic flags, which isn’t relevant to your point. Your second post told us that you are insulted by my point and are not interested in refinement or improvement of your argument in the face of my point. As long as you are engrossed in emotional responses like this, and more interested in letting us know how to act online than how to prove claims and be convincing, then I agree: You seem to have little interest in discourse.

                                                                                                          2. 6

                                                                                                            Ads have been an important source of revenue for the publishing industry since forever;

                                                                                                            This is very, very true. To this very day, the most effective way to make any kind of money off of digital content creation comes from advertisements. The author talks about the fact that they’ve not seen a YouTube advert in years, and yet all of those years they’ve been consuming the content created by YouTube video makers and hosted by Google without contributing anything back. If it weren’t for advertisements, YouTube and the massive ecosystem of diverse content it hosts wouldn’t exist; the author is depending entirely on the people still viewing those ads to support their selfish consumption.

                                                                                                            YouTube Premium exists for this exact use case. For $10/month, you can choose to rid YouTube of advertisements completely; that subscription fee is then partially passed on to the content creators you watch. Similar options exist for many different sites. However, for the massive number of small or independent sites or content creators out there operating outside of the umbrella or a large corporation like Google, that’s not really an option.

                                                                                                            If you create some kind of website that provides entertainment, information, or utility to people, the only real ways to monetize it are:

                                                                                                            1. Put advertisements/sponsored content on it
                                                                                                            2. Charge people subscriptions
                                                                                                            3. Rely on donations
                                                                                                            4. Harvesting user data which is often used for - you guessed it - targeting advertisements

                                                                                                            There are a few exceptions to this for things like Google-scale companies providing things like Gmail for free in order to capture market share and funnel users into their ecosystem, but that doesn’t apply to the vast majority of independent or small-scale content creators out there.

                                                                                                            The scope or utility of a piece of web content has to be way higher to justify charging people to use it; any kind of monthly charge is going to turn away well over 90% of your users, probably closer to 99%. Freemium can be a good fit for some things, but it takes non-trivial overhead to engineer and set up that system, and that’s assuming that people even care enough to do it.

                                                                                                            When it comes down to it, the advertising industry is really one of the most direct methods of corporate patronage out there. These companies are convinced that they’re being provided incredible amounts of business value from showing their branding or products everywhere, but

                                                                                                            I think ads are an incredibly inefficient and overall undesirable thing, but they’re absolutely critical to the rich ecosystem of free content that the internet provides today. People have proposed alternatives like browser-integrated cryptocurrency microtransactions (which are dystopian enough in and of themselves from the right perspective, but that’s a different conversation), but the fact is that there really are very few other paths out there to sustainably provide something for free on the internet without advertising.

                                                                                                            Personally, I think that the advertising economy is going to collapse in the coming ~10-20 years. So many online advertising providers throw metrics at the advertisers that make it look like they’re capturing incredible amounts of value and seeing huge returns on their ad spend, but in reality they’re just paying to take up the search space they’d get for free organically[1]. Companies are going to start to realize that spending millions of dollars to show users ads for the same vacuum cleaner that they bought for two weeks after they bought it isn’t providing them any value at all. I don’t know what this will mean for the world of digital media, but I do know whatever does end up happening will require a fundamental shift in the way that


                                                                                                          1. 39

                                                                                                            I work for Cloudflare, so I have a bit more insight into how it operates (I’m speaking for myself, that’s not an official response).

                                                                                                            • Free customers at Cloudflare are a really cool hack. You are the product, but not in the Google/Facebook way you’d expect. The more Cloudflare caches, the more it helps ISPs save on costs of their outgoing traffic, and in return Cloudflare can negotiate better peering agreements. That’s a win-win, because Cloudflare gets cheaper bandwidth, and ISPs on other continents are very happy they don’t have to fetch everything from us-east-1.

                                                                                                              The free tier is also used for testing rollouts and customer acquisition. You an read about it in Cloudflare’s S-1:

                                                                                                            • There are customers who really want and pay good money for features like WAF and blocking of “bad” traffic. Sure it sounds dumb, but “just don’t have SQL injection vulnerabilities” doesn’t work for everyone. There are some customers who have thousands of sites, and are at risk of being pwned just because one of marketing teams might have set up a Wordpress microsite for a promotion 5 years ago and forgot about it. Cloudlfare has an entire team that monitors attacks happening in the wild, and keeps updating WAF in response, so you have much smaller chance of being hit by the CVE of the day.

                                                                                                            • Aggressiveness of bot blocking, e-mail filtering, etc. are controlled by users. Harassment of users with CAPTCHAs doesn’t help anyone. It’s just that classification of traffic is a very hard problem.

                                                                                                            • Cookies […] Since Cloudflare definitely has assets in the EU — it has to, it’s a CDN — it’s also pretty egregiously violating EU law here.

                                                                                                              If it was a pretty egregious violation then wouldn’t you think that some law enforcement would have happened?

                                                                                                            • The mysterious reason why U.S Govt allows Cloudflare to “violate copyright” (and so do all other governments in the world! — wow, Cloudflare is in bed with all of them!) is that users click “Agree” on Terms of Service.

                                                                                                            1. 36

                                                                                                              Free customers at Cloudflare are a really cool hack. You are the product, but not in the Google/Facebook way you’d expect. The more Cloudflare caches, the more it helps ISPs save on costs of their outgoing traffic, and in return Cloudflare can negotiate better peering agreements.

                                                                                                              Meaning, Cloudflare gets more power and more say in who gets to have a website. 8chan is still offline, two and a half months later, as a direct result of Cloudflare’s actions. (I understand many people are happy about that, though.)

                                                                                                              But, it’s just the world we live in now.

                                                                                                              I think power grabs aren’t a cool hack. I’ve seen too many of them go badly to be comfortable with immense centralization.

                                                                                                              1. 16

                                                                                                                I think fewer genocide fan sites is always better, and I regret Cloudflare drags its feet dropping them.

                                                                                                                1. 33

                                                                                                                  You’ll feel that way right up until they ban a site you like. And the distance between today and that day is getting smaller.

                                                                                                                  It’s not about 8chan. It’s about the fact that they can choose who gets to be a part of the internet. You know, that thing that we used to believe everyone should have a say in.

                                                                                                                  1. 14

                                                                                                                    To put things in perspective: even in the very recent past government censorship in the US and Europe was much much more intense than Cloudflare kicking two sites (StormFront and 8chan) off the internet for literal support of literal terrorism. We’re probably living in the most free era that has ever been known.

                                                                                                                    1. 9

                                                                                                                      We’re probably living in the most free era that has ever been known.

                                                                                                                      Unless you measure government and corporate surveillance. In that case we are certainly living in the most surveilled era that has ever been known.

                                                                                                                      1. 7

                                                                                                                        We’re probably living in the most free era that has ever been known

                                                                                                                        Obligatory disclaimer: if you’re fortunate enough to live in a liberal democracy.

                                                                                                                        But I agree with your other statements!

                                                                                                                        American “cultural imperialism” has many faces - the normalization of US norms of free speech to the world’s internet is one of them.

                                                                                                                        1. 7

                                                                                                                          We’re probably living in the most free era that has ever been known

                                                                                                                          Obligatory disclaimer: if you’re fortunate enough to live in a liberal democracy.

                                                                                                                          Yeah, obviously. I’m currently living in Indonesia and things are different here; I can’t go on Reddit for example as it’s all blocked :-/

                                                                                                                          Living abroad in general is one of the things that gives you some perspective by the way, to give a different example, I used to complain about the Dutch public transport system, but after having lived in several different countries I can report that the Dutch public transport is actually really good compared to almost every other country.

                                                                                                                          1. 3

                                                                                                                            Americans who value freedom should build technologies that prevent their own speech from being censored by foreigners (or other Americans) who value freedom of speech less than some other political goal. It’s no imperialism worth opposing if non-Americans also make use of those technologies to secure their own speech.

                                                                                                                        2. 6

                                                                                                                          You know, that thing that we used to believe everyone should have a say in.

                                                                                                                          I think most of who said that never believed that nazism could come back. At least I did, and now that it has come back, I’m reconsidering my position. The weaponizing of masses for digital terrorism was another thing I didn’t foresee at all, but that’s what we have now.

                                                                                                                          If your point was that it’s weird that single companies have to carry the responsibility to make these decisions, that I can agree with.

                                                                                                                            1. 22

                                                                                                                              I think you’re stretching that comic a bit there. If you mean it as “you have free speech but I don’t have to listen to you”, I agree with you. You’re not stopping anyone else from listening to him by ignoring him.

                                                                                                                              But being able to remove a platform in the blink of an eye is a very powerful tool. It should not fall in the wrong hands. As long as Cloudflare is upfront about what is acceptable and what not, and upholds those standards in a publicly verifyable way, I don’t see an issue, but the way 8chan was handled is less than ideal.

                                                                                                                              Today it was 8chan that suddenly was denied service, tomorrow it could be something that I care about.

                                                                                                                              1. 18

                                                                                                                                I am worried about decisions of platforms that capture audience and control attention of large numbers of people (YouTube, Twitter, Facebook), because when they drop someone, they disconnect them from their audience. When they promote someone, they amplify their voice.

                                                                                                                                With Cloudflare none of that happens. It doesn’t bring you an audience. You use your own domain, so when Cloudflare drops you, you can go elsewhere and reconnect with your audience. But if nobody else is willing to host 8chan, that’s the xkcd situation.

                                                                                                                                In either case, when a platform makes a wrong judgement that’s very unfortunate, but IMHO it should not be an excuse for not making any judgements at all.

                                                                                                                                1. 6

                                                                                                                                  If your service is Denial of Service prevention, and you can at your own discretion stop providing service to sites you don’t like (or even prevent certain demographics from reaching a certain site), you’re effectively saying that you protect from Denial of Service, except your own.

                                                                                                                                  Most of your customers are not actually in need of DoS-protection, but some are. For those, you suddenly denying them service is a huge blow. I have no sympathy for 8chan, let that be clear, but some day in the future a case might show up that is not so black and white, and do we trust Cloudflare to make the right call then? Remember they got it wrong with 8chan before - the site was not taken online as soon Cloudflare learned about it.

                                                                                                                                  And equally important (you’d almost forget about it with all this talk about 8chan), do we trust Cloudflare not to abuse their close-to-monopoly on web traffic?

                                                                                                                                  1. 4

                                                                                                                                    because when they drop someone, they disconnect them from their audience.

                                                                                                                                    If CF dropping 8chan didn’t disconnect them from their audience, what was the point in dropping them?

                                                                                                                                    But if nobody else is willing to host 8chan, that’s the xkcd situation.

                                                                                                                                    How do you feel about the Hollywood Blacklist?

                                                                                                                                    1. 5

                                                                                                                                      I think refusing to cooperate with those who you believe to be harmful/immoral/corrupting/otherwise unacceptable is a good non-violent method of suppressing such views and behaviors. It doesn’t mean I agree with motivations of all people who use this method.

                                                                                                                                      1. 7

                                                                                                                                        Operating a hosting service doesn’t mean you can somehow be apolitical. Saying that you will host anything is itself a political statement.

                                                                                                                                        Choosing to enable hate-speech is a political action. With 8chan, it appears that no one wanted that publicly associated with the site, and so it is offline. I’d say that’s a good thing. You can disagree. That’s politics.

                                                                                                                                        I disapprove of the Hollywood Blacklist and similar McCarthyist nonsense. Those people should not have been harassed because those people were not violent or dangerous. This is consistent with wanting hate sites (which do appear to encourage copycat attacks, radicalise others, etc) to be shut down.

                                                                                                                                        Ideally, the users would be identified and encouraged to take part in counselling and sensitivity training to try to stop them being such racists.

                                                                                                                                    2. 8

                                                                                                                                      Even if the same form of a rule (ban X from Y) can be used both for good (ban Nazis from Twitter) and for bad (ban women from public places) we aren’t obliged to throw the rule in all of its forms away. We can apply the rule in ways that reduce suffering, and refuse to apply it in ways that increase suffering.

                                                                                                                                      This is obvious. We don’t abandon wholesale the concept of laws and punitive justice (if you assault someone the state may confine you) even though it can be misapplied (if you commit adultery the state may execute you).

                                                                                                                                      1. 6

                                                                                                                                        Whoa, this is not about whether censorship is good, this is about wheter it’s a good idea to do it at the discretion of a single company. At a state level the lawmaker is supposed to be separate from the justice system. Cloudflare is responsible for a large chunck of the internet; do we want to trust them now and in the future not to abuse that responsibility at some point?

                                                                                                                                        1. 4

                                                                                                                                          They do not have a monopoly, so they’re not censoring. The other site CF blocked is hosted again, for example.

                                                                                                                                          Yes, it would be nice if this kind of thing were done democratically, and CF highlight that in their blog, but the occasional refusal of service to literal fascists is hardly the most compelling argument for democratic governance of the internet.

                                                                                                                                          1. 4

                                                                                                                                            Is your point that since you agree with their action this time, we don’t need oversight because next time you will also agree?

                                                                                                                                            Can I ask if you protested Cloudflare when they defended hosting 8chan?

                                                                                                                                            1. 3

                                                                                                                                              I didn’t protest, but if I had heard about it on here or reddit I might have expressed disapproval.

                                                                                                                                              My point is that it would be nice to have democratic oversight of this kind of thing, but it’s also not really that big a deal because there are competitors to use. If CF was a monopoly, this would be more of an issue and a democratic body should take action (regulate CF or break it up).

                                                                                                                                              Because CF is in a competitive market, the situation is more like this one: In the UK some hotels refused service to gay people and were then sued under anti-discrimination laws because sexual orientation is a protected characteristic. If a country passes anti-discrimination laws protecting hate-speech, then the administrators of 8chan could sue in that jurisdiction.

                                                                                                                                              Indeed, if CF refused to host Stonewall, then they could probably be sued in the UK on that basis. That’s the current democratic consensus and I’m mostly fine with it.

                                                                                                                                    3. 7

                                                                                                                                      No need to re-iterate, we understand what you’re saying: free speech is only for opinions you approve of. You’re just wrong, is all.

                                                                                                                                      1. 11

                                                                                                                                        We understand what you’re saying: free speech is only for opinions you approve of. You’re just wrong, is all.

                                                                                                                                        It is disingenuous in the extreme to handwave away white supremacy or Nazi ethno-nationalism as mere “opinions you don’t approve of”, or “political speech”, or whatever other weasel phrase you want to use. That the New England Patriots are a good football team is an opinion I don’t approve of. The efficacy of Austrian economic policy is political speech I don’t subscribe to. The notion that a society should be a white ethno-state is fundamentally different, different in kind, an antisocial cancer that deserves complete and contemptuous eradication.

                                                                                                                                        1. 2

                                                                                                                                          [ethno-state stuff]

                                                                                                                                          How do you feel about non-white ethno states?

                                                                                                                                      2. 4

                                                                                                                                        Hey, I have an idea. How about a central registry of naughty opinions? If you’re on the list, you’re not allowed to have a website or social media presence. It could be like a modern day sex offender registry: It’ll track when you say something disagreeable, and any time you pop up online it’ll automatically post a link to it for everyone to see.

                                                                                                                                        I’m a bit sad that this seems like a viable idea. Also sad that people seem to want this future.

                                                                                                                                        1. 8

                                                                                                                                          Hey, I have an idea. How about a central registry of naughty opinions? If you’re on the list, you’re not allowed to have a website or social media presence.

                                                                                                                                          It is disingenuous in the extreme to handwave away white supremacy or Nazi ethno-nationalism as mere “naughty opinions”, or whatever other weasel phrase you want to use. Disliking cilantro, or enjoying EDM, might be naughty opinions. The notion that a society should be a white ethno-state is fundamentally different, different in kind, an antisocial cancer that deserves complete and contemptuous eradication.

                                                                                                                                          1. 2

                                                                                                                                            China is pretty much this.

                                                                                                                                            1. 0

                                                                                                                                              We already have that, its called Twitter and its cancel culture.

                                                                                                                                    4. 10

                                                                                                                                      Aggressiveness of bot blocking, e-mail filtering, etc. are controlled by users.

                                                                                                                                      It’s the defaults that are terrible! Tons of fully static blogs have the stupid “bot protection” for GET requests which has no security purpose whatsoever. Because users do not bother to change defaults.

                                                                                                                                      1. 4

                                                                                                                                        That’s a fair point. I’ll ask if we can change the defaults.

                                                                                                                                        I suppose it’s tricky, because when we create an account, we don’t really know if it’s going to be used for a dumb static site. And there are some origins (e.g. Wordpress on low-end hosting) that can go down if they’re crawled less than gently, so they do need protection even for GET.

                                                                                                                                      2. 9

                                                                                                                                        Thanks for taking the time to respond to this.

                                                                                                                                        There are customers who really want and pay good money for features like WAF and blocking of “bad” traffic.

                                                                                                                                        It’s not news to me that WAFs are snake oil sold to enterprises who are determined to see security as a kind of product they can buy, or a box to be ticked. It remains a fundamentally broken practice.

                                                                                                                                        Aggressiveness of bot blocking, e-mail filtering, etc. are controlled by users.

                                                                                                                                        As far as I’m aware Cloudflare reserves the ability to disable all meddling to paid tiers, unless this has changed. And in any case most sites leave this stuff enabled, leading to the various issues I raise in the article. The fact that some of these sites have their own AJAX calls broken does not suggest to me that site operators are fully understanding the caveats of Cloudflare’s product.

                                                                                                                                        If it was a pretty egregious violation then wouldn’t you think that some law enforcement would have happened?

                                                                                                                                        Honestly, no. For a law as vague and open-ended as EU privacy law, there’s always going to be more violations than enforcement actions. GDPR for example is sufficiently pervasive in its implications I doubt enforcement action will be taken against even 1% of its violations. Enforcement is prioritized against the biggest or most publicly visible harms. Though of course, I’d be interested if Cloudflare has its own legal arguments with regards to this tracking cookie.

                                                                                                                                        The mysterious reason why U.S Govt allows Cloudflare to “violate copyright” (and so do all other governments in the world! — wow, Cloudflare is in bed with all of them!) is that users click “Agree” on Terms of Service.

                                                                                                                                        You’re misinterpreting my argument. Yes, of course Cloudflare can and does receive permission from a website owner to redistribute their content. However, this assumes that the website owner has permission to distribute everything on their website, which isn’t necessarily the case.

                                                                                                                                        The Pirate Bay is an instructive example because, although it doesn’t host anything illegal directly, its purpose is to engage in contributory copyright infringement by linking to infringing material. Under US law, it would be obliged to process 17 USC 512(c) takedown notices in exactly the same way that Google, a search engine, is obliged to process such takedown notices for mere links to infringing material in its search results.

                                                                                                                                        In order to be exempt from liability for contributory copyright infringement, Cloudflare needs to fall under one of the exemptions from liability provided for under 17 USC 512, presumably 512(b). However, they cannot because they modify the content they transmit. This suggests, unless I am mistaken, that Cloudflare’s activities do not fall under any 17 USC 512 exemption. This is no problem for Cloudflare’s redistribution of content which a website operator had permission to distribute and thus gave to Cloudflare, but it poses a big problem if Cloudflare provides service to any website which itself violates copyright law… which it does, namely TPB.

                                                                                                                                        1. 1

                                                                                                                                          GDPR for example is sufficiently pervasive in its implications I doubt enforcement action will be taken against even 1% of its violations.

                                                                                                                                          Have you tried raising your concerns with your local data protection agency?

                                                                                                                                          1. 1

                                                                                                                                            When filtering we try to observe MIME types, so AJAX calls shouldn’t break, unless sites incorrectly label their responses. File bugs with customer support, these get passed on to devs. We’re in the process of upgrading our HTML rewriter, so we may be able to fix many edge cases.

                                                                                                                                            Cloudflare has a ton of lawyers who review everything we do. I can’t even make a blog post without presenting evidence for all claims to our legal, so I’m pretty sure the main functionality of our main product has been carefully reviewed. Illegal stuff is taken down if Cloudflare is ordered to do so. There’s an entire overworked dept for dealing with law enforcement.

                                                                                                                                            IANAL, but the cookie is not tied to any PII, and its siloed to DoS protections. As a dev I don’t have access to it, so I can’t use it for other products (even though it’d be useful for things like smart H/2 push or RUM metrics).

                                                                                                                                            We don’t have infrastructure to do any major tracking. Almost everything is per request and distributed and stateless. Log aggregation is per zone (customer) for billing and performance metrics.

                                                                                                                                          2. 7

                                                                                                                                            Harassment of users with CAPTCHAs doesn’t help anyone

                                                                                                                                            Agreed. So when will it stop?

                                                                                                                                            Since Cloudflare definitely has assets in the EU — it has to, it’s a CDN — it’s also pretty egregiously violating EU law here.

                                                                                                                                            If it was a pretty egregious violation then wouldn’t you think that some law enforcement would have happened?

                                                                                                                                            It isn’t a violation because you haven’t been fined? By that logic I’ve never driven past the speed limit, because I’ve never received a speeding ticket.

                                                                                                                                            1. 3

                                                                                                                                              There’s ongoing work on improvement of bot detection accuracy, but it’s an endless cat and mouse game.

                                                                                                                                              Cloudflare has nothing against Tor, but when actual attackers use Tor, and legit users use Tor, and both do everything they can to make their traffic look the same, we have no way of telling them apart.

                                                                                                                                              IIRC Cloudflare proposed some solutions that were meant to preserve privacy while carrying a “I’m not a bot” proof, but unsurprisingly Tor users are not receptive to changing anything about their traffic, so that’s probably a stalemate.

                                                                                                                                              I’ve just checked the Tor bug tracker about it, and the thread ends with users linking to Hitler memes.

                                                                                                                                              1. 15

                                                                                                                                                This has already been adressed in the article. I quote:

                                                                                                                                                Cloudflare’s inexplicable inability to implement HTTP in a sane, transparent manner, despite this incapability being seemingly unshared by every other CDN service in existence, became even more ridiculous when Cloudflare reached out to the Tor project to request that they make changes to Tor to accommodate their own problematic practices.

                                                                                                                                                Or to say it another way: Allow GET requests from low-reputation IPs.

                                                                                                                                            2. 5

                                                                                                                                              What’s your take on the argument that the NSA must have compromised Cloudflare and is using it as a convenient tap to become a Global Active Adversary? (Because the NSA is many things, but it ain’t dumb.) I know you can’t speak to specific countermeasures you may or may not have in place against such things, but… this has always seemed like a really important point to address.

                                                                                                                                              I appreciate that Cloudflare has made some credible efforts at working with Tor, especially the Privacy Pass initiative (which is the first concrete step I’ve seen towards the blinded reputation system we really need.) But… there’s still a long, long way to go. I don’t know if you’ve tried using the web through Tor, but Cloudflare is becoming increasingly problematic. :-/

                                                                                                                                              1. 5

                                                                                                                                                Cloudfare already does monitoring of raw traffic for security (esp DDOS), availability, and competitive insights into improving their own business. If backdooring Cloudfare, NSA would use systems that already intercept and/or redirect lots of traffic using patterns or firewall rules (“targeting criteria”) substituting their own. The information will be sent to them either directly in a way Cloudfare normally sends external traffic or back to collection points such a national or regional HQ’s or backbones. They’ll likely be sent to some NSA controlled system that, AT&T-style, has an extra connection that sends traffic outside the building without Cloudfare’s systems seeing that. They might even use master-master systems in HA configuration with the redirected data said to be testing those systems. Even fail them over periodically when intel wasn’t needed. Many ways to do it.

                                                                                                                                                At most, there would be 1-3 executives/managers and a few specialists that need to know what’s actually going on. The equipment and systems would look like any others for the stated purpose. Their traffic patterns could look different if one looks closely at them but crypto could obscure it. Trusted systems that don’t do anything outside their bounds might also never get traffic inspection by a human. A subversion of a Cloudfare-scale organization would take a handful of people keeping the rest in the dark. NSA might also provide the specialists, too, since they’d be cleared for it. Just with fake resumes.

                                                                                                                                                And you should already assume it happened due to Core Secrets saying NSA asked FBI to “compel” U.S. companies to “SIGINT-enable” their systems. And, since it’s TS/SCI, lie to their employees and customers about that. It’s straight-up a felony with 15 years imprisonment for them to tell you the truth if they were coerced into one of those programs. However, the other leaks were clear that NSA paid tens of millions to companies with lots of reach. Around $100 mil each to big telecoms. It’s more likely that Cloudfare, a startup with a huge bill for physical assets, took a large pile of cash to rapidly grow the business faster than those just taking VC money. Also, they made the tradeoff knowing the alternative was being fined out of existence or the executives doing time. There’s few, actual choices if one lives in a police state like America. Liking it or not, I’d understand if a for-profit, small startup took the money instead of declaring war on the U.S. government.

                                                                                                                                                1. 2

                                                                                                                                                  I can’t prove a negative. We have our own hardware and people familiar with the entire hardware and software stack, so I think a non-targeted/high-volume attack would be detected quickly. There’s a healthy level of paranoia about security. There’s also an option of signing TLS sessions from a remote machine, so that we don’t even have a key to compromise:

                                                                                                                                                  1. 8

                                                                                                                                                    When you mention a “non-targeted, high-volume attack”, you’re referring to hypothetical processing and exfiltration of all or nearly all traffic metadata, right? (E.g. the NSA extracting all Tor traffic for analysis.) I agree that that’s unlikely, and that barring a goodly number of employees actually being in the pay of the NSA, it would be extremely difficult for it to remain undetected. I’d be more concerned about a sequence of targeted attacks on specific endusers.

                                                                                                                                                    I know you can’t prove a negative. I suppose I’m asking you to justify helping create a large MITM system, knowing that it will inevitably be a huge target for state-level adversaries, rather than working to design something without this danger to society.

                                                                                                                                                    (Keyless SSL is indeed cool, but it doesn’t change you being a MITM.)

                                                                                                                                                    1. 4

                                                                                                                                                      Cloudflare wants to be in the business of delivering data quickly, protecting sites from attacks, implementing cutting-edge protocols and performance optimizations. MITM isn’t a goal, and it would be fantastic if all these features could be delivered without liability of key management.

                                                                                                                                                      Cloudflare is a big target, because it grew big offering useful MITM. I don’t know what you expect Cloudflare to do about it? Drop customers? Shut down? Let proletariat seize the means of content distribution?

                                                                                                                                                      1. 3

                                                                                                                                                        Let proletariat seize the means of content distribution?

                                                                                                                                                        Yes. Entities as powerful as CloudFlare are not healthy for the internet.

                                                                                                                                                        The power could be spread thin administratively. You could become a non-profit foundation and govern your own code and infrastructure through consensus-driven mechanisms that the public participates in, a la the IETF.

                                                                                                                                                        The power could be spread thin technically. You could split up billing so that each datacenter bills customers individually and set up each of your datacenters to be an independent node that has zero trust in the others and is configured to discover and interact with any other datacenter that implements the same protocols. This would allow third parties to participate–assuming that it behaves as it should in the network. (I recommend requiring nodes to spit out their own source code on demand.) A federation of CDN providers. Indeed, marketplace of competition among CDN providers.

                                                                                                                                                        Sorry for the word salad! I’m sure what I suggest makes no sense–I don’t know how CDNs work or how your company is organized. But, I repeat my answer to your question: Yes!

                                                                                                                                                        Same goes for Google, Facebook, Comcast, Level3 Communications, etc. I would happily run a couple Google nodes in my basement if I could just apt-get install google-daemon and get paid for converting electricity into services. I can even offer very low latency to my neighbors!

                                                                                                                                                        1. 1

                                                                                                                                                          I can’t say I expect Cloudflare to be upfront about what their service really is, but I think they would have fewer customers if the customers understood what the service is and whether they really need it.

                                                                                                                                                          Most of your customers don’t need the “delivering data quickly”, “performance optimizations” (it would’ve been quick anyway), “protect sites from attacks” (if there’s nothing to attack on a static page) and “cutting-edge protocols”. Through very good marketing they make technical novices think that they need the service, and that they get a good deal by getting it for free.

                                                                                                                                                          Consider, not behind Cloudflare, more users than a lot of the sites behind Cloudflare free tier, and yet it’s not slow or regularly down due to attacks.

                                                                                                                                                          1. 0

                                                                                                                                                            Using cloudflare may be good for the environment, depending on how they’re set up. Networking is expensive and you do less of it if you hit a local CDN instead of us-east or whatever.