1. 2

    On the HN version, syrusakbary chimed in to add they were working on something similar with a focus on maintainability. They also got Ngnix running. Reposting here in case any wasm fans find it interesting.

    1. 3

      Their claims seem to be exaggerated. They don’t have fork() or sigaction() working, it’s not a very useful server without them. Wasmjit is also focused on maintainability, I’m not sure what they mean by that.

      1. 1

        Wasmjit: it promised something similar to our goals, but after digging and trying to contribute to it we realized that all the architecture-specific instructions were hardcoded into the runtime itself. This meant a very steep process to implement WebAssembly instructions, and a considerable effort when reaching other architectures (basically the same effort as creating an IR engine like LLVM by hand). And sadly, the library didn’t have either a single test.

        Seems pretty clear-cut to me, though I can’t comment on its veracity.

        1. 2

          I was referring to their claims on “creating the first WebAssembly runtime able to run Nginx” which is exaggerated since they haven’t implemented a whole slew of critical functionality.

      2. 3

        They also posted something today about it and a quick comparaison to the other project including wasmjit: https://medium.com/@syrusakbary/running-nginx-with-webassembly-6353c02c08ac

      1. 5

        I would argue the point that since Blink Edge was a closed-source blob supported on a small subset of users’ platform, the negative impact of losing it is much much less than say Servo. Could it be possible moving over to Chromium’s engine will make it easier for Microsoft to influence the web since it can push its changes upstream to the most widely used engine and being open-source, they are much easier to be picked up and reproduced by other engine as well?

        1. 1

          I think you mean Edge, not Blink :-)

          1. 1

            That’s right! Need more coffee.

        1. 27

          I think people talking about inspecting the source before installing dependencies are being unreasonable to some degree.

          1. The malicious code was present only in the minified version of the code. I suppose the red flag that tipped the reporter was the lack of history/popularity of the repository in question, but it doesn’t have to be like that
          2. It can be released to npm in a way that’s not evident to casually browsing the github repo
          3. There isn’t even any guarantee that the code on npm matches what’s on github at all

          Meaning the ways to be safe are:

          1. Hand-inspect the code in your node_modules directory (including — or especially— those that may be minified); or
          2. Don’t use npm at all.

          I don’t see these people (nor myself) doing either. From which it follows:

          Any company desiring to buy into the so-called “modern” front end development (be it for productivity, performance or hiring purposes) does so by making itself vulnerable to attacks such as this.

          I don’t know if that’s a reasonable price to pay to use, say, React, but it sure isn’t reasonable to me to pay that to use Node (versus, say, Golang, which can reasonably be used to build the same kinds of apps using little more than the standard library).

          1. 21

            The malicious code was present only in the minified version of the code. I suppose the red flag that tipped the reporter was the lack of history/popularity of the repository in question, but it doesn’t have to be like that

            One more reason for reproducible builds… minified JS should be treated like compiled code and automated mechanisms should check if it matches the unminified version…

            1. 6

              This, a thousand times this. I can’t comprehend the reasoning that goes into committing derived code into source control. It’s a pain to remember to update it every time you commit, it’s hard to verify the code matches the original source and just pollutes the history. Diffing is mostly undoable too.

              1. 3

                I think the reasoning is to avoid build dependency. For some time, it was a usual practice to include Autoconf-derived configure script in release artifacts, so that users can avoid installing Autoconf.

                1. 1

                  Yeah, that’s annoying too (and a lot of projects still do it even though it’s not really good practice), but at least configure scripts don’t tend to/need to change with every single code change like these minified files do.

                  1. 1

                    generated autoconf configure scripts are pretty easy to read, I can say there were times I preferred them over the m4 source.

              2. 11

                It would be really nice if the package repositories (npm/pypi/rubygems/etc) did something:

                • try to automatically detect obfuscated code
                • stop letting maintainers upload packages from their dev machines, make sure any compilation happens on a public CI environment from a known git tag (this would also encourage non-compiled packages, i.e. just direct snapshots of git tags)
                • have some popularity threshold for packages beyond which manual review from a trusted group of reviewers is required for each new release
                • (also why not require the git tags to be gpg signed for these popular packages)
                • maybe rethink the whole package handover thing, maybe only allowing “deprecation in favor of [a fork]” (i.e. requiring every update to be manual) is good
                1. 3

                  I wouldn’t even check the node_modules output either as package installation can execute arbitrary code (node-gyp, other node bindings producing code)

                  1. 4

                    I agree with you!

                    People seems to like to hit on npm, but I don’t see how the core issue is different than say Pypi, Cargo or Go (Other than the issues you raised). I personnaly take easy and simple dependancies management over C/C++ fragmented package management because most of my project are not security critical anyway or my threat model doesn’t include targeted code injection in my stack.

                    I find it annoying when people look at those issues and some fault is put on the maintainers. Maybe the issue is not that one of your application’s thousands of dependancies compromition, but the fact that your risk management for your wallet application relies on thousands of unvetted dependancies…

                    Meaning the ways to be safe are:

                    I guess a first start would be to gather a bunch of useful and common repositories and ensure they and all their dependancies are well vetted and signed by the maintainers for each release and prevent any new dependancies from being pulled in without proper review and ensuring those dependancies use the same process. Documenting and enforcing such process for a subset of widely used dependancies would allow to trust a few organization and avoid to code review any dependancies I pull in in my own project. I guess most distribution core repositories has similar process like Arch maintained packages vs AUR.

                    1. 8

                      Pypi absolutely has the the same potential issues, though in practice I think the dependency trees for popular projects are way smaller than what you get in the node ecosystem. So you’re much less likely to be hit by a transitive vulnerability. To me this is one of the advantages of a fairly comprehensive standard library, and a relatively small number (compared to node, at least) of popular, high quality third-party libraries that get a lot of eyeballs.

                      1. 11

                        On top of that, a lot of Python code is deployed to production by system engineers. Often it’s vetted, built, tested and baked in by distributions - and the same is true for other non-web languages.

                        javascript, on the other hand, is more often deployed by the upstream developer and thrown at web browsers straight away without any 3rd party review.

                        1. 3

                          Definitely! But that somehow happened to be this way. It would be nice to look at the social side as to why Python ended up this way while nothing prevented it from ending up like NPM. Maybe some key aspect of the tooling drive the trend one way or the other or it might be just the community (Python being much older and the tooling has seen a lot of changes over the years).

                          I would be looking forward to a someone doing a graph analysis of a few package repositories across languages and find some way to rate them and put some risk on packages. How many and how deep does their dependancies go? How many of them are maintained by external maintainer? Sounds like I found myself a new week-end project…

                          1. 12

                            Python has a decent class library. Good libraries that have general use migrate back into that class library, in some fashion or another. Thus, third party libraries don’t have to have long dependency chains to do anything.

                            What NPM forgot was that this was the fundamental idea that made package management useful. This stretches back to the early days of Java, at least, and I’m sure you can find other similar examples. By having a rich class library which already provides most of what you need, you’re simply going to layer on dependencies to adapt that framework to your specific business needs. Java, .NET, Ruby, Python- they all have that going for them. JavaScript simply does not. So half the Internet unwittingly depends on leftpad because a dependency of a dependency of a dependency needed it, and there wasn’t anything in the core library which could do it.

                            1. 1

                              Maybe some key aspect of the tooling drive the trend one way or the other or it might be just the community (Python being much older and the tooling has seen a lot of changes over the years).

                              I think this is a big part of it — Python’s tooling is generally less capable than the Node ecosystem’s.

                              To this day Pip doesn’t have a dependency resolver, so the result of installing a dependency tree with conflicts at the transitive dependency level isn’t an error, but an arbitrary version getting installed. You can only have a single version of a Python module installed, too, because they are global state. Contrast with how npm has historically (still does?) install multiple versions of a package, effectively vendoring each dependency’s dependency tree, transitively.

                              Additionally, publishing Python packages has long been messy and fraught. Today there is decent documentation but before that you were forced to rely on rumors, hearsay, and Stack Overflow. Putting anything nontrivial on PyPI (e.g., a C extension module) is asking for a long tail of support requests as it fails to install in odd environments.

                              I think the end result was a culture that values larger distributions to amortize packaging overhead. For example, the popular Django web framework long had a no-dependencies policy (if dependencies were required, they were vendored — e.g. simplejson before it entered the standard library).

                              Regardless of the reasons for it, I think that this is healthier than the Node culture of tiny dependencies with single contributors. More goes into distributing software than just coding and testing — documentation, support, evangelism, and legal legwork are all important — but tiny libraries have such limited scope that they’ll never grow a social ecosystem which can persist in the long term (of course, even Django has trouble with that).

                              1. 1

                                You can only have a single version of a Python module installed, too, because they are global state.

                                That’s actually a pretty good point I think. I have fought myself a few time against Pip due to conflicting versions. It does benefits library with fewer dependancies.

                          2. 1

                            While I’m not generally a fan of it, I think minimal version selection that’s planned for the future go package manager would make this attack spread much more slowly.

                        1. 41

                          Wow, that’s pretty terrible.

                          On the other hand, I can’t help but to feel sorry about Dominic, we all make mistakes, this public shaming is pretty violent.

                          I guess we should sometimes take some time off to read the license before using a library:

                          THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

                          (F)OSS is not a consumer good.

                          1. 11

                            I agree that shaming people is toxic and unproductive. No one wants to be shamed and no one is perfect.

                            But I see another dimension to the negative responses Dominic has received. Non-hierarchical, self-governing communities like open source software are organized by social norms. Social norms work through peer pressure - community members conform to the norms of the community not because they are compelled to by law but because it would cost them standing in the community not to. This isn’t inherently good. Some norms are toxic and self-policing via peer pressure can lead to shaming. What I see in some of the critical comments addressed to Dominic is an attempt to establish a clear social norm about what to do when you are ready to abandon a package. The norm is desirable because it increases the general level of trust. Even if the landscape is generally untrustworthy, you can have some confidence that people aren’t handing their packages off to strangers because it’s the norm not to do that. The desire for some norm here, whatever it is in the end, is reasonable.

                            Ending the discussion with “don’t worry about it Dominic, everyone makes mistakes, and anyways you’re not liable for it” signals to everyone that they’re not responsible for the consequences of what they do. In a strictly legal sense, that might be true. Even then, I’m skeptical that the warranty clause would cover negligence in the distribution of the software rather than the software itself. But in either case, don’t we want a community where people do feel responsible for the actions they take and are open to receiving feedback when an action they’ve taken has a bad result? This dialogue can occur without shaming, without targeting anyone personally, and can be part of the same give-and-take process that produces the software itself.

                            1. 6

                              Blaming people in any security issue is toxic, no matter what happens. In any organization with paid people where you should expect better, the most important rule of a post-mortem is to remain blameless. It doesn’t get anyone anywhere and doesn’t get remotely close to actual root cause. Instead of asking about why Dominic gave away a critical package, people should be asking why some random maintainer were able to give away a critical package.

                              Ending the discussion with “don’t worry about it Dominic, everyone makes mistakes, and anyways you’re not liable for it” signals to everyone that they’re not responsible for the consequences of what they do.

                              By putting blame on Dominic, people are not taking responsibilities. The main issue is that many core libraries in the JavaScript ecosystems still depends on external, single-file, non-core, likely unmaintained library. People who should take responsabilities are the ones who chose to add a weak single point of failure by depending on event-stream.

                              1. 2

                                It depends what you mean by blame. If you mean assigning moral responsibility, especially as a pretext for shaming them, then I agree it’s toxic. I think I was clear that I agree this shouldn’t happen. But if blame means asserting a causal relationship between Dominic’s actions and this result, it’s hard to argue that there isn’t such a relationship. The attack was only possible because Dominic transferred the package. This doesn’t mean he’s a bad person or that he should be “in trouble” or that anything negative should happen to him as a consequence. A healthy social norm would be to avoid transferring packages to un-credentialed strangers when you’re ready to abandon the package because we’ve seen this opens an attack vector. Then what’s happened here is instructive and everyone benefits from the experience. And yes, ideally these dilemmas are prohibited by the system. Until that is the case, it helps to have norms around the best way to act.

                                1. 1

                                  I understand you don’t condone the attacks and shaming going around. However I would argue that even if you agree that the blaming is toxic, that building some social norm around it is better than nothing, I believe that even hinting that it was somehow Dominic’s fault is a net negative.

                                  The attack was only possible because Dominic transferred the package.

                                  This is exactly what I’m condoning. By looking at individual and their action you scope the issue at that level. The attack was taking over a dependancy. It is possible to do so in so many way, especially for packages such as Dominic’s. This time it was a case of social engineering, next time it might as well be a case of credential hijacking, phishing or maintainer going rogue.

                                  A healthy social norm would be to avoid transferring packages to un-credentialed strangers when you’re ready to abandon the package because we’ve seen this opens an attack vector.

                                  I would say pushing this rethoric is actually unhealty and only lead people to rely on those social norm and use it as an excuse to disown their accountability. It would be much healthier to set expectation right and learn proper risk assessment about dependancies management.

                                  Then what’s happened here is instructive and everyone benefits from the experience. And yes, ideally these dilemmas are prohibited by the system. Until that is the case, it helps to have norms around the best way to act.

                                  The same issue have come up so many time in the past few years, especially in the NPM ecosystem, it should be well past the “learn from the experience” and I believe it’s time the relevant actors actually move toward a solution.

                            2. 17

                              I’ve done a similar thing before. After leaving the Elm community, I offered to transfer most of my repos over to the elm-community organisation. They accepted the most popular ones, but not elm-ast (and maybe one or two others). A few months later I received an e-mail from @wende asking if he could take over so I took a look at his profile and stuff he’s done in the past and happily gave him commit access thinking users would continue getting updates and improvements without any hassle. Now, @wende turns out to be a great guy and I’m pretty sure he hasn’t backdoored anyone using elm-ast, but I find it hilarious that people somehow think that maintainers should be responsible for vetting who they hand over control of their projects to or that they could even do a good job of it OR that it would even make any sort of a difference. Instead of trusting one random dude on the internet (me) you’re now trusting another.

                              Don’t implicitly trust random people on the internet and run their code. Vet the code you run and keep your dependency tree small.

                              1. 25

                                Vet the code you run

                                Or trust well-known, security-oriented distributions.

                                keep your dependency tree small

                                Yes, and stay away from environment, frameworks, languages that force dependency fragmentation on you.

                                1. 4

                                  Or trust well-known, security-oriented distributions.

                                  That too! :D

                                  1. 3

                                    and stay away from […] frameworks

                                    I wouldn’t say that as absolutely for the web. I suspect that things would likely go a lot more haywire if people started handling raw HTTP in Python or Ruby or what have you. There’s a lot of stuff going on under the hood such as content security policies, CSRF protection and the like. If you’re not actively, consciously aware of all of that, a web framework will probably still end up providing a net security benefit.

                                    1. 5

                                      Please don’t quote words without context:

                                      […] that force dependency fragmentation on you

                                      Frameworks and libraries with few dependencies and a good security track record are not the problem. (If anything, they are beneficial)

                                      1. 2

                                        I interpreted “Yes, and stay away from environment, frameworks, languages that force dependency fragmentation on you.” as (my misunderstandings in brackets) “Yes, and stay away from [(a) integrated development] environments, [(b)] frameworks, [(c)] languages that force dependency fragmentation on you.” with a and b being separate from the “that” in c.

                                        I apologize for the misunderstanding caused.

                                    2. 2

                                      Isn’t it the case that reputable, security-focused distributions acquire such status and the continuity thereof by performing extensive vetting of maintainers?

                                      The responsible alternative being abandoning the project and letting the community fork it if they want to.

                                      1. 1

                                        Or trust well-known, security-oriented distributions.

                                        Then how do You deal with things like that: “The reason the login form is delivered as web content is to increase development speed and agility” ?

                                        1. 2

                                          As a distribution? Open a bug upstream, offer a patch, and sometimes patch the packaged version.

                                          1. 1

                                            That’s a good idea in general but sometimes the bug is introduced downstream.

                                    3. 9

                                      Most proprietary software also comes with pretty much the same warranty disclaimer. For example, see section 7c of the macOS EULA:

                                      https://images.apple.com/legal/sla/docs/macosx107.pdf

                                      I mean, have we held accountable Apple or Google or Microsoft or Facebook in any substantial ways for their security flaws?

                                      1. 4

                                        In many other products accountability is enforced by law and it overrides any EULA. And that is tied to profit in the broad sense: sales or having access to valuable customer data & so on.

                                        Software companies got away with zero responsibility and this only encourages bad software.

                                        1. 1

                                          And how have we enforced that by law for those companies, regardless of what those EULAs have said? When macOS allowed anyone to log in as root, what were the legal consequences it faced?

                                          1. 3

                                            other products

                                            e.g. selling cars without safety belts, electrical appliances without grounding…

                                      2. 2

                                        It is a security disaster given how easy it is for js stuff to hijack cookies and sessions.

                                        1. 1

                                          It really isn’t if a well thought out CORS policy is defined.

                                      1. 12

                                        I find Safari’s “Reader View” to be acceptable most of the time, although it’s nice to see someone taking Medium back to what it was regardless.

                                        1. 17

                                          Firefox has the same feature which I use extensively without any issues.

                                          1. 4

                                            I find myself usually just sending long Medium pieces directly to Pocket, and then I’ll read them later on my phone during a spare moment.

                                            The Pocket reformatter gives up too easily in my experience but does an adequate job at Medium (only really breaks pull quotes).

                                            1. 4

                                              On mobile the reader mode is a blessing. Instead of accepting the cookie policy, closing the app bar at the top, closing the other modal windows and then reading, one click and it’s all gone, clear text, large font an no distractions.

                                            1. 1

                                              One of my current issue on top of limited and noisy Github notification systems is that I use Gitlab in addition to Github, doubling my amount of effort required to track all my stuff. Any plan to support more than Github in the future?

                                              1. 2

                                                I had a look to see if it was possible with the GitLab API about a year ago and there were some pretty big pieces missing, must revisit it soon to see if they’ve added anything similar to the notifications API

                                              1. 3

                                                I find Bug Bounty Programs quite an interesting process.

                                                Imagine having a bank/supermarket and telling people to test the security by attempting to rob it and report back.

                                                1. 4

                                                  Banks do actually do that—at least some of them. Sneakers’ portrayal is a bit overdramatic, but that concept is very real. But you have some very specific contracts in place when you do that stuff.

                                                  1. 1

                                                    That’s cool! I didn’t know that :D

                                                  2. 3

                                                    Although while on bug bounty you should not attempt to actually rob the bank, but maybe finds way to rob and report the plans that might actually work. If possible you should build your own bank from the plan of your target and try to rob your own banks before reporting back to the actual bank. Bug bounties program are not free pass to run actual exploit against a system.

                                                  1. 2

                                                    supported by the Mozilla WebRender rendering engine

                                                    So… electron.rs? ☹️

                                                    But, no javascript? 😀

                                                    I’m so conflicted.

                                                    1. 12

                                                      So… electron.rs? ☹️

                                                      Doesn’t seem so: https://hacks.mozilla.org/2017/10/the-whole-web-at-maximum-fps-how-webrender-gets-rid-of-jank/ & https://github.com/servo/webrender & https://github.com/servo/webrender/wiki

                                                      As I seem to understand, WebRender is nowhere close to be an Electron alternative. Seems to be an efficient and modern rendering engine for GUI and provide nothing related to JavaScript/Node/Web API.

                                                      So it looks like you can be free of conflict and enjoy this interesting API :) . I personally definitely keep an eye on it for my next pet project and find it refreshing to have an API for UI that look both usable and simple.

                                                      1. 4

                                                        If you’re a fan of alternative, Rust-native GUI’s, you might want to have a look at xi-win-ui (in the process of being renamed to “druid”). It’s currently Windows-only, because it uses Direct2D to draw, but I have plans to take it cross-platform, and use it for both xi and my music synthesizer project.

                                                        1. 1

                                                          Please, give us some screenshots! ;)

                                                          1. 1

                                                            Soon. I haven’t been putting any attention into visual polish so far because I’ve been focusing on the bones of the framework (just spent the day making dynamic mutation of the widget graph work). But I know that screenshot are that important first impression.

                                                            1. 1

                                                              Please do submit a top-level post on lobste.rs once you add the screenshots :)

                                                              1. 1

                                                                Will do. Might not be super soon, there’s more polishing I want to do.

                                                                1. 1

                                                                  Thanks! :) And sure, take your time :)

                                                      2. 6

                                                        If comparing with Chromium stack, Webrender is similar to Skia, and this is GUI toolkit on top of it, instead of on top of whole browser. BTW, there’s example of app that has whole (non-native) UI on top of Skia: Aseprite.

                                                        (AFAIK, Skia is something a la Windows GDI, immediate mode, and Webrender is scene graph-style lib, more “retained-mode”)

                                                        And seems that, despite there’s no components from real browser, Azul has DOM and CSS. So, Azul is something in the spirit of NeWS and Display Postscript, but more web-ish, instead of printer-ish?

                                                        1. 4

                                                          There is also discussion of making an xml format for specifying the dom, like html.

                                                          1. 1

                                                            It’s using Mozilla’s WebRender, so how about XUL?

                                                            1. 1

                                                              Considering that Mozilla is actively trying to get rid of XUL, doing anything new with it seems like a bad idea.

                                                              But also, if I understand what XUL is correctly it’s mostly a defined list of widgets over a generic XML interface, if I understand that proposal properly it’s to make the list of widgets completely user controllable (though there will no doubt be some default ones, including HTML like ones).

                                                        2. 1

                                                          WebRender is basically a GPU-powered rectangle compositor, with support for the kinds of settings / filters you can put on HTML elements. It’s nowhere near the bloated monstrosity that is electron.

                                                        1. 2

                                                          On one hand: I agree that DNS-over-HTTPS is a silly and convoluted solution.

                                                          On the other hand: DNS-over-TLS is a bad solution for the reason pointed out: it lives on its own port.

                                                          Question: Why do we need ports any more at all? It seems like if we didn’t have dedicated port numbers, but instead referred to resources by subdomain or subdirectory beneath the main hostname, then all traffic would be indiscriminate when secured by TLS.

                                                          1. 4

                                                            Could it have been possible for DNS-over-TLS to use 443 and make the server able to route DNS and HTTP request appropriately? I’m not very knowledgable of TLS. From what I understand its just a transport layer so a server could simply read the beginning of an incoming message and easily detect if it is an HTTP or DNS header?

                                                            1. 9

                                                              Yes, much like http2 works. It complicates the TLS connection because now it passes a hint about the service it wants, but that bridge is already crossed.

                                                            2. 4

                                                              IP addresses allow two arbitrary computers to exchange information [1], whereas ports allow to arbitrary programs (or processes) to exchange information. Also, it’s TCP and UDP that have ports. There are other protocols that ride on top of IP (not that anyone cares anymore).

                                                              [1] Well, in theory anyway, NAT breaks that to some degree.

                                                              1. 3

                                                                Ports are kinda central to packet routing, if my understanding is correct, as it has been deployed.

                                                                1. 5

                                                                  You need the concept of ports to route packets to the appropriate process, certainly. However, with DNS SRV records, you don’t need globally-agreed-upon port assignments (a la “HTTP goes to port 80”). You could assign arbitrary ports to services and direct clients accordingly with SRV.

                                                                  Support for this is very incomplete (e.g. browsers go to port 80/443 on the A/AAAA record for a domain rather than querying for SRVs), but the infrastructure is in place.

                                                                  1. 5

                                                                    On what port do I send the DNS query for the SRV record of my DNS server?

                                                                    1. 1

                                                                      Obviously, you look up an SRV record to determine which port DNS is served over. ;)

                                                                      I don’t know if anyone has thought about the bootstrapping problem. In theory, you could deal with it the same way you already bootstrap your DNS (DHCP or including the port with the IP address in static configurations), but I don’t know if this is actually possible.

                                                                    2. 2

                                                                      You need the concept of ports to route packets to the appropriate process

                                                                      Unless we assign an IP address to every web facing process.

                                                                  2. 1

                                                                    Problem: both solutions to private DNS queries have downsides related to the DNS protocol fundamentally having failed to envision a need for privacy

                                                                    Solution: radically overhaul the transport layer by replacing both TCP and UDP with something portless?

                                                                    The suggested cure is worse than the disease, in this case, in terms of sheer amount of work, and completely replaced hardware and software, it would require .

                                                                    1. 2

                                                                      I don’t think DNS is the right place to do privacy. If I’m on someone’s network, he can see what IP addresses I’m talking to. I can hide my DNS traffic, but he still gets to see the IP addresses I ultimately end up contacting.

                                                                      Trying to add privacy at the DNS stage is doing it at the wrong layer. If I want privacy, I need it at the IP layer.

                                                                      1. 4

                                                                        Assuming that looking up an A record and making a connection to that IP is the only thing DNS is used for.

                                                                        1. 3

                                                                          Think of CDN or “big websites” traffic. If you hit Google, Amazon, Cloudflare datacenters, nobody will be able to tell if you were reaching google.com, amazon.com, cloudflare.com or any of their costumers.

                                                                          Currently, this is leaking through SNI and DNS. DoH and Ecrypted SNI (ESNI) will improve on the status quo.

                                                                          1. 2

                                                                            And totally screws small sites. Or is the end game centralization of all web sites to a few hosts to “protect” the privacy of users?

                                                                            1. 2

                                                                              You can also self-host more than one domain on your site. In fact, I do too. It’s just a smaller set :-)

                                                                              1. 1

                                                                                End game would be VPNs or Tor.

                                                                              2. 2

                                                                                Is that really true? I though request/response metadata and timing analysis coud tell them who we were connecting to.

                                                                                1. 2

                                                                                  Depends who they are. I’m not going to do a full traffic dump, then try to correlate packet timings to discover whether you were loading gmail or facebook. But tcpdump port 53 is something I’ve actually done to discover what’s talking to where.

                                                                                  1. 1

                                                                                    True. maybe ESNI and DoH are only increasing the required work. Needs more research?

                                                                                    1. 1

                                                                                      Probably to be on safe side. Id run it by experts in correlation analyses on network traffic. They might already have something for it.

                                                                                  2. 2

                                                                                    nobody will be able to tell if you were reaching google.com, amazon.com, cloudflare.com or any of their costumers.

                                                                                    except for GOOGL, AMZN, et al. which will happily give away your data, without even flinching a bit.

                                                                                    1. 1

                                                                                      Yeah, depends on who you want to exclude from snooping on your traffic. The ISP, I assumed. The Googles and Amazons of the world have your data regardless of DNS/DoH.

                                                                                      I acknowledge that the circumstances are different in every country, but in the US, the major ISPs actually own ad networks and thus have a strong incentive not to ever encrypt DNS traffic.

                                                                                      1. 1

                                                                                        Yeah, depends on who you want to exclude from snooping on your traffic. The ISP, I assumed. The Googles and Amazons of the world have your data regardless of DNS/DoH.

                                                                                        so i’m supposed to just give them full access over the remaining part which isn’t served by them?

                                                                                        I acknowledge that the circumstances are different in every country, but in the US, the major ISPs actually own ad networks and thus have a strong incentive not to ever encrypt DNS traffic.

                                                                                        ISPs in the rest of the world aren’t better, but this still isn’t a reason to shoehorn DNS into HTTP.

                                                                                        1. 1

                                                                                          No, you’re misreading the first bit. You’re already giving iit to them, most likely, because of all those cloud customers. This makes their main web property indistinguishable from their clients, once SNI and DNS is encrypted.

                                                                                          No need to give more than before.

                                                                                          1. 1

                                                                                            You’re already giving iit to them, most likely, because of all those cloud customers.

                                                                                            this is a faux reason. i try to not use these things when possible. just because many things are there, it doesn’t mean that i have to use even more stuff of them, quite the opposite. this may be an inconvenience for me, but it is one i’m willing to take.

                                                                                            This makes their main web property indistinguishable from their clients, once SNI and DNS is encrypted.

                                                                                            indistinguishable for everybody on the way, but not for the big ad companies on whose systems things are. those are what i’m worried about.

                                                                                            1. 1

                                                                                              Hm I feel were going in circles here.

                                                                                              For those people who do use those services, there is an immediate gain in terms of hostname privacy (towards their ISP), once DoH and ESNI are shipped.

                                                                                              That’s all I’m saying. I’m not implying you do or you should.

                                                                                              1. 1

                                                                                                I’m not implying you do or you should.

                                                                                                no, but the implications of DoH are that i’ll end up using it, even if i don’t want to. it’ll be baked into the browsers, from there it’s only a small step to mandatory usage in systemd. regarding DoH in general: if you only have http, everything looks like a nail.

                                                                                2. 1

                                                                                  Alternative solution: don’t use DNS anymore.

                                                                                  Still lots of work since we need to ditch HTTP, HTTPS, FTP, and a host of other host-oriented protocols. But, for many of these, we’ve got well-supported alternatives already. The question of how to slightly improve a horribly-flawed system stuck in a set of political deadlocks becomes totally obviated.

                                                                                  1. 3

                                                                                    That’s the biggest change of all of them. The whole reason for using DoH is to have a small change, that improves things, and that doesn’t require literally replacing the entire web.

                                                                                    1. 1

                                                                                      Sure, but it’s sort of a waste of time to try to preserve the web. The biggest problem with DNS is that most of the time the actual hostname is totally irrelevant to our purposes & we only care about it because the application-layer protocol we’re using was poorly designed.

                                                                                      We’re going to need to fix that eventually so why not do it now, ipv6-style (i.e., make a parallel set of protocols that actually do the right thing & hang out there for a couple decades while the people using the old ones slowly run out of incremental fixes and start to notice the dead end they’re heading toward).

                                                                                      Myopic folks aren’t going to adopt large-scale improvments until they have no other choice, but as soon as they have no other choice they’re quick to adopt an existing solution. We’re better off already having made one they can adopt, because if we let them design their own it’s not going to last any longer than the last one.

                                                                                      DNS is baked into everything, despite being a clearly bad idea, because it was well-established. Well, IPFS is well-established now, so we can start using it for new projects and treating DNS as legacy for everything that’s not basically ssh.

                                                                                      1. 8

                                                                                        Well, IPFS is well-established now

                                                                                        No it’s not. Even by computer standards, IPFS is still a baby.

                                                                                        Skype was probably the most well-established P2P application in the world before they switched to being a reskinned MSN Messenger, and the Skype P2P network had disasters just like centralized services have, caused by netsplits, client bugs, and introduction point issues. BitTorrent probably holds the crown for most well-established P2P network now, and since it’s shared-nothing (the DHT isn’t, but BitTorrent can operate without it), has never had network-wide disasters. IPFS relies on the DHT, so it’s more like Skype than BitTorrent for reliability.

                                                                                        1. 0

                                                                                          It’s only ten years old, sure. I haven’t seen any reliability problems with it. Have you?

                                                                                          DHT tech, on top of being an actually appropriate solution to the problem of addressing static chunks of data (one that eliminates whole classes of attacks by its very nature), is more reliable now than DNS is. And, we have plenty of implementations and protocols to choose from.

                                                                                          Dropping IPFS or some other DHT into an existing system (like a browser) is straightforward. Opera did it years ago. Beaker does it now. There are pure-javascript implementations of DAT and IPFS for folks who can’t integrate it into their browser.

                                                                                          Skype isn’t a good comparison to a DHT, because Skype connects a pair of dynamic streams together. In other words, it can’t take advantage of redundant caching, so being P2P doesn’t really do it any favors aside from eliminating a single point of failure from the initial negotiation steps.

                                                                                          For transferring documents (or scripts, or blobs, or whatever), dynamicism is a bug – and one we eliminate with named data. Static data is the norm for most of what we use the web for, and should be the norm for substantially more of it. We can trivially eliminate hostnames from all asset fetches, replace database blobs with similar asset fetches, use one-time pads for keeping secret resources secret while allowing anyone to fetch them, & start to look at ways of making services portable between machines. (I hear DAT has a solution to this last one.) All of this is stuff any random front-end developer can figure out without much nudging, because the hard work has been done & open sourced already.

                                                                                          1. 4

                                                                                            IPFS is not ten years old. Its initial commit is five years ago, and that was the start of the paper, not the implementation.

                                                                                            1. 1

                                                                                              Huh. I could have sworn it was presented back in 2010. I must be getting it confused with another DHT system.

                                                                                        2. 7

                                                                                          Sure, but it’s sort of a waste of time to try to preserve the web.

                                                                                          This is letting Perfect be the enemy of Good thinking. We can incrementally improve (imperfectly, true) privacy now. Throwing out everything and starting over with a completely new set of protocols is a multi-decade effort before we start seeing the benefits. We should improve the situation we’re in, not ignore it while fantasizing about being in some other situation that won’t arrive for many years.

                                                                                          The biggest problem with DNS is that most of the time the actual hostname is totally irrelevant to our purposes & we only care about it because the application-layer protocol we’re using was poorly designed.

                                                                                          This hasn’t been true since Virtual Hosting and SNI became a thing. DNS contains (and leaks) information about exactly who we’re talking to that an IP address doesn’t.

                                                                                          1. 2

                                                                                            This is letting Perfect be the enemy of Good thinking. We can incrementally improve (imperfectly, true) privacy now.

                                                                                            We can also take advantage of low-hanging fruit that circumvent the tarpit that is incremental improvements to DNS now.

                                                                                            The perfect isn’t the enemy of the good here. This is merely a matter of what looks like a good idea on a six month timeline versus what looks like a good idea on a two year timeline. And, we can guarantee that folks will work on incremental improvements to DNS endlessly, even if we are not those folks.

                                                                                            Throwing out everything and starting over with a completely new set of protocols is a multi-decade effort before we start seeing the benefits.

                                                                                            Luckily, it’s an effort that started almost two decades ago, & we’re ready to reap the benefits of it.

                                                                                            DNS contains (and leaks) information about exactly who we’re talking to that an IP address doesn’t.

                                                                                            That’s not a reason to keep it.

                                                                                            Permanently associating any kind of host information (be it hostname or DNS name or IP) with a chunk of data & exposing that association to the user is a mistake. It’s an entanglement of distinct concerns based on false assumptions about DNS permanence, and it makes the whole domain name & data center rent-seeking complex inevitable. The fact that DNS is insecure is among its lesser problems; it should not have been relied upon in the first place.

                                                                                            The faster we make it irrelevant the better, & this can be done incrementally and from the application layer.

                                                                                          2. 2

                                                                                            But why would IPFS solve it?

                                                                                            Replacing every hostname with a hash doesn’t seem very user-friendly to me and last I checked, you can trivially sniff out what content someone is loading by inspecting the requested hashes from the network.

                                                                                            IPFS isn’t mature either, it’s not even a decade old and most middleboxes will start blocking it once people start using it for illegitimate purposes. There is no plan to circumvent blocking by middleboxes, not even after that stunt with putting wikipedia on IPFS.

                                                                                            1. 1

                                                                                              IPFS doesn’t replace hostnames with hashes.It uses hashes as host-agnostic document addresses.

                                                                                              Identifying hosts is not directly relevant to grabbing documents, and so baking hostnames into document addresses mixes two levels of abstractions, with undesirable side effects (like dependence upon DNS and server farms to provide absurd uptime guarantees).

                                                                                              IPFS is one example of distributed permanent addressing. There are a lot of implementations – most relying upon hashes, since hashes provide a convenient mechanism for producing practically-unique addresses without collusion, but some using other mechanisms.

                                                                                              The point is that once you have permanent addresses for static documents, all clients can become servers & you start getting situations where accidentally slashdotting a site is impossible because the more people try to access it the more redundancy there is in its hosting. You remove some of the hairiest problems with caching, because while you can flush things out of a cache, the copy in cache is never invalidated by changes, because the object at a particular permanent address is immutable.

                                                                                              Problems (particularly with web-tech) that smart folks have been trying to solve with elaborate hacks for decades become trivial when we make addresses permanent, because complications like DNS become irrelevant.

                                                                                              1. 1

                                                                                                And other problems become hard like “how do I have my content still online in 20 years?”.

                                                                                                IPFS doesn’t address the issues it should be addressing, using hashes everywhere being one of them making it particularly user-unfriendly (possibly even user-hostile).

                                                                                                IPFS doesn’t act like a proper cache either (unless their eviction strategy has significantly evolved to be more cooperative) in addition to leaking data everywhere.

                                                                                                Torrent and dat:// solve the problem much better and don’t over-advertise their capabilities.

                                                                                                Nobody really needs permanent addressing, what they really need is either a Tor onion address or actually cashing out for a proper webserver (where IPFS also won’t help if your content is dynamic, it’ll make things only more javascript heavy than they already are).

                                                                                                1. 1

                                                                                                  how do I have my content still online in 20 years?

                                                                                                  If you want to guarantee persistence of content over long periods, you will need to continue to host it (or have it hosted on your behalf), just as you would with host-based addressing. The difference is that your host machine can be puny because it’s no longer a single point of failure under traffic load: as requests increase linearly, the likelihood of any request being directed to your host decreases geometrically (with a slow decay via cache eviction).

                                                                                                  IPFS doesn’t address the issues it should be addressing, using hashes everywhere being one of them making it particularly user-unfriendly (possibly even user-hostile).

                                                                                                  I would absolutely support a pet-name system on top of IPFS. Hashes are convenient for a number of reasons, but IPFS is only one example of a relatively-mature named-data-oriented solution to permanent addressing. It’s minimal & has good support for putting new policies on top of it, so integrating it into applications that have their own caching and name policies is convenient.

                                                                                                  IPFS doesn’t act like a proper cache either (unless their eviction strategy has significantly evolved to be more cooperative) in addition to leaking data everywhere.

                                                                                                  Most caches have forced eviction based on mutability. Mutability is not a feature of systems that use permanent addressing. That said, I would like to see IPFS clients outfitted with a replication system that forces peers to cache copies of a hash when it is being automatically flushed if an insufficient number of peers already have it (in order to address problem #1) as well as a store-and-forward mode (likewise).

                                                                                                  Torrent and dat:// solve the problem much better and don’t over-advertise their capabilities.

                                                                                                  Torrent has unfortunately already become a popular target for blocking. I would personally welcome sharing caches over DHT by default over heavy adoption of IPFS since it requires less additional work to solve certain technical problems (or, better yet, DHT sharing of IPFS pinned items – we get permanent addresses and seed/leech metrics), but for political reasons that ship has probably sailed. DAT seems not to solve the permanent address problem at all, although it at least decentralizes services; I haven’t looked too deeply into it, but it could be viable.

                                                                                                  Nobody really needs permanent addressing,

                                                                                                  Early web standards assume but do not enforce that addresses are permanent. Every 404 is a fundamental violation of the promise of hypertext. The fact that we can’t depend upon addresses to be truly permanent has made the absurd superstructure of web tech inevitable – and it’s unnecessary.

                                                                                                  what they really need is either a Tor onion address

                                                                                                  An onion address just hides traffic. It doesn’t address the single point of failure in terms of a single set of hosts.

                                                                                                  or actually cashing out for a proper webserver

                                                                                                  A proper web server, though relatively cheap, is more expensive and requires more technical skill to run than is necessary or desirable. It also represents a chain of single points of failure: a domain can be siezed (by a state or by anybody who can social-engineer GoDaddy or perform DNS poisoning attacks), while a host will go down under high load (or have its contents changed if somebody gets write access to the disk). Permanent addresses solve the availability problem in the case of load or active threat, while hash-based permanent addresses solve the correctness problem.

                                                                                                  where IPFS also won’t help if your content is dynamic,

                                                                                                  Truly dynamic content is relatively rare (hence the popularity of cloudflare and akamai), and even less dynamic content actually needs to be dynamic. We ought to minimize it for the same reasons we minimize mutability in functional-style code. Mutability creates all manner of complications that make certain kinds of desirable guarantees difficult or impossible.

                                                                                                  Signature chains provide a convenient way of adding simulated mutability to immutable objects (sort of like how monads do) in a distributed way. A more radical way of handling mutability – one that would require more infrastructure on top of IPFS but would probably be amenable to use with other protocols – is to support append-only streams & construct objects from slices of that append-only stream (what was called a ‘permascroll’ in Xanadu from 2006-2014). This stuff would need to be implemented, but it would not need to be invented – and inventing is the hard part.

                                                                                                  it’ll make things only more javascript heavy than they already are

                                                                                                  Only if we stick to web tech, and then only if we don’t think carefully and clearly about how best to design these systems. (Unfortunately, endemic lack of forethought is really the underlying problem here, rather than any particular technology. It’s possible to use even complete trash in a sensible and productive way.)

                                                                                                  1. 1

                                                                                                    The difference is that your host machine can be puny because it’s no longer a single point of failure under traffic load: as requests increase linearly, the likelihood of any request being directed to your host decreases geometrically (with a slow decay via cache eviction).

                                                                                                    I don’t think this is a problem that needs addressing. Static content like the type that IPFS serves can be cheaply served to a lot of customers without needing a fancy CDN. An RPi on a home connection should be able to handle 4 million visitors a month easily with purely static content.

                                                                                                    Dynamic content, ie the content that needs bigger nodes, isn’t compatible with IPFS to begin with.

                                                                                                    Most caches have forced eviction based on mutability

                                                                                                    Caches also evict based on a number of different strategies that have nothing to do with mutability though, IPFS’ strategy for loading content (FIFO last I checked) behaves poorly with most internet browsing behaviour.

                                                                                                    DAT seems not to solve the permanent address problem at all, although it at least decentralizes services; I haven’t looked too deeply into it, but it could be viable.

                                                                                                    The public key of a DAT share is essentially like a IPFS target with the added bonus of having at tracked and replicated history and mutability, offering everything an IPNS or IPFS hash does. Additionally it’s more private and doesn’t try to sell itself as censorship resistant (just look at the stunt with putting Wikipedia on IPFS)

                                                                                                    Every 404 is a fundamental violation of the promise of hypertext.

                                                                                                    I would disagree with that. It’s more important that we archive valuable content (ie, archive.org or via the ArchiveTeam, etc.) than having a permanent addressing method.

                                                                                                    Additionally the permanent addressing still does not solve content being offline. Once it’s lost, it’s lost and no amount of throwing blockchain, hashes and P2P at it will ever solve this.

                                                                                                    You cannot stop a 404 from happening.

                                                                                                    The hash might be the same but for 99.999% of content on the internet, it’ll be lost within the decade regardless.

                                                                                                    Truly dynamic content is relatively rare

                                                                                                    I would also disagree with that, in the modern internet, mutable and dynamic content are becoming more common as people become more connected.

                                                                                                    CF and Ak allow hosters to cache pages that are mostly static like the reddit frontpage as well as reducing the need for georeplicated servers and reducing the load on the existing servers.

                                                                                                    is to support append-only streams & construct objects from slices of that append-only stream

                                                                                                    See DAT, that’s what it does. It’s an append-only log of changes. You can go back and look at previous versions of the DAT URL provided that all the chunks are available in the P2P network.

                                                                                                    Only if we stick to web tech, and then only if we don’t think carefully and clearly about how best to design these systems.

                                                                                                    IPFS in it’s current form is largely provided as a Node.js library, with bindings to some other languages. It’s being heavily marketed for browsers. The amount of JS in websites would only increase with IPFS and likely slow everything down even further until it scales up to global, or as it promises, interplanetary scale (though interplanetary is a pipedream, the protocol can’t even handle satellite internet properly)

                                                                                                    Instead of looking at pipedreams of cryptography for the solution, we ought to improve the infrastructure and reduce the amount of CPU needed for dynamic content, this is a more easy and viable option than switching the entire internet to a protocol that forgets data if it doesn’t remember it often enough.

                                                                                  1. 7

                                                                                    Climbing, keep my fit and sane. With my new business I don’t have time for much hobbies, but I promised myself to always make time for climbing and even manage to keep doing climbing competition and use conference travelling to climb more rock. This is definitely my best way to disconnect from everything else and it’s also a good reason to meet up with old and new friends.

                                                                                    1. 3

                                                                                      This seems to be a trend with major browsers recently — I definitely take issue with it, but I’m guessing there’s some rationale behind the decisions.

                                                                                      1. 9

                                                                                        Because URLs are hard to understand, apparently: https://www.wired.com/story/google-wants-to-kill-the-url/

                                                                                        1. 7

                                                                                          I don’t think URLs are working as a good way to convey site identity

                                                                                          That’s because they are supposed to convey a location, not an identity.

                                                                                          But it’s important we do something, because everyone is unsatisfied by URLs

                                                                                          Who’s “everyone”? Never heard anybody say they were unsatisfied with URLs. Typical google-speak where they claim they are working for the greater good, while they are simply trying to twist the web to make it easier for their algorithms to process.

                                                                                          1. 7

                                                                                            That’s because they are supposed to convey a location, not an identity.

                                                                                            A URL is a URI, so they are definitely also identifiers.

                                                                                            1. 6

                                                                                              Who’s “everyone”?

                                                                                              I’m pretty sure the numbers speak loud enough about people not understanding URLs or its shortcoming only with all the successful phishing going around and all the confusion about the meaning of the padlock (Could be argued this is not an URL issue, but IMO still relies on the user understanding what is a domain).

                                                                                              Domain and URLs should be abstracted away to the average user. The user wants to go on Facebook or Google, not https://facebook.com or https://google.com.

                                                                                          2. 5

                                                                                            I prefer what a lot of browsers do where they gray out most of the URL and show the domain name in full white/black

                                                                                          1. 7

                                                                                            Looks like a dream scenario for phishing, with the opportunity to create legit-looking domain names, plus the secure padlock right next to the address bar.

                                                                                            I’m curious what they were trying to optimise for when coming up with this.

                                                                                            1. 14

                                                                                              I’m curious what they were trying to optimise for when coming up with this.

                                                                                              Consumer lock-in is my guess. In conjunction with their other remarks about URLs, I think they want to make URLs unpredictable and hence scary, leading users to trust Google to tell them how to get to Apple’s website more than they trust ‘https://apple.com/’ to.

                                                                                              This gives them more power to advertise, more power to redirect, and more power to censor. From their point of view it’s pure win; from ours, not so much.

                                                                                              1. 12

                                                                                                I think they want to scrap the URL bar all together so you can only make searches and click links (which go to google AMP pages) googles dream web is just one big google.

                                                                                                1. 5

                                                                                                  …so you can only make searches and click links…

                                                                                                  That’s just catching up to what everyone is doing anyway. Even commercials eschew a domain name and tell the listener to search the company. Back to the ye olde “AOL Keyword” days.

                                                                                                  1. 3

                                                                                                    because the domain name system is broken in the first place.

                                                                                                    it’s invented by network engineers for network engineers.

                                                                                                    1. 2

                                                                                                      This has been the case for ages in Japan now, where ads often feature a search-like bar and the thing to type into said search bar.

                                                                                                    2. 4

                                                                                                      This. My hypothesis is that they are deliberately trying to break the URL bar with “improvements” such as these so that they can later justify removing it altogether.

                                                                                                      As much as I’m annoyed with Firefox breaking DNS, this is arguably much worse. And what’s said is that all of the other major browsers will probably follow suit because imitating chrome is just what they all do now.

                                                                                                    3. 4

                                                                                                      I’ll be shocked if they don’t replace the address bar with a search-only box.

                                                                                                    4. 5

                                                                                                      I fail to see how this make phishing any easier. Given an attacker own a domain, he’s free to use whatever legit-looking subdomain names he wants. And even if somehow an attacker took control of www subdomain of a target, user are so used to www being aliased to @, I don’t see anyone thinking they might be phished due to that.

                                                                                                      I’m curious what they were trying to optimise for when coming up with this.

                                                                                                      My guess is they are trying to rethink the way people navigate the web. URLs are coming from somewhere with quite different application and users. Maybe we can do better for the average user (People on lobste.rs are not the average users). Hopefully those small changes can be easily driven by user testing and UX researches.

                                                                                                      1. 1

                                                                                                        A phisher who does obtain access to a domain can now quietly point WWW where they want and just one more thing will work out for their benefit. That isn’t a large difference, maybe, but could be quite confusing.

                                                                                                    1. 34

                                                                                                      I’m impressed by the lack of testing for this “feature”. It may have a huge impact for end users, but they have managed it to ship with noob errors like the following:

                                                                                                      Why is www hidden twice if the domain is “www.www.2ld.tld”?

                                                                                                      Who in their right mind misses that, and how on Earth wasn’t it caught at some point before it made it to the stable branch?

                                                                                                      1. 11

                                                                                                        url = url.replace(/www/g, '') - job well done!

                                                                                                        1. 21

                                                                                                          Worse

                                                                                                          What’s really eye-opening is that comment just below wrapped in the pre-processor flag! Stunning.

                                                                                                          1. 9

                                                                                                            Wow, so whoever controls www.com can disguise as any .com page ever? And, as long as it’s served with HTTPS, it’ll be “secure”? That’s amazing.

                                                                                                            1. 5
                                                                                                              1. 5

                                                                                                                Not just .com. On any TLD so you could have lobster.www.rs

                                                                                                              2. 3

                                                                                                                If I may ask, how is this worse than url = url.replace(/www/g, '')? If anything, the current implementation use a proper tokenizer to search and replace instead of a naive string replace.

                                                                                                                1. 2

                                                                                                                  That’s just my hyperbole.

                                                                                                            2. 10

                                                                                                              Right, the amateurishness of Google here is stunning. You’d think with their famed interview process they’d do better than this.

                                                                                                              On a tangential rant, one astonishing phenomenon is the helplessness of tech companies with multibillion capitalizations on relatively simple things like weeding out obvious bots or fixing the ridiculousness of their recommendation engines. This suggests a major internal dysfunction.

                                                                                                              1. 14

                                                                                                                To continue off on the tangent, it sounds like the classic problem with any institution when it reaches a certain size. No matter which type (public, private, government…), at some point the managerial overhead becomes too great and the product begins to suffer.

                                                                                                                Google used to have a great search engine. It might even still be great for the casual IT user, but the signal-to-noise ratio has tanked completely within the past ~2 years. Almost all of my searches are now made on DuckDuckGo and it’s becoming increasingly rare that I even try Google, and when I do it’s mostly an exercise in frustration and I spend the first 3-4 searches on quoting and changing words to get proper results.

                                                                                                                1. 5

                                                                                                                  Large institutions collapsing under their own managerial weight is more of a ‘feature’ in this case.

                                                                                                                  1. 1

                                                                                                                    What are a few examples of queries for which DDG produces better results than Google?

                                                                                                                    1. 2

                                                                                                                      I’m not able to rattle off any examples, sorry. I’ll try to keep it in mind and post an example or two, but don’t hold your breath :)

                                                                                                                      I’ve been using DDG as my primary search engine for 2-3-4 years now, and have tried to avoid Google more and more in that same time frame. This also means that all the benefits of Google having a full profile on me are missing from the equation, and I don’t doubt that explains a lot of the misery I experience in my Google searches. However, I treat DDG the same and they still manage to provide me with better search results than Google…

                                                                                                                      In general every search that includes one or more common words tend to be worse on Google. It seems to me that Google tries to “guess” the intent of the user way too much. I don’t want a “natural language” search engine, I want a search engine that searches for the words I type into the search field, no matter how much they seem like misspellings.

                                                                                                              1. 11

                                                                                                                I’m one of the VerneMQ developers, so if you have any questions about VerneMQ I’d be happy to try to answer.

                                                                                                                1. 2

                                                                                                                  I saw you addressed the difference with RabbitMQ. Do you know about Malamute (ZeroMQ broker)? How would it compare against? I really like the philosophy behind ZeroMQ, and use Malamute internally, but its a bit of a pain to install outside of Linux and I’ve hit bugs that left me with a feeling of “I’m the first user of this”.

                                                                                                                  1. 1

                                                                                                                    I haven’t heard about Malamute before, so can’t say anything about it, I’m afraid. I did work with ZeroMQ briefly some years back and it seemed pretty nice. I’ll have to check out Malamute!

                                                                                                                  2. 2

                                                                                                                    What are the pros/cons of VerneMQ vs Mosquitto?

                                                                                                                    1. 3

                                                                                                                      I guess what’s a pro and what’s a con is in the eye of the beholder. The biggest difference is that VerneMQ is built from the start to be a distributed broker, while Mosquitto is a stand-alone broker. The clustering makes VerneMQ horizontally scalable, so that would be a pro if you need that. Another difference which may be an important pro or con, depending on what one fancies, is that Mosquitto is written in C and hence plugins has to be written in C (correct me if I’m wrong here). VerneMQ plugins can be written in Erlang, Elixir or Lua or as HTTP endpoints. There are of course lots of other details, but those are, I think the main ones.

                                                                                                                      1. 2

                                                                                                                        emqtt is another one I have run across a few times (haven’t tried it out yet).

                                                                                                                    1. 6

                                                                                                                      He asked: why is there no argument to memcpy() to specify the maximum destination length?

                                                                                                                      That’s the third one.

                                                                                                                      If you really insist, #define safe_memcpy(d, s, dn, sn) memcpy(d, s, min(dn, sn))?

                                                                                                                      1. 4

                                                                                                                        Yeah, also, I don’t understand why would they want that.

                                                                                                                        Imagine calling memcpy(d, 10, s, 15), and having your data not copied entirely, having your d buffer with cropped data. Garbage, essentially. How would that be better?

                                                                                                                        edit: to be clear, I’m not complaining about your suggestion, but about the reasoning of the presenter on this.

                                                                                                                        1. 4

                                                                                                                          Yeah, also, I don’t understand why would they want that.

                                                                                                                          Imagine calling memcpy(d, 10, s, 15), and having your data not copied entirely, having your d buffer with cropped data. Garbage, essentially. How would that be better?

                                                                                                                          Cropped data would be a logic error in your application. With standard memcpy the additional 5 bytes overwrite whatever is in memory after the d buffer. This can even enable an attacker to introduce execution of their own code. That’s why ie. Microsoft ships a memcpy_s.

                                                                                                                          Reading materials:

                                                                                                                          1. 8

                                                                                                                            But the unanswered question is why you’re calling memcpy(d, s, 15) instead of memcpy(d, s, 10)? At some level the problem is calling the function with the wrong argument, and adding more arguments maybe doesn’t help.

                                                                                                                            1. 4

                                                                                                                              Every security exploit can be drilled down to “why were you doing this!”. If there was an obvious answer, security exploit would have been a thing of the past. Meanwhile advocating harm reduction is as good as we can get because even if calling memcpy with a smaller destination is wrong to begin with, truncated data still has a more chance to end up with non-exploitable crash than plain old buffer overflow that often end up with reliable code exec.

                                                                                                                              1. 3

                                                                                                                                But why do we assume this extra parameter is better than the other parameter which we have assumed is incorrect? Why not add another extra parameter? memcpy_reallysafe(dest, src, destsize, srcsize, destsize_forserious, doublechecksize)

                                                                                                                                1. 3

                                                                                                                                  Because in ten years a line of code can change and the assumptions that made one variable the right one will break. Suddenly you got the wrong variable in there. Personally, I think this is where asserts belong, to codify the assumptions over a long span of time and multiple developers.

                                                                                                                                  1. 3

                                                                                                                                    A common use case of memcpy is to copy a buffer over another. The way program are structure we often end up with srcsize and dstsize that matches their buffer. The error come from the implicit contract that srcsize is always at least bigger than dstsize. Sure, good code would ensure this is always true. Actual code had many instance where it is not. Adding dstsize to memcpy means that this contract is now explicit and can be asserted by the actual function that put this contract in place.

                                                                                                                                    I mean, at this point we are not arguing of hypothetical scenario, we have a whole history of this bug class happening over and over again. Simply keeping track of the semantic (Copy one buffer to the other) and asking for all the properties required (Buffer and their size) is a low effort and easy way to prevent many of those bug.

                                                                                                                                    1. 1

                                                                                                                                      Yeah, keeping track of the buffer size is a very good idea. But if you want it to always be correct, it should be done without requiring the programmer to manually carry the buffer size along in a separate variable from the buffer pointer.

                                                                                                                                      Either something like “Managed C++”, where the allocator data structures are queried to figure out the size of the buffer, or something like Rust slices:

                                                                                                                                      typedef struct {
                                                                                                                                          char *ptr;
                                                                                                                                          size_t len;
                                                                                                                                      } slice_t;
                                                                                                                                      slice_t slice(slice_t slice, size_t start, size_t end) {
                                                                                                                                          assert(start <= end);
                                                                                                                                          assert((end - start) <= slice.len);
                                                                                                                                          slicet.ptr += start;
                                                                                                                                          slice.len = end - start;
                                                                                                                                          return slice;
                                                                                                                                      }
                                                                                                                                      slice_t slice_front(slice_t slice, size_t start) {
                                                                                                                                          assert(start <= slice.len);
                                                                                                                                          slice.ptr += start;
                                                                                                                                          slice.len -= start;
                                                                                                                                          return slice;
                                                                                                                                      }
                                                                                                                                      slice_t slice_back(slice_t slice, size_t end) {
                                                                                                                                          assert(end <= slice.len);
                                                                                                                                          slice.len = end;
                                                                                                                                          return slice;
                                                                                                                                      }
                                                                                                                                      void slicecpy(slice_t dest, slice_t src) {
                                                                                                                                          assert(dest.len == src.len);
                                                                                                                                          memcpy(dest, dest.len, src);
                                                                                                                                      }
                                                                                                                                      

                                                                                                                                      The point being to make it harder to mix up which len goes with which ptr, plus providing a assert-assisted pointer manipulation in addition to the safe memcpy itself. A safe abstraction needs to account for the entire life cycle of its bounds check, not just the point of use.

                                                                                                                                      Also, this would really, really benefit from templates.

                                                                                                                        1. 2

                                                                                                                          I stumbled upon this project a few week ago. It seems there’s a protocol to redeem and use tokens when filling the first captcha and skip later ones. The protocol also allow user to stay anonymous.

                                                                                                                          1. 2

                                                                                                                            The issues with mruby have been to use it as a sandbox itself. While some language like Lua (or JavaScript) has been made to be embeddable and expect to run arbitrary scripts, most application language implementation such as mruby, CRuby, CPython, etc. are not. The fix is simply to sandbox the process and not the script. By using seccomp, code execution in mruby doesn’t lead anywhere, so it makes sense for Shpoify to reduce the bounty at 10% of its original price (And that’s still a generous amount for useless bugs). I’m glad to be one of the early participant of the bounty, there was a lot of low hanging fruit to exploit :)

                                                                                                                            1. 3

                                                                                                                              I think this kind of stories will appear more and more often as time goes by and wasm is becoming more and more accessible.

                                                                                                                              1. 2

                                                                                                                                Yeah. Having seen this all before (we called it JavaScript then), is it worth posting them?

                                                                                                                                1. 6

                                                                                                                                  I think the author left a lot of details in the README about the process he went through that made it quite interesting. This is much more involved than just CC=emscripten make.

                                                                                                                              1. 3

                                                                                                                                Do you by any chance have a MOBI (Or even EPUB) format? Would be nicer to read it from an e-reader.

                                                                                                                                1. 1
                                                                                                                                  1. 1

                                                                                                                                    Awesome, thanks!

                                                                                                                                  2. 1

                                                                                                                                    Unfortunately not! I just can’t seem to find a way to properly generate it from markdown, including the code samples with the nice syntax highlighting. I probably didn’t try enough, I’m sure there’s a simple way out there.

                                                                                                                                    1. 4

                                                                                                                                      pandoc can do this, but it’s not especially simple.

                                                                                                                                      1. 2

                                                                                                                                        Thanks, will check it out :)

                                                                                                                                        1. 1

                                                                                                                                          That would be awesome

                                                                                                                                        2. 1

                                                                                                                                          Why shouldn’t it be simple? Shouldn’t pandoc input1.md input2.md ... final.md -t epub3 -o book.epub. More epub options are listed in the man page, such as for specifying custom fonts, stylesheets and cover images.

                                                                                                                                          Then, producing a mobi file is trivial using kindlegen or calibre.

                                                                                                                                          1. 1

                                                                                                                                            It’s not simple because about six seconds later I want to start tinkering with the code highlighting and maybe bump up that one margin and should that footnote have a slightly different font and…

                                                                                                                                            1. 1

                                                                                                                                              If you can edit .zip files and save it’s internal files, it’s quite simple to play around with the embedded stylesheet, since it’s just regular CSS.

                                                                                                                                              But one should also say that Ebooks shouldn’t be overcustomized, IMO, but kept simple for the sake of compatibility and an ease of reading.

                                                                                                                                        3. 3

                                                                                                                                          I can help you with this if you upload the book source on github, i have a template for generating epub from markdown here if you are interested.

                                                                                                                                      1. 1

                                                                                                                                        I use Linux for pretty much everything, but one of my last job required Windows for development. My setup end up being Cygwin to replicate pretty much my entire Linux environment, sharing mostly the same dotfiles. With Cygwin you can install X11 and then use your favorite Linux terminal (I did use urxvt without any issue). Then you can use Vim, tmux, ssh, irssi just as you would on Linux. I did run into a few crash every now and then due to forking not always working on Windows, but otherwise I was able to live in my terminals the exact same way I was on Linux.