1.  

    Its a bit of a tough choice. With the current state of things, most users would see a massive improvement from switching from ISP DNS servers that admit to collecting and selling your data and switching to cloudflare who has agreed to protect privacy.

    In the end, you have to trust someone for your DNS. Mozilla could probably host it themself but they also dont have the wide spread of server locations that a CDN company has.

    1. 5

      While I agree that, you need to trust someone to your DNS, it shouldn’t be a specific app making that choice for you. A household or even a user with multiple devices benefits from their router caching DNS results for multiple devices, every app on every device doing this independently is foolish. If Mozilla wants to help users then they can run an informational campaign, setting a precedent for apps each using their own DNS and circumventing what users have set for themselves is the worst solution.

      1.  

        It isn’t ideal that firefox is doing DNS in app but it’s the most realistic solution. They could try and get microsoft, apple and all linux distros to change to DNS over HTTPS and maybe in 5 years we might all have it or they could just do it themself and we all have it in a few months. Once firefox has started proving it works really well then OS vendors will start adding it and firefox can remove their own version or distros will patch it to use the system DoH.

        1.  

          They could try and get microsoft, apple and all linux distros to change to DNS over HTTPS

          I don’t WANT DNS over HTTPS. I especially don’t want DNS over HTTP/2.0. There’s a lot of value in having protocols that are easy to implement, debug, and understand at a low level, and none of those families of protocols are that.

          Add TLS, maybe – it’s also a horrendous mess, but since DNSCURVE seems to be dead, it may get enough traction. Cloudflare, if they really want, can do protocol sniffing on port 443. But please, let’s not make the house of card protocol stack that is the internet even more complex.

          1. 8

            DNS is “easy to implement, debug, and understand”? That’s news to me.

            1.  

              it’s for sure easier than when tunneled over HTTP2 > SSL > TCP, because that’s how DoH works. The payload of the data being transmitted over HTTP is actual binary DNS packets so all this does is adding complexity overhead.

              I’m not a big fan of DoH because of that and also because this means that by default intranet and development sites won’t be available any more to users and developers, invalidating an age-old concept of having private DNS.

              So either you now need to deploy customized browser packages, or tweak browser’s configs via group policy or equivalent functionality (if available), or expose your intranet names to public DNS which is a security downgrade from the status quo.

              1.  

                It is when you have a decent library to encode/decode DNS packets and UDP is nearly trivial to deal with compared to TCP (much less TLS).

              2.  

                Stacking protocols makes things more simple. Instead of having to understand a massive protocol that sits on its own, you now only have to understand the layer that you are interested in. I haven’t looked in to DNS but I can’t imagine it’s too simple. It’s incredibly trivial for me to experiment and develop with applications running on top of HTTP because all of the tools already exist for it and aren’t specific to DoH. You can also share software and libraries so you only need one http library for a lot of protocols instead of them all managing sending data over TCP.

                1.  

                  But the thing transmitted over HTTP is binary DNS packets. So when debugging you still need to know how DNS packets are built, but you now also have to deal with HTTP on top. Your HTTP libraries only give you a view into the HTTP part of the protocol stack but not into the DNS part, so when you need to debug that, you’re back to square one but also need your HTTP libraries

                  1.  

                    And don’t forget that HTTP/2 is basically a binary version of HTTP, so now you have to do two translation steps! Also, because DoH is basically just the original DNS encoding, it only adds complexity. For instance, the spec itself points out that you have two levels of error handling: One of HTTP errors (let’s say a 502 because the server is overloaded) and one of DNS errors.

                    It makes more sense to just encode DNS over TLS (without the unnecessary HTTP/2 stuff), or to completely ditch the regular DNS spec and use a different wire format based on JSON or XML over HTTP.

                    1.  

                      And don’t forget that HTTP/2 is basically a binary version of HTTP

                      If only it was that simple. There’s server push, multi-streaming, flow control, and a huge amount of other stuff on top of HTTP/2, which gives it a relatively huge attack surface compared to just using (potentially encrypted) UDP packets.

                      1.  

                        Yeah, I forgot about all that extra stuff. It’s there (and thus can be exploited), even if it’s not strictly needed for DoH (I really like that acronym for this, BTW :P)

            2.  

              it shouldn’t be a specific app making that choice for you

              I think there is a disconnect here between what security researchers know to be true vs what most people / IT professionals think is true.

              Security, in this case privacy and data integrity is best handled with the awareness of the application, not by trying to make it part of the network or infrastructure levels. That mostly doesn’t work.

              You can’t get any reasonable security guarantees from the vast majority of local network equipment / CPE. To provide any kind of privacy the application is the right security barrier, not your local network or isp.

              1.  

                I agree that sensible defaults will increase security for the majority of users, and there is something to be said for ones browser being the single most DNS hungry app for that same majority.

                If its an option that one can simply override (which appears to be the case), then why not. It will improve things for lots of people, and those which choose to have the same type of security (dnscrypt/dnssec/future DNS improvements) on their host or router can do so.

                But I can’t help thinking its a bit of a duct tape solution to bigger issues with DNS overall as a technolgy and the privacy concerns that it represents.

          1. 3

            I like the highly technical nature of posts and the great community which tends towards useful comments that actually go into detail about topics instead of just posting a quick gut reaction. And I also like the fact that the update frequency is rather low, you don’t get this “attempting to drink from a firehose” vibe that you get with certain other websites.

            1. 2

              Comment quality is a big one. I always have to stop myself if I go to make a comment before reading the linked article. Its a bad habit from using reddit where the linked article usually has no content.

            1. 1

              I am curious. I have been looking at a few job postings and they say be expert in SQL for a role like data analysis or such. Practically, I have only ever used basic statements for querying directly. If I have to do a complicated task I end up handing the control over to the ORM paradigm and in this way I end up never learning SQL well enough. Can somebody else provide some insight whether I am doing it the right way? People who got good at query languages, was it on the job or before grtting such a job which mandated SQL like languages?

              1. 3

                I learned most of my SQL skills on the job. I struggled a lot with SQL in the university course I took, and I didn’t grok it. But when I started to need SQL on the job, it started to “click” and I learned it properly. To be honest, I committed the sin of rolling my own framework way back when, but it meant I was somewhat forced to learn a lot of underlying stuff. When you have to generate SQL from your ORM, that’s a fantastic way to learn the nitty-gritty of SQL. You’ll even encounter the limitations of your RDBMS, as you’re almost guaranteed to run into them whilst “compiling” SQL.

                I’ve also noticed that as you start to need more advanced features, many ORMs don’t support it and you’ll need to drop down to the SQL level more. This is especially true of PostgreSQL, as it has a very extended feature set. Django has some contributed support for lots of Postgres features, but other ORMs tend to cater only to the lowest common denominator, so you can’t do fancy things. Especially something like a complex UPDATE statement with a FROM query on a related table or doing an INSERT of a complex SELECT where you dynamically generate values based on another query, while returning the inserted values.

                Of course, when you aren’t even aware of advanced features, you’re not likely to even want to use them. To get an awareness of what’s possible, I’d suggest reading Use the Index, Luke and just the Postgres manual, which is quite well-written and can be read like a book.

                When making data migrations you’ll often want to do it directly in SQL for performance and memory usage reasons (doing it all in the procedural language is a no-go if you have a huge data set). Big data sets are the key: if you have lots and lots of data, you can’t simply slurp everything into memory and iterate over it, updating records one at a time. Bulk updates of non-static values tend to require dropping down to SQL pretty quickly unless you have a very powerful ORM.

                1. 2

                  I picked up 99% if my SQL knowledge on the job. My first job offered to pay for Microsoft certificates test taking, so I went and got the MCTS in SQL Server 2008. It was a good introduction to all the various features of SQL Server. Not sure that this qualified me as an expert, but I definitely have a decent feel for the SQL server feature set. I suspect there are similar resources for MySql, Postgresql and so on.

                  Since then, it’s mostly just been the odd thing here or there. If SQL interests you, I definitely recommend either getting a book, or reading up on things like window functions, (recursive) common table expressions materialized/indexed views, window functions, how aggregates work, and so on. Digging into the exclusive functionality of the database you use is also recommend There’s some interesting stuff under the hood if you take a look.

                  Probably the most frustrating thing about SQL, however is that most abstraction features, like functions, stored procedures, or views do not compose well from a performance perspective. One layer of them usually works well, two or more is often trouble.

                  1. 2

                    On the job I learned how to use SQL to get order-of-magnitude speedups over ORM. Turns out databases are really good at querying data! For me, common-table expressions were the gateway drug and window functions were what gave me the most power.

                  1. 3

                    Author here. This post is a little outdated regarding the calling convention; now we pass in arguments to Scheme procedures via an array of arguments (the C varargs ABI turned out to be a little fiddly on some platforms; our new style of argument passing is 100% portable and, as it turns out, not slower).

                    If there are any questions I’d be happy to answer them.

                    1. 2

                      I’m going to put in some time to fix remaining bugs in CHICKEN Scheme 5.0.0rc1 so we can make a second release candidate. I’ll also go to the weekly market to buy some food for next week on the cheap :)

                      1. 2

                        It’s really sweet that these types of articles pop up from time to time. There’s a lot of power here. Problem is that so many people use ORMs which don’t support all the types, or have to support different databases, that we end up in a suboptimal position :(

                        1. 6

                          That is one of the thing I love about SQLAlchemy, we use most of these without problem. JSONB is really nice to use, fast, efficient

                          1. 7

                            When people bemoan ORMs and the “impedance mismatch” and it being hard to formulate queries or to optimize them I used to think “what are you talking about, there’s nothing wrong with ORMs” and eventually I came to realize that I was probably very lucky to have mostly only used SQLAlchemy (and LINQ way back) or raw SQL for talking to DBs.

                          2. 4

                            The CHICKEN PostgreSQL egg, which is not an ORM but “just” a database access library (a wrapper for libpq with several additional features, much like Python’s PsycoPG) has rich support for types. I’ve written a so-called “omelette recipe” (a tutorial) about this a couple of years ago.

                            1. 3

                              Most recent project I did, the SQL and the data access code was done in plain JDBC access code in Scala. The SQL uses postgres types and one or two more specialized bits. It’s worked out reasonably well; a compare and contrast with the FRM and ORMs from Scala that I did later on revealed roughly about as many lines of code, but without the support for Postgres capabilities; adding those capabilities got very fiddly and brittle, so I closed down that line of work.

                              Fortunately, I didn’t have to try to target multiple SQL databases. That would be Very Irritating.

                              1. 1

                                I have always maintained that the “but multiple databases!” rationale for ORM is meaningless – how often do large projects change their entire data persistence strategy? How often ought they?

                                1. 3

                                  It may be rare for a particular installation to change, but “works with the database you are already running” is a compelling sales bullet.

                                  1. 1

                                    Several times a day. By which I mean, using SQLite in unit tests and so forth, while using PostgreSQL or similar in acceptance testing and production. If I couldn’t do that, I’d have to write much, much more mocking code to have the level of code coverage I have.

                                    1. 1

                                      You would only have to run postgres locally instead of sqlite. The coverage would be unchanged, and you’d be able to take advantage of more powerful database features in your application.

                                    2. 1

                                      It’s pretty much as @tedu said. Good luck breeding those lions (as the meme says) if you leverage Postgres heavily but many of your pitential customers run Oracle and their DBAs don’t want to add something new to the mix. An ORM is the easy way out.

                                      Even then you may encounter interesting situations with the Oracle driver and long text, unless that’s fixed or not applicable in your ORM of choice.

                                      It does suck people don’t usually run tests against their target db. I’ve seen big companies fsck this up with Django. It really isn’t that hard. Turn off fsync and Postgres isn’t even slow.