And what, exactly, is so wrong about MitM yourself, on your own network? Have we been so gaslit by “security specialists” that doing so on our own equipment is considered unthinkable? Or am I just an old man yelling at clouds?
There’s lots of research on the prevalence of people screwing up TLS interception like this (I recently looked for some so my team at work would have ammunition for refusing to do so on work laptops, which we manage).
That being said there’s a lot going for this approach:
Go’s TLS library is probably pretty reasonable and is likely to prevent a lot of common footguns here - not passing on certificate validation failures from the upstream origin, etc.
You’re only doing it for one website, t.co, instead of generic TLS connections which significantly reduces attack surface (and complexity!).
You’re not doing this at scale/you’re probably not a target. Yeah someone could do Bad Things™ with your root CA certificate if they got onto your network, but on a typical home network you’ve got bigger problems then. So meh?
🤷 seems PROBABLY okayish even though it makes me sweat a little! Not that I am an expert.
Admittedly, if you’re only hijacking one hostname, you might as well self-sign an entity certificate for the target hostname and directly add it to your trust stores (without creating a self-signed CA).
There’s a standard that exists for that. I was party toimplementations of that, but I don’t think it got much traction on the internet at large. The easiest mainstream way is to certify it using a root that you control and add name constraints, but for that to be secure (in a general way) you need to own both CAs.
Not to say this is a bad solution, but what happens when your friends come over and ask to use your wifi? Presumably they haven’t installed your CA’s root cert. (Ignore for a moment the fact that obviously any TRUE friend would install their friend’s root cert.)
Anyway the benefits outweigh the downsides, but it’s something to think about.
A much better solution is to abolish t.co altogether, which is now a lot closer to happening than I would have dared to hope six months ago! I haven’t followed a t.co link in months, and with any luck never will again, but I understand others might not be so fortunate at this time.
For my situation I actually don’t have Adguard as the DNS resolver on my router, mainly because I’ve never been able to get it to work, so I just update the DNS manually on devices instead - so friends and family won’t be affected (but they wouldn’t be able to use this tool) unless they specifically set the DNS resolver on their phones.
It’s a fair point though, I guess creating an isolated VLAN/guest network for guests would be another way around this.
Another valid reply is that t.co is inherently sketchy as hell, and getting a warning when you’re accessing it isn’t necessarily a bad thing. (But it would be better if the warning were clearer about the specific problem.)
I’m shocked that the example t.co link took 9 hops to get to nytimes.com. That must make for a horrible user experience, introducing ~1 second of lag just to resolve the URL. I also find it unnerving that it redirects to an intermediate website “trib.al”, which a user would likely never see when clicking on the link directly. The only articles on the surface web that I could find describing Tribal Analytics are vapid and shallow, which only serves to make me want to avoid them more.
I commend the author for writing some software that at least allows the user to take back some control from these link shortening services, even though it was done with some… questionable security practices.
For most of the nytimes links I saw 3 levels of nesting, t.co -> nytim.es -> trib.al
However there were a few 6 level deep ones and the 9 level deep posted in the blog.
I suspect it’s probably people pasting short links that link to other short links in the software that posts out tweets etc, probably not realising the effect it causes.
Would be fun to see if there are longer chains out there!
I’ve noticed this while browsing twitter a lot. t.co links are unbearably slow. Not only are they collecting data about users, they’re substantially degrading the simple act of clicking a link.
With the “real” link URL now avaialble it’s often polluted with various query parameters used for tracking or affiliate descriptors, so the service also strips these out and presents a “cleaned” version of the link alongside the original
I started to write a comment to ask how this works—are you stripping all query params, or is there a block list, or an allow list?—when I realized that the code is just on GitHub. It looks like there is a block list for parameter names, and another block list for parameter name prefixes (e.g., “strip any parameter whose name starts with utm_”). Neat!
Yeah, I put all those in after experimenting around a bit. There are browser extensions that do similar and probably have a more comprehensive “block/allow” list, I just threw these together based on what I saw.
And what, exactly, is so wrong about MitM yourself, on your own network? Have we been so gaslit by “security specialists” that doing so on our own equipment is considered unthinkable? Or am I just an old man yelling at clouds?
There’s lots of research on the prevalence of people screwing up TLS interception like this (I recently looked for some so my team at work would have ammunition for refusing to do so on work laptops, which we manage).
That being said there’s a lot going for this approach:
t.co, instead of generic TLS connections which significantly reduces attack surface (and complexity!).🤷 seems PROBABLY okayish even though it makes me sweat a little! Not that I am an expert.
On point #3, the Name Constraints Extension appears to be a good mitigation toward someone hijacking the root CA.
http://pkiglobe.org/name_constraints.html and https://www.sysadmins.lv/blog-en/x509-name-constraints-certificate-extension-all-you-should-know.aspx have interesting notes on how these constraints get applied to entity certificates by clients.
Unfortunately it seems some browsers only apply name constraints to intermediate CAs (https://bugs.chromium.org/p/chromium/issues/detail?id=1072083), so even this might not be a silver bullet.
Admittedly, if you’re only hijacking one hostname, you might as well self-sign an entity certificate for the target hostname and directly add it to your trust stores (without creating a self-signed CA).
Relatedly, I wish there was a way to add a CA to your trust base BUT only for certain specific domains and subdomains.
There’s a standard that exists for that. I was party toimplementations of that, but I don’t think it got much traction on the internet at large. The easiest mainstream way is to certify it using a root that you control and add name constraints, but for that to be secure (in a general way) you need to own both CAs.
I didn’t know CA name constraints are a thing. Thank you.
Hehe, I guess I was just preparing for the deluge of disapprovals so spent a good while explaining myself!
Agree. This is not a terrible solution and I don’t see why it wouldn’t be recommended. This is a great hack and I love it.
Not to say this is a bad solution, but what happens when your friends come over and ask to use your wifi? Presumably they haven’t installed your CA’s root cert. (Ignore for a moment the fact that obviously any TRUE friend would install their friend’s root cert.)
Anyway the benefits outweigh the downsides, but it’s something to think about.
A much better solution is to abolish
t.coaltogether, which is now a lot closer to happening than I would have dared to hope six months ago! I haven’t followed a t.co link in months, and with any luck never will again, but I understand others might not be so fortunate at this time.I operate an open wifi for friends to use, but that’s a fair point
So you would only do this interception on the private WiFi then?
This is a fair point.
For my situation I actually don’t have Adguard as the DNS resolver on my router, mainly because I’ve never been able to get it to work, so I just update the DNS manually on devices instead - so friends and family won’t be affected (but they wouldn’t be able to use this tool) unless they specifically set the DNS resolver on their phones.
It’s a fair point though, I guess creating an isolated VLAN/guest network for guests would be another way around this.
Another valid reply is that t.co is inherently sketchy as hell, and getting a warning when you’re accessing it isn’t necessarily a bad thing. (But it would be better if the warning were clearer about the specific problem.)
I’m shocked that the example t.co link took 9 hops to get to nytimes.com. That must make for a horrible user experience, introducing ~1 second of lag just to resolve the URL. I also find it unnerving that it redirects to an intermediate website “trib.al”, which a user would likely never see when clicking on the link directly. The only articles on the surface web that I could find describing Tribal Analytics are vapid and shallow, which only serves to make me want to avoid them more.
I commend the author for writing some software that at least allows the user to take back some control from these link shortening services, even though it was done with some… questionable security practices.
For most of the nytimes links I saw 3 levels of nesting, t.co -> nytim.es -> trib.al
However there were a few 6 level deep ones and the 9 level deep posted in the blog.
I suspect it’s probably people pasting short links that link to other short links in the software that posts out tweets etc, probably not realising the effect it causes.
Would be fun to see if there are longer chains out there!
I’ve noticed this while browsing twitter a lot. t.co links are unbearably slow. Not only are they collecting data about users, they’re substantially degrading the simple act of clicking a link.
I started to write a comment to ask how this works—are you stripping all query params, or is there a block list, or an allow list?—when I realized that the code is just on GitHub. It looks like there is a block list for parameter names, and another block list for parameter name prefixes (e.g., “strip any parameter whose name starts with
utm_”). Neat!Yeah, I put all those in after experimenting around a bit. There are browser extensions that do similar and probably have a more comprehensive “block/allow” list, I just threw these together based on what I saw.
I loved the solution and the simple approach. Going to explore and extend this to my home network. Thanks.
Even better is the explanation and the associated diagrams. You are indeed a smart one – very few have this talent.
For those using NextDNS, it also lets you rewrite DNS answers.
Available in settings.