The suggested solution might work on iOS, where Apple has banned non-safari web engines entirely, but on any platform that’s not locked down in the same way you can’t really enforce this – if an app wants to pull in a library to display a web page, and they pick one that just doesn’t respect this request, you’re kindof SOL.
You don’t even need a separate library. On iOS, for example, you can use NSURL to fetch the page content and then provide the resources to the WKWebView, so the web view ever see the headers.
The problem here is that it was an explicit design goal for the web to be agnostic to user agents. How to render a web page was always supposed to be under the control of the client, not the server. It’s almost impossible to change that without completely rearchitecting the whole thing.
This article has it backwards. The user controls their device, and how they want to view content, not the author of the website. To say it another way the device and app is a user-agent, not a website-agent. The author of the website has no more right to demand that I view their page in a “normal” browser than they have to demand that I do jumping jacks while reading their website, or that I don’t use an adblocker (which is to say that they could make it a contractual requirement, but that they would have to get me to sign that contract before giving me the content, and that they have no right to impinge on everyones devices to ensure that people are following their strange contract).
There is something to be a bit concerned about here, but it’s not that apps are somehow unfairly hurting websites. It’s that apps that are abusing their position as a source of links to other content to coerce users into using them to view the content when they might prefer to view the content via another user agent. Probably the appropriate medium for resolving this is regulation - since Google does this themselves it seems unlikely that it will be resolved by the platform just deciding to ban apps that do it from their app store.
I have the freedom to set the terms on which I will offer access to a website of mine.
If you do not like those terms, you may reject them and not access it. If you reject them and then attempt to access it anyway, you are the one violating my freedom: the freedom to decide how I will run my site and by whom and on which terms it will be accessed.
The Web is not built around advance informed consent; there’s no agreement to terms before downloading a public file (besides basic protocol negotiations). This is one reason why “by using this site, you agree to our cookies, privacy policy, kidney harvesting, etc” notices won’t fly under the GDPR.
A website admin can’t set terms for downloading a linked document; the user-agent just makes a request and the server works with that data to deny or accept it. There’s no obligation for the UA to be honest or accurate.
Ultimately, nobody is forcing you to run a Web server; however, plenty of people have to use the Web. Respect for the UA is part of the agreement you make when joining a UA-centric network.
Should you disagree with the precedent set by the HTML Living Standard, nearly every Web Accessibility Initiative standard (users must be able to override and replace stylesheets, colors, distracting elements), the exceptions to e.g. the Content Security Policy in Webappsec standards to allow UA-initiated script injection, etc.: you’re always free to build your own alternative to the Web with your own server-centric standards.
Who said anything about advance consent? I can put up a splash page laying out terms and tell you to either accept them and continue, or reject them and leave. Or I can login-wall things. And if you try to work around it and access anyway, I have every right to use both technical and legal-system measures to try to prevent you, or to hold you accountable afterward for the violation.
Or plenty of other low-level tricks and techniques are fair game, too; for example, I believe Jamie Zawinski at least used to (I don’t know if he still does) serve a famous obscene image to any inbound request with a referer from Hacker News.
But before you go too far into citing standards and accessibility at me, do keep in mind that what we’re discussing here is whether sites should be able to object to Instagram literally MITM’ing users and injecting potentially malicious script. And the original parent comment’s suggestion of regulating this away is actually contradictory to the absolutist “browser is a user agent” moral stance, since that stance requires rejection of any imposed limitation on what the “user agent” may do. After all, some person out there might actively want an “agent” to MITM and inject Instagram trackers for them, so banning the practice by law is as hostile to user freedom as is any technical measure which attempts to prevent it.
Also, the absolutist “user agent” stance is still hostile to the freedom of a site owner to decide who to offer access to, as I originally pointed out, and that has nothing to do with accessibility or usability or any of the other things you tried to steer the argument off-topic to. If I want to make a secret online club and decide who I do and don’t let in and on what terms, I can do that and you don’t get to tell me otherwise.
And the original parent comment’s suggestion of regulating this away is actually contradictory to the absolutist “browser is a user agent” moral stance, since that stance requires rejection of any imposed limitation on what the “user agent” may do
It does not. There are all sorts of restrictions on what one make available to consumers. Whether that’s baby toys covered in lead, or products that abuse their monopoly position to gain monopolies on other unrelated markets (anti-trust law, which is the closest analogy to the regulation I proposed IMO).
It merely means that you should be making such restrictions to benefit the user, not some third party with no rights to the users device whatsoever.
hostile to the freedom of a site owner to decide who to offer access to
The site owner has a freedom to do whatever he likes, such as your examples of serving the user with a contractual agreement that they must agree to before the site owner serves them the actual content. The site owner has no right to have every user attempting to access his site (prior to agreeing to any contract) do it in any particular manner though, it is up to him to not give content away to people that coming asking for it if he wants to require them to agree to contractual limitations before they get the content.
So if a government were to pass an enforceable law saying that any site which sends an X-Frame-Options with a “deny” value must be opened in the user’s default browser rather than an app-embdded one, would you be OK with that? There are user-centric reasons for doing so, after all, so it would be a law with benefit to the user.
But it’s also exactly the thing you previously attacked.
No. Nor did I say so. Rather I have continuously been attacking that idea, and will continue to do so below.
It is in the users interest to be able to view websites however they want. Rather your suggestion would be the government gifting control over how users view documents on their devices they they lawfully own (the actual instance of the bits, not the copyright, same as owning a book) to website owners. To the extent that there is user harm resulting from the current app ecosystem, it is extremely minimal compared to the utterly draconian measure you are proposing.
Moreover there is the much less invasive, well tested and understood method of requiring users be given the choice of how to open links (see for example the similar laws for payment providers that are cropping up, and the much older consent decree related to internet explorer). I don’t think the harm is great enough the government necessarily even needs to do something about this, but I wouldn’t mind if they did because it is really just a slight extension to existing anti-trust law and unlike your suggestion does very minimal harm users freedoms to use their devices how they want to.
Edit:
I think you generally misunderstand the nature of the relationships here. In order of “should have the most control over how the content is viewed” to “who should have the least control” it goes
User > Creator of The App that the User chose to install on their device and view the website in > Website Owner
Not as you seem to have it, Website Owner > User > App Creator, or even the unreasonably charitable reading of your posts of User > Website Owner > App Creator.
The website owner bears no special relationship to the user, is not trusted, and did nothing but supply some data which they no longer have any relevant rights to once the transfer of data is complete (they continue to own the copyright if they did in the first place, but nothing restricted by copyright is being done to the data). The app creator supplied software that the user chose to run on their device, in a relatively privileged manner, and is far more trusted to act in the users interest.
I want to be absolutely crystal clear here. I posed a hypothetical where sending a certain header would be required to use “the user’s default browser rather than an app-embedded one”, And your description of this is “utterly draconian”.
How, exactly, is it “utterly draconian” to use the user’s default browser?
Because your hypothetical has just given website owners the ability to legally require that users only view their website through their default browser when they have absolutely no right to demand users do anything of the sort.
It has made the decision the users aren’t entitled to view news articles in their news app and social media sites in their social media app.
It has made it next to impossible to make a huge variety of tools from simple ones like curl and youtube-dl to complex ones like citation managers and privacy respecting replacement apps for YouTube and Facebook without either the cooperation of website owners or breaking the law.
It is fundamentally seizing a fairly significant degree of control of the device from the users, and handing it to the people who serve the content.
Maybe you only have the users best intentions at heart (I sort of doubt it given that we’re discussing the under an article whose whole premise is that users aren’t entitled to view websites how they choose because it violates some supposed right of the website owners), but the policy you’re proposing is not going to be only used for good.
This is an inconsistent position, though. The status quo is user-hostile. Any technical solution would also be user-hostile by your definition. And so too would any regulatory solution – no matter how it’s implemented, it will place restrictions on what a “user agent” is allowed to do, or which “user agents” are allowed, and that appears to be anathema to you.
Even something like “app must ask” can be turned user-hostile and anticompetitive, as in the case of the iOS Gmail app, which – I don’t know if it still does, but I know it did, upon a time – would “helpfully” ask if you wanted to open a link in your default browser, or install Chrome. With a “remember my choice” that only “remembered” for that single link in that single email message, and would prompt again for the next link it encountered, all in hopes you’d finally give in to its badgering and install Chrome.
So I simply don’t see how any position, consistent with the moral values about the user that you keep citing, can be built which would also allow any type of regulation to solve this. All solutions will, by your definitions, end up taking away some freedom from the user, which is something you seem absolutely unwilling to budge even the slightest bit on, and regulatory solutions will do so by force.
A related thing that I’d like to see added in browsers in a meta tag that force-disables any javascript on the page. This is useful when you’ve got a perfectly fine and functional page that doesn’t use javascript, but someone else decides to inject javascript into it - be it Facebook, American ISPs or whatever. With the meta value set, any tracking junk would be unable to do its thing.
Yes, a sufficiently malicious embed can retrieve the source HTML as a string via network call, strip out any anti-embed headers or tags, and then pass that source HTML on to be rendered.
The suggested solution might work on iOS, where Apple has banned non-safari web engines entirely, but on any platform that’s not locked down in the same way you can’t really enforce this – if an app wants to pull in a library to display a web page, and they pick one that just doesn’t respect this request, you’re kindof SOL.
You don’t even need a separate library. On iOS, for example, you can use NSURL to fetch the page content and then provide the resources to the WKWebView, so the web view ever see the headers.
The problem here is that it was an explicit design goal for the web to be agnostic to user agents. How to render a web page was always supposed to be under the control of the client, not the server. It’s almost impossible to change that without completely rearchitecting the whole thing.
This article has it backwards. The user controls their device, and how they want to view content, not the author of the website. To say it another way the device and app is a user-agent, not a website-agent. The author of the website has no more right to demand that I view their page in a “normal” browser than they have to demand that I do jumping jacks while reading their website, or that I don’t use an adblocker (which is to say that they could make it a contractual requirement, but that they would have to get me to sign that contract before giving me the content, and that they have no right to impinge on everyones devices to ensure that people are following their strange contract).
There is something to be a bit concerned about here, but it’s not that apps are somehow unfairly hurting websites. It’s that apps that are abusing their position as a source of links to other content to coerce users into using them to view the content when they might prefer to view the content via another user agent. Probably the appropriate medium for resolving this is regulation - since Google does this themselves it seems unlikely that it will be resolved by the platform just deciding to ban apps that do it from their app store.
I have the freedom to set the terms on which I will offer access to a website of mine.
If you do not like those terms, you may reject them and not access it. If you reject them and then attempt to access it anyway, you are the one violating my freedom: the freedom to decide how I will run my site and by whom and on which terms it will be accessed.
The Web is not built around advance informed consent; there’s no agreement to terms before downloading a public file (besides basic protocol negotiations). This is one reason why “by using this site, you agree to our cookies, privacy policy, kidney harvesting, etc” notices won’t fly under the GDPR.
A website admin can’t set terms for downloading a linked document; the user-agent just makes a request and the server works with that data to deny or accept it. There’s no obligation for the UA to be honest or accurate.
Ultimately, nobody is forcing you to run a Web server; however, plenty of people have to use the Web. Respect for the UA is part of the agreement you make when joining a UA-centric network.
Should you disagree with the precedent set by the HTML Living Standard, nearly every Web Accessibility Initiative standard (users must be able to override and replace stylesheets, colors, distracting elements), the exceptions to e.g. the Content Security Policy in Webappsec standards to allow UA-initiated script injection, etc.: you’re always free to build your own alternative to the Web with your own server-centric standards.
POSSE note from https://seirdy.one/notes/2022/08/12/user-agents-set-the-terms/
Who said anything about advance consent? I can put up a splash page laying out terms and tell you to either accept them and continue, or reject them and leave. Or I can login-wall things. And if you try to work around it and access anyway, I have every right to use both technical and legal-system measures to try to prevent you, or to hold you accountable afterward for the violation.
Or plenty of other low-level tricks and techniques are fair game, too; for example, I believe Jamie Zawinski at least used to (I don’t know if he still does) serve a famous obscene image to any inbound request with a referer from Hacker News.
But before you go too far into citing standards and accessibility at me, do keep in mind that what we’re discussing here is whether sites should be able to object to Instagram literally MITM’ing users and injecting potentially malicious script. And the original parent comment’s suggestion of regulating this away is actually contradictory to the absolutist “browser is a user agent” moral stance, since that stance requires rejection of any imposed limitation on what the “user agent” may do. After all, some person out there might actively want an “agent” to MITM and inject Instagram trackers for them, so banning the practice by law is as hostile to user freedom as is any technical measure which attempts to prevent it.
Also, the absolutist “user agent” stance is still hostile to the freedom of a site owner to decide who to offer access to, as I originally pointed out, and that has nothing to do with accessibility or usability or any of the other things you tried to steer the argument off-topic to. If I want to make a secret online club and decide who I do and don’t let in and on what terms, I can do that and you don’t get to tell me otherwise.
It does not. There are all sorts of restrictions on what one make available to consumers. Whether that’s baby toys covered in lead, or products that abuse their monopoly position to gain monopolies on other unrelated markets (anti-trust law, which is the closest analogy to the regulation I proposed IMO).
It merely means that you should be making such restrictions to benefit the user, not some third party with no rights to the users device whatsoever.
The site owner has a freedom to do whatever he likes, such as your examples of serving the user with a contractual agreement that they must agree to before the site owner serves them the actual content. The site owner has no right to have every user attempting to access his site (prior to agreeing to any contract) do it in any particular manner though, it is up to him to not give content away to people that coming asking for it if he wants to require them to agree to contractual limitations before they get the content.
So if a government were to pass an enforceable law saying that any site which sends an
X-Frame-Optionswith a “deny” value must be opened in the user’s default browser rather than an app-embdded one, would you be OK with that? There are user-centric reasons for doing so, after all, so it would be a law with benefit to the user.But it’s also exactly the thing you previously attacked.
No. Nor did I say so. Rather I have continuously been attacking that idea, and will continue to do so below.
It is in the users interest to be able to view websites however they want. Rather your suggestion would be the government gifting control over how users view documents on their devices they they lawfully own (the actual instance of the bits, not the copyright, same as owning a book) to website owners. To the extent that there is user harm resulting from the current app ecosystem, it is extremely minimal compared to the utterly draconian measure you are proposing.
Moreover there is the much less invasive, well tested and understood method of requiring users be given the choice of how to open links (see for example the similar laws for payment providers that are cropping up, and the much older consent decree related to internet explorer). I don’t think the harm is great enough the government necessarily even needs to do something about this, but I wouldn’t mind if they did because it is really just a slight extension to existing anti-trust law and unlike your suggestion does very minimal harm users freedoms to use their devices how they want to.
Edit:
I think you generally misunderstand the nature of the relationships here. In order of “should have the most control over how the content is viewed” to “who should have the least control” it goes
User > Creator of The App that the User chose to install on their device and view the website in > Website Owner
Not as you seem to have it, Website Owner > User > App Creator, or even the unreasonably charitable reading of your posts of User > Website Owner > App Creator.
The website owner bears no special relationship to the user, is not trusted, and did nothing but supply some data which they no longer have any relevant rights to once the transfer of data is complete (they continue to own the copyright if they did in the first place, but nothing restricted by copyright is being done to the data). The app creator supplied software that the user chose to run on their device, in a relatively privileged manner, and is far more trusted to act in the users interest.
I want to be absolutely crystal clear here. I posed a hypothetical where sending a certain header would be required to use “the user’s default browser rather than an app-embedded one”, And your description of this is “utterly draconian”.
How, exactly, is it “utterly draconian” to use the user’s default browser?
Because your hypothetical has just given website owners the ability to legally require that users only view their website through their default browser when they have absolutely no right to demand users do anything of the sort.
It has made the decision the users aren’t entitled to view news articles in their news app and social media sites in their social media app.
It has made it next to impossible to make a huge variety of tools from simple ones like curl and youtube-dl to complex ones like citation managers and privacy respecting replacement apps for YouTube and Facebook without either the cooperation of website owners or breaking the law.
It is fundamentally seizing a fairly significant degree of control of the device from the users, and handing it to the people who serve the content.
Maybe you only have the users best intentions at heart (I sort of doubt it given that we’re discussing the under an article whose whole premise is that users aren’t entitled to view websites how they choose because it violates some supposed right of the website owners), but the policy you’re proposing is not going to be only used for good.
This is an inconsistent position, though. The status quo is user-hostile. Any technical solution would also be user-hostile by your definition. And so too would any regulatory solution – no matter how it’s implemented, it will place restrictions on what a “user agent” is allowed to do, or which “user agents” are allowed, and that appears to be anathema to you.
Even something like “app must ask” can be turned user-hostile and anticompetitive, as in the case of the iOS Gmail app, which – I don’t know if it still does, but I know it did, upon a time – would “helpfully” ask if you wanted to open a link in your default browser, or install Chrome. With a “remember my choice” that only “remembered” for that single link in that single email message, and would prompt again for the next link it encountered, all in hopes you’d finally give in to its badgering and install Chrome.
So I simply don’t see how any position, consistent with the moral values about the user that you keep citing, can be built which would also allow any type of regulation to solve this. All solutions will, by your definitions, end up taking away some freedom from the user, which is something you seem absolutely unwilling to budge even the slightest bit on, and regulatory solutions will do so by force.
A related thing that I’d like to see added in browsers in a meta tag that force-disables any javascript on the page. This is useful when you’ve got a perfectly fine and functional page that doesn’t use javascript, but someone else decides to inject javascript into it - be it Facebook, American ISPs or whatever. With the meta value set, any tracking junk would be unable to do its thing.
Content Security Policies are flexible enough to do this. They can be specified either through a HTTP header or as a meta tag in the page.
If somebody can inject javascript into your page, though, aren’t they also in a position to strip any such tags or headers?
Yes, a sufficiently malicious embed can retrieve the source HTML as a string via network call, strip out any anti-embed headers or tags, and then pass that source HTML on to be rendered.
Thanks, I’ll look into that!
Until they don’t just inject, but also filter your meta tag, I suppose.