I found this article a bit painful to read, and there are some inaccuracies. Namely REST is similar to Plan 9/Unix in that it has the “Uniform Interface Constraint”
That is basically “everything is a file” – modeling APIs as a bunch of nouns with fixed verbs - GET, POST, PUT, DELETE
Rather than an open-ended set of verbs:
GetIssue(), GetUser(), UpdateUser(), etc.
it’s
GET /issue/$x, GET /user/$y, POST /user/$y, etc.
This isn’t a trivial point, and IME most people try to do the former before they get the latter.
In Unix, open() read() write() close() works on plain files, but also devices like /dev/mem and /dev/tty etc.
However what I will say is that “HATEOS” is confusing and I agree it’s not always followed in practice.
The issue there is:
Does the client jump directly to sourcehut.org/issues/32
Or do they have to GET sourcehut.org/issue-list/, for an opaque URL to issue 32, and then retrieve that URL (whatever it is)
What that does is decouple the client and server – the client only has to know the home page, and then it can dynamically discover other things. The server can upgrade without breaking the client.
This is useful up to a point, but obviously after awhile everyone is just going to jump directly to sourcehut.org/issues/32 for reasons of performance.
I agree REST has a lot of confusing terminology, but it’s real, and it makes things even worse to say “restful is a scam”. I think programmers are sometimes not comfortable with fuzzy architecture. REST is a bit fuzzy, but it doesn’t mean it’s not real.
Re: your HATEOAS example: I’d say the shibboleth is not how the client goes where they go, but how the application showed them where they are and where they can go.
In other words, does the application (a) REST: Transfer a REpresentation of the application State in hypertext and (b) HATEOAS: in that same hypertext show what the user can do to change the state or the view (so that the Hypertext, like an Engine, can drive the Application State)
Some examples, increasingly less Web-like:
does the Issue 32 page contain a form+link [Add Comment](POST /issues/32/comment)?
Any browser can display this page, and any user who understands the words “Add Comment” can use it. The current state is represented in the hypertext, and so is the ‘engine’ of things the user can do.
Or does the page have a link like [Comment](POST /Comment), whose response depends on some invisible (outside the hypertext!) state ‘current issue=32’?
This is discoverable, but stateful: correct usage of POST /Comment requires knowing session state that is not present in the hypertext. Imagine having two issue pages open at once… Or imagine a stateful GET /NextIssue: such a URL could not be cached.
Or does the user have to know the magic word/page/command NextIssue, and invoke it with parameter issue=32?
State-free, but not discoverable. Every such web application needs its own user training.
Or worse, does the application require a non-uniform verb or network packet like Type=command,Length=20:NEXTISSUE /issue/32\0?
This is state-free, but not a uniform interface: now the web browser must know about the custom protocol, and in addition to specific user training we need a custom client. Every application that does this needs its own client.
Yeah, I think it’s basically his “fault”. After 15 years, I’d boil it down to the adage
What’s interesting about REST isn’t true (people don’t use those things)
What’s true about REST isn’t interesting
I read his thesis more than 10 years ago, and probably read that blog post when he published it in 2008.
I largely think it’s not a useful framing, and the world has passed it by. REST means something else now.
These ideas are interesting and fundamental to the web:
“Everything is a resource identified by a URL” (again very similar to Unix / Plan 9)
Statelessness
middleboxes
Remote code evaluation
But we never needed REST to explain them.
I don’t know how the term came to be common, but it doesn’t really matter either way.
I kinda wish Tim Berners-Lee had written an authoritative architectural description of the web, and brushed Fielding’s aside.
It just seems to cause bad arguments and confusion, and that’s about it. His clarification is honestly just adding to the problem!
e.g. the first point is just dumb, sorry:
A REST API should not be dependent on any single communication protocol, though its successful mapping to a given protocol may be dependent on the availability of metadata, choice of methods
It’s abstraction when there’s no need for any. It doesn’t explain anything, and doesn’t produce useful engineering properties.
To clarify, this doesn’t contradict what I previously wrote, because the accepted meaning for REST is real, but Roy’s meaning isn’t! We fixed it for him by actually using it :)
bunch of nouns with fixed verbs […] This isn’t a trivial point
Precisely! It seems trivial, but turns out to be quite profound.
Separating the verbs from the names that they act on is very, very powerful, for one thing it actually enables the Universal Access Principle that OOP puts forward but fails to achieve (see Polymorphic Identifiers [pdf]). OOP was right in demanding the interfaces be polymorphic. It was wrong in its contention that therefore, interfaces must be procedural and state must be hidden. State access can be just as polymorphic, and when it is it is not just perfectly fine to have state-based interfaces, it is in many cases (far) superior.
Once you have that, you realise that it’s not just suitable for byte-oriented external communication, but can just as well be used in-process. Not having to access everything through the POSIX filesystem API (or HTTP) makes the Plan 9 idea of mediating access to most resources via these sorts of path-based interfaces a lot more palatable.
And of course the fact that you have only a very small number of verbs means that these elements highly composable. I call them Storage Combinators [pdf].
Finally, when you have language support for URI and scheme handlers and stores, you no longer have the architectural mismatch between procedural interfaces in your program and REST-based interfaces to the world. A mismatch that’s always painful and that is reasonable to try and resolve by making the external interface RPC-based (looking at you gRPC), despite the fact that we know it’s not a good idea.
IMHO, it’s better to resolve the mismatch by making the internal interfaces REST-based as well. This happens to be beneficial even when there were no mismatch to resolve, and when you get to use uniform REST-based APIs throughout your stack things flow really smoothly.
One thing that I haven’t seen mentioned explicitly yet, not even in Roy Fielding’s thesis: REST as a medium/design, or at least the HATEOS (Hypertext as the enigine of application state) part, assumes and serves human users who can read and understand the huge diversity of webpages.
Most parts of REST, you can implement them and get benefits without a human in the loop:
Stateless, self-contained requests simplify monitoring->visibility, error recovery, and implementation
->scalability
Caching is always useful
Uniform representation allows any browser to retrieve HTTP and render HTML
BUT. The thing Roy Fielding is trying to explain is why Web technology is so good that a user can use the same browser for webmail, online banking, and buying books, despite these being completely different tasks. This requires
A medium that need not know about these tasks… (Uniform representation allows any browser to retrieve HTTP and render HTML)
…that can feed the information and available actions…
(HTML allows representing nearly any content in a way human readers can understand.)
(HATEOAS says: also represent the available commands / state transitions, so the user doesn’t have to play guess-the-verb)
…to someone that knows how to do any of these tasks: a human.
Wikipedia, Stack Overflow, banks, webmail, do NOT have a uniform task or representation between them
but humans can read and understand any and all of these interfaces
and have a goal in mind, and heuristics on how to get there
and can pathfind their way across links and actions from here (the current state they see represented) to their goal.
There is no point in representing a multitude of resource types if your user cannot understand them. REST/HATEOAS is for web resources used interactively by humans.
Very well put! True REST/HATEOAS adds unnecessary baggage to an RPC and if I’m company A and I want to make a request to company B whenever my users do X, that’s RPC, and I’d like a stable interface, and not REST. On the other hand, for users, REST is great because you can just click the X button, and who cares how it works behind the scenes.
I don’t know where the acronym first appeared, but what this article says is that Rest is just whatever technology makes a bunch of bytes remotely retrievable and usable by a browser.
In short, HTTP, HTML with a bunch of expectations on what happens when you follow/click on a link, submit a form, etc.
In that case, it doesn’t even make sense to start talking about rest APIs, but wasn’t exactly it there where the acronym sprang to existence? (honest question)
I suppose the reason why they’re not commonly referred to as HTTP APIs, is simply to exclude other styles like soap, xml-rpc, graphql.
An API consumer does specific things, things that the API explicitly supports. It is the opposite case of a browser. By design, a browser shows HTML content and lets the human consume and interact with it as they wish at the moment.
Obviously, if the consumer is a computer program, the author of such program must define ahead of time what the program does.
The article mentioned a web scraper and that’s no coincidence. That is the closest (and only?) Use case of a program doing generic things on API responses. Other than that, a consumer is interesting in doing specific things. It is silly to retrieve a page to follow a specific link if you just know thr link ahead of time.
The article repeatedly refers to a thesis; Roy Fielding’s Ph.D thesis, where he coined the REST and HATEOAS acronyms and described their respective concepts.
The ‘industry’ connotation of REST/restful has really run away from the original ideas of the thesis. I currently hear folks say REST API when describing something closer to JSON RPC over HTTP.
One of the programs I’ve developed at work used to have an HTTP API (not JSON API, JSON RPC, HTTP RPC, REST etc). When you sent data that couldn’t be validated but was structurally valid, you received a 422, and maybe the body as an error message. Apparently the “correct” approach is to always return a 200 OK, with JSON { "valid": false } if the data couldn’t be validated…
That latter approach is what you do when you have incompetent API clients writing programs in languages that raise exceptions on non-200 responses but you really need them to actually read the error messages you’re giving them.
Yeah I agree with your comment there, except I wouldn’t call it “pop culture”
I would just say architecture should be descriptive of the world, not prescriptive, and Roy’s definition of REST has been passed by and is no longer useful
In that case, it doesn’t even make sense to start talking about rest APIs, but wasn’t exactly it there where the acronym sprang to existence? (honest question)
HTTP+hypermedia formats offer a way to do things like, say, provide a list of types of available resources, and then drill down into one of those for a list of available instances of that resource, and then operations available on those instances, etc.
The thing is that the native way to do this is “old-fashioned” HTML: a list of links to different types of resources. Click one, get a list of instances of that resource, and links to view, edit, delete, create new, etc.
You can kinda-sorta emulate this with JSON as the data transfer format and put JSON representations in there with URLs to navigate to, but as with the HTML + links version, it requires some intelligent entity on the client side to read those JSON structures and make sense of what’s being offered by them.
And that’s really what trips people up with trying to do “REST APIs” – you still need to effectively hard-code some amount of knowledge and logic into an automated client, and that client will need to be maintained in parallel with whatever it consumes (since the link structures might change in a way an intelligent agent could navigate, but an automated client would keep trying to navigate the way it was coded to navigate, and fail).
But by using HTTP+hypermedia formats in a web-native way, you can produce a setup where the process of writing the “API client” mostly consists of recording what look like normal web-browsing events: fetch this URL, follow this link, fill out this form.
This means that if you do ever build a properly theoretically pure “REST API” and client, it’s going to look an awful lot like a traditional website being hit by a screen scraper.
So. What if that’s really what we should’ve been doing all along – building human-usable systems where the automated client consists of, basically, recorded macros generated from browsing sessions, rather than trying to build “machine readable” APIs that automated clients will somehow magically understand and navigate without needing human intervention (an effort that has largely failed)?
the thesis is confusing: hypermedia as the whatsit of thingy
Overall the REST push was good for connected systems because what came before it was really quite bad. When people first wanted to argue these things I read Fielding’s paper figuring the appeal to their authority would settle things quickly. It is so much more about HTTP being a universal protocol relative to what came before it. There was definitely discussion of hypermedia but what I felt like I actually learned from the paper was that HTML was created in service of HTTP as much or more than HTTP was created in service of HTML. Since the HTTP side of the paper was proven out and has thus become irrelevant to most people the hypermedia side is all anyone really wants to argue about, which is a big part of why later converts to REST dive so hard into hypermedia.
It’s interesting that the acronym “REST” changed its meaning from Fielding’s original design to contemporary “JSON over HTTP”, which is almost the exact opposite of the original intention (this is not to say that one approach is better than the other).
I found this article a bit painful to read, and there are some inaccuracies. Namely REST is similar to Plan 9/Unix in that it has the “Uniform Interface Constraint”
https://en.wikipedia.org/wiki/Representational_state_transfer
That is basically “everything is a file” – modeling APIs as a bunch of nouns with fixed verbs - GET, POST, PUT, DELETE
Rather than an open-ended set of verbs:
GetIssue(),GetUser(),UpdateUser(), etc.it’s
GET /issue/$x,GET /user/$y,POST /user/$y, etc.This isn’t a trivial point, and IME most people try to do the former before they get the latter.
In Unix,
open() read() write() close()works on plain files, but also devices like/dev/memand/dev/ttyetc.However what I will say is that “HATEOS” is confusing and I agree it’s not always followed in practice.
The issue there is:
sourcehut.org/issues/32sourcehut.org/issue-list/, for an opaque URL to issue 32, and then retrieve that URL (whatever it is)What that does is decouple the client and server – the client only has to know the home page, and then it can dynamically discover other things. The server can upgrade without breaking the client.
This is useful up to a point, but obviously after awhile everyone is just going to jump directly to
sourcehut.org/issues/32for reasons of performance.I agree REST has a lot of confusing terminology, but it’s real, and it makes things even worse to say “restful is a scam”. I think programmers are sometimes not comfortable with fuzzy architecture. REST is a bit fuzzy, but it doesn’t mean it’s not real.
Re: your HATEOAS example: I’d say the shibboleth is not how the client goes where they go, but how the application showed them where they are and where they can go. In other words, does the application (a) REST: Transfer a REpresentation of the application State in hypertext and (b) HATEOAS: in that same hypertext show what the user can do to change the state or the view (so that the Hypertext, like an Engine, can drive the Application State)
Some examples, increasingly less Web-like:
[Add Comment](POST /issues/32/comment)?[Comment](POST /Comment), whose response depends on some invisible (outside the hypertext!) state ‘current issue=32’?POST /Commentrequires knowing session state that is not present in the hypertext. Imagine having two issue pages open at once… Or imagine a statefulGET /NextIssue: such a URL could not be cached.Type=command,Length=20:NEXTISSUE /issue/32\0?Roy himself posted a nice clarification:
https://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven
It’s amusing how all of his points are almost exact opposite of typical “RESTful” APIs.
Yeah, I think it’s basically his “fault”. After 15 years, I’d boil it down to the adage
I read his thesis more than 10 years ago, and probably read that blog post when he published it in 2008.
I largely think it’s not a useful framing, and the world has passed it by. REST means something else now.
These ideas are interesting and fundamental to the web:
But we never needed REST to explain them.
I don’t know how the term came to be common, but it doesn’t really matter either way.
I kinda wish Tim Berners-Lee had written an authoritative architectural description of the web, and brushed Fielding’s aside.
It just seems to cause bad arguments and confusion, and that’s about it. His clarification is honestly just adding to the problem!
e.g. the first point is just dumb, sorry:
It’s abstraction when there’s no need for any. It doesn’t explain anything, and doesn’t produce useful engineering properties.
To clarify, this doesn’t contradict what I previously wrote, because the accepted meaning for REST is real, but Roy’s meaning isn’t! We fixed it for him by actually using it :)
Roy’s REST is basically HTML and browsers. That abstraction sorta works over file:// too!
So REST works and is widely used, just not for APIs.
Yeah that’s a decent point, although POST PUT DELETE don’t work on file://
My main beef is that the way he writes about it doesn’t help at all
It’s very prescriptive, not descriptive
He could have said that “APIs aren’t something REST applies to”, but he didn’t
So he’s prescribing basically the wrong thing, so his definition of REST is left in the dustbin of history :)
Precisely! It seems trivial, but turns out to be quite profound.
Separating the verbs from the names that they act on is very, very powerful, for one thing it actually enables the Universal Access Principle that OOP puts forward but fails to achieve (see Polymorphic Identifiers [pdf]). OOP was right in demanding the interfaces be polymorphic. It was wrong in its contention that therefore, interfaces must be procedural and state must be hidden. State access can be just as polymorphic, and when it is it is not just perfectly fine to have state-based interfaces, it is in many cases (far) superior.
Once you have that, you realise that it’s not just suitable for byte-oriented external communication, but can just as well be used in-process. Not having to access everything through the POSIX filesystem API (or HTTP) makes the Plan 9 idea of mediating access to most resources via these sorts of path-based interfaces a lot more palatable.
And of course the fact that you have only a very small number of verbs means that these elements highly composable. I call them Storage Combinators [pdf].
Finally, when you have language support for URI and scheme handlers and stores, you no longer have the architectural mismatch between procedural interfaces in your program and REST-based interfaces to the world. A mismatch that’s always painful and that is reasonable to try and resolve by making the external interface RPC-based (looking at you gRPC), despite the fact that we know it’s not a good idea.
IMHO, it’s better to resolve the mismatch by making the internal interfaces REST-based as well. This happens to be beneficial even when there were no mismatch to resolve, and when you get to use uniform REST-based APIs throughout your stack things flow really smoothly.
Another lovely post by Tef.
One thing that I haven’t seen mentioned explicitly yet, not even in Roy Fielding’s thesis: REST as a medium/design, or at least the HATEOS (Hypertext as the enigine of application state) part, assumes and serves human users who can read and understand the huge diversity of webpages.
Most parts of REST, you can implement them and get benefits without a human in the loop:
BUT. The thing Roy Fielding is trying to explain is why Web technology is so good that a user can use the same browser for webmail, online banking, and buying books, despite these being completely different tasks. This requires
There is no point in representing a multitude of resource types if your user cannot understand them. REST/HATEOAS is for web resources used interactively by humans.
Very well put! True REST/HATEOAS adds unnecessary baggage to an RPC and if I’m company A and I want to make a request to company B whenever my users do X, that’s RPC, and I’d like a stable interface, and not REST. On the other hand, for users, REST is great because you can just click the X button, and who cares how it works behind the scenes.
I don’t know where the acronym first appeared, but what this article says is that Rest is just whatever technology makes a bunch of bytes remotely retrievable and usable by a browser. In short, HTTP, HTML with a bunch of expectations on what happens when you follow/click on a link, submit a form, etc.
In that case, it doesn’t even make sense to start talking about rest APIs, but wasn’t exactly it there where the acronym sprang to existence? (honest question)
I suppose the reason why they’re not commonly referred to as HTTP APIs, is simply to exclude other styles like soap, xml-rpc, graphql.
An API consumer does specific things, things that the API explicitly supports. It is the opposite case of a browser. By design, a browser shows HTML content and lets the human consume and interact with it as they wish at the moment. Obviously, if the consumer is a computer program, the author of such program must define ahead of time what the program does.
The article mentioned a web scraper and that’s no coincidence. That is the closest (and only?) Use case of a program doing generic things on API responses. Other than that, a consumer is interesting in doing specific things. It is silly to retrieve a page to follow a specific link if you just know thr link ahead of time.
The article repeatedly refers to a thesis; Roy Fielding’s Ph.D thesis, where he coined the REST and HATEOAS acronyms and described their respective concepts.
https://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm
The ‘industry’ connotation of REST/restful has really run away from the original ideas of the thesis. I currently hear folks say REST API when describing something closer to JSON RPC over HTTP.
One of the programs I’ve developed at work used to have an HTTP API (not JSON API, JSON RPC, HTTP RPC, REST etc). When you sent data that couldn’t be validated but was structurally valid, you received a 422, and maybe the body as an error message. Apparently the “correct” approach is to always return a 200 OK, with JSON
{ "valid": false }if the data couldn’t be validated…That latter approach is what you do when you have incompetent API clients writing programs in languages that raise exceptions on non-200 responses but you really need them to actually read the error messages you’re giving them.
[Comment removed by author]
Previously: https://lobste.rs/s/ngiz7c/restful_is_antimemetic_least_successful
Yeah I agree with your comment there, except I wouldn’t call it “pop culture”
I would just say architecture should be descriptive of the world, not prescriptive, and Roy’s definition of REST has been passed by and is no longer useful
https://lobste.rs/s/5wnigy/restful_is_scam_browsers_are_cool#c_oyqb1t
Sure, but if you do that it’s an RPC API not hypermedia.
HTTP+hypermedia formats offer a way to do things like, say, provide a list of types of available resources, and then drill down into one of those for a list of available instances of that resource, and then operations available on those instances, etc.
The thing is that the native way to do this is “old-fashioned” HTML: a list of links to different types of resources. Click one, get a list of instances of that resource, and links to view, edit, delete, create new, etc.
You can kinda-sorta emulate this with JSON as the data transfer format and put JSON representations in there with URLs to navigate to, but as with the HTML + links version, it requires some intelligent entity on the client side to read those JSON structures and make sense of what’s being offered by them.
And that’s really what trips people up with trying to do “REST APIs” – you still need to effectively hard-code some amount of knowledge and logic into an automated client, and that client will need to be maintained in parallel with whatever it consumes (since the link structures might change in a way an intelligent agent could navigate, but an automated client would keep trying to navigate the way it was coded to navigate, and fail).
But by using HTTP+hypermedia formats in a web-native way, you can produce a setup where the process of writing the “API client” mostly consists of recording what look like normal web-browsing events: fetch this URL, follow this link, fill out this form.
This means that if you do ever build a properly theoretically pure “REST API” and client, it’s going to look an awful lot like a traditional website being hit by a screen scraper.
So. What if that’s really what we should’ve been doing all along – building human-usable systems where the automated client consists of, basically, recorded macros generated from browsing sessions, rather than trying to build “machine readable” APIs that automated clients will somehow magically understand and navigate without needing human intervention (an effort that has largely failed)?
Amen!
Overall the REST push was good for connected systems because what came before it was really quite bad. When people first wanted to argue these things I read Fielding’s paper figuring the appeal to their authority would settle things quickly. It is so much more about HTTP being a universal protocol relative to what came before it. There was definitely discussion of hypermedia but what I felt like I actually learned from the paper was that HTML was created in service of HTTP as much or more than HTTP was created in service of HTML. Since the HTTP side of the paper was proven out and has thus become irrelevant to most people the hypermedia side is all anyone really wants to argue about, which is a big part of why later converts to REST dive so hard into hypermedia.
https://bower.sh/dogma-of-restful-api
It’s interesting that the acronym “REST” changed its meaning from Fielding’s original design to contemporary “JSON over HTTP”, which is almost the exact opposite of the original intention (this is not to say that one approach is better than the other).