These are some great general guidelines; there isn’t one I didn’t nod my head at.
Building a web API isn’t easy. One of the things I pushed for at Zapier was to publish a best-practices guide. It never got prioritized, but I’ve always thought they were in a great position to put that together with authority. We really saw everything there - the good, the bad, and the ugly. For everything that made sense, there were weird combinations of query params that would cancel each other out, or disappearing response fields, or APIs that didn’t follow HTTP specs, but we had to adjust our end to get it working.
Anyway, hopefully a definitive guide like that exists some day. Might do some folks good (if they read it).
This advice all looks great. One thing to consider for the ‘someone’s ESP8266 buried in a forest’ use case is that there’s nothing stopping someone proxying your API. For example, a number of IoT devices with 10+ year expected deployment lifetimes are now starting to think about post-quantum crypto. Unfortunately, there isn’t yet a post-quantum key exchange protocol that’s standardised and robust and the ones that do exist are very memory intensive. The typical proposed solution is to implement such a protocol in software at boot and use it to exchange a per-device AES key. The device can then use TLS with a pre-shared key. This is quite easy to support in a proxy but you almost certainly don’t want to support PSK for your API. At the same time, if there is a proxy then your API stability guarantees can be weaker - as long as you can implement the old API in terms of the new one, the proxy can handle the difference.
I tend to like to recommend https://aip.dev as a great recommendations set with lots of hard-learnt lessons baked in. (Though many of the ones in the OP link are not there yet.)
Don’t avoid it dogmatically, either. My team pumped the brakes on an ADR that was essentially reinventing OAuth and just did OAuth instead. The server-side implementation is quite simple if you don’t need all the bells and whistles.
Host the API on its own hostname
Serve your API at api.example.com, never at example.com/api.
That makes sense, however. For $WORK we have api.$WORK as the main published api, but the web-stuff has $WORK/api as alias available. Using that alias from the browser saves all those CORS requests and that absolutely makes a huge difference in perceived speed.
I just truncate the message to 1,024 characters and process it anyway (assuming it wasn’t so large that it hit the web server’s request message size limit). The user still receives something, and if they care that it’s truncated, they can investigate why.
If this guy thinks it’s anything but completely fucked up to deliberately drop customer data and make them “investigate why” then I won’t ever touch one of his APIs.
This entirely depends on the service though, and they are describing a service for push notifications. I would say that delivering a truncated message is better than not delivering it at all in that case. If the data is supposed to be stored long term then it’s another matter.
He could be hardline and 400 all requests that don’t conform, and I bet that was the case, originally.
I’m curious what you would do? How would you balance correctness, support concerns / customer happiness, operational concerns to protect the service (eg rate limiting of legitimately bad/abusive requests), and all the other things as a single person providing this service?
I’m curious what you would do? How would you balance correctness, support concerns / customer happiness, operational concerns to protect the service (eg rate limiting of legitimately bad/abusive requests), and all the other things as a single person providing this service?
It all depends on what the API is doing.
It looks like the messages in question are basically status updates delivered to end-user devices. I guess it’s unlikely that the message consumers will be doing anything other than printing them to a screen, and it’s entirely possible that message producers wouldn’t know about the size limit for a single recipient. In this (specific) case, truncating the message is at least arguably beneficial vs. rejecting it outright.
But this feels to me like an exceptional use case. If these messages were expected to be machine-readable, or any of a hundred other variables were different, then truncating them would make them unreliable, likely useless. As you note, you usually want to reject bad requests (with a 400 or whatever) by default, and carve out exceptions based on use case.
I drew the same conclusion as you and the sibling. I want to know what @caboteria thinks since they are hardline against it. I am assuming they didn’t think much about the nuance of the problem and stopped at “truncate” before aggressively stomping their foot and writing a mean spirited comment void of any substance.
TBH, I wasn’t familiar with the Pushover service so when I read that data gets truncated I recoiled a bit. @caboteria might have been similarly ignorant. That section might be a bit better with some context on the service.
Great post! Always wondered what’s up with the prefix tokens, good to know how they’re used :)
When a failure count is nearing the hard block limit, Pushover’s API will respond with a 200 status code
Why not go with 429 Too Many Requests directly in this case? responding with a 200 will make the code inspect the body while dealing with these in status code seems more ideal. I think jcs is assuming that the programmer will be well behaved and use a client side rate limiting, in which case what’s written in the post makes more sense. Also does hard block here mean IP based blocking once it hits API limits?
The API already starts returning 429s once the application reaches its API limit. If it continues trying to send at a fast rate, then the IP gets soft blocked and receives 429s for any API call it makes until the block is lifted. If it keeps sending quickly while soft blocked then it starts receiving 200s as a last ditch effort, then it gets hard blocked completely from connecting.
Also does hard block here mean IP based blocking once it hits API limits?
Only if it keeps trying to send messages at a fast rate. If it keeps trying at a normal pace it will just get 429s until its API limits reset.
These are some great general guidelines; there isn’t one I didn’t nod my head at.
Building a web API isn’t easy. One of the things I pushed for at Zapier was to publish a best-practices guide. It never got prioritized, but I’ve always thought they were in a great position to put that together with authority. We really saw everything there - the good, the bad, and the ugly. For everything that made sense, there were weird combinations of query params that would cancel each other out, or disappearing response fields, or APIs that didn’t follow HTTP specs, but we had to adjust our end to get it working.
Anyway, hopefully a definitive guide like that exists some day. Might do some folks good (if they read it).
Oh man, Zapier definitely would be in a position to write wine sommelier style guide to HTTP APIs: this one is nutty with hints of blood.
This advice all looks great. One thing to consider for the ‘someone’s ESP8266 buried in a forest’ use case is that there’s nothing stopping someone proxying your API. For example, a number of IoT devices with 10+ year expected deployment lifetimes are now starting to think about post-quantum crypto. Unfortunately, there isn’t yet a post-quantum key exchange protocol that’s standardised and robust and the ones that do exist are very memory intensive. The typical proposed solution is to implement such a protocol in software at boot and use it to exchange a per-device AES key. The device can then use TLS with a pre-shared key. This is quite easy to support in a proxy but you almost certainly don’t want to support PSK for your API. At the same time, if there is a proxy then your API stability guarantees can be weaker - as long as you can implement the old API in terms of the new one, the proxy can handle the difference.
I tend to like to recommend https://aip.dev as a great recommendations set with lots of hard-learnt lessons baked in. (Though many of the ones in the OP link are not there yet.)
Don’t avoid it dogmatically, either. My team pumped the brakes on an ADR that was essentially reinventing OAuth and just did OAuth instead. The server-side implementation is quite simple if you don’t need all the bells and whistles.
That makes sense, however. For $WORK we have api.$WORK as the main published api, but the web-stuff has $WORK/api as alias available. Using that alias from the browser saves all those CORS requests and that absolutely makes a huge difference in perceived speed.
so $WORK/api points to api.$WORK right? can you share more on the reverse proxy setup used? made some notes around this here https://mogoz.geekodour.org/posts/20230302210256-cors/#when-use
Just the normal nxing we have running already anyway.
ok, it’s the weird kubernetes nginx version, but same idea:
If this guy thinks it’s anything but completely fucked up to deliberately drop customer data and make them “investigate why” then I won’t ever touch one of his APIs.
You might want to check the about page to see who created lobster.rs before saying you’ll not use his stuff 🙃
This entirely depends on the service though, and they are describing a service for push notifications. I would say that delivering a truncated message is better than not delivering it at all in that case. If the data is supposed to be stored long term then it’s another matter.
He could be hardline and 400 all requests that don’t conform, and I bet that was the case, originally.
I’m curious what you would do? How would you balance correctness, support concerns / customer happiness, operational concerns to protect the service (eg rate limiting of legitimately bad/abusive requests), and all the other things as a single person providing this service?
It all depends on what the API is doing.
It looks like the messages in question are basically status updates delivered to end-user devices. I guess it’s unlikely that the message consumers will be doing anything other than printing them to a screen, and it’s entirely possible that message producers wouldn’t know about the size limit for a single recipient. In this (specific) case, truncating the message is at least arguably beneficial vs. rejecting it outright.
But this feels to me like an exceptional use case. If these messages were expected to be machine-readable, or any of a hundred other variables were different, then truncating them would make them unreliable, likely useless. As you note, you usually want to reject bad requests (with a 400 or whatever) by default, and carve out exceptions based on use case.
edit: basically a +1 to Brekkjern’s sibling comment
I drew the same conclusion as you and the sibling. I want to know what @caboteria thinks since they are hardline against it. I am assuming they didn’t think much about the nuance of the problem and stopped at “truncate” before aggressively stomping their foot and writing a mean spirited comment void of any substance.
TBH, I wasn’t familiar with the Pushover service so when I read that data gets truncated I recoiled a bit. @caboteria might have been similarly ignorant. That section might be a bit better with some context on the service.
That’s a fair point!
Great post! Always wondered what’s up with the prefix tokens, good to know how they’re used :)
Why not go with 429 Too Many Requests directly in this case? responding with a 200 will make the code inspect the body while dealing with these in status code seems more ideal. I think jcs is assuming that the programmer will be well behaved and use a client side rate limiting, in which case what’s written in the post makes more sense. Also does hard block here mean IP based blocking once it hits API limits?
The API already starts returning 429s once the application reaches its API limit. If it continues trying to send at a fast rate, then the IP gets soft blocked and receives 429s for any API call it makes until the block is lifted. If it keeps sending quickly while soft blocked then it starts receiving 200s as a last ditch effort, then it gets hard blocked completely from connecting.
Only if it keeps trying to send messages at a fast rate. If it keeps trying at a normal pace it will just get 429s until its API limits reset.
I’ve missed those toasters. And stay off my lawn.