This crate should really be more well-known in the Rust ecosystem! The following quote on OpenAPI support resonates a lot with me (source):
[An] important goal for us was to build something with strong OpenAPI support, and particularly where the code could be the source of truth and a spec could be generated from the code that thus could not diverge from the implementation. We weren’t sure that would be possible, but it seemed plausible and worthwhile if we could do it. None of the crates we found had anything like this.
I use something like this in the Go ecosystem, and generating an OpenAPI spec from source code is much better than generating source code from an OpenAPI spec.
One under-appreciated reason why it’s better: you don’t have to handle the entirely of the OpenAPI spec, just the features your server framework wants to produce. We took advantage of this by also writing our own code gen for clients too, that’s also easier to make good because you only need to handle that same subset.
Happy to answer questions about Dropshot, I use it every day.
Specifically, Dropshot can produce OpenAPI documents, and then https://github.com/oxidecomputer/oxide.ts can generate a typescript client from that diagram. I personally am also using sqlx, so I get types the whole way from my database up through into the browser.
It’s not a perfect web server, but it serves us really well.
A common pattern I see in HTTP APIs is to treat responses like sum types where the status code determines the structure of the body and so on. For example, the Matrix protocol’s media APIs use 200 when media can be served directly and 307/308 if a redirect is necessary. Is it possible to model this in Dropshot such that it’s reflected in the generated OpenAPI document?
It seems like it probably isn’t, because each request handler function is permitted to have exactly 1 successful response code and several error codes (which must be in the 4XX and 5XX range, see ErrorStatusCode), and each error code shares the same body structure. Looking at the API design, ApiEndpointResponse has an Option<StatusCode> field, HttpCodedResponse has a StatusCode associated constant, and HttpResponseError only has a method for getting the current status code of an error response value. Looking at the generated OpenAPI document for the custom-error example seems to support this conclusion. Am I missing something?
Not 100% sure but I believe that’s true today, yes – HttpCodedResponse has a const status code.
Modeling this is an interesting challenge – I can imagine accepting an enum with variants and then producing the corresponding sum type in the OpenAPI document.
Yeah, that’s the solution I came up with for a similar project I attempted that was more warp-shaped than axum-shaped; here’s an example. The derive macro generates metadata about the response variants and uses the variant names to decide the HTTP status code to use. Maybe useful as prior art, I dunno. The obvious downside is that ? won’t work in this situation without either Try stabilizing or defining a second function where ? works and doing some mapping in the handler function from that function.
In the design I posted, there aren’t really “successful” or “unsuccessful” responses, just responses. Responses of all status codes go into the same enum, and each handler function directly returns such an enum, not wrapped in Result or anything. So if you want to use ? to e.g. handle errors, you have to define a separate function that returns e.g. Result which uses ? internally, and then call that function in the handler function and then use match or similar to convert the Result‘s inner values into your handler function’s enum.
I believe that’s a long winded way of answering “yes”, but I thought I’d elaborate further to try to make it clearer what I was trying to say originally just in case.
Ah I see. What do you think of separating out successful and unsuccessful variants into a result type? 2xx and 3xx on one side, 4xx and 5xx on the other.
That could work. I think doing it that way could be more convenient for users because ? would be usable (though probably still require some map and map_err), but at the cost of some library-side complexity. Using a single enum obviates the need for categorizing status codes. Dropshot has already solved that problem with its ErrorStatusCode (but you’ll probably also need a SuccessStatusCode, which I’m not sure currently exists).
Personally, I don’t think either way would make much difference for me as user. When working with HTTP libraries, I generally implement actual functionality in separate functions from request handlers anyway, so that such functions have no knowledge of any HTTP stuff. There are various reasons for this, the relevant one being that this minimizes the amount of code that has to actually care how the HTTP library is designed. Not everyone operates this way, though.
Something about the headline doesn’t quite click for me: is this a web framework? Namely, does it provide the web server loop as well? Or is it just related to the data model mapping between types and endpoints?
Yeah, it has the web server loop too. It’s maybe a bit too bare bones to be a “framework,” like it’s closer to a flask/sinatra than a Django/Rails. But it’s focused on “I want to produce a JSON API with OpenAPI” as a core use case.
Happy to answer questions about Dropshot as well. Over the last year one of my contributions to it was support for APIs defined via Rust async traits (RFD 479). This came soon after async traits were first stabilized in Rust, and was (I believe) one of the first major uses of them. I hope Rust projects more generally adopt this pattern, since it helps extract API information without needing to compile (or even have) a concrete implementation at hand.
I’m into it. I’m a big fan of the typestate pattern, and even though this can feel a bit repetitive for endpoints with less logic, I like that it’s so straightforward. No more worrying about the order various handlers run…
Interesting. So is the idea with regards to typestate that you’d ensure that your routes/apis do X and Y and Z steps before calling into some function F(DidZ) ?
I haven’t open sourced my codebase yet, but yeah. So like, instead of saying “this API call is guarded by a is_logged_in? handler, my “save this Foo to the database” function requires an Authorization struct. How can you get one of those? Well, the only way is to call into the authorization subsystem. And doing that requires a User. And you can only get a User by calling into the authentication subsystem. And that happens to take a request context.
You can see how a lot of these handlers have the general form “grab a nexus instance from the Dropshot context, construct a Context from the request context, then call some nexus method, passing the context in.” Same basic idea, except a little cleaner; I’m sort of in a “embrace a little more boilerplate than I’m used to” moment and so rather than the Context stuff I’m doing the same idea but a bit more “inline” in the handlers. I might remove that duplication soon but I want to sit with it a bit more before I overly refactor.
Anyway, I think that in Nexus, that these handlers have the same sort of shape but are also a bit different is the strength of this approach. I’ve worked on rails apps where the before, after, and around request middleware ended up with subtle dependencies between them, and ordering issues, and “do this 80% of the time but not 20% of the time” kinds of things. Doing stuff this way eliminates all of that; it’s just normal code that you just read in the handler. I’ve also found that this style is basically the “skinny controller” argument from back in the day, and comes with the same benefits. It’s easier to test stuff without needing Dropshot at all, since Dropshot itself is really not doing any business logic whatsoever, which middlewares can often end up doing.
Is there a specific reason the README is worded in a way that people (see someone’s comment here) have to ask: Is this a web framework?
Might be nitpicky, but when I just saw this, I also thought of something slightly different. And yes, of course web framework is just as loose a term as any. Kinda glad the README is not “THIS IS THE NEW BEST WEB FRAMEWORK”, although that would have given away that a) it is indeed a web framework and b) it’s probably not the best.
I think it reminded me of https://docs.postgrest.org/en/v12/ - the “ turns your PostgreSQL database directly into a RESTful API” part, but I’m not claiming I make sense :P
This crate should really be more well-known in the Rust ecosystem! The following quote on OpenAPI support resonates a lot with me (source):
I use something like this in the Go ecosystem, and generating an OpenAPI spec from source code is much better than generating source code from an OpenAPI spec.
One under-appreciated reason why it’s better: you don’t have to handle the entirely of the OpenAPI spec, just the features your server framework wants to produce. We took advantage of this by also writing our own code gen for clients too, that’s also easier to make good because you only need to handle that same subset.
Happy to answer questions about Dropshot, I use it every day.
Specifically, Dropshot can produce OpenAPI documents, and then https://github.com/oxidecomputer/oxide.ts can generate a typescript client from that diagram. I personally am also using sqlx, so I get types the whole way from my database up through into the browser.
It’s not a perfect web server, but it serves us really well.
A bit of a random one, but is the name a reference to dropwizard by any chance?
Fishpong is https://github.com/fishpong/docs/wiki/Primer, it’s a ping-pong variant that a lot of folks at Oxide love. “drop shot” is a concept that’s across various racket sports. https://en.wikipedia.org/wiki/Drop_shot
It also has the advantage of being short and wasn’t taken at the time.
I’m not sure! I’ll ask.
A common pattern I see in HTTP APIs is to treat responses like sum types where the status code determines the structure of the body and so on. For example, the Matrix protocol’s media APIs use 200 when media can be served directly and 307/308 if a redirect is necessary. Is it possible to model this in Dropshot such that it’s reflected in the generated OpenAPI document?
It seems like it probably isn’t, because each request handler function is permitted to have exactly 1 successful response code and several error codes (which must be in the 4XX and 5XX range, see
ErrorStatusCode), and each error code shares the same body structure. Looking at the API design,ApiEndpointResponsehas anOption<StatusCode>field,HttpCodedResponsehas aStatusCodeassociated constant, andHttpResponseErroronly has a method for getting the current status code of an error response value. Looking at the generated OpenAPI document for the custom-error example seems to support this conclusion. Am I missing something?Not 100% sure but I believe that’s true today, yes –
HttpCodedResponsehas aconststatus code.Modeling this is an interesting challenge – I can imagine accepting an enum with variants and then producing the corresponding sum type in the OpenAPI document.
Yeah, that’s the solution I came up with for a similar project I attempted that was more warp-shaped than axum-shaped; here’s an example. The derive macro generates metadata about the response variants and uses the variant names to decide the HTTP status code to use. Maybe useful as prior art, I dunno. The obvious downside is that
?won’t work in this situation without eitherTrystabilizing or defining a second function where?works and doing some mapping in the handler function from that function.re
?not working, you can always map a successful response into the corresponding enum variant explicitly, right?In the design I posted, there aren’t really “successful” or “unsuccessful” responses, just responses. Responses of all status codes go into the same enum, and each handler function directly returns such an enum, not wrapped in
Resultor anything. So if you want to use?to e.g. handle errors, you have to define a separate function that returns e.g.Resultwhich uses?internally, and then call that function in the handler function and then usematchor similar to convert theResult‘s inner values into your handler function’s enum.I believe that’s a long winded way of answering “yes”, but I thought I’d elaborate further to try to make it clearer what I was trying to say originally just in case.
Ah I see. What do you think of separating out successful and unsuccessful variants into a result type? 2xx and 3xx on one side, 4xx and 5xx on the other.
That could work. I think doing it that way could be more convenient for users because
?would be usable (though probably still require somemapandmap_err), but at the cost of some library-side complexity. Using a single enum obviates the need for categorizing status codes. Dropshot has already solved that problem with itsErrorStatusCode(but you’ll probably also need aSuccessStatusCode, which I’m not sure currently exists).Personally, I don’t think either way would make much difference for me as user. When working with HTTP libraries, I generally implement actual functionality in separate functions from request handlers anyway, so that such functions have no knowledge of any HTTP stuff. There are various reasons for this, the relevant one being that this minimizes the amount of code that has to actually care how the HTTP library is designed. Not everyone operates this way, though.
Something about the headline doesn’t quite click for me: is this a web framework? Namely, does it provide the web server loop as well? Or is it just related to the data model mapping between types and endpoints?
Yeah, it has the web server loop too. It’s maybe a bit too bare bones to be a “framework,” like it’s closer to a flask/sinatra than a Django/Rails. But it’s focused on “I want to produce a JSON API with OpenAPI” as a core use case.
Happy to answer questions about Dropshot as well. Over the last year one of my contributions to it was support for APIs defined via Rust async traits (RFD 479). This came soon after async traits were first stabilized in Rust, and was (I believe) one of the first major uses of them. I hope Rust projects more generally adopt this pattern, since it helps extract API information without needing to compile (or even have) a concrete implementation at hand.
How has this design choice played out? It’s been a few years, I’m curious to hear lessons learned.
I’m into it. I’m a big fan of the typestate pattern, and even though this can feel a bit repetitive for endpoints with less logic, I like that it’s so straightforward. No more worrying about the order various handlers run…
Interesting. So is the idea with regards to typestate that you’d ensure that your routes/apis do X and Y and Z steps before calling into some function F(DidZ) ?
I haven’t open sourced my codebase yet, but yeah. So like, instead of saying “this API call is guarded by a is_logged_in? handler, my “save this Foo to the database” function requires an Authorization struct. How can you get one of those? Well, the only way is to call into the authorization subsystem. And doing that requires a User. And you can only get a User by calling into the authentication subsystem. And that happens to take a request context.
Nexus (the control plane API) does it slightly differently, but they have significantly more complexity to their auth than I do. See 24-39 here: https://github.com/oxidecomputer/omicron/blob/main/nexus/auth/src/context.rs
And some examples of using it here: https://github.com/oxidecomputer/omicron/blob/main/nexus/src/external_api/http_entrypoints.rs
You can see how a lot of these handlers have the general form “grab a nexus instance from the Dropshot context, construct a Context from the request context, then call some nexus method, passing the context in.” Same basic idea, except a little cleaner; I’m sort of in a “embrace a little more boilerplate than I’m used to” moment and so rather than the Context stuff I’m doing the same idea but a bit more “inline” in the handlers. I might remove that duplication soon but I want to sit with it a bit more before I overly refactor.
Anyway, I think that in Nexus, that these handlers have the same sort of shape but are also a bit different is the strength of this approach. I’ve worked on rails apps where the before, after, and around request middleware ended up with subtle dependencies between them, and ordering issues, and “do this 80% of the time but not 20% of the time” kinds of things. Doing stuff this way eliminates all of that; it’s just normal code that you just read in the handler. I’ve also found that this style is basically the “skinny controller” argument from back in the day, and comes with the same benefits. It’s easier to test stuff without needing Dropshot at all, since Dropshot itself is really not doing any business logic whatsoever, which middlewares can often end up doing.
Yep, perfect. Makes sense to me. I’m wondering about composability though. (edit: nvm, code explained my questions)
Is there a specific reason the README is worded in a way that people (see someone’s comment here) have to ask: Is this a web framework?
Might be nitpicky, but when I just saw this, I also thought of something slightly different. And yes, of course web framework is just as loose a term as any. Kinda glad the README is not “THIS IS THE NEW BEST WEB FRAMEWORK”, although that would have given away that a) it is indeed a web framework and b) it’s probably not the best.
I’ve never seen this reaction before, so I’m not sure. Worth thinking about.
I think it reminded me of https://docs.postgrest.org/en/v12/ - the “ turns your PostgreSQL database directly into a RESTful API” part, but I’m not claiming I make sense :P
Interesting. I love REST Api.