-
At this point we might believe that Rust just analyses the whole program and check the validity of all borrows when in assembles the program. But whole program analysis is costly and hard to communicate well (“hey, this borrow doesn’t work because something at the edge of nowhere”).
And it’s not what happens.
This was a real moment of clarity for me. I think I ultimately knew that my mental model of lifetimes (which was exactly what you wrote I “might believe” :P) was at least incomplete, since I haven’t understood exactly what the
'lifetimetags are doing. Having that explicitly shut down as wrong feels really freeing to actually understand how else the compiler is working it out!And I feel like I’ve now got a great start on that. Nice examples, clearly written and explained. Thanks for the great post!
-
The first time I released Monocypher, I was wildly over-confident:
Monocypher is probably already bug-free.
Something tells me this might be the second round of “wildly over-confident”
-
It’s not obvious from the way it’s styled, but that quote is Loup quoting themselves from that first time around, not a present claim. The text of that quote in the article links to its original context.
-
Sure but the line below implies he still feels that way.
my crypto library, is done and ready for production
He speaks about how auditing is important but nothing about how it has been done with his software. I’m sorry but if your crypto has not been audited it is not ready for production.
-
Oh, I’m on board with your point! …
we now have a crypto library that could displace Libsodium itself
And, re-parsing your comment now, I think I’m reading it the way you meant, which is not as unfair as the way I first understood it. I think the quote-in-quote threw me off. Sorry!
-
-
-
-
-
-
That’s cool but until it happens it’s pretty irresponsible to say that it’s production ready.
-
-
-
-
-
Yeah the ‘dominated by sponsorship’ thing is a total turn off for me as well.
However don’t paint all programming meetups with the same brush. The author was simply describing the less successful ones.
I would point to groups like Boston Python - high quality presentations, vibrant packed project nights, and organizers (like Ned Batchelder who exemplify the community spirit with their tireless efforts.
Also, with respect to the author, are you really belly aching because you didn’t get your free pizza? Seems a bit entitled and, forgive me, but obnoxious to me.
And maybe rather than slagging the meet ups, why not pitch in and make some awesome?
-
are you really belly aching because you didn’t get your free pizza? Seems a bit entitled and, forgive me, but obnoxious to me.
I don’t fully disagree, but a lot of events happen at night, after work. In NYC (at least), it’s super common to just stay at work a little longer before going to the meetup, because the trek home can be long. That means if you were told there’d be pizza, and there’s not, you haven’t eaten yet at 7:30. This isn’t fun.
-
Yeah, breaking a food expectation that people plan around is not great, though the OP’s phrasing might have been a bit obnoxious about it.
Beyond just expectation-setting, I think the free pizza we provided at my local code club was actually important (though certainly not essential) in a few other ways:
- We were mostly students on fairly tight food budgets. A free meal every week is a nice thing.
- The food’s arrival provides a shake-up part-way through the night. It gets people on their feet and moving around, talking to other people.
- It was a fun way to engage people who wandered by. “Have some pizza!” creates a space to talk for a bit. It’s hard to get new people (and especially new programmers!) to actually come out for their first time, so this sort of bypasses that.
…I may also just be post-rationalizing since, at least to me as an organizer, it was a small thing every week that kept me going!
-
I think you’re right and I totally recognize that the food is important. I guess you’ve also hit the nail on the head in that the author’s tone was a bit - I can’t think of any other way to say it than “entitled”.
“Gee all these people coming together and trying to create a community and spread ideas and lift each other up? It’s CRAP. And I didn’t get my free food!”.
I can totally sympathize with what everyone’s saying - going a long evening with no food is un-fun, but as the recipient of all this effort and care by the organizers, I think there are more gracious and helpful ways of dealing with that than the way the author chose to purvey.
-
-
-
I hope this isn’t too ranty or light on technical content for lobsters. I hope it’s a useful heads up to anyone who uses RDS for hobby projects and relies on the automatic snapshots as a backup strategy.
-
If the point of the api is to get data in an out of the db, you could use PostgREST (also haskell :))
-
-
That’s a dismissive comment. Have you actually looked at PostgREST?
Here’s what I’ve found from two months of hacking on a project built on it:
- Extend it: I haven’t found myself held back while adding features like OAuth for authentication and a Places Search that talks to Nominatium. I haven’t found anything about PostgREST that makes this harder. I still write back-end code, PostgREST just takes care of the data stuff.
- Gate access: PostgreSQL actually has great authorization features via roles, including Row Level Security. I’m actually finding myself feeling much happier about my the security of my data now that I’m writing rules in declarative SQL, instead of manually writing app logic. Recent comment
- do various other things and putting that logic: I mean, sure, I have a login function as a Stored Procedure. The horror! It’s 8 lines of nicely-formatted plpgsql. I have an autocomplete endpoint that does some pretty smart querying as a Stored Procedure, which is longer, but is just a plain parameterizied sql query.
I won’t be surprised if I hit pain points if this project becomes a hit and grows to be huge… but from here, that day feels like a long way away, and PostgREST feels really valuable.
(edited to clarify a point)
-
My comment was not intended as “the thing the article describes is not that good, try this”. The article describes a traditional way of building a rest api, it just happens to be in Haskell, nothing controversial about that. Since it’s traditional, of course it works and there is nothing wrong with it, but since the example shown was about how to get data in and out the db, it’s a good place to use PostgREST. It’s also ok that people dismiss it, the docs do not explain (yet) “the big picture” and how postgrest is only a part of the new stack, not the whole stack. The only objection i have to your comment is the part about “logic in the db” :) because that’s not dismissive of postgrest but of the databases in general (postgresql). Literally decades of work have gone into implementing views/stored procedures/triggers/role system/rls … and you are basically saying that was all for nothing since “putting that logic in your database is fraught” :) If the logic is very closely related to the data being handled, the database is exactly the place to put it.
-
-
-
I’ve been working on my latest project with PostgreSQL and PostgREST. No production experience yet, but it’s been a thorough (and ongoing) exercise in learning how to leverage roles for user-level authorization.
I keep meaning to write this stuff down as a blog post, but for now, an obnoxiously long comment is what I’m going with :)
Column security: I’m a fan. So far it has been easy to read, write, and reason about in SQL. For each table I create I’ve been doing pretty explicit column grants, like:
/* restrict even yourself from seeing access tokens */ GRANT SELECT (id, created, name, fb_id) ON TABLE api.members TO member; GRANT UPDATE (name) ON TABLE api.members TO member; /* let loggerinner do its thing */ GRANT SELECT ON TABLE api.members TO loggerinner; GRANT INSERT (fb_id, name, fb_token, fb_expires) ON TABLE api.members TO loggerinner; GRANT UPDATE (name, fb_token, fb_expires) ON TABLE api.members TO loggerinner;Row level security: So far I’ve only used it for the member system, but upcoming features are going to have more interesting privacy rules that will make me start using it more:
/* restrict members to only updating themselves */ ALTER TABLE api.members ENABLE ROW LEVEL SECURITY; CREATE POLICY own_account_only ON api.members USING (current_user <> 'anon') WITH CHECK (current_user = 'loggerinner' -- loggerinner upserts OR id = auth.user_id()); -- update yourselfIt’s nice how the row and column rules combine, so the RLS allowing people to create/update their own rows only actually lets them update their name.
However….
Views are footgunsIn postgres, any data seen through a view is seen through the eyes of the role that created the view. Since my schema migrations are running as a superuser, that means that no rules are being applied at all, and if I’m not careful, a view can be exposing arbitrarily sensitive data.
I can get column rules back for a single role by
SET ROLEing to it before creating the view, or I can make copies of the view for each role I want toGRANT, doingSET ROLEto the right view before each. Even if I do that and don’t make mistakes, RLS is not applied.Instead, I revert to Just Being Careful with views, and thinking through all rules that “should” be applied to the rows and columns being returned, and trying to make sure that they are.
Just as a word of caution :)
-
This bit of sample code jumps out at me (as a total crypto noob, pretty nervous about writing any crypto-related code):
$nonce = random_bytes(24); $ciphertext = sodium_crypto_box($plaintext, $nonce, $message_keypair);Would it make sense for a safe-defaults-oriented crypto api to generate
$nonceby itself, and return it along with the ciphertext? That way I can’t screw it up by re-using it or using something predictable.
Meta: I can’t stand “first” claims. Title suggest:
PHP 7.2 Adds Modern Cryptography to its Standard Library-
The nonce does not necessarily have to be randomly generated, and in fact it can have some semantic meaning if you want it to. For example, in “Practical Cryptography,” Schneier and Ferguson suggest using the message number plus some additional information as a nonce, assuming the messaging system has the concept of a message number. The most important aspect of a nonce is that it is used only once (hence, “Number used once”), and does not necessarily have to be high-entropy. If you want it to have some kind of meaning, then having the crypto library choose it for you is not always what you want.
-
Just to be pedantic and because I’m taking a break at work:
I’m like 90% sure that the etymology isn’t “Number used once”, but rather from the Middle English “nonce”, meaning “current occasion”, via the linguistics term “nonce word”, meaning a word that is expected to only occur once.
Why yes, I am fun at parties.
-
I won’t argue with you at all, as I had absolutely no idea what the etymology of the word is :). Assuming you are right, I still find my mental link of “nonce” == “number used once” useful because it helps me to remember the actual purpose behind the use of the value.
-
No, no, your version is immediately more useful. I just like linguistics. :)
-
-
-
Oh, interesting! That makes me feel better, so hopefully the docs will include similar guidance :)
I wonder then, if
sodium_crypto_boxis stateful, and will refuse to work if$nonceis reused? I didn’t find relevant libsodium docs with a quick search, but I think I just don’t know the right keywords.-
Secretbox is an abstraction of the concept: “given a
key x noncepair, push data into a thing and get the encrypted data on the other side”. An application could potentially have thousands of secretboxes at once and they really shouldn’t know anything about each other.Think of a secretbox being stateful the way an iterator is: if you give it all the same parameters its going to replay exactly the same behavior.
Finally, the nonce space can be very large (depending on the cartographic algorithm being used and what it specifies). In the case of Salsa20, it is 64 bits. It isn’t realistic to track a space that size. (When using libsodium you should really stick to the symbolic constants, like
crypto_secretbox_NONCEBYTES, and not, say,24.)As long as you never use the same
key x noncepair, you’re fine. As @untothebreach said, using an application invariant that guarantees the nonce will be unique is often sufficient. For many algorithms, the nonce space is large enough to randomly generate the nonce each time and not worry about it (if that sounds fishy to you, recall that the nature of cryptography is to make bad things really unlikely and then accept that it’s fine).
-
-
-
-
-
-
I am so, so excited about this! It had me at
#[get("/hello/<name>/<age>")] fn hello(name: &str, age: u8) -> String {...Seriously, it’s hard to find safe routing patterns among any other request-oriented rust library. Most of the time, you have to
.get().unwrap()parameters from a Map or something. If you don’t like the.unwrap(), you need to handle theNonecase, which is a huge anti-pattern IMO, since you defined the route such that it’s impossible to enter the handler if it’sNone. Gah!I wrote my own horribly hacky macro library for my own routing purposes. The code it generates isn’t pretty, but it’s at least validating the routes at compile time, no
.unwrap()necessary.But this look leaps and bounds better. Can’t wait to build more stuff with rust now!
-
Huh, browser architecture is over 90% 32-bit. Anyone know why?
-
I believe the main reason so many Firefox users run a 32 bit version is because of legacy browser plugins, some of which may never have been released in a 64 bit version. Since they are binary blobs, the browser ABI has to match the plugin’s ABI for the plugin to work.
At any rate, many users should be able to switch these days. There is work scheduled to migrate Windows users with 32 bit Firefox to 64 bit Firefox (assuming they’re on 64 bit OS, of course).
-
-
I don’t like most of this, but I like this:
the default hour long meetings dropped to 30 minutes instead
-
uh oh we just added these hot 4 features, better bump up the major version!
-
Well, it’s not like Firefox follows SemVer.
Firefox used to adhere to a strict 6-week train model of promoting features to release. Now they publish a 6-to-8 week schedule for the year, for the same process. It’s actually quite cool, I’d recommend checking out the first link if you’re not familiar with the train model.
See also:
- Announcement of rust’s train model
- A recent lobsters submission about deploying an app on a train release schedule
-
-
Indeed they changed their model after Chrome sped up the release process. As I understand, they did it to match the rate of evolution in the web. Firefox’s release model before then was probably too long, too slow to keep up with the rate of change in Javascript APIs, etc. Hurray?
-
-
-
-
The idea that by limiting the options available in the downvote you are positively and proactively controlling users and promoting community is bad UX and a slightly broken way of thinking about interface design and social interaction.
Let’s get real about the process and the user experience that’s really at play here:
- User reads a comment they really don’t like. They have some self awareness that it is not factually incorrect, but it is grating and the user feels that the comment should have a downvote.
- User is not thinking carefully or wanting to express himself creatively, user is caught in emotional reflexive response state.
- User clicks downvote
- User can not find a downvote option that describes his feeling
- User chooses closest option from the ones availble that matches his strong feeling for the need to downvote.
If you force a user to not be able to mash the downvote button because she feels like it, Bad Things will happen when feelings are forced to be quantized.
The solution here is to make more options available in the downvote, including one that is ‘bah i don’t like this!’, and for users to be able to customise their view of the data according to the downvote types they want to take into account, or even users whos votes they want to take into account.
Also, why is this anonymous? The lobste.rs code knows the user that has downvoted, would making it public change behaviour?
-
That’s an interesting take – a having
disagreeoption in the downvote menu could capture most of the disagree downvotes that are currently being funneled intotrollorincorrect. And then we have options:- ignore them
- show the “disagree” count separately
- show separately and de-anonymize disagreers
- pop up a modal when “disagree” is clicked to say “We don’t downvote for that here, please explain why you disagree in a reply comment instead”
- …
-
At least a few of these are good books (Algorithm Design Manual is my favorite algorithms book), but the wishy-washy headline/title is really annoying.
-
Ha, well, the author used their introduction to kind of pre-emptively defend the title while introducing the post:
There are a lot of “12 CS books every programmer must read” lists floating around out there. That’s nonsense. The field is too broad for almost any topic to be required reading for all programmers, and even if a topic is that important, people’s learning preferences differ too much for any book on that topic to be the best book on the topic for all people.
This is a list of topics and books where I’ve read the book, am familiar enough with the topic to say what you might get out of learning more about the topic, and have read other books and can say why you’d want to read one book over another.
I have to say, especially preceding so many good-sounding recommendations, the title helped ease my oh-no-so-much-to-learn-so-little-time anxiety that comes up with every link like this. I might consider these. Everything will be ok :)
-
-
olivier
1 year ago
|
link
| on:
400,000 GitHub repositories, 1 billion files, 14 terabytes of code: Spaces or Tabs?
I frankly don’t get why people insist on spaces. Please, someone, enlighten me. Please. I wanna understand why a single resizeable character is inferior to a small pile of unresizeable ones.
-
- tabs (and all other invisible characters, see dos2unix) have consistently proven to be difficult-to-understand and error-prone among computer users
- resizing is easy in an environment you’re familiar with, but in practice people often operate in environments that they are unfamiliar with, so the lowest common denominator (i.e. what I can do without thinking in Notepad.exe) wins.
-
The biggest problem is that they get mixed inadvertently. Go has shown that a program can actually fix it. As a result, my editor reads tabs, I often type spaces, and gofmt makes it standard tabs on save. This is legitimately the best feature of the Go tool chain.
Years ago when I programmed in Python, there would always be that one committer who would insist on tabs, or use 3 spaces instead of 4 in some places, and all hell would break loose in the commit history–annoying, but not the end of the world. The shameful person, btw, didn’t do the cleanup, cause they didn’t give a shit.
In summary, this is the least of our problems and can be automated incredibly effectively if it means enough to you, but if not, who cares?
-
Nowadays, tooling has improved in Python, too. You can also use tools like autopep8 or yapf (https://github.com/google/yapf) to autoformat Python code (yes, that is a thing ;) )
-
-
-
The problem is the assumption that allowing it to be resized is even desirable in the first place. If you are working on a specific codebase, you need to stick with the style that the codebase dictates. Imagine if people thought it was a good idea to allow “flexible brace styles” to let you use your preference. Why must indentation be different?
If you want the code to be displayed differently according to your preference, it should be implemented as a rendering-time hack (for instance, showing “lambda” as “λ”, etc) rather than changing the actual bytes on disk. Rendering 4 spaces on disk as 8 spaces on screen is a trivial config change in any decent editor.
-
Some languages are whitespace sensitive and a tab doesn’t always represent the same number of characters so making everything done with spaces is the simple, easy way out when dealing with vertical alignment. Gofmt does vertical alignment pretty nicely and it works well across different editors and even with doublewidth characters involved etc but most developers don’t like using a formatting tool outside of what its editor does on its own and most languages don’t even have an agreed upon format so there lacks formatters for those languages, because adoption would stumble.
-
-
In order to allow tab characters in your code, you have to disallow certain coding styles to prevent misalignment when tab sizes change. It’s much easier to simply disallow all tab characters, especially as the team size increases.
-
-
I’d say no.
Many (most?) articles on JavaScript will also be Node-related. Consider a new JavaScript library intended for the browser that is only available via npm. Is that JavaScript only? What about a tutorial about JavaScript UIs that uses npm as it’s build system? Conversely, literally every
nodejsstory would be a validjavascriptstory.I think that there is sufficient ambiguity that the
nodejstag would likely end up being useless.-
Does
npmimplynodejs? I don’t think it does.edit for clarity: …so I don’t think your examples would be correctly tagged
nodejs. Just because there’s a related piece of infrastructure, doesn’t mean a story has to be tagged with that infrastructure if it’s not the subject.
-
-
Oh no. This is exactly the way not to do things. Humans need to learn that computers are fast. Every time they are not, there is someone to blame.
When people do not trust an algorithm comparing phone plans, the correct way would be to present a breakdown of the analysis instead of just the result. Every aspect of the winning plan should have been compared with it’s neighbors and scores presented. See for example cpuboss.com.
Making the software feel wise is returning from science and education back to beliefs and superstitions.
-
These aren’t CLI tools.
Humans need to learn…
Why? This does not sound like the way to start designing anything that puts humans first.
When people do not trust an algorithm comparing phone plans…
I would like to see a counter-example to support the idea that a instantaneous-but-more-detailed comparison tool will be judged just as trustworthy. cpuboss is not this counter-example for two different reasons:
cpuboss is not instantaneous in the first place. Measuring time-to-first-render on my fast computer / reasonable internet, I get 1.22 seconds. Subsequent page loads with a warm cache are faster, but it’s still doing a network round-trip to get them, coming in at hundreds of milliseconds.
It’s a completely different tool. cpuboss is comparing only two CPUs in a head-to-head. The examples in the article were computing a recommendation based on many available options.
“But when we tested with people, they assumed it was all marketing bullshit because it was instantaneous. They’d say, ‘This was obviously a canned result, I’m just gonna shop myself.’”
I find it really hard to imagine any design solution that could make me, personally, trust random site X that I stumble across when looking for recommendations for Y, if I feel like I’m being served a canned result that might be paid for by the recommended option. A detailed feature matrix? Why would I trust that? Something else?
-
Why? This does not sound like the way to start designing anything that puts humans first.
Let me rephrase that. Humans, in aggregate, need to learn about computers. When we resign on understanding the features and limitations of the tools we use, we might end up using them incorrectly. Not ideal when we let those tools decide for us. More on the topic in chapter 4 of Summa Technologiae or even the renowned movie I, Robot.
I would like to see a counter-example to support the idea that a instantaneous-but-more-detailed comparison tool will be judged just as trustworthy.
Haven’t there recently been a wave of “Right to Explanation” links?
If such a study found that the tool would not be trusted as much, I would argue against slowdowns anyway. It’s not only morally wrong, it’s the wrong trend in general. We can’t let our tools use such low psychological tricks on ourselves!
cpuboss is not this counter-example for two different reasons:
Of course, I was pointing at the way the results are presented. It’s still far from ideal by not presenting individual benchmark scores and confidence but still way better than if it just took long.
It’s a completely different tool. cpuboss is comparing only two CPUs in a head-to-head. The examples in the article were computing a recommendation based on many available options.
Not really, it’s all about delegating the decision to a tool.
-
-
-
https://github.com/uniphil/results is a library for that lets you use ADTs like this inside JS, it’s kind of fantastic!
-
Results author here, thanks for mentioning it, @maxbittker!
We use Results pretty heavily in our 50k JS client app at OpenRide, for exactly the kinds of cases mentioned in the article. It’s not magic, and there are hard things we’re still figuring out, especially around modeling async data.
In particular, we found this type,
type RemoteData e a = NotAsked | Loading | Failure e | Success afairly troublesome:
NotAskedis just hard. It’s super-annoying to have to handle that case in component views, and if we don’t have bugs, components should just never be rendered with aNotAsked. We settled on{ Done, Pending, Errored }where
Erroredis a subsequent ADT/Union/whatever ofRequestError, which includesNotStarted.This is all cool, and can extended to model optimistic updates, but it’s easy to let it get very messy, miss cases, and confuse data-modeling requirements for stuff we give to view components vs. requirements for the client storage, which we’ve found are quite distinct.
-
-
Computer scientists should really be spending their time developing new libraries rather than inventing new programming languages.
This assumes that the purpose of computer scientists is to make professionals in industry more efficient programmers in the short term. I disagree with that assumption – the purpose of computer scientists is to further our knowledge of computer science, and any benefit to professionals in industry is a secondary effect.
-
For others who might read these comments before clicking through to the article: that quote is not a position taken by author. The author relays what a friend told them for the first two paragraphs (including that quote), and then in the third says,
Being a language designer myself, I, of course, don’t share this opinion.
and goes on for the rest of the article to defend why they think language design is worthwhile. Note that even the title of the article is in quotation marks.
On the other hand, I agree with @jmelesky’s comment after the quote and think it is still relevant to the article. The article does try to justify language design efforts in terms of making industry programmers more efficient and happier – the author’s own language was motivated by limitations he ran into as a library author. But this doesn’t mean all CS efforts are or should be motivated by industry.
-
For others who might read these comments before clicking through to the article: that quote is not a position taken by author.
Thanks for that.
I skipped this article because of jmelesky’s comment – this isn’t something that I want to argue about:
There’s tonnes of Java systems and Java programmers and if the world doesn’t need another language, then it’s Java, and I’m simply not okay with that.
-
-
-
Awesome! I have a similar library for JavaScript called Results, and it’s encouraging to see another implementation choose a similar API.
At OpenRide, on the web client, we model a lot of data as discriminated unions with Results, to force it to handle all the cases. Of course, since it’s JS and we don’t have static typechecking to enforce it, bugs can still sneak in wherever tests don’t cover a path that run through a
.match. Despite that, it has still felt like a significant win so far.Does anyone know of any good research/studies/blog posts or anything that look at discriminated unions, either as language features, or as libraries bolted on?
(edit: whoops, hit Update instead of Preview the first time)
I can see what the author’s getting at here -
RemoteDatadoesn’t capture more complex use cases, but I don’t think it’s intended to.RemoteDatamodels a single loading event. Dropping it completely for more complex use cases strikes me a bit as throwing the baby out with the bathwater. I suspect a cleaner option would be to composeRemoteDatawith additional state and data structures to build up to the use cases he describes.For existing data and a refresh request, that might look something like:
For a large number of requests, you could do something like:
And it becomes pretty easy to derive the states from the list:
Of course, in these examples, the states aren’t represented as union types, so you lose come compiler checking that you’ve handled all states. That said, I’ve worked on some pretty complex interfaces, and I have never needed or wanted something that would validate that we had code to handle all of:
That said, if you really wanted it, you could take put your composition of RemoteData with additional state into it’s own module, make it an opaque type and enforce correct transistions between states by limiting the exposed API.
I think all of this would be clearer with a specific use case in mind. The exercise in the article strikes me as a case of premature generalization. It seems like it’s trying to solve all possible problems rather than anything specific.
I also have questions about what kind of cache is being referenced in the article, as I have some fairly strong opinions about caching data in client-side applications. (TL;DR: Don’t, the browser can already do this for you.)
I would love to hear more about this, because two problems have me stuck with client-side JS data caches in my apps. And I love deleting code.
Embedded documents. Every app I’ve worked on denormalizes the data we fetch to cut down on HTTP requests. Is it cheap enough with HTTP/2 to send lots and lots of requests? Even then, it would mean a bunch of sequential round-trips for each child document that depends on its parent.
Consistency. If two parts of the app load the same data at different times, we can get different responses, and have weirdness in the UI. With a JS cache, when we update a document, we can have every dependent piece of UI re-render and be consistent.
You should take this all with a big grain of salt, because I have not had the bandwidth to implement most of these ideas in practice, due to the usual constraints around priorities and limited time. I’ve just been increasingly bothered by the complexity of implementing caches in the client, or alternatively the ugly behavior that results when they’re implemented naively, and on reflection, I think it’s mostly unnecessary. I have probably missed some corner cases and I suspect there are many apps where some amount of specific, targeted caching might still be useful for a subset of APIs or pages.
With all of that in mind, I will say that #1 is probably the best reason I’ve seen for having a client side cache. I think in that case it’s worth looking at usage patterns to be sure it’s really providing benefit. If the individual requests your app is making in between big de-normalized requests don’t overlap much with the de-normalized data, the client side cache isn’t going to buy you much, although neither is the browser cache. Or if you’re always making large de-normalized requests, you’re still probably not getting a caching win, unless you have a way of structuring those requests to specify what you already have data on.
I think there’s a lot of promise with HTTP/2. The single TCP connection is nice on it’s own, but there’s also the potential to do interesting things like make a request for a de-normalized structure that only contains the relationships between resources, and then have the server push the actual data from the resources individually. That way they’re cached individually, and the browser will actually cancel the push for resources it has already cached. Running some experiments with that is somewhere medium-high on my TODO list.
#2 is tricky. If you data doesn’t change often, it’s not as big of a deal, or if you don’t often/never show the same data twice on the same page. One thing you can do if it’s still a problem after taking both of those into consideration is to track ongoing requests at the API layer. If you’re using something like promises, that means when a request comes in while another is still outstanding, you should be able to return the promise from the first request to the second caller, and just share the API call. If the first request has completed already, the browser should have the data in it’s cache (assuming the data has a time-based cache rather than something like etags).
This is awesome, thank you so much for taking the time to share your thoughts @mcheely!
Solving the round-trip part of #1 with HTTP/2 server push seems like it could be so damn magical and cool. In my most common case of hard-to-cache embedding – “load a list of items” and then “load a detailed view of one item” – it seems like a drop-in solution.
For #2, I actually hadn’t thought about races! I was thinking more about the case where data rendered on one part of the screen becomes stale, but there’s no way for the browser cache to tell that part of the UI to re-render, so it stays stale. I guess, since we hope it to be cached, maybe I just need to adjust my thinking to re-render more things more often. Cheap most of the time, since it’s cached, and expensive when it should be expensive anyway. Huh.
(solving the races by having a client API layer managing promises across the whole app starts to feel like a dangerously tempting place to add new features like… caching :P )
I think it all comes back to this, for me. Building an app usually feels like a process of discovery to me; top-down plans don’t survive long. The usage patterns can be pretty unstable, and it can get painful surprisingly quickly to be completely naïve about loading data.
…It’s appealing to imagine that a client-side cache, made hopefully robust through explicit modeling of all the possible states of each piece of data, can provide a 90% solution in a general way. Coupled with things like graphQL or PostgREST, you just build stuff and it works reasonably well, for free-ish.
Thanks for reading and thanks for the feedback.
As I say in the post, “States, events, and transitions should reflect the needs of the application”. I listed those states as an example because the last four applications I’ve built have needed all of these states. I tried to use RemoteData for two of those applications and ran into the problems I describe in the post. These apps do not strike me as complex and three years of dealing with these states led me to assume they were common. One example is an app that lists financial transactions. The app periodically refreshes the list. A loading icon is shown next to the list header during each refresh. Error messages are shown above the list if a refresh fails. That’s half of the states in that list already. On top of that, the user can make inline edits to the transaction details. When the updates are committed, a loading icon displays next to the transaction title. Errors related to the update (e.g. network failure) are displayed under the transaction title. That’s all of the states in that list. But the point of the article is not to dictate states to the reader. Again, “States, events, and transitions should reflect the needs of the application”. The point is that you cannot oversimplify the problem just because the result of that oversimplification looks nice in a blog post or a tweet.
I agree that these states map closely to the HTTP request/response lifecycle. As I said in the post, “RemoteData models a stateful cache of data in terms of a stateless transfer of data, REST.” The original RemoteData post clearly states that the pattern is intended to model the cache, not the request/response lifecycle. That is why that post starts by evaluating existing patterns for modeling cached data and then offers RemoteData as an alternative. Notice that these posts place RemoteData in the model and that the view functions consume RemoteData - cache state, not request state.