I agree with the README-driven approach. As soon as you start imagining interactions, all kinds of questions arise, and the high level design can adapt to changes very quickly - much more quickly than code. Taken a little further, the longer you defer writing code at all, the better the result tends to be.
I like to start with a set of text files. Describe what the project is, and is not. One might contain data format, another some objects I think are needed, another with some process flow and so on. If I can keep all those consistent with each other while incorporating various ideas, the result is a specification, or as close as I want to get to one.
You’ve described a process that is eerily close to something I’m trying right now.
Worst case, if I get bored of this project, I can shelve it and come back to it later and the docs will help me onboard myself again.
I usually write out a design on paper (I love paper) with lots of lists and diagrams/flows. Then I do classic test driven development. Then I write out a readme based on what the API now is, that is the step I hardly ever/never get to in my personal projects (Good luck to people trying to use my projects and sorry!).