Hey Lobsters,
During the corona times I’ve had a lot of opportunities to observe how I go about day-to-day programming tasks. One thing I’ve noticed is that I can have quite strong focus toward a goal, but that focus is thwarted by recursively following newer subgoals such that the path to the original goal is lost.
As an example: I might not be able to solve a tricky bug, so I research how to integrate a debugger into my text editor, which leads to diagnosing a problem in how an interpreter is installed on my system, which leads to switching from homebrew to the nix package manager, which leads to an afternoon of reading up on the nix language syntax instead of solving my bug.
I suspect that I’m not alone, and wondered if anyone here had advice on how to handle deeply nested thought-stacks. I’ve often heard that keeping a written journal is a good start, but for some reason I’m not good at these kinds of diligent bookkeeping tasks. Regardless, anything would be appreciated.
Breadth-first, not depth. Defer relentlessly. Check with your primary goal regularly. Time-boxing.
The trick with making meaningful progress and not spinning out on these tangents is pausing to recognize them as tangents. Only execute on a sub-task if it is necessary to complete your immediate goal. If a sub-tasks can be deferred, do that; You can evaluate if they are still useful later. Capturing them to get it out of your head should alleviate some of the pull that they have on you – they won’t be forgotten, but don’t need to be done now.
And always be asking the question “Is this helping me solve my immediate problem?”. Why did you want the interactive debugger? Probably for more context. For debugging specifically, always be asking is there a dumber/simpler way to find concrete information? Just sitting and thinking through the specific context that you think you need may have allowed you to continue with print debugging might have short-circuited the tangent.
The other tactic that can help is, when you start a sub-goal, estimate how much time it is worth to you, and set a timer. Had you valued the interactive debugger at 20m, and then the timer went off and you realized that you were about to re-install your interpreter. That is a good moment to re-evaluate. And having the concrete time box prevents you from losing an entire afternoon to a chain of those.
As a reminder, maybe put a sticky note in front of you with your current goal. And keep checking in that you are still really working towards it.
As for tooling, omnifocus and dynalist.io both have quick capture features for things you can defer until later. And dynalist.io and workflowy literally let you nest these tangents, which can be a visual signal when you’ve gone too far. But I think the crux of your question is more about focus and process and less about the tools.
I’ve learned this recently and changing from depth first has been tough but very helpful. How did you land upon this concept? and how do you practice it?
I’ve spent a lot of time in my own rabbit holes. Learning to slow down and take a step back is mindfulness and practice. Something I still strive for, and sometimes fail at.
Practically speaking ideas like MVPs and Tracer Bullets are have pushed me in this direction – See Pragmatic Programmer. Getting something thing in front of people as a way to understand what is important, and to quickly change things that aren’t working without spending a ton of time on them has worked well for me. Doing things this way you go wide, shallow, and simple for each feature. Inevitably when you’re writing a feature a hundred ideas and edge cases and thoughts come up, and you just have to throw them at the end of the list – If the feature survives or is important you can go deeper on it. That has meant a lot of practice adding to the end of my todo list rather than the middle.
As for “Breadth-first”, the original question’s use of recursion just made me think of depth-first graph traversal. So breadth-first being the opposite sprung to mind, though I was basically describing it as a FIFO queue, not so different I suppose.
I love the idea of this, how a lot of people are think of and realizing the same thing at the same time as a product of the culture they love in.
(I ordered para prog v2 two weeks ago)
Thanks for your answer!
First, keep notes. You don’t have to be incredibly thorough, you don’t have to produce anything polished. Start with a big piece of paper under your keyboard or a notes.txt in your editor of choice. When I really have to focus, I find that it often helps to make a checklist for every mundane step I know about for the thing I’m currently working on, and update it as I go. Yeah, this is bookkeeping in a sense, but really it’s just offloading some of your working memory. When your working memory is as crap as mine is, you need all the help you can get.
Second, when you’re tempted to spin off into a new thing, put it on a checklist somewhere else, and then go back to your primary task. I keep notes in a vimwiki, so I’ve got a file called
yak-shaving.wiki. Here’s the current top of the list for flavor:Some of that stuff I’ll revisit and some I won’t, but capturing the thought and setting it aside usually lets me get back to whatever I was actually working on at the time.
I recently learned that solving a problem requires bread-first effort, asking how can i get a working solution as soon as possible. Once that is ready, find the bottleneck, fix that in a bread first way.
Keep a notebook where you start with what you want to do and what are the necessary tasks, it’s ok to deviate but always keep the bigger picture in mind.
I definitely approve of eating bread before doing work. It’s hard to concentrate on an empty stomach.
(You meant breadth-first. Sorry for making your response the butt of the
breadjoke.)you found my typo, dough! You can Loaf at me all you want.
This might seem strange, but I have a mediawiki instance (hosted by miraheze, they are awesome) that I use as a kind of extension of my brain. I basically have a TODO.md for most personal project and trello boards for most $WORK projects that keep me focussed on what needs to be done. But then, every once in a while you’ll get sidetracked, get an awesome idea, read something you might want to investigate later (typically programming languages) or anything like that. That all goes into a big master list of my personal wiki. Usually, I just write stuff down there and never get back to it, but if an idea is good, I will maybe (at an appropriate time) look into it. But it just helps me a lot to “dump” these ideas somewhere so I know they aren’t lost but I also don’t have to expend cognitive energy keeping track of them.
I keep a written lab notebook. It’s got a faint graph, which keeps me organized. I make a list of things I need to get done, and write down notes along the way (dependencies or related tasks). If things take too much time, I check (either with myself or our tech lead) if it needs to be done right now or can be deferred/ignored.
Meta: prioritize. If you’re doing programming for work, then you’ll have some tasks that are more important and some that are less, but the most important thing is to get them done. Convince yourself to save the yak-shaving for later, or at least concentrate on the high-value tooling optimization (you REALLY need a debugger in your editor). However, if you’re programming for fun, then why not go down the rabbit-trail?
First: figure out how to “save state” for anything you want to work on - maybe just a folder full of text files like
nix-notes.txtwhere you write down ideas, links, what progress you’ve made, etc. - kind of “comments” that should allow you to continue where you left off, even years after you last worked on the thing. Writing down these things helps your brain to let go of the desire to hack on something immediately, which will in turn reduce the emotional desire to yak-shave. I suggest reading up on the Getting Things Done methodology.Second: don’t think of these as being “stacks”, but instead a graph. You’ll probably have more than one reason you want to try nix. Thinking of them as stacks encourages you to do things in dependency order for a particular task instead of the order which is most useful for your work (e.g. maybe your work uses nix internally and you’ve deferred setting it up) or that is most interesting to you personally.
Third: work on your ability to do those “diligent bookkeeping tasks” - aside from keeping “project files”, journaling is pretty beneficial, and more importantly, the meta-skills of introspection and state-saving are important for way more than just programming.
Fourth: step back and think. This will sometimes help you to realize that you don’t really NEED (in case of work) or WANT (in case of play) to yak-shave down to the next level. Maybe instead of switching from homebrew to nix, a few more minutes of searching on the internet would have allowed you to fix your problem with the interpreter directly.
If your subgoal really needs to be solved for the current goal to proceed, just use a stack. Get a set of sticky notes, then any time you want to go deep into a subgoal, write down your current goal with as much clarity as you can, and paste it to your monitor right top corner. Now, any time you want to go into a subsub goal, paste the idea you were working on on top of the previous sticky note. Peel it off when done, or move it aside if you can accomplish the same thing in another less painful way.
If your stickies start getting thicker and falling off from the screen, then it is time to see whether you are solving your original problem or yak shaving.
If you’re in learning/exploration mode, recursively following tangents is more useful than actually getting your ostensible projects done.
If you’re in exploit mode (i.e., somebody is paying you to solve problems & you’ll get fired if you don’t solve them on time, or you are paying someone to make you solve problems & you will get expelled if you don’t solve them on time) then learning is a lower priority & a tactic of repeated limited depth-first is better – in other words, identify what tangents you might want to go on, but then create a dumb and easy solution that works before going on any of them. Once the problem is solved, if you have time, revisit the problem and see if you can solve it better with only incrementally more research.
This is advice about how to avoid unnecessary yak shaving rather than how to manage thought stacks, but the core of your problem seems to be that the yak shaving never ends (and you’ve just sort of assumed that any solution will involve shaving the same number of yaks in a more organized way). I don’t know of a way to keep track of hairy yaks that’s any better than just keeping a list. Yaks are not inherently recursive – they are random access (and densely intertwingled through connections you can choose not to follow), and it’s just your framing of the problem that makes them seem recursive – so you can shave them at your leisure & then use them later as they become available.
If nothing else, try keeping a running stream of consciousness journal. Nothing crazy, just leave yourself mental waypoints so you can keep your thought process checked. This also helps give you time to evaluate the necessity of it all.
I’ve found that having a “I’ll write this idea down for later” place is also really helpful.
I’m trying to solve this problem by integrating my window manager with a note-taking solution. This is super difficult problem and I think it’s equivalent to knowledge management in general. I’ll post something here when/if I have something useful.
For now, all I can tell you is to just keep plugging along and see if you can develop some good habits.