I agree. I think it’s fair to take this to the next step: which tools and languages automatically reduce load? But even in pure FP, you can end up with symbol leakage from underlying systems that you trust that do things you didn’t expect. One of the nice things about the onion architecture is that it forces you to come to terms with all of that right up-front. One of the things I don’t like about railway programming is that you can endlessly put off making business decisions because you’re not forced to think through leakage right away, at the point of consumption.
I love the idea of a cognitive load metric, but it surely has nothing to do with symbols per line; understanding what a line or expression does to the first degree is never really the problem. It’s the subsequent degrees that cause headaches. Was this method I’m calling monkey-patched by something else in the component graph that I have no way of knowing about, and thus do something subtly different than I expect? Does twiddling this bit of global state have implications that aren’t obvious unless I read some other seemingly unrelated package? It’s about correctly modeling the application behavior, not parsing expressions.
I think “code quality”, or “cognitive load” can be measured on two different levels, the microlevel and macrolevel.
On the microlevel, it’s all about “can I read what I see on the screen?” I think we’ve all seen horribly formatted/dense code where we struggle even understanding it. On the macrolevel, it’s all about “can I understand how everything fits together?”, and this is what you’re talking about.
To give an analogy from language: I bought Hobbes’ Leviathian a few years ago, and found the old-fashioned flowery English very hard to read, but I had no trouble understanding the greater ideas presented in it. On the other hand, there are arguments presented in very clear English that I nontheless have trouble compreheding, such as quantum mechanics.
I think “code quality”, or “cognitive load” can be measured on two different levels, the microlevel and macrolevel.
On the microlevel, it’s all about “can I read what I see on the screen?” I think we’ve all seen horribly formatted/dense code where we struggle even understanding it.
Absolutely, but this is a solved problem in any modern programming language (langfmt) and/or sane organization (standards docs). The only interesting level is the macro level.
The point of the essay is that you cannot identify whether it is the micro or macro level simply by looking at a line of code. We keep doing this and it never works. An overall metric is the only thing that is tractable.
The point of the essay is that you cannot identify whether it is the micro or macro level simply by looking at a line of code. We keep doing this and it never works.
Yes, it’s nontrivial, and requires semantic analysis of the code. But it’s the only thing worth doing, because anything less isn’t a useful proxy.
An overall metric is the only thing that is tractable.
I suppose “overall” is carrying some weight in this sentence, but I don’t understand what it is. Can you clarify?
“Yes, it’s nontrivial, and requires semantic analysis of the code. But it’s the only thing worth doing, because anything less isn’t a useful proxy.”
I am happy to argue this at length, but no, I believe you are wrong. This is just stuff I could have come up with 20 years ago. The question is here is about humans maintaining pieces of programming that have business value. Not about semantic analysis. The only semantic analysis that matters happens between the ears of the maintenance programmer.
Is there a number that can give us insight into how difficult or easy it might be for a random human to maintain some application? If you’re okay with that question then we can argue specifics.
You’re over-analyzing this. What’s the job, what’s keeping us from the job, how do we use the minimum amount of metrics to measure that. That’s it.
Is there a number that can give us insight into how difficult or easy it might be for a random human to maintain some application? If you’re okay with that question then we can argue specifics.
I don’t know, but I can identify some generalizable properties of programs which would go into such a number. State mutations. Distance of state from logic. Identifiers referenced implicitly.
You’re over-analyzing this. What’s the job, what’s keeping us from the job, how do we use the minimum amount of metrics to measure that. That’s it.
I am completely failing to grok your point, apparently in the essay itself and I think also in these comments.
I don’t know, but I can identify some generalizable properties of programs which would go into such a number. State mutations. Distance of state from logic. Identifiers referenced implicitly.
Yay. Cool beans. Welcome to the club. Let’s work on the numbers and formula.
I see “Cognitive Load” being discussed as a neurological/psychological term. I see nothing like this being discussed in programming. But anybody, including you, will say that it’s very important. Are you grokking now?
I am sorry. The fault is mine. Makes sense to me, but it obviously needs a lot of work to make sense to others. Thank you.
I don’t know, but I can identify some generalizable properties of programs which would go into such a number. State mutations. Distance of state from logic. Identifiers referenced implicitly.
I’ll try one more time. Perhaps second time is the charm. Apologies for being a bit cranky. I know better than to dive right in to feedback after I write something. I’m too much of a drama queen. (grin)
I believe we’re talking past one another. You are addressing the technical question: what are the ways code becomes dangerous and can’t we evaluate that on a per-instance basis. Technically, yes. We all understand the mechanisms involved. Hopefully I’m not explaining mutability, overloads, or so forth. And by knowing these things, we could certainly make a more explicit metric for a piece of code.
I am addressing the psychological relationship between the programmer and the IDE/code. I would argue that most all programmers coding in OO/mutable-state are manipulating symbols in such a way that they believe they are making much higher-level abstract decisions than they actually are. Your idea is a great one, but it fails the minute we move past the compilation unit. As an example, if I’m coding in Typescript, and have only used that, the minute I run into a problem where I have to bring up a DOM tree or a browser debug window, I’m now being exposed to Javascript, html, css, and so forth. Even though I may have strongly-typed code, and from the standpoint of the compiler I have limited the ability to be misled, I’ve suddenly got an extreme amount of more mental work involved. I’m looking at the mental load of the coder. Unfortunately with all of the loose bindings and multiple abstraction layers with holes in them, it does directly relate to the technical aspects of how things are coded. (Long discussion here about various platforms where this is much less than modern web development)
Maybe that’s better. Hope so. Thanks for the comment.
Over 95% of every-day programming problems are “solved” by “just” using the right combination of tools and a “sane” organisation. Turns out that those two preconditions are non-trivial to achieve in practice, not in the least because many can’t agree on what “sane” is in the first place.
And while stuff like gofmt does make it impossible to write truly unreadable dense code – like, say, procmail.c – there’s still plenty of oppertunity to make the code on the screen hard to read: lack of newlines combined, non-standard obscure variable names, confusing conditionals/loops, no comments about what the goal of that 15 line function even is, etc. make some Go code much harder to read than it could be.
Over 95% of every-day programming problems are “solved” by “just” using the right combination of tools and a “sane” organisation. Turns out that those two preconditions are non-trivial to achieve in practice, not in the least because many can’t agree on what “sane” is in the first place.
Ah. I haven’t worked in an organization where microlevel stuff has even come up in a code review in many, many years. So, I guess I just don’t feel these pains.
Well, even worse - the entire site is written with Vue and doesn’t load without JavaScript enabled. For some bizarre reason, the scrollbar for that div with the content is padded to be obscured outside of view, so it isn’t apparent that it’s scrollable!
I’m trying to imagine how this relates to more traditional measurements of complexity, particularly Kolmogorov complexity. We can draw a quick and rough correspondence, generalizing line count to symbol count. The symbol maths is not simple, unfortunately; we need entropy. Where the CCL formula says freq, we must provide a probability distribution, and we’ll likely want to convert everything to bits.
I don’t know whether I believe in the subjective parts of CCL, though. And if we drop them, to obtain an objective metric, perhaps as computed by a linter or line-counter, then I think that we are merely approximating Kolmogorov complexity by choosing an approximate probability distribution. Perhaps that could be effective on a language with a rich-enough corpus to draw upon and a rich-enough symbol library to give high bursts of entropy as needed.
Would be nice to come up some concrete algorithm to calculate the “Code Cognitive Load”. It is already well-known, language and tool designer need to incorporate human-computer-interface knowledge to improve their design. There are couple of communities and conferences working on this front:
I like the article, but the formatting is really getting in my way. The article title and author masthead is enormous, taking up over a third of the vertical screen real estate on my laptop. The whole of the content is for some reason constrained to the left two thirds of my screen … which would be fine, except I can’t scroll the text at all unless I’m moused over the relatively small window where the article itself is.
Thanks. Like the Constellation in the Doomsday Machine, I had not expected to take this blog engine into a real situation. I plan on moving all of this to LeanPub very soon.
I have a proposal: we string up everybody who puts an animated favicon on their site, and hang them in gibbets out the front of browser developers’ shops as a warning for all.
There have been so many great comments I am going to write a follow-up.
Like everything else in analysis, the more smart people you have asking honest, direct, penetrating questions, the better the terms are understood by everybody. This is exactly what I needed.
I’m having trouble parsing out the main theme here. Is it that “cognitive load is bad”? Is it the specific way of measuring that? Is it in how cognitive load happens? Is it that cognitive load is always unnecessary?
Thanks for the correction. I freely chose from several languages and just winged it. It’s been fixed. Also I’m on vacation and have no access to a printer. This essay would be much tighter if I had my regular writing flow in place. Instead it feels to me more like an extended tweet.
Is a high Code Cognitive Load always bad? I can’t answer your question. Code Cognitive Load exists, whether we want to ignore it or not. There is a certain amount of mental work that it takes to understand and make changes to any application codebase, and that mental work can be measured empirically. It’s fair enough to argue about how to do that. It’s also fair enough to argue that some Code Cognitive Load is necessary for some domains.
The purpose of the essay was to introduce the term, provide an initial definition, and invite the community to start thinking about the implications of measuring how well we’re coding for other people. I think there are a ton of useful and quite dramatic implications here, but I don’t want to get ahead of things. Right now I’m just looking for people to tell me that I’m wrong. Thanks for the comment.
For me one of the main benefits of strongly typed functional programming (e.g. Haskell) is precisely that it reduces cognitive load.
In pure code, the inputs and outputs are clearly defined, and you know there can be no side effects in called functions.
For code with effects, this is made apparent, and so can be looked at more carefully.
I agree. I think it’s fair to take this to the next step: which tools and languages automatically reduce load? But even in pure FP, you can end up with symbol leakage from underlying systems that you trust that do things you didn’t expect. One of the nice things about the onion architecture is that it forces you to come to terms with all of that right up-front. One of the things I don’t like about railway programming is that you can endlessly put off making business decisions because you’re not forced to think through leakage right away, at the point of consumption.
I love the idea of a cognitive load metric, but it surely has nothing to do with symbols per line; understanding what a line or expression does to the first degree is never really the problem. It’s the subsequent degrees that cause headaches. Was this method I’m calling monkey-patched by something else in the component graph that I have no way of knowing about, and thus do something subtly different than I expect? Does twiddling this bit of global state have implications that aren’t obvious unless I read some other seemingly unrelated package? It’s about correctly modeling the application behavior, not parsing expressions.
I think “code quality”, or “cognitive load” can be measured on two different levels, the microlevel and macrolevel.
On the microlevel, it’s all about “can I read what I see on the screen?” I think we’ve all seen horribly formatted/dense code where we struggle even understanding it. On the macrolevel, it’s all about “can I understand how everything fits together?”, and this is what you’re talking about.
To give an analogy from language: I bought Hobbes’ Leviathian a few years ago, and found the old-fashioned flowery English very hard to read, but I had no trouble understanding the greater ideas presented in it. On the other hand, there are arguments presented in very clear English that I nontheless have trouble compreheding, such as quantum mechanics.
Absolutely, but this is a solved problem in any modern programming language (langfmt) and/or sane organization (standards docs). The only interesting level is the macro level.
The point of the essay is that you cannot identify whether it is the micro or macro level simply by looking at a line of code. We keep doing this and it never works. An overall metric is the only thing that is tractable.
Yes, it’s nontrivial, and requires semantic analysis of the code. But it’s the only thing worth doing, because anything less isn’t a useful proxy.
I suppose “overall” is carrying some weight in this sentence, but I don’t understand what it is. Can you clarify?
“Yes, it’s nontrivial, and requires semantic analysis of the code. But it’s the only thing worth doing, because anything less isn’t a useful proxy.”
I am happy to argue this at length, but no, I believe you are wrong. This is just stuff I could have come up with 20 years ago. The question is here is about humans maintaining pieces of programming that have business value. Not about semantic analysis. The only semantic analysis that matters happens between the ears of the maintenance programmer.
Is there a number that can give us insight into how difficult or easy it might be for a random human to maintain some application? If you’re okay with that question then we can argue specifics.
You’re over-analyzing this. What’s the job, what’s keeping us from the job, how do we use the minimum amount of metrics to measure that. That’s it.
I don’t know, but I can identify some generalizable properties of programs which would go into such a number. State mutations. Distance of state from logic. Identifiers referenced implicitly.
I am completely failing to grok your point, apparently in the essay itself and I think also in these comments.
Yay. Cool beans. Welcome to the club. Let’s work on the numbers and formula.
I see “Cognitive Load” being discussed as a neurological/psychological term. I see nothing like this being discussed in programming. But anybody, including you, will say that it’s very important. Are you grokking now?
I am sorry. The fault is mine. Makes sense to me, but it obviously needs a lot of work to make sense to others. Thank you.
I’ll try one more time. Perhaps second time is the charm. Apologies for being a bit cranky. I know better than to dive right in to feedback after I write something. I’m too much of a drama queen. (grin)
I believe we’re talking past one another. You are addressing the technical question: what are the ways code becomes dangerous and can’t we evaluate that on a per-instance basis. Technically, yes. We all understand the mechanisms involved. Hopefully I’m not explaining mutability, overloads, or so forth. And by knowing these things, we could certainly make a more explicit metric for a piece of code.
I am addressing the psychological relationship between the programmer and the IDE/code. I would argue that most all programmers coding in OO/mutable-state are manipulating symbols in such a way that they believe they are making much higher-level abstract decisions than they actually are. Your idea is a great one, but it fails the minute we move past the compilation unit. As an example, if I’m coding in Typescript, and have only used that, the minute I run into a problem where I have to bring up a DOM tree or a browser debug window, I’m now being exposed to Javascript, html, css, and so forth. Even though I may have strongly-typed code, and from the standpoint of the compiler I have limited the ability to be misled, I’ve suddenly got an extreme amount of more mental work involved. I’m looking at the mental load of the coder. Unfortunately with all of the loose bindings and multiple abstraction layers with holes in them, it does directly relate to the technical aspects of how things are coded. (Long discussion here about various platforms where this is much less than modern web development)
Maybe that’s better. Hope so. Thanks for the comment.
Over 95% of every-day programming problems are “solved” by “just” using the right combination of tools and a “sane” organisation. Turns out that those two preconditions are non-trivial to achieve in practice, not in the least because many can’t agree on what “sane” is in the first place.
And while stuff like gofmt does make it impossible to write truly unreadable dense code – like, say,
procmail.c– there’s still plenty of oppertunity to make the code on the screen hard to read: lack of newlines combined, non-standard obscure variable names, confusing conditionals/loops, no comments about what the goal of that 15 line function even is, etc. make some Go code much harder to read than it could be.Ah. I haven’t worked in an organization where microlevel stuff has even come up in a code review in many, many years. So, I guess I just don’t feel these pains.
The website won’t let me scroll normally in several different browsers…
EDIT: my mouse has to be over the text in order to scroll the website, if it’s in the white space beside it I can’t scroll. This is bad.
Well, even worse - the entire site is written with Vue and doesn’t load without JavaScript enabled. For some bizarre reason, the scrollbar for that div with the content is padded to be obscured outside of view, so it isn’t apparent that it’s scrollable!
I’m trying to imagine how this relates to more traditional measurements of complexity, particularly Kolmogorov complexity. We can draw a quick and rough correspondence, generalizing line count to symbol count. The symbol maths is not simple, unfortunately; we need entropy. Where the CCL formula says
freq, we must provide a probability distribution, and we’ll likely want to convert everything to bits.I don’t know whether I believe in the subjective parts of CCL, though. And if we drop them, to obtain an objective metric, perhaps as computed by a linter or line-counter, then I think that we are merely approximating Kolmogorov complexity by choosing an approximate probability distribution. Perhaps that could be effective on a language with a rich-enough corpus to draw upon and a rich-enough symbol library to give high bursts of entropy as needed.
Would be nice to come up some concrete algorithm to calculate the “Code Cognitive Load”. It is already well-known, language and tool designer need to incorporate human-computer-interface knowledge to improve their design. There are couple of communities and conferences working on this front:
I like the article, but the formatting is really getting in my way. The article title and author masthead is enormous, taking up over a third of the vertical screen real estate on my laptop. The whole of the content is for some reason constrained to the left two thirds of my screen … which would be fine, except I can’t scroll the text at all unless I’m moused over the relatively small window where the article itself is.
The blinking browser icon is kind of cool though.
Thanks. Like the Constellation in the Doomsday Machine, I had not expected to take this blog engine into a real situation. I plan on moving all of this to LeanPub very soon.
I have a proposal: we string up everybody who puts an animated favicon on their site, and hang them in gibbets out the front of browser developers’ shops as a warning for all.
This has been tremendous. Thanks guys.
There have been so many great comments I am going to write a follow-up.
Like everything else in analysis, the more smart people you have asking honest, direct, penetrating questions, the better the terms are understood by everybody. This is exactly what I needed.
I’m having trouble parsing out the main theme here. Is it that “cognitive load is bad”? Is it the specific way of measuring that? Is it in how cognitive load happens? Is it that cognitive load is always unnecessary?
Also, possible mistake:
Should that be
cout << "Hello World"?Thanks for the correction. I freely chose from several languages and just winged it. It’s been fixed. Also I’m on vacation and have no access to a printer. This essay would be much tighter if I had my regular writing flow in place. Instead it feels to me more like an extended tweet.
Is a high Code Cognitive Load always bad? I can’t answer your question. Code Cognitive Load exists, whether we want to ignore it or not. There is a certain amount of mental work that it takes to understand and make changes to any application codebase, and that mental work can be measured empirically. It’s fair enough to argue about how to do that. It’s also fair enough to argue that some Code Cognitive Load is necessary for some domains.
The purpose of the essay was to introduce the term, provide an initial definition, and invite the community to start thinking about the implications of measuring how well we’re coding for other people. I think there are a ton of useful and quite dramatic implications here, but I don’t want to get ahead of things. Right now I’m just looking for people to tell me that I’m wrong. Thanks for the comment.