REPLs are nice but they work well only for reasonably isolated code with few dependencies.
Which is exactly the reason to have all your code broken up in this way. Even if you don’t use REPL.
It’s hard to set up a complex object to pass into a function.
Avoid complex objects then. Everything wrapped in bespoke objects with methods that each don’t even need all of the object’s state is one of the most prominent things that’s wrong about OO.
“Avoid complex objects” is a good rule of thumb to keep code clean, but if you’re modeling a complex real-world concept, the complexity needs to be reflected in your model somewhere or it will be an incorrect representation.
Ah-hah! A friend of mine once put it very nicely when, in a fit of rage caused by trying to refactor a 4,000-line drive, he just yelled fsck it, this DMA controller here is super murky so of course my driver for it is going to be super murky, sillicon don’t care I gotta write clean code.
Not all things admit a “beautiful” model. A set of tools that only works with things that admit a beautiful model makes for great blog posts but lots and lots of frustration when they meet the messy real world of here are the 14 steps you have to perform in order to get a byte off this bus, yes the writes at steps 6, 7 and 10 are at aliased, unaligned memory locations, why do you ask?
But isn’t this what abstraction… does? We have an ugly inconsistent machine that doesn’t solve our problem very elegantly (hardware of some sort) so we sandwich in other, abstract machines of higher and higher level of abstraction until the thing we want to express almost falls right out of just asking the question of the highest level?
(And each level abstract machine does mask over a few of the most glaring problems of the machine just underneath it, so it’s low-dependency and trivial to test and reason about.)
When the underlying hardware is an ugly, inconsistent machine, the first layer of abstract machines will have to mimic its model, you don’t really have a choice about that. If it consists of 12 non-consecutive memory-mapped registers that can be written independently, but not concurrently, then you’ll probably have 12 pointers and a mutex (at best – sometimes you’ll need 12…) whether you like it or not, because ultimately data will have to be written in memory. If getting a byte off a bus requires you to do fifteen ugly steps, then your code will have to do them.
Now, in addition to that, you can sometimes wrap this is a unified, ellegant interface. That won’t save the first layer of code but it will at least allow third parties to use it without having to deal with its peculiarities.
If you poke around Linux’ driver tree (I’m mentioning it just because it’s probably the one that it’s easiest to find docs about, it’s not necessarily my favourite), you’ll find plenty of cases where the driver authors had to go to incredible lengths, and sometimes do a bunch of horrendous hacks, in order to get the driver to conform to the unified interface while still keeping the hardware happy.
But even that doesn’t always work. Abstractions inevitably leak, for example – you can’t augment the “unified” interface to account for every hardware quirk in existence, otherwise you end up with an interface so large that nobody implements it correctly. That’s true outside the world of device drivers, too, I’ve seen plenty of codebases where people really wanted ellegant, unified interfaces, and they ended up with 120-function interfaces to “middleware” interfaces that plugged to the real interfaces of the underlying implementation. Most programmers, including yours truly, just aren’t smart enough to navigate the innards of that kind of a behemoth, which is why it looks great in the docs but it’s a bug-ridden piece of junk if you try to run it.
Speaking strictly for device drivers, you also can’t pile up abstractions forever, at some point you have to stop at the unified API everyone expects. E.g. if you pick ten devices from the same family, you could probably come up with a higher-level, more ellegant abstraction than the one their device drivers implement on a given platform, but it would be completely useless because that’s not the interface the next layer in the kernel expects. Sometimes it’s literally impossible to change the next layer (e.g. on Windows, unless you’re Microsoft), but even if it’s technically possible, that’s a whole other can of worms.
This seems like it would only work for pure functional code. And by pure functional, I mean at least Haskell without the IO monad. Any code that touches shared mutable state or the outside world in any way can’t fulfill this requirement. Arguably, most code should be able to run in such an isolated fashion in tests, but how much real code lives up to that standard?
Abstract interpretation and program slicing work just fine on imperative programs. Here’s a concrete example, the Wisconsin Program Slicing Tool, which works on C programs: https://research.cs.wisc.edu/wpis/slicing_tool/
I agree. Outside state makes things way too unpredictable. Furthermore, the behaviour the author describes (what if X was 3 here?) has never happened to me while writing Haskell (and Scheme). In such languages, it almost seems a tool like this is unnecessary. It’s a shame imperative programming still is the norm today.
While the write-up is certainly written with an imperative bias, I think the general idea applies to Haskell-like and other languages as well. The difference is only which questions are answered by such a system. I’m not convinced that in Haskell one does not sit in front of the computer and mentally simulate the language semantics to determine the outcome given some imagined inputs. That is the kind of mental work that needs to be be easy to offload to the computer.
This seems like it would only work for pure functional code. And by pure functional, I mean at least Haskell without the IO monad. Any code that touches shared mutable state or the outside world in any way can’t fulfill this requirement.
Eh, depends. I’m super-used to having dummy input from stdin or a socket or the like; they’re just file descriptors. And people have used mocks to fake out external mutable state in unit tests for years. Extending that to a concept of an implicit mock so you can see what happens in real time when a certain input of your choice theoretically comes over the wire doesn’t even seem like a big leap; just a logical extension of what we’re doing, couched in better tooling.
In other words, I’m very unconvinced this is as difficult as you’re saying, at least on that particular level.
From the title, I thought the article would be about writing declarative code when you can. Code takes fewer steps to understand when instead of having to deduce the result from the steps, you read the result right in the code. Don’t tell me the recipe, show me the sandwich.
But a high-minded practice like that is no help if the code is already written. If you’re maintaining a legacy system — and sooner or later, don’t we all? — you could benefit immensely from this kind of tool. I’d love to see it realized.
+1. I would love to implement this for my language. The open question for me is, what does the user interface look like, and are there existing systems to copy the UI from? You mention a what_if primitive as an example; any other language changes?
More on the what_if idea: I think it should be composable. It should also start from the primitives of the system. One idea is below, based on refinement types.
If you have a type
type Circle:
Point center
Color color
I should be able to define circles in a certain ‘range’, e.g.
# all circles with center x=between 1 and 10; y between 1 and 20
circles1 = Circle(center=Point(0..10, 0..20), color=...)
# all circles that are red
circles2 = Circle(color=Red, center=...)
Now I should be able to compose these
# all red circles centered at x=between 1 and 10; y between 1 and 20
circles3 = circles1 & circles2
So the what_if is just the invocation of an program with a refinement type. If you consider trace3 is “within” trace1, these what_ifs form a tree. Our base assumptions for an execution trace at the higher levels of this tree and our later assumptions are the children. Some good way to visualize these would be nice. So you could click on one of these and tweak it independently, while still keeping the overall tree composition intact.
The other thing implied above that I think is required is an easy way to express and visualize ‘refinement types’. Above I use ellipsis to show missing information. We need a good way to visualize the traced values within a composed what_if as we well.
I think this is a classic case of „ideas are nothing, execution is everything“.
Also I like to think about code as text which tells a human what a computer should do. Code is communication. Hard to read code should be refactored as a very hard to understand sentences should be rephrased.
Which is exactly the reason to have all your code broken up in this way. Even if you don’t use REPL.
Avoid complex objects then. Everything wrapped in bespoke objects with methods that each don’t even need all of the object’s state is one of the most prominent things that’s wrong about OO.
“Avoid complex objects” is a good rule of thumb to keep code clean, but if you’re modeling a complex real-world concept, the complexity needs to be reflected in your model somewhere or it will be an incorrect representation.
Ah-hah! A friend of mine once put it very nicely when, in a fit of rage caused by trying to refactor a 4,000-line drive, he just yelled fsck it, this DMA controller here is super murky so of course my driver for it is going to be super murky, sillicon don’t care I gotta write clean code.
Not all things admit a “beautiful” model. A set of tools that only works with things that admit a beautiful model makes for great blog posts but lots and lots of frustration when they meet the messy real world of here are the 14 steps you have to perform in order to get a byte off this bus, yes the writes at steps 6, 7 and 10 are at aliased, unaligned memory locations, why do you ask?
But isn’t this what abstraction… does? We have an ugly inconsistent machine that doesn’t solve our problem very elegantly (hardware of some sort) so we sandwich in other, abstract machines of higher and higher level of abstraction until the thing we want to express almost falls right out of just asking the question of the highest level?
(And each level abstract machine does mask over a few of the most glaring problems of the machine just underneath it, so it’s low-dependency and trivial to test and reason about.)
When the underlying hardware is an ugly, inconsistent machine, the first layer of abstract machines will have to mimic its model, you don’t really have a choice about that. If it consists of 12 non-consecutive memory-mapped registers that can be written independently, but not concurrently, then you’ll probably have 12 pointers and a mutex (at best – sometimes you’ll need 12…) whether you like it or not, because ultimately data will have to be written in memory. If getting a byte off a bus requires you to do fifteen ugly steps, then your code will have to do them.
Now, in addition to that, you can sometimes wrap this is a unified, ellegant interface. That won’t save the first layer of code but it will at least allow third parties to use it without having to deal with its peculiarities.
If you poke around Linux’ driver tree (I’m mentioning it just because it’s probably the one that it’s easiest to find docs about, it’s not necessarily my favourite), you’ll find plenty of cases where the driver authors had to go to incredible lengths, and sometimes do a bunch of horrendous hacks, in order to get the driver to conform to the unified interface while still keeping the hardware happy.
But even that doesn’t always work. Abstractions inevitably leak, for example – you can’t augment the “unified” interface to account for every hardware quirk in existence, otherwise you end up with an interface so large that nobody implements it correctly. That’s true outside the world of device drivers, too, I’ve seen plenty of codebases where people really wanted ellegant, unified interfaces, and they ended up with 120-function interfaces to “middleware” interfaces that plugged to the real interfaces of the underlying implementation. Most programmers, including yours truly, just aren’t smart enough to navigate the innards of that kind of a behemoth, which is why it looks great in the docs but it’s a bug-ridden piece of junk if you try to run it.
Speaking strictly for device drivers, you also can’t pile up abstractions forever, at some point you have to stop at the unified API everyone expects. E.g. if you pick ten devices from the same family, you could probably come up with a higher-level, more ellegant abstraction than the one their device drivers implement on a given platform, but it would be completely useless because that’s not the interface the next layer in the kernel expects. Sometimes it’s literally impossible to change the next layer (e.g. on Windows, unless you’re Microsoft), but even if it’s technically possible, that’s a whole other can of worms.
To clarify, I do still think the described tooling would be useful.
This seems like it would only work for pure functional code. And by pure functional, I mean at least Haskell without the IO monad. Any code that touches shared mutable state or the outside world in any way can’t fulfill this requirement. Arguably, most code should be able to run in such an isolated fashion in tests, but how much real code lives up to that standard?
Abstract interpretation and program slicing work just fine on imperative programs. Here’s a concrete example, the Wisconsin Program Slicing Tool, which works on C programs: https://research.cs.wisc.edu/wpis/slicing_tool/
I agree. Outside state makes things way too unpredictable. Furthermore, the behaviour the author describes (what if X was 3 here?) has never happened to me while writing Haskell (and Scheme). In such languages, it almost seems a tool like this is unnecessary. It’s a shame imperative programming still is the norm today.
While the write-up is certainly written with an imperative bias, I think the general idea applies to Haskell-like and other languages as well. The difference is only which questions are answered by such a system. I’m not convinced that in Haskell one does not sit in front of the computer and mentally simulate the language semantics to determine the outcome given some imagined inputs. That is the kind of mental work that needs to be be easy to offload to the computer.
Eh, depends. I’m super-used to having dummy input from
stdinor a socket or the like; they’re just file descriptors. And people have used mocks to fake out external mutable state in unit tests for years. Extending that to a concept of an implicit mock so you can see what happens in real time when a certain input of your choice theoretically comes over the wire doesn’t even seem like a big leap; just a logical extension of what we’re doing, couched in better tooling.In other words, I’m very unconvinced this is as difficult as you’re saying, at least on that particular level.
Elm code does. We can’t write anything else than pure functional!
From the title, I thought the article would be about writing declarative code when you can. Code takes fewer steps to understand when instead of having to deduce the result from the steps, you read the result right in the code. Don’t tell me the recipe, show me the sandwich.
But a high-minded practice like that is no help if the code is already written. If you’re maintaining a legacy system — and sooner or later, don’t we all? — you could benefit immensely from this kind of tool. I’d love to see it realized.
+1. I would love to implement this for my language. The open question for me is, what does the user interface look like, and are there existing systems to copy the UI from? You mention a
what_ifprimitive as an example; any other language changes?Hi Doug,
More on the
what_ifidea: I think it should be composable. It should also start from the primitives of the system. One idea is below, based on refinement types.If you have a type
I should be able to define circles in a certain ‘range’, e.g.
Now I should be able to compose these
This could be extended for full traces, e.g
So the
what_ifis just the invocation of an program with a refinement type. If you consider trace3 is “within” trace1, these what_ifs form a tree. Our base assumptions for an execution trace at the higher levels of this tree and our later assumptions are the children. Some good way to visualize these would be nice. So you could click on one of these and tweak it independently, while still keeping the overall tree composition intact.The other thing implied above that I think is required is an easy way to express and visualize ‘refinement types’. Above I use ellipsis to show missing information. We need a good way to visualize the traced values within a composed what_if as we well.
I wish I could turn off the part that does simulation of code sometimes. It’s brutal.
I think this is a classic case of „ideas are nothing, execution is everything“. Also I like to think about code as text which tells a human what a computer should do. Code is communication. Hard to read code should be refactored as a very hard to understand sentences should be rephrased.