I’m not even 100% what to call it, hence the poor phrasing. In Ruby it’s often called “monkeypatching” but emacs lisp also has it: any code in the program can replace existing procedures/variables anywhere else in the program.
This may seem like a bad thing, and sometimes it is, but it’s super useful also: for example in Emacs I can patch a core module with an improved version by adding a definition of a single core procedure to my config file without having to overwrite the actual core file itself.
So I’m wondering if people know of other languages/environments with similar features? Probably more LISPs, but I think not Schemes quite because of the module systems?
I’d say the term you look for is late binding. The bad thing is it violates the Open-closed principle.
Unix shells also qualify. You can replace binaries in /bin and the behavior of lots of shell scripts changes.
Dynamic libraries allow monkeypatching for C (and whatever). You can override any libc function with LD_PRELOAD, for example.
Shell (especially using functions) and LD_PRELOAD are excellent examples I hadn’t considered, thanks!
But LD_PRELOAD is only used to select which function definition is used at startup. Lisps, Smalltalk, etc. allow redefining functions in a running program.
Depending on what you mean by “in the program”, code from dynamic libraries may not count.
Do you know if some sort of dynamic scoping is required to implement this?
Yes, you could say shells and dynamic libraries use dynamic scoping.
No, it is not required in general. Smalltalk, Python, Ruby, and others allow you to modify lexically scoped namespaces by monkeypatching.
Smalltalk is an easy answer. :-)
In Smalltalk, you can replace the contents of any method in any class of the system. Obviously, you’d better not be wrong.
Almost, but not quite: certain methods are actually opcodes (in ST80, arithmetic, ifTrue, and a couple of others), and modern smalltalks will open-code them, so overriding the method will do nothing unless the primitive fails. Even the ST80 blue book interpreter handles ifTrue directly in the bytecode, and the 32 “special” messages with their own opcodes result in directly calling a primitive without message lookup. Now, you’re not likely to want to override integer addition or
Object>>class, but it still bothers me that you can’t.Objective-C. It’s frowned upon but it’s doable. (Back in 2010 Chromium on Mac used to have an example of this, as part of its then-very-unusual UI with the tabs sticking into the title bar. This required messing with NSWindow.)
Io, does, if I recall.
Forth, after a fashion, does as well, though in a proper forth, you can overload a definition going forward, but not backward (aka previously defined words don’t usually late-binding).
Javascript allows some level of this, which is often used for polyfills.
This is a feature that can be very useful, but it’s also a good way to build out a system that’s very nonstandard.
Late binding or monkeypatching is a great tool to have when you need to mock functionality for tests. Dependency injection is not fun
On the JVM you can override anything on the classpath really. The simplest method is just overriding it using another file. If you have a library that exposes a class with a namespace
com.foo.bar.Frobyou can place a file with that namespace and your application will see that one first, since it will look first at the immediate classpath and only then the libraries. So you can just make asrc/main/java/com/foo/bar/Frob.javaand put whatever you want there. I have patched multiple libraries this way in order to fix things quickly before upstreaming them.Not for JDK libraries, and not for modules anymore. :-)
This lets your override entire packages or classes, but not individual methods on a class, I think?
I do it in Python occasionally.
Have been bitten by this in python when a library I imported redefined a global object. It was not a fun debugging session.
Yeah, can imagine. I usually only use it under extreme circumstances, and
witha context manager, to minimize the impact. Kind of similar to what patchy does (although patchy is even more extreme and less fun to debug)Python only allow it to a certain extent. You can monkeypatch any
module, even stdlib ones, but you can’t monkeypatch the builtin types or instances of builtin types. modules from C extensions are not monkeypatcheable as well, I think?Erlang, to some extent. There is also Prolog.
I’m not sure why you would say “to some extent” here. Erlang takes the art of redefinition to a whole new level that no other runtime in the world even comes close to.
When you want to reload a module in Erlang, the VM keeps both the old and the new modules in memory at the same time so as not to interrupt currently-running processes using the old module. When a given process makes a call that the VM can determine makes it safe to do so, it swaps out the module with the new one. But since the definitions of records, etc, may have changed in the new module, every new module can provide an “upgrade” function which takes values in the format of the old version and allows them to be converted into the new version, for instance by adding new fields that previously didn’t exist.
This is how you get continuously-evolving systems that have been running without a restart for years and years. As far as I know, no other system offers reloading like this outside academia.
That Erlang feature sounds very cool, and very in line with my discovery direction! Does it always require full module replacements, though? Or can I overwrite a single member of a module without editing the original source and reloading the whole thing?
The bit where it keeps both versions intact and intelligently hot-swaps specifically requires module-level granularity. It’s been a while since I’ve done Erlang; there may be other forms of reloading which don’t have that safety built-in.
Well it depends on the meaning of “overriding any existing definition”. Modules are mutable in some languages (the OP mentioned Ruby) so it’s really easy to monkey patch functions for testing. For example, it’s a common pattern to mock like this the function that returns the current timestamp, run some automated tests and revert the default implementation.
I don’t know if it’s possible to do this with Erlang’s hot code reloading but in anyway it works very differently than monkey patching in Ruby/Python/JavaScript, so there are probably different limitations.
PS: It seems that it is possible: https://github.com/eproxus/meck
PostScript allows that and according to the creators this is deliberate to be able to patch buggy PostScript words in printers by allowing documents to redefine them with fixed versions.
Julia will do this. One can specialize or outright override existing function methods. Julia’s JIT compiler will even go back and recompile any code that may have depended on the old method to use the new.
Note of course this is dangerous when used carelessly - the system is intended for extending or specializing functions for new input types, rather than replacing methods. One fun thing to do is replace something fundamental like integer addition and watch everything immediately break, via e.g.
Base.:+(a::Int64, b::Int64) = 0.(While much of Julia’s compiler is written in Julia, it is itself is “immune” to such redefinitions because it lives in it’s own fixed “world”, but so much of the system depends on the standard libary that this instantly destroys any REPL session).
Julia is actually at the top of my list! I just finished reading the manual, and I find it a bit unclear about this. It seems to indicate that sometimes an already-compiled expression or incrementally-compiled module won’t see overrides that come later?
Golang does it as well.
(Sorry for the X-Y-ing in advance.)
I think a core idea that would deal with some of the problems people rightfully brought up here is focusing from
to
Which in turn raises the question, which I think is most helpful to your endeavor:
Which languages has a mechanism that allows supplying overriding definitions in the style of
If I understand, you are asking about metaprogramming? If so, Tcl has neat features for that:
See also:
F# lets you define a module with the same name as a different module (or class) in a different namespace, bring them both in scope, and use all definitions as if from a single module. There’s also
type String with ...which I believe C# has a version of as well.Clojure does as well, as probably most lisps.
Yup, Scheme allows it too
I’m curious about this now. I had thought from my research that Scheme would not allow this in the presence of modules. If I have a Scheme module that defines a and exports b which calls a and c which calls a and b, can I overwrite a from outside the module so that b’s behaviour changes? Or b so that c’s behaviour changes?
If the procedure is not exported, you can’t access it from the outside, which means you can’t change the value of the identifier. However, from inside the module you can change it. If it is exported, it’s undefined behaviour if you may change it or not. CHICKEN allows redefinition of exported identifiers though.
Correct, with the exception of special forms (for example, you can’t re-def
let).That’s something I’ve never actually tried, but makes sense.
I did at some point realise that python allows you to override the
__plus__method, which can lead to… interesting results.The ML family and Haskell allow shadowing any bindings—though it has no effect on code that used old definitions before the new module was opened, due to strict lexical scoping.
It’s perfectly good for redefining any part of any library for the scope of your program though. Reinventing the standard library or resurrecting one from another era is a rather popular pastime.
But that’s still not quite the same thing. If library L contains definition A that uses definition B, you can redefine B all you want, even in the same module as your call to L.A, but A will still use the old definition of B.