1. 14
  1. 2

    Very cool idea! I have some questions about the semantics, which don’t seem very comprehensively described:

    What is kept in memory, what is not? How are changes in types handled, with regards to the things that are kept in memory? What happens to globals defined in reloaded modules (don’t 100% remember now if OCaml has those)? How can I control which side-effects are re-run on reload, vs. which are not? How can Hotlink.is_hot_unloaded () ever be true? Does it mean the module will be unloaded, but not any running code from that module, so it needs to check this flag to self-terminate gracefully and do some custom communication if needing to “pass the baton” to a newly loaded version of the module? Also, a “module is unloaded” phrase, does it mean the old version of the module (possibly upgraded to a newer version), or any version of the module is now completely gone?

    1. 4

      Hello, I am the author.

      After compilation, an OCaml module is basically a record of values (some mutable ones and some that are functions/closures, if needed) paired with an initialisation function. During loading, the record is allocated, the initialisation function is called, and then we are done with that module. The module name now refers to the record.

      Unloading the module simply means forgetting the association between the module name and the record. Other parts of the program can still refer to values (and functions) defined in the module, those are not going to disappear.

      So the granularity of side effects that are rerun during reloading is the module (allocation of a new record, execution of the initialisation function). Modules that have changed and the whole cone of their reverse dependencies will be reevaluated.

      Does it mean the module will be unloaded, but not any running code from that module, so it needs to check this flag to self-terminate gracefully and do some custom communication if needing to “pass the baton” to a newly loaded version of the module?

      Yes.

      Also, a “module is unloaded” phrase, does it mean the old version of the module (possibly upgraded to a newer version), or any version of the module is now completely gone?

      It mean that the old version of the module is gone. Either a new version has been provided, or the dependency has disappeared.

      How are changes in types handled

      Changes in types do not receive any special treatment (as I said, the granularity is the module). There are good reasons to avoid that: OCaml type definitions are not structural, they aren’t even just nominal, they are generative. The action of defining a type is side effectful. Trying to handle changes in any clever way could easily result in breaking modular abstraction.

      It is not as bad as it sounds: it means that it is the responsibility of the developer to opt in for a special treatment. This does not have to be built-in Hotcaml. It can safely be built on top of existing primitives, for instance by using some reification of types (e.g. using https://github.com/thierry-martinez/refl or https://github.com/pqwy/tpf), and then automating some of the migration process by defining a compatibility relation between types (by induction on their structure). It is not easy, but orthogonal to the Hotcaml machinery.

      That being said, a ML-like language designed with reloading in mind would likely do things very differently.

      1. 1

        Thanks for the reply!

        A couple more questions if I may:

        Once execution of the entrypoints is done, the interpreter will watch the disk for changes.

        (Above quoted from the readme.) Does this mean that an entrypoint must not end with some blocking call, like main_loop ()? From a quick glance at hotcaml.ml, I seem to understand hotcaml.exe will ensure that the main thread never ends, instead of how this would (I presume) in regular OCaml code be the responsibility of the main app’s entry point?

        How are changes in types handled

        Changes in types do not receive any special treatment (…) OCaml type definitions are not structural, they aren’t even just nominal, they are generative. (…)

        I’m not good at type-theory at all, so just to make sure I got it: after every reload, any types are incompatible with their “previous versions”, regardless if the structure changed or not, yes? (Unless some extra explicit migration is done, e.g. through the libs you linked, or presumably some explicit serialize+deserialize.) Though, I now understand it would also typically probably not be that much of a problem, because any modules depending on the reloaded one will also be reloaded.

        1. 2

          (Above quoted from the readme.) Does this mean that an entrypoint must not end with some blocking call, like main_loop ()? From a quick glance at hotcaml.ml, I seem to understand hotcaml.exe will ensure that the main thread never ends, instead of how this would (I presume) in regular OCaml code be the responsibility of the main app’s entry point?

          Yes, it should not end with a blocking call. This is where “Lwt” comes in handy: it provides an alternative main-loop that allows concurrent and cooperative computations (async/promise/coroutines/,,,).

          I’m not good at type-theory at all, so just to make sure I got it: after every reload, any types are incompatible with their “previous versions”, regardless if the structure changed or not, yes? (Unless some extra explicit migration is done, e.g. through the libs you linked, or presumably some explicit serialize+deserialize.) Though, I now understand it would also typically probably not be that much of a problem, because any modules depending on the reloaded one will also be reloaded.

          Yes, types are incompatibles. Even without reload, if you do:

          type t = { a : int }
          type t = { a : int }
          

          you have two incompatible type t that have the same definition.

          Exhibiting an explicit migration function or as you suggest serialization+deserialization is the solution. With the proper framework, this can be done in a rather lightweight way.

          The migration function is actually a proof that the types are, in some sense, compatible, and that the user agreed for this compatibility to be exploited (you could also have, for instance, two types that are actually integers but should not be seen as compatible, for instance a delay in seconds and an amount of dollars). It is quite similar to serialize+deserialize, but it can skip the actual transformation and directly share compatible values.

      2. 2

        there was a discussion on hackernews where the author chimed in