I read the article that this is in response to and wondered if regular CLOS could have been used with some metaclass magic, as well, so I’m glad to see this. I like the changes proposed here, but I feel like there’s still many missing pieces in my understanding. I’m not sure the variable bindings position-x and position-y are coming from in the image drawing example snippet. And since ‘draw’ takes no arguments, I guess it’s implied that there’s global state of all entities? I don’t love that. I’m not sure how I’d put together a scene graph in such a system.
Thanks for the feedback. I lifted most of the code from the original post without thinking and the result is that some things are screwed up. Accessors should take the instance as argument and therefore position-x should be (position-x position). I will see if I can sneak it in with an edit.
I think the fact that draw reads from global state is a red-herring. There’s nothing preventing you from changing it to get the underlying storage as parameters but you would have to lift some control flow somewhere else, e.g.read only/read write access to the underlying storage must still be taken.
I also don’t like to have global state around but I’m not going to be too formal in a blog-post because otherwise I may as well implement it already.
I’m not sure either how to put together a scene-graph but that’s on me not knowing rendering techniques :)
Thanks for the follow-up! I’ll adjust and read the example code more like pseudo-lisp. I wasn’t sure how much of this was actually prototyped.
I think it would make sense for the component class to store the memory pool/arena/whatever for all instances and to have a world/scene class with an instance slot for a collection of entities. The draw procedure could then take a world/scene as an argument to query for components.
Hey, author of original ECS piece here. Thanks for your thoughs! Actually when starting that ECS library of mine, I’ve considered using CLOS+MOP, but gave up that idea pretty quickly since I’m nowhere near being MOP expert.
You’re right that the CLOS approach looks syntactically clearer and does not involve building your own DSL, instead relying on things in CL standard. However, using CLOS I’m always concerned about performance, and I wanted this library to be as fast as possible (hence the name, cl-fast-ecs). For instance, if we’re redefining how slots are allocated on object and how they’re accessed, can slot access be inlined? Or can it be at least a tiny function to call and not a full-blown generic dispatch using slot-value-using-class or something? (sorry, I’m definitely no MOP expert)
Another issue I’m interested in is redefinition of components. As you accurately pointed out, I was forced to implement runtime redifinition, and CLOS contains such machinery out of the box. I’ve actually implemented quite a lot of it: not only the storage arrays for component slots are properly reallocated when the component is redefined, but the code using that component would still function (well, if the type of slots haven’t been changed that is). This is achieved by having tiny 20-something byte function that serves as slot accessor every time it is needed, and in release mode (which should be turned on explicitly) even this accessor is inlined and optimized away by SBCL into just a few machine instructions. I wonder if one could achieve something like that with MOP.
Next, there’s a matter of portability: right now my library passes its test suite using 6 different CL implementations (there were 7, but Clasp does not like complex macros defining other macros). I know there are some portability issues with MOP, hence the wrapper closer-mop exists; I’m not sure doing something such deep and low-level as changing how slots are accessed and where their values are stored can be done portably.
Finally, I was going to ask if you know some good MOP resources so I can educate myself a little bit and do try out approach you’ve outlined.
For instance, if we’re redefining how slots are allocated on object and how they’re accessed, can slot access be inlined? Or can it be at least a tiny function to call and not a full-blown generic dispatch using slot-value-using-class or something?
This is definitely compiler dependent. There is no way that all your targets will support all optimizations. I would expect this to be optimized away in a modern compiler.
This is achieved by having tiny 20-something byte function that serves as slot accessor every time it is needed, and in release mode (which should be turned on explicitly) even this accessor is inlined and optimized away by SBCL into just a few machine instructions. I wonder if one could achieve something like that with MOP.
So you actually have proof that SBCL is very good at this kind of optimization. Take a swing at it by starting with make-class and make-instance and see where it goes. I would expect this to work. This simple test will actually be easier than reading the documentation (?).
Next, there’s a matter of portability:
It’s very hard to get portability and performance. Different compilers will make different choices and there isn’t much that you can do. The closer-mop is more about bridging the implementation of the CLOS with the presentation of the MOP that came later (?).
For instance, if we’re redefining how slots are allocated on object and how they’re accessed, can slot access be inlined? Or can it be at least a tiny function to call and not a full-blown generic dispatch using slot-value-using-class or something? (sorry, I’m definitely no MOP expert)
You can do some MOP-ery to make something non-redefinable. With a bit of work, this can be done in a way that gives the optimizer a lot to work with.
I know there are some portability issues with MOP, hence the wrapper closer-mop exists; I’m not sure doing something such deep and low-level as changing how slots are accessed and where their values are stored can be done portably.
99% of what closer-mop does is just re-exporting symbols from different packages, since there’s no standard saying which package the MOP symbols should live in. I’ve seen many people do stuff like change how slots are stored, so I don’t think it poses any issues.
I read the article that this is in response to and wondered if regular CLOS could have been used with some metaclass magic, as well, so I’m glad to see this. I like the changes proposed here, but I feel like there’s still many missing pieces in my understanding. I’m not sure the variable bindings position-x and position-y are coming from in the image drawing example snippet. And since ‘draw’ takes no arguments, I guess it’s implied that there’s global state of all entities? I don’t love that. I’m not sure how I’d put together a scene graph in such a system.
Thanks for the feedback. I lifted most of the code from the original post without thinking and the result is that some things are screwed up. Accessors should take the instance as argument and therefore
position-xshould be(position-x position). I will see if I can sneak it in with an edit.I think the fact that draw reads from global state is a red-herring. There’s nothing preventing you from changing it to get the underlying storage as parameters but you would have to lift some control flow somewhere else, e.g.read only/read write access to the underlying storage must still be taken.
I also don’t like to have global state around but I’m not going to be too formal in a blog-post because otherwise I may as well implement it already.
I’m not sure either how to put together a scene-graph but that’s on me not knowing rendering techniques :)
Thanks for the follow-up! I’ll adjust and read the example code more like pseudo-lisp. I wasn’t sure how much of this was actually prototyped.
I think it would make sense for the component class to store the memory pool/arena/whatever for all instances and to have a world/scene class with an instance slot for a collection of entities. The draw procedure could then take a world/scene as an argument to query for components.
Hey, author of original ECS piece here. Thanks for your thoughs! Actually when starting that ECS library of mine, I’ve considered using CLOS+MOP, but gave up that idea pretty quickly since I’m nowhere near being MOP expert.
You’re right that the CLOS approach looks syntactically clearer and does not involve building your own DSL, instead relying on things in CL standard. However, using CLOS I’m always concerned about performance, and I wanted this library to be as fast as possible (hence the name, cl-fast-ecs). For instance, if we’re redefining how slots are allocated on object and how they’re accessed, can slot access be inlined? Or can it be at least a tiny function to call and not a full-blown generic dispatch using slot-value-using-class or something? (sorry, I’m definitely no MOP expert)
Another issue I’m interested in is redefinition of components. As you accurately pointed out, I was forced to implement runtime redifinition, and CLOS contains such machinery out of the box. I’ve actually implemented quite a lot of it: not only the storage arrays for component slots are properly reallocated when the component is redefined, but the code using that component would still function (well, if the type of slots haven’t been changed that is). This is achieved by having tiny 20-something byte function that serves as slot accessor every time it is needed, and in release mode (which should be turned on explicitly) even this accessor is inlined and optimized away by SBCL into just a few machine instructions. I wonder if one could achieve something like that with MOP.
Next, there’s a matter of portability: right now my library passes its test suite using 6 different CL implementations (there were 7, but Clasp does not like complex macros defining other macros). I know there are some portability issues with MOP, hence the wrapper closer-mop exists; I’m not sure doing something such deep and low-level as changing how slots are accessed and where their values are stored can be done portably.
Finally, I was going to ask if you know some good MOP resources so I can educate myself a little bit and do try out approach you’ve outlined.
This is definitely compiler dependent. There is no way that all your targets will support all optimizations. I would expect this to be optimized away in a modern compiler.
So you actually have proof that SBCL is very good at this kind of optimization. Take a swing at it by starting with
make-classandmake-instanceand see where it goes. I would expect this to work. This simple test will actually be easier than reading the documentation (?).It’s very hard to get portability and performance. Different compilers will make different choices and there isn’t much that you can do. The closer-mop is more about bridging the implementation of the CLOS with the presentation of the MOP that came later (?).
You can do some MOP-ery to make something non-redefinable. With a bit of work, this can be done in a way that gives the optimizer a lot to work with.
99% of what closer-mop does is just re-exporting symbols from different packages, since there’s no standard saying which package the MOP symbols should live in. I’ve seen many people do stuff like change how slots are stored, so I don’t think it poses any issues.