This is IMO a) the best reason to deliver a high quality project and b) the biggest pitfall for a project’s success.
I had the privilege to spend a year working on a passion project, failing to make it work at first. One thing I learned: perhaps the biggest contributor to successful projects is ruthless pragmatism, putting personal pride at the second place.
The very short story:
I wanted to create the best possible solution for the problem that in music shows and parties, visuals were always either fully pre-rendered or never in sync with the music.
With a bunch of experience in realtime graphics programming and building music software, and newly added ways to connect to Ableton Live, I knew I could make it work. I built prototypes, toured with artists, proved it worked beautifully. Smoothly micro-adjusting movie to transport alignment solving video streaming delays, a recursive routing and mixing graphics pipeline, beat-based envelopes, so much good stuff was in there. Everyone I showed it to said they wanted to use it once it was done. However there was so much that it didn’t yet do. Non-realtime rendering, cross-platform support, flexible licencing, etc etc. I wanted to be proud before I considered it a product and thus never got to call it done. After a year of mostly fulltime work on it, and then many parttime hours in the following years, it was still not done. Life happened, I got kids, and it was still not done. I wasn’t selling the product so I couldn’t sustain development. I got a steady job and that would have been the end of it. Me wanting to be proud before calling it done ultimately would have made the project fail.
If it wasn’t for the other talented contributors that the project attracted, who got more space as my involvement dwindled. They were much more pragmatic, used SaaS for all code that was not core functionality, accepted many hiatuses (all three mentioned above, initially), built a website, and.. we just started selling.
As soon as a 1.0 was out, adoption began, income was being generated. Now, several years later, user numbers and income are steadily growing, initial hiatuses are filled; the feature set is still not complete, but the project is alive and well. I wouldn’t have started the project if it wasn’t for pride, but it wouldn’t have survived if pride would have stayed the main measure of success.
I appreciate you taking the time to share that story and the advice. Ruthless pragmatism is a good term and I’m going to keep that in mind this year. :)
I had the privilege to spend a year working on a passion project, failing to make it work at first. One thing I learned: perhaps the biggest contributor to successful projects is ruthless pragmatism and putting personal pride at the second place.
I am hoping to ship a passion project of my own this year and will be keeping this in my thoughts. It is also music related, with some ideas for integrating with FL Studio (probably using FL’s SDK(?) and JUCE’s new WebViews).
I saved up and took January off to work on my own personal project. Do it. Full speed ahead. Don’t regret it, and don’t justify it to others (who aren’t dependent on you financially). When you quit your job, suddenly 100% of your time belongs to you. Use your precious time on Earth to work on the things that matter to you.
As another Clojure fan of ten years for all the same reasons, I’m excited to see what’s in store for Jank this year. Best of luck to you.
Also, seems like @kristof has taken a month off from work, which is different from quitting your job, which comes with a lot of uncertainty. You really can’t dedicate 100% of your time and mental energy on blue sky personal projects with existential uncertainty after it.
Congratulations on getting to a point where you can make this move.
One question:
Seamless C++ interop
Would it be feasible to do seamless Rust interop as well, that is, some way of easily using Rust libraries from Jank? If your bet is that all of the interesting libraries are C or C++ libraries, or at least that they’ll have C or C++ APIs, then I understand.
Seamless Rust interop is going to be tough. Tougher than with C++, mainly due to the tooling, at this point. I’ve been working with some LLVM folks who’ve been building tooling specifically for other langs to interact with C++ in a JIT fashion. Until we have something like that for Rust, there’s not much of an opening.
There are people thinking about a stable Rust-specific ABI, which would do a lot for interoperability with rust if/when it gets implemented. Unfortunately I can’t seem to find the specific proposal I remember seeing for a named alternative to repr(C) that would fulfill this function.
There’s far more than just the ABI for seamless interop.
We built some proof-of-concept prototypes for C++ interop in Verona. The abstract machine is that C++ code is confined to a region (all C++ code operates on objects in a single region, C++ has no notion of regions it thinks that all objects live in a single global memory). We used clang to build an AST from a set of C++ modules and could export types, and instantiate templates so we would be able to surface C++ templates as Verona generics where the mapping made sense. We could then use clang libraries to generate wrapper functions (every argument that isn’t a primitive is passed as a pointer and in the Verona world it’s just an opaque blob of stuff that we have a pointer to) that we could call with a simple ABI (these would then be inlined during code generation, so clang handled all of our C++ ABI issues). Accessing fields in C++ objects worked the same way: generate an accessor function, clang will error if you try to access a private field, forward the error to the user rewriting source locations, LLVM inlines the accessor function and generates a single load or store plus whatever address calculation is needed.
All of this was simple because the C++ abstract machine is very permissive. You might want to handle standard-library unique and shared pointers as special things, but that’s about it. This is also why things like Sol3 make it easy to create C++ / Lua interop: Lua’s GC just takes ownership of a copy of a C++ object. If that object is a smart pointer, deallocating it when GC runs may deallocate the underlying C++ object.
In contrast, there’s a lot of stuff in Lua that wouldn’t be valid Rust. Lua lets you take a reference to an object that’s reachable from another object. If everything is shared / weak pointers, that’s fine. In Rust, that is not permitted but there’s no simple way of surfacing the borrow checker into Lua. With a lot of work, you could make Lua do something dynamic to check borrows, but that’s basically reimplementing the Rust borrow checker as a dynamic graph-walking thing, and that’s going to be painful.
My intuition is that while there’s no impedance mismatch between C++ and Rust’s data models, you have to duplicate all of the interop work that has to do with the mapping between the languages, not the data. For C++ this involves parsing headers, not object files. Rust can produce C-headers and a C-compatible FFI but consuming that directly loses the Rust data model, for instance. You could reconstruct it on the other end if you know exactly how Rust maps its types to C’s, but round-tripping like that is bound to be error-prone. Lastly, Rust’s runtime and C’s runtime are actually different; Rust’s panic unwinding is a footgun you have to manually avoid when writing FFIs.
So, seamless interop? I don’t think so. You have to redo most of the interop work and be able to consume the objects Rustc produces.
I’d like to hear people’s experience with and ideas for using Rust from other languages. I agree with jeaye that at this point it’s a lot harder to do than C++ or C.
I took a cursory look at using it from Common Lisp and didn’t get very far. My ground rules were no going through C, and no modification of Rust code (i.e. no adding #[pyfunction] like PyO3).
Basically, I want cl-autowrap, but instead of (autowrap:include “something.h”), I want (rffi:use “some.crate”)
I’d have something like “import ”, and then have the functions and data types available in a package named after the crate, like (crate-name:functionname …). For bonus points, I’d like to do it in such a way that users of my Lisp library only need to install the Rust binary, and not the whole Rust build system (in C and Debian terms, they need libfoo, but not libfoo-dev).
Congrats, welcome to the club. I suspect this day will mark two halves of your life.
Oh, hey! Much appreciated, you stopping by. I suspect you’re right. :)
This is IMO a) the best reason to deliver a high quality project and b) the biggest pitfall for a project’s success.
I had the privilege to spend a year working on a passion project, failing to make it work at first. One thing I learned: perhaps the biggest contributor to successful projects is ruthless pragmatism, putting personal pride at the second place.
The very short story:
I wanted to create the best possible solution for the problem that in music shows and parties, visuals were always either fully pre-rendered or never in sync with the music.
With a bunch of experience in realtime graphics programming and building music software, and newly added ways to connect to Ableton Live, I knew I could make it work. I built prototypes, toured with artists, proved it worked beautifully. Smoothly micro-adjusting movie to transport alignment solving video streaming delays, a recursive routing and mixing graphics pipeline, beat-based envelopes, so much good stuff was in there. Everyone I showed it to said they wanted to use it once it was done. However there was so much that it didn’t yet do. Non-realtime rendering, cross-platform support, flexible licencing, etc etc. I wanted to be proud before I considered it a product and thus never got to call it done. After a year of mostly fulltime work on it, and then many parttime hours in the following years, it was still not done. Life happened, I got kids, and it was still not done. I wasn’t selling the product so I couldn’t sustain development. I got a steady job and that would have been the end of it. Me wanting to be proud before calling it done ultimately would have made the project fail.
If it wasn’t for the other talented contributors that the project attracted, who got more space as my involvement dwindled. They were much more pragmatic, used SaaS for all code that was not core functionality, accepted many hiatuses (all three mentioned above, initially), built a website, and.. we just started selling.
As soon as a 1.0 was out, adoption began, income was being generated. Now, several years later, user numbers and income are steadily growing, initial hiatuses are filled; the feature set is still not complete, but the project is alive and well. I wouldn’t have started the project if it wasn’t for pride, but it wouldn’t have survived if pride would have stayed the main measure of success.
I appreciate you taking the time to share that story and the advice. Ruthless pragmatism is a good term and I’m going to keep that in mind this year. :)
I’m glad yours ended up working out.
I am hoping to ship a passion project of my own this year and will be keeping this in my thoughts. It is also music related, with some ideas for integrating with FL Studio (probably using FL’s SDK(?) and JUCE’s new WebViews).
I saved up and took January off to work on my own personal project. Do it. Full speed ahead. Don’t regret it, and don’t justify it to others (who aren’t dependent on you financially). When you quit your job, suddenly 100% of your time belongs to you. Use your precious time on Earth to work on the things that matter to you.
As another Clojure fan of ten years for all the same reasons, I’m excited to see what’s in store for Jank this year. Best of luck to you.
As a husband and father I find this take pretty hilarious :-)
Also, seems like @kristof has taken a month off from work, which is different from quitting your job, which comes with a lot of uncertainty. You really can’t dedicate 100% of your time and mental energy on blue sky personal projects with existential uncertainty after it.
No, I quit my job.
Wish you the best of luck then! I hope it turns out to be a great decision.
Nice! I’m glad this worked out for you. I’m quite excited to have all of that time, though it does come with a lot of anxiety. Thanks for stopping by.
Just remember to rest, or you’ll find out you have no energy to do anything.
Source: ive lost count on my burnouts
Have fun! The two months I spent working full-time on my database project were the happiest of my career.
Thanks! I’m glad you tried it out and enjoyed it, too. I’ve had several people convey the same sentiment.
Congrats Jeaye! Excited to continue watching jank progress.
Thanks, Jason. Apologies for the late follow up on X; entirely missed your message last year. Let’s chat more!
Congratulations on getting to a point where you can make this move.
One question:
Would it be feasible to do seamless Rust interop as well, that is, some way of easily using Rust libraries from Jank? If your bet is that all of the interesting libraries are C or C++ libraries, or at least that they’ll have C or C++ APIs, then I understand.
Seamless Rust interop is going to be tough. Tougher than with C++, mainly due to the tooling, at this point. I’ve been working with some LLVM folks who’ve been building tooling specifically for other langs to interact with C++ in a JIT fashion. Until we have something like that for Rust, there’s not much of an opening.
I’m very interested, though.
There are people thinking about a stable Rust-specific ABI, which would do a lot for interoperability with rust if/when it gets implemented. Unfortunately I can’t seem to find the specific proposal I remember seeing for a named alternative to
repr(C)that would fulfill this function.crabi: https://github.com/rust-lang/rust/issues/111423
Thanks! That’s it.
There’s far more than just the ABI for seamless interop.
We built some proof-of-concept prototypes for C++ interop in Verona. The abstract machine is that C++ code is confined to a region (all C++ code operates on objects in a single region, C++ has no notion of regions it thinks that all objects live in a single global memory). We used clang to build an AST from a set of C++ modules and could export types, and instantiate templates so we would be able to surface C++ templates as Verona generics where the mapping made sense. We could then use clang libraries to generate wrapper functions (every argument that isn’t a primitive is passed as a pointer and in the Verona world it’s just an opaque blob of stuff that we have a pointer to) that we could call with a simple ABI (these would then be inlined during code generation, so clang handled all of our C++ ABI issues). Accessing fields in C++ objects worked the same way: generate an accessor function, clang will error if you try to access a private field, forward the error to the user rewriting source locations, LLVM inlines the accessor function and generates a single load or store plus whatever address calculation is needed.
All of this was simple because the C++ abstract machine is very permissive. You might want to handle standard-library unique and shared pointers as special things, but that’s about it. This is also why things like Sol3 make it easy to create C++ / Lua interop: Lua’s GC just takes ownership of a copy of a C++ object. If that object is a smart pointer, deallocating it when GC runs may deallocate the underlying C++ object.
In contrast, there’s a lot of stuff in Lua that wouldn’t be valid Rust. Lua lets you take a reference to an object that’s reachable from another object. If everything is shared / weak pointers, that’s fine. In Rust, that is not permitted but there’s no simple way of surfacing the borrow checker into Lua. With a lot of work, you could make Lua do something dynamic to check borrows, but that’s basically reimplementing the Rust borrow checker as a dynamic graph-walking thing, and that’s going to be painful.
My intuition is that while there’s no impedance mismatch between C++ and Rust’s data models, you have to duplicate all of the interop work that has to do with the mapping between the languages, not the data. For C++ this involves parsing headers, not object files. Rust can produce C-headers and a C-compatible FFI but consuming that directly loses the Rust data model, for instance. You could reconstruct it on the other end if you know exactly how Rust maps its types to C’s, but round-tripping like that is bound to be error-prone. Lastly, Rust’s runtime and C’s runtime are actually different; Rust’s panic unwinding is a footgun you have to manually avoid when writing FFIs.
So, seamless interop? I don’t think so. You have to redo most of the interop work and be able to consume the objects Rustc produces.
I’d like to hear people’s experience with and ideas for using Rust from other languages. I agree with jeaye that at this point it’s a lot harder to do than C++ or C.
I took a cursory look at using it from Common Lisp and didn’t get very far. My ground rules were no going through C, and no modification of Rust code (i.e. no adding #[pyfunction] like PyO3).
Basically, I want
cl-autowrap, but instead of (autowrap:include “something.h”), I want (rffi:use “some.crate”)I’d have something like “import ”, and then have the functions and data types available in a package named after the crate, like (crate-name:functionname …). For bonus points, I’d like to do it in such a way that users of my Lisp library only need to install the Rust binary, and not the whole Rust build system (in C and Debian terms, they need libfoo, but not libfoo-dev).
Previously, on Lobsters there was discussion of Vale’s exploration of seamless Rust interop.