I like this idea of including a table of contents and making it semantic and more than just a doc string (if you put a breakdown of the codebase in the README it might get missed if someone jumps to the code and is likely to fall out of date). An extension of this idea is to enforce documentation coverage through unit tests: https://simonwillison.net/2018/Jul/28/documentation-unit-tests/
Personally I’ve experimented with generating example-based documentation from the tests themselves. This works really well and you can ask contributors to write a test for their new feature and it’s covered in the documentation right away!
Using software professionally isn’t about having a chic, boutique experience - it’s about getting the job done as quickly and efficiently as possible.
This, I think, cuts to the heart of the issue. Product designers are busy trying to design “experiences” and I just want stuff that I can treat like a utility. I want my software to be like my dumb fridge – it’s a means to an end. Just keep my dang food cold. If I’m conscious of how I’m using my fridge, somebody screwed up.
Even though Apple is better at design than most companies, I completely blame this “experience” trend on them. They used to be better. Try using an iPod from a decade ago and notice how much more of a utility-like experience it is. It feels more like a tool to listen to music rather than some flashy app that’s trying to impress me.
However, I think the existing advise comes from seeing people get stuck microoptimizing things. There are so many layers between your code and it’s actual execution, you can’t reasonably guess performance without actually profiling your full software. And when you do that, a very small part of your code is actually performance sensitive.
If you are starting a new project, you should have a sense for your performance goals and pick the tools that make that roughly feasible. When you do architecture, you have to do the same. But when you are writing actual code, I think there is value in making perf the last step often.
I think just keeping “don’t make it slow” in your mind with the same steps works. Because the only thing you really need to do is to not paint yourself into a slow architecture. But you can have a fast architecture, but with the code still being slow - but that just means that you need to check which exact parts are slow, and start improving them.
More generally, the “YAGNI” advice has a similar problem that it focuses on the wrong thing. You don’t have to build full support for it now, but you also shouldn’t make any decisions now that make it very hard to add support for it later.
Ahh, I like that. “Don’t make it slow” is a more helpful cue that emphasizes a better thought process. Thinking roughly about what I expect the data to look like, I’ll try to use a good data structure that best represents the sparse or dense data I’ll be dealing with. I try to pick something with good access patterns and save the profiling and more intensive tuning work for later (if it ever even comes up).
Yup. My rule of thumb here is to avoid doing anything quadratic. But anything less than that is probably fine. (Although you need to be careful about O(n) + O(n) sneaking up on you). Past that then I can optimize the bits that are slow.
There are so many layers between your code and it’s actual execution, you can’t reasonably guess performance without actually profiling your full software.
No, I’m sorry, but this is abdicating our responsibility for our craft. There are some things that we know are going to be vastly slower than the alternative–multiplying matrices the wrong way, hitting the data for every single write instead of doing bulk inserts/updates where possible, hitting the database at all if a memory cache is available, repeatedly setting up and tearing down SSL connections, chasing pointers excessively, excessive use of locking and unlocking, writing a character at a time to stdout, etc.
Profiling is super important, sure, but let’s not pretend we are blind babes in the woods needing to bang our noses on everything until a profiler tells us otherwise.
When we talk about optimization or performance or whatever, there are two broad categories. One is “not writing dumb code”, and the other is “writing specifically optimized code”. You’re describing the first category, but the article is talking about the second category.
If you are starting a new project, you should have a sense for your performance goals and pick the tools that make that roughly feasible. When you do architecture, you have to do the same. But when you are writing actual code, I think there is value in making perf the last step often.
Completely agreed. I think the problem people have with this advice is that they conflate the design and implementation steps. Both should go through the ‘make it work, make it correct, make it fast’ sequence. Your design starts with the minimum that you need to solve the problem in the common case, then adds the constraints that you need to solve it in the general case, then adds performance constraints. You end up with something that permits a fast and correct implementation, then you start writing code.
When you start writing code, you can make shortcuts to get something that works well enough to start testing, then to quickly turn that into something that actually works correctly, but you document missed optimisation opportunities along the way. Then you profile and see which of the places where you used an inefficient algorithm or data structure (cleanly encapsulated, because you designed it to be amenable to fast implementation) is actually hurting performance. Then you optimise those cases. Importantly, at this point you have the slow-but-correct version to test your fast-and-hopefully-correct version against, and you have tests for annoying corner cases, so it’s much lower risk to do the clever thing.
I think the OP’s point is that often, it’s not a very small part of the code that’s performance sensitive, but a lot of code that’s each slows you down by a tiny bit until the whole thing is slow. So you might not be focusing on performance at first, but you always need to keep it in mind.
I dunno, it’s always been my experience that as long as you’re not doing anything obviously dumb — batch your requests, avoid O(n^2), etc. — the bottleneck is always small, and (most importantly) always unpredictable.
I made a major breakthrough yesterday on my REPL CAD thing, so I can now stream models to a viewer when saving. I have to say, raylib saved me a ton of time on the viewer side. Being able to throw up something that can eat 3D data and give you a camera is amazing. It’s super fast because it starts with a minimum resolution and doubles the res until it hits a target. Changes between code don’t reset the camera or anything, so I can be focused on a part of the model and continually edit :D
This week is cleaning up the experience a bit and hopefully posting a demo / MVP :)
Raylib/raygui is great for tools development (honesty it’s faster for me than anything for web at this point)! What language are you using to implement the CAD stuff?
NodeJS + libfive bindings for writing the code CAD and meshing, and Zig for the 3D viewer. The viewer simply reads bytes over the unix socket so you can use ANY underlying technology to visualize, as long as it outputs 3D data.
It’s pretty dope. I open kakoune, type :cad, and it’ll start the viewer, and on save, it’ll stream progressively increasing resolution meshes to the viewer.
After this stage, I’ll probably stop, since that solves the last core code CAD issue I have. Anything will be just making it more robust. It would be nice if someone then comes in to add on-screen or in-space measurement tools, but this isn’t a huge concern, just a “nice to have”.
After I see this is working well for awhile, I’ll switch out libfive for something even better coming ;)
What I love about my system is it’s completely conceptually simple, and doesn’t rely on a massive browser stack to work. From what I’ve seen, the stack I’ve chosen can run on almost any system that has a C++ compiler and supports raylib.
The whole project started off as just NodeJS bindings to libfive, so I’m going to have to come up with a name sometime soon other than “node-libfive” :p I’m thinking I don’t want “CAD” part of the name, but it’s gotta be obvious it’s for constructing geometry / solids / models / objects using code.
For those wondering (most likely no one :), I think the time-to-this-point has been about 8 months, but only because in those 8 months I maybe touched the code base less than 20 times.
I have played a little bit with CadQuery recently, which is apparently based on OpenCascade / OCCT which I understand is big and old and powerful. (I care about support for importing and exporting standard CAD files, which are also fiendishly complicated.)
Yes, Im familiar with most of these open systems. Cadquery is best for your needs based on what youve just said. You should check out Replicad which also uses OCCT, but I like the code interface better.
The bit near the end is the key bit. I think it’s unfair to Godot to claim that it was designed to be slow. It was[1] designed to be easy to use, with other goals being secondary. GDScript is incredibly concise. Using C# with Godot requires a lot more boilerplate. The late binding dynamic behaviour of GDScript makes it easily approachable by people with little programming experience, perhaps some JavaScript.
In particular, it isn’t designed as a tool for experienced game developers to write high-performance systems, it’s designed as a way for people that want to build games to delegate the high-performance parts to the experienced game developers who write the engine. As someone who is not an experienced game developer, this appeals to me. The goal for the authors was, as I understand produce something that let them quickly write games for customers.
It sounds as if the author wants instead a set of building blocks that they can reuse to build their own game engine. Decomposing Godot so that it can expose those building blocks may attract more contributors and so might be a worthy goal.
The examples from the article had a bit of a ‘you’re holding it wrong’ feel to me. Why are you doing ray casting from the scripting layer on the hot path? You have a game engine, it knows how to do the rendering and exposes physics things for object collisions. This kind of code seems like something that you’d provide as a shader, not write in the scripting layer. But, as I said, I’m not a game developer, so maybe there’s a good reason.
[1] From the perspective of someone who has read the docs but never got around to using it.
Yeah, the conclusion sounds like some of the “Python slow” FOMO I sometimes hear. Slow interpreted languages don’t affect the overall performance much when they’re just glue code. Computers are fast, most small indie game logic is trivial, and the expensive parts of the engine are usually implement in highly optimized kernel code that’s internal to the engine just like Numpy for Python.
You might run up against a handful of high-level performance issue sooner than you would in a compiled language (e.g. some naïve AI logic will only run at acceptable speeds on 10 units rather than 100 units as it would with a compiled language), but when the focus is prototyping and experimentation that’s really not a primary concern. If you need a new high-performance kernel, it’s quite easy to drop into C# or even C and implement a tight loop or two that you can call in your glue code in the scripting layer.
Why are you doing ray casting from the scripting layer on the hot path?
Raycasts are quite common in game logic, since it’s often used for checks that find nearby objects, line of site calculation for AI units, hit detection for bullet paths, and character controller code, which isn’t usually driven by rigid-body physics simulation in most games but is hand crafted with non-physical logic so controls feel responsive and snappy. But I don’t think it’s much of a hot path relative to what else an engine does.
That’s not to say I wouldn’t like a scripting language with good performance characteristics for games! Spare compute every frame would still be appreciated. I’d love something with greater control over memory layout via ownership semantics. Late binding isn’t often useful, and there’s probably some low-hanging fruit these interpreters could take advantage of, but my gamedev pain points are rarely caused by poor scripting language speed.
FWIW, I’ve done several game jams in Python (with Raylib) and never had an issue with performance, even when doing realtime streaming audio processing in a Python callback in a simple 3D game! From my extremely unscientific experiments, Python is usually around 10x slower than an equivalent impl in C/Zig if a lot of the time is spent in Python loops. If you’re using 60% of the frame time on game logic, it might be worth porting some segment of that to a lower level language, but if it’s <10% (as it always has been in my games), you don’t gain much.
Why are you doing ray casting from the scripting layer on the hot path?
Raycasts are quite common in game logic, since it’s often used for checks that find nearby objects, line of site calculation for AI units, hit detection for bullet paths, and character controller code, which isn’t usually driven by rigid-body physics simulation in most games but is hand crafted with non-physical logic so controls feel responsive and snappy. But I don’t think it’s much of a hot path relative to what else an engine does.
But most of those can be moved out for hot path. Checks for hit detection for bullet paths happens only when a bullet is shot - that happens less than once per frame in not bullet-hell type games, and can easily be implemented with the node solution, without even changing the positions of it (as the post complained). Line of sight calculations for AI units can be implemented similarly with a raycast colider, only triggering a response when an appropriate target has been hit. Checks for nearby objects should mostly be doable with the same approach. Character control does depend more on your code, but even then, you can easily use the node solution, again, without changing the positions of it (rather, the raycast would usually be needed in a fixed position in relation to the character, and that’s trivial to do in Godot).
It is IMO trivial to offload most of the times whatever you want to do with raycasts to be mostly done in-engine.
Implementations of languages with similar ergonomic properties, like js and lua, have fairly good performance (with fast gc, shape inference, and jit compilation). The language is not the principal issue here.
The issue at hand is not the language but the performance of the interoperability layer with C++. This remains hard. As the article says, Unity has a custom C# compiler for this. C++ interop is often a bottleneck in JavaScript (with v8, it’s often faster to rewrite something in JavaScript than call into C++: the JavaScript is a good fraction of C++ performance and anything on a hot path that goes through the interop layer is a bad idea). Surfacing objects into the managed language from the unmanaged world often incurs multiple levels of indirection in both cases.
The issue at hand is that the performance profile of the engine is inadequate for a would-be user’s needs. If gdscript were fast enough, there would be no need to use c#. (Let me add, I can imagine alternate ways of structuring the implementation that would enable fast interoperation with other languages without harming gdscript ergonomics—even using a naive interpreter for the latter—still only an implementation issue.)
Even if that was the bottleneck, a truly mind-boggling amount of effort went into making JS fast.
And because of its niche Lua was designed so it could be made relatively light and fast, and luajit is still a heroic amount of effort by Mike Pall.
I have no idea what informed the design of gdscript. But language design absolutely matters when it comes to the effort necessary to making a fast implementation (ignoring other issues such as exposed APIs / FFI, which can also severely hobble the effort).
In what respects is lua easier to implement performantly than js? I think it has most, if not all, of the same interesting dynamic characteristics. I am pretty sure it was originally designed as a configuration language—and hence not expected to be fast at all—and grew organically into its current role.
I think there are a good number of people who could produce something not much worse in quality than luajit given appropriate time (0.5–2 years) and interest/incentive. (I count myself in that number.) Especially considering that luajit already exists and has done a good amount of exploration of its design space (you could even just use its code as is…)
FWIW, the screencast community on YouTube seems to have universally adopted manim (created by 3blue1brown) as the tool of choice for programable animation in presentations. In fact, with some additional tooling, it’s possible to create an entire video programmatically with voiceovers. But yeah, if you’re just embedding videos in a powerpoint anyway, I’m sure manim is easier to use than animated svgs.
Or, if you’re not tied to powerpoint, you could always just use a 2D game engine which introduces a whole new world of animation, visualization, and interactivity. Godot’s UI framework isn’t terrible, and you could probably whip up a slideshow project in a couple of hours. Or if you want something lighter weight, Raylib + Raygui is all you really need. Assuming you’re crazy enough to want to write your slides in C :P
TIL (from this book’s chapter on Unix’s lousy documentation) that you can use %cmdname to foreground a process by name when it’s been backgrounded. e.g.:
vim
^Z
top
^Z
fg %vim
I’ve been using bash regularly for a decade and didn’t know that (and man fg certainly doesn’t tell you how to do this, instead helpfully referring you to the job spec defined in Base Definitions volume of POSIX.1‐2008, Section 3.204, Job Control Job ID which afaik isn’t available on a base Linux system and isn’t accessible via man). I guess they proved the point :P
I routinely ^Z and fg (although I didn’t know about fg %name; I look at jobs and try to remember numbers), but I was under the impression it was a fairly obscure habit these days.
Yes, it used to be relevant when all you had was a dumb terminal, and some BOFHs didn’t keep a helpful copy of GNU screen around and you had disk quotas so you didn’t have space to keep your own, so you only had your one VT220 thing to look at so managing jobs made sense.
I still use it all the time for switching between apps, whenever I don’t want to commit to opening an entire new window/tab/buffer/pane. It’s often faster and reduces clutter.
Though only recently I learned about fg %<number>, before that I only used fg <number>, which turned out to be a custom thing in bash. In some shells it also works with PID.
Yeah, that’s what I’ve always done in the past, but it requires an extra invocation of jobs and/or some mental stack juggling which is less convenient.
I’ve solved this problem by ditching the smartphone and switching to a nokia dumbphone instead. It provides a similar level of irritation with fewer interruptions. Smartphones make certain things convenient, but the only productive tasks they excel at are casual photography, providing navigation directions, and serving as portable music players. But mostly, they excel at being a source of constant distraction. I want a smartphone that’s a tool to help me get useful work done, but that smartphone simply doesn’t exist.
Pull based notifications are great. The FOMO is strong, but once you overcome it, you realize didn’t need push-based notifications in the first place.
I still have a Smartphone, but I’ve always had a default-deny policy for notifications. The only things that are allowed to send me notifications are Signal and Team, and Teams may do so only during the working day (my global notification settings in Teams also prevent any notifications going to my phone when I’m at my computer).
I have Sieve rules that file marketing emails in a marketing folder. I skim it once or twice a week to see if there’s anything interesting. The worst people for this are GOG.com. They often send messages where all of the content is images (no, I am not going to download trackers, so I will never see anything in the images) and they have time-limited discounts that have often expired by the time that I read the email. Their loss, I just don’t spend any money with them as a result of the emails.
Being overwhelmed by notifications always seems like a self-inflicted problem. Who, when prompted by a UI element that says ‘Hi, I want to annoy you’ says ‘Oh, great!’?
Being overwhelmed by notifications always seems like a self-inflicted problem. Who, when prompted by a UI element that says ‘Hi, I want to annoy you’ says ‘Oh, great!’?
For me, it’s mostly apps I want or need some notifications from for some reason, but that insist on pushing marketing stuff or other bs, without a setting to just have the notification kinds I actually want. In theory, that’s against Apple‘s rules, in practice these are never enforced…
One program that I love is Syncthing. Who would’ve guessed that the sentences “continuous peer-to-peer file synchronization” and “it just works” could ever appear together.
One that I certainly couldn’t recommend without caveats but nevertheless makes me very happy is the alternative Discord and Slack client Ripcord. It looks great and presents a remarkably better interface to these two proprietary services. Long may it live.
You mean like how if the admins of any of the “servers” you use find out you’re using this on discord, they’re required to narc on you and get your account banned? =(
Syncthing is excellent. 3+ years, 4 machines (3 Mac, 1 FreeBSD), 120GB here. No hiccups. Love it. I’ve even built a SwiftBar plugin that works against the localhost API to show my machines and their sync states.
That’s a pleasant surprise to hear. I used Syncthing quite heavily a long time ago (6-8 years?) and it worked great up until it didn’t, and then required a lot of manual screwing around or a full repo wipe to get a wedged state un-wedged. Not a common occurrance, but not an uncommon one either. It did what I needed but when I had the opportunity to ditch it and just use a NAS instead I had no regrets. Sounds like it might be worth taking another look at for wandering laptops and such.
I have exactly the opposite opinion. I’ve worked on projects that use submodules and ones that do almost all of the proposed alternatives to avoid using submodules. Submodules have some rough edges (mostly caused by the tools, like many other bits of git, having stupid defaults) but the alternatives have always caused more problems than they’ve solved. The one exception is using a proper module system: I’d favour something like vcpkg over sumbodules for third-party dependencies where I’m tracking releases, though having to maintain overlay ports for when I discover that there’s a fix I need that isn’t in the release is painful).
The benefits of submodules are:
They are supported by a lot of third-party tooling. For example, GitHub’s file web UI will happily navigate into them, most CI systems support cloning them with the initial clone.
Closely-related, the suffering involved with submodules is the same as with any other project using them and so people learn to work around it (see also: everything else involving git). The suffering involved in not using submodules is bespoke special suffering that is likely to be unique to your ptoject.
It’s easy to use insteadOf in your git config to redirect them all to your local mirror.
It’s easy to move between a release, a current version, or a private clone without touching anything other than the top level.
You can default to a depth-1 fetch for them (saving disk space and network bandwidth) but go and grab the history later if you need it.
Changes you make locally are already in the right git clone for upstreaming.
You can easily do automated testing for new commits in a separate repo and then update the submodule when you’re happy.
Changes you make locally are already in the right git clone for upstreaming.
You can easily do automated testing for new commits in a separate repo and then update the submodule when you’re happy.
For languages like Zig/Odin/C that don’t have package managers, my preferred workflow is to use submodules in my projects for precisely this reason. When I have a patch to make to one of my dependencies, I can test it inline in my project before pushing it to my fork for a PR to upstream into the original project.
The author mentioned subtrees which I’ve not used, but will have to try out. I think this workflow should be supported by them too.
Ok. That’s really cool. Surprised there’s not more excitement about this here. The real interesting thing is how this is all working so quickly, i.e. jumping to a specific timestamp and the hot reloading / incremental compiles. That has to be a lot of frame data under the hood.
This would all be great to have, even for a non-game app.
That has to be a lot of frame data under the hood.
Yes! This. I really wonder how this works under the hood. I guess I can imagine running the entire game under a ptrace equivalent, and recording an execution trace of every change, storing only a diff, of sorts. But, I can’t quite see how you go back? I guess you undo every diff? But what if you free pointers? Does this not work with memory allocation?
The other way I think of it, is like call/cc in scheme. Essentially, building an efficient snapshot continuation tree, where all heap objects are forever live as they are referenced in continuations. But how would this not require immense memory, and, worse… allow you to debug memory allocation problems? One example shown was a null panicy thing. Their tooling works fine there.
Surely, there’s prior art for things like this that I could read about. I’ll have to investigate
From what I’ve pieced together of the youtube comments and this twitter thread, it looks like the replay is using a combination of state snapshotting and deterministic replay of inputs. Your point about freeing pointers is a good one, and it indicates that something lower level is going on.
I’m not familiar with most Windows tooling, so I’ll use the Linux equivalences for my guesses. Since it’s possible to change memory layouts and free/allocate memory, I’m assuming they’re either snapshotting the whole process memory (using something like the Windows equivalent of CRIU) or dumping a ton of introspection data in their custom compiler. To avoid snapshotting every frame, they seem to rely on replay of game inputs (mouse position/clicks). Still, I wonder how big the traces can get when you seem to be snapshotting large chunks of memory…
How they decide when to snapshot vs rely on input replay is unclear to me, but there seems to be a “loading” step when scrobbling through the replay (where the engine finds the nearest snapshot before and replays inputs to catch up) so it looks like the snapshot’s are relatively sparse. This is the biggest question for me, because if I knew what triggered a snapshot and what exactly was being snapshot, it might be a lot clearer what’s going on. In one of the comments on the Twitter thread, Allan says:
The technical basis for this though came out of work I did previously to build a debugger to reverse engineer NES games. Overhead is low(er) if you stick to only mutable data.
So perhaps its only snapshotting writes to non-stack memory? Again, I’m not sure if this is using some kind of dynamic instrumentation technique, but because the compiler stack is built from scratch, it would be pretty easy to add static instrumentation to each binary flagging each write that happens. This might be possible to bolt-on to other compiled languages with something like DynamoRIO which is a dynamic instrumentation toolkit that allows you to modify traces of code similar to a tracing JIT (albeit with a pretty high performance overhead, unfortunately…).
It’s also unclear what constraints there are on the code (other than it must be single threaded). If the game is written roughly as:
Then the update function is essentially pure, and that simplifies the problem a bit. This is the way I’ve attempted hot-reload in my game experiments in the past, but it’s still surprisingly hard to get right, so I’d really love a more in-depth writeup.
I have a plan to read the technical paper on rr, though, I think there’s gotta be a lot more going on. Your point about the pure update function and recording inputs makes sense to me. They didn’t talk a lot about the language, but we did see lots of mutation happening, so I can’t imagine it’s just “write everything functionally, and you get the world.”
I can’t edit my comment anymore, but realized there was a follow up comment from the original developer that explains some more details:
The act of going forward and back is not itself recorded – just the evolution of the game’s state.
The snapshots happen according to 2 different schedules – a coarse grain schedule that records a new snapshot every so often based on time (we do every 2 minutes currently) and a fine grain limited set of snapshots that move around depending on where you are currently working on the timeline. That’s why the initial reverse debugger step causes a brief pause and then becomes fast – the first one seeks back to the most recent coarse grain snapshot, simulates forward creating fine grained snapshots that are exponentially spaced out backwards from your seek target, and then the subsequent steps will tend to have a snapshot that is right on the frame you need (or very close – unless you step back far enough to need to go create more snapshots but that is the rare case.)
The state capture is mostly just a memcpy of the game’s heap (snapshots only happen on frame boundaries so the stack is never needed.) It for sure could be too big to keep as many snapshots as we currently do – that will just be game dependent. Something to use to calibrate what you expect is possible though is to remember that games like Metroid Prime, LOZ The Wind Waker, RE4, Mario Sunshine, etc. all ran on a system that basically had 24MB of RAM to use for your game. And it wasn’t just game state filling up that 24MB, it was your code and other read only resources – the kind of stuff that we don’t have to include in our snapshots. So while it’s true that this system is not a general purpose solution for any and all kinds of games, it’s also true that you can make some pretty incredible games with not a ton of actual game state.
Yes in theory you could totally fork the timeline in the past and create a new session based off of the old one up to that point. That is a feature that I had in the reverse engineering debugger I made before this because it was good for creating what were basically tool assisted speed run videos for code coverage purposes. For our current system though it hasn’t been something that I thought we would actually use enough to justify spending the time to implement it.
The code gen is totally custom but keep in mind that this toolchain only needs to run on our development platform which is Windows. To ship the finished game we will transpile the code to C and then compile it with the native toolchains on whatever platforms we’re targeting.
Python: seconding numpy and to a lesser extent matplotlib, and requests. Pandas is good because dataframes are great, not because the library is especially well designed :|. Also cffi + hypothesis means I can write property-based tests for any language that can expose an FFI interface. pwntools, scapy + a lot of the stdlib makes it the best choice for infosec/ctfs/reverse engineering.
C/C++: Raylib, Raygui, Imgui, SDL2, OpenGL, ffmpeg (you can’t do much multimedia programming without at least touching some of this stuff)
Zig: @cImport + all the C libraries out there. Technically not a library, but it enables the use of all the killer libraries in C with a more modern language without having to write FFI bindings by hand.
Lots of these design decision overlap with Odin, and Zig to a lesser extent. If this seems interesting to folks, I’d recommend checking them out as well. Odin has:
the implicit context in every function with a default allocator and temp allocator
compiler directives (different ones though, including #soa vectors which allows you to write code as if it’s an AoS but uses SoA under the hood!)
nice reflection
very similar syntax to Jai
built in mat/vec types that autovectorize allowing you to do some basic array programming, really nice for graphics programming code
auto generated string formatter
Zig has:
a scriptable build system in the same langauge
comptime programming (although comptime memory allocation is handled separately in Zig, I’m curious how Jai manages to use the same allocators – I’m guessing it compiles and runs the code block as part of the compilation step?)
auto generated string formatter
nice optional type/error type syntax
a ridiculously nice cross-compilation setup
Interesting to see what features Jai includes though. I’m glad to see more languages pop up in this “C+” space. There’s a lot of small improvements you can make to C while still keeping the language lightweight.
Re. manual memory management: I kinda feel that, if we’re in a metaphorical epidemic of razor-slashings, then it would be good if language designers stopped coming out with new and exciting varieties of double-edged razor blades, and maybe considered product safety. I mean, even C++ has managed to wrap tape around one of the blades so you can hold it safely, most of the time, if you don’t slip.
I realize Jai is aimed at game dev, not something with bigger security concerns like OS kernels or banking systems, but given the well-known endemic problems in the game industry, it might be better to pay some attention to making development more reliable so teams don’t have to spend all their time in crunch-mode fixing bugs, rather than cowboying memory management to eke out another frame-per-second. (But then, I don’t work in games.)
That being said, there do seem to be some cool ideas. The “context” thing would be useful — it looks basically like Go’s ‘context’ package but with the context passed invisibly. I wonder, though, whether the ABI passes it as an invisible parameter to everything (which will take a toll on performance) or if it’s effectively a thread-local global (in which case you can pretty easily do the same thing in C/C++/Rust/etc.)
stopped coming out with new and exciting varieties of double-edged razor blades, and maybe considered product safety
Eh, I feel like slices + bounds checking + maybe types + no (easy) pointer arithmetic gives you 90% of the memory safety that you need for problems with relatively simple memory-management schemes. Double-frees aren’t common in arena-allocated stuff like game code.
That being said, I’d be interested in seeing a memory-safe(r) language use a region-based or arena-based allocation scheme [1]. I wonder if that would be enough to convince more game devs to switch. C#’s structs are a huge help in reducing the amount of temporary short-lived allocations a program makes (and I regularly wish we had them in Java – garbage-free Java is not the most pleasant thing to write).
[1]: Although this might be a hard thing to combine with GC – I can’t have been the first person with this idea, but I can’t think of any languages that do this.
This is a large reason I’ve been drawn more to Zig and Python than Rust in my personal graphics/audio/gamedev related projects. I like to prototype numerical algorithms quickly, and numpy is killer for that. When I want control over memory (which absolutely matters in certain prototypes), I’ll drop to Zig. It provides better guardrails than C or C++, gives me access to all my favorite C libs through @cImport, but is still much faster to prototype with than Rust. [1]
Like the article mentions if you’re writing an OS kernel (or filesystem, or database, or a cryptography library, or a mission-critical scheduler, or any number of other multithreaded + safety + performance sensitive code [2]), Rust is a great choice. In established organizations, Rust also has an edge since larger companies are more focused on risk-mitigation, have larger teams, and benefit from stronger SDLC constraints to keep code consistent when dozens of programmers get their grubby paws on everything. But I can definitely see how a small startup that needs a bunch of short-term wins to survive would struggle with Rust. I’m sure there are startups building products where safety and performance are main features, and I bet Rust would provide an edge in some of those cases as well. But for a SaaS CRUD app with a handful of developers, it probably wouldn’t rank high on my list of languages to use.
[1]: I’d pick Rust even for my prototypes if worked on my problems that were multithreaded, but the safety guarantees of Zig are strong enough for the small, single-threaded programs I usually write. I’ve really enjoyed contributing a few small Rust patches to open source projects though, so I might also pick it if I wanted to make collaboration easy.
[2]: Although, it might even be tricky with some of these examples since they could require a decent chunk of unsafe code which is another sticking point for Rust.
I know this may be a simple thing for many but I wanted to share it just in case someone else struggles with similar problem, and I believe they have easier time finding it from here than stumble upon my blog :)
I’m going to be rating games for 32bitjam since I submitted a PS1-inspired action game for it built with Raylib/Python earlier this week. I’ll probably do a small writeup on my blog with lessons learned. Maybe I’ll clean up the music I wrote for the game and publish it on soundcloud too (even though it was rushed, I am starting to like aspects of it :P). I’m also planning on playing around with Zig 0.10 to see if I can setup web builds with Raylib for the next game jam I do.
I like this idea of including a table of contents and making it semantic and more than just a doc string (if you put a breakdown of the codebase in the README it might get missed if someone jumps to the code and is likely to fall out of date). An extension of this idea is to enforce documentation coverage through unit tests: https://simonwillison.net/2018/Jul/28/documentation-unit-tests/
Personally I’ve experimented with generating example-based documentation from the tests themselves. This works really well and you can ask contributors to write a test for their new feature and it’s covered in the documentation right away!
This, I think, cuts to the heart of the issue. Product designers are busy trying to design “experiences” and I just want stuff that I can treat like a utility. I want my software to be like my dumb fridge – it’s a means to an end. Just keep my dang food cold. If I’m conscious of how I’m using my fridge, somebody screwed up.
Even though Apple is better at design than most companies, I completely blame this “experience” trend on them. They used to be better. Try using an iPod from a decade ago and notice how much more of a utility-like experience it is. It feels more like a tool to listen to music rather than some flashy app that’s trying to impress me.
Totally get where the author is coming from.
However, I think the existing advise comes from seeing people get stuck microoptimizing things. There are so many layers between your code and it’s actual execution, you can’t reasonably guess performance without actually profiling your full software. And when you do that, a very small part of your code is actually performance sensitive.
If you are starting a new project, you should have a sense for your performance goals and pick the tools that make that roughly feasible. When you do architecture, you have to do the same. But when you are writing actual code, I think there is value in making perf the last step often.
I think just keeping “don’t make it slow” in your mind with the same steps works. Because the only thing you really need to do is to not paint yourself into a slow architecture. But you can have a fast architecture, but with the code still being slow - but that just means that you need to check which exact parts are slow, and start improving them.
More generally, the “YAGNI” advice has a similar problem that it focuses on the wrong thing. You don’t have to build full support for it now, but you also shouldn’t make any decisions now that make it very hard to add support for it later.
Ahh, I like that. “Don’t make it slow” is a more helpful cue that emphasizes a better thought process. Thinking roughly about what I expect the data to look like, I’ll try to use a good data structure that best represents the sparse or dense data I’ll be dealing with. I try to pick something with good access patterns and save the profiling and more intensive tuning work for later (if it ever even comes up).
Yup. My rule of thumb here is to avoid doing anything quadratic. But anything less than that is probably fine. (Although you need to be careful about O(n) + O(n) sneaking up on you). Past that then I can optimize the bits that are slow.
No, I’m sorry, but this is abdicating our responsibility for our craft. There are some things that we know are going to be vastly slower than the alternative–multiplying matrices the wrong way, hitting the data for every single write instead of doing bulk inserts/updates where possible, hitting the database at all if a memory cache is available, repeatedly setting up and tearing down SSL connections, chasing pointers excessively, excessive use of locking and unlocking, writing a character at a time to stdout, etc.
Profiling is super important, sure, but let’s not pretend we are blind babes in the woods needing to bang our noses on everything until a profiler tells us otherwise.
When we talk about optimization or performance or whatever, there are two broad categories. One is “not writing dumb code”, and the other is “writing specifically optimized code”. You’re describing the first category, but the article is talking about the second category.
Completely agreed. I think the problem people have with this advice is that they conflate the design and implementation steps. Both should go through the ‘make it work, make it correct, make it fast’ sequence. Your design starts with the minimum that you need to solve the problem in the common case, then adds the constraints that you need to solve it in the general case, then adds performance constraints. You end up with something that permits a fast and correct implementation, then you start writing code.
When you start writing code, you can make shortcuts to get something that works well enough to start testing, then to quickly turn that into something that actually works correctly, but you document missed optimisation opportunities along the way. Then you profile and see which of the places where you used an inefficient algorithm or data structure (cleanly encapsulated, because you designed it to be amenable to fast implementation) is actually hurting performance. Then you optimise those cases. Importantly, at this point you have the slow-but-correct version to test your fast-and-hopefully-correct version against, and you have tests for annoying corner cases, so it’s much lower risk to do the clever thing.
I think the OP’s point is that often, it’s not a very small part of the code that’s performance sensitive, but a lot of code that’s each slows you down by a tiny bit until the whole thing is slow. So you might not be focusing on performance at first, but you always need to keep it in mind.
I dunno, it’s always been my experience that as long as you’re not doing anything obviously dumb — batch your requests, avoid O(n^2), etc. — the bottleneck is always small, and (most importantly) always unpredictable.
I made a major breakthrough yesterday on my REPL CAD thing, so I can now stream models to a viewer when saving. I have to say, raylib saved me a ton of time on the viewer side. Being able to throw up something that can eat 3D data and give you a camera is amazing. It’s super fast because it starts with a minimum resolution and doubles the res until it hits a target. Changes between code don’t reset the camera or anything, so I can be focused on a part of the model and continually edit :D
This week is cleaning up the experience a bit and hopefully posting a demo / MVP :)
Raylib/raygui is great for tools development (honesty it’s faster for me than anything for web at this point)! What language are you using to implement the CAD stuff?
NodeJS + libfive bindings for writing the code CAD and meshing, and Zig for the 3D viewer. The viewer simply reads bytes over the unix socket so you can use ANY underlying technology to visualize, as long as it outputs 3D data.
It’s pretty dope. I open kakoune, type :cad, and it’ll start the viewer, and on save, it’ll stream progressively increasing resolution meshes to the viewer.
After this stage, I’ll probably stop, since that solves the last core code CAD issue I have. Anything will be just making it more robust. It would be nice if someone then comes in to add on-screen or in-space measurement tools, but this isn’t a huge concern, just a “nice to have”.
After I see this is working well for awhile, I’ll switch out libfive for something even better coming ;)
What I love about my system is it’s completely conceptually simple, and doesn’t rely on a massive browser stack to work. From what I’ve seen, the stack I’ve chosen can run on almost any system that has a C++ compiler and supports raylib.
The whole project started off as just NodeJS bindings to libfive, so I’m going to have to come up with a name sometime soon other than “node-libfive” :p I’m thinking I don’t want “CAD” part of the name, but it’s gotta be obvious it’s for constructing geometry / solids / models / objects using code.
For those wondering (most likely no one :), I think the time-to-this-point has been about 8 months, but only because in those 8 months I maybe touched the code base less than 20 times.
Sounds fun!
I have played a little bit with CadQuery recently, which is apparently based on OpenCascade / OCCT which I understand is big and old and powerful. (I care about support for importing and exporting standard CAD files, which are also fiendishly complicated.)
Do you know how libfive compares to OCCT?
Yes, Im familiar with most of these open systems. Cadquery is best for your needs based on what youve just said. You should check out Replicad which also uses OCCT, but I like the code interface better.
The bit near the end is the key bit. I think it’s unfair to Godot to claim that it was designed to be slow. It was[1] designed to be easy to use, with other goals being secondary. GDScript is incredibly concise. Using C# with Godot requires a lot more boilerplate. The late binding dynamic behaviour of GDScript makes it easily approachable by people with little programming experience, perhaps some JavaScript.
In particular, it isn’t designed as a tool for experienced game developers to write high-performance systems, it’s designed as a way for people that want to build games to delegate the high-performance parts to the experienced game developers who write the engine. As someone who is not an experienced game developer, this appeals to me. The goal for the authors was, as I understand produce something that let them quickly write games for customers.
It sounds as if the author wants instead a set of building blocks that they can reuse to build their own game engine. Decomposing Godot so that it can expose those building blocks may attract more contributors and so might be a worthy goal.
The examples from the article had a bit of a ‘you’re holding it wrong’ feel to me. Why are you doing ray casting from the scripting layer on the hot path? You have a game engine, it knows how to do the rendering and exposes physics things for object collisions. This kind of code seems like something that you’d provide as a shader, not write in the scripting layer. But, as I said, I’m not a game developer, so maybe there’s a good reason.
[1] From the perspective of someone who has read the docs but never got around to using it.
Yeah, the conclusion sounds like some of the “Python slow” FOMO I sometimes hear. Slow interpreted languages don’t affect the overall performance much when they’re just glue code. Computers are fast, most small indie game logic is trivial, and the expensive parts of the engine are usually implement in highly optimized kernel code that’s internal to the engine just like Numpy for Python.
You might run up against a handful of high-level performance issue sooner than you would in a compiled language (e.g. some naïve AI logic will only run at acceptable speeds on 10 units rather than 100 units as it would with a compiled language), but when the focus is prototyping and experimentation that’s really not a primary concern. If you need a new high-performance kernel, it’s quite easy to drop into C# or even C and implement a tight loop or two that you can call in your glue code in the scripting layer.
Raycasts are quite common in game logic, since it’s often used for checks that find nearby objects, line of site calculation for AI units, hit detection for bullet paths, and character controller code, which isn’t usually driven by rigid-body physics simulation in most games but is hand crafted with non-physical logic so controls feel responsive and snappy. But I don’t think it’s much of a hot path relative to what else an engine does.
That’s not to say I wouldn’t like a scripting language with good performance characteristics for games! Spare compute every frame would still be appreciated. I’d love something with greater control over memory layout via ownership semantics. Late binding isn’t often useful, and there’s probably some low-hanging fruit these interpreters could take advantage of, but my gamedev pain points are rarely caused by poor scripting language speed.
FWIW, I’ve done several game jams in Python (with Raylib) and never had an issue with performance, even when doing realtime streaming audio processing in a Python callback in a simple 3D game! From my extremely unscientific experiments, Python is usually around 10x slower than an equivalent impl in C/Zig if a lot of the time is spent in Python loops. If you’re using 60% of the frame time on game logic, it might be worth porting some segment of that to a lower level language, but if it’s <10% (as it always has been in my games), you don’t gain much.
But most of those can be moved out for hot path. Checks for hit detection for bullet paths happens only when a bullet is shot - that happens less than once per frame in not bullet-hell type games, and can easily be implemented with the node solution, without even changing the positions of it (as the post complained). Line of sight calculations for AI units can be implemented similarly with a raycast colider, only triggering a response when an appropriate target has been hit. Checks for nearby objects should mostly be doable with the same approach. Character control does depend more on your code, but even then, you can easily use the node solution, again, without changing the positions of it (rather, the raycast would usually be needed in a fixed position in relation to the character, and that’s trivial to do in Godot).
It is IMO trivial to offload most of the times whatever you want to do with raycasts to be mostly done in-engine.
Implementations of languages with similar ergonomic properties, like js and lua, have fairly good performance (with fast gc, shape inference, and jit compilation). The language is not the principal issue here.
The issue at hand is not the language but the performance of the interoperability layer with C++. This remains hard. As the article says, Unity has a custom C# compiler for this. C++ interop is often a bottleneck in JavaScript (with v8, it’s often faster to rewrite something in JavaScript than call into C++: the JavaScript is a good fraction of C++ performance and anything on a hot path that goes through the interop layer is a bad idea). Surfacing objects into the managed language from the unmanaged world often incurs multiple levels of indirection in both cases.
The issue at hand is that the performance profile of the engine is inadequate for a would-be user’s needs. If gdscript were fast enough, there would be no need to use c#. (Let me add, I can imagine alternate ways of structuring the implementation that would enable fast interoperation with other languages without harming gdscript ergonomics—even using a naive interpreter for the latter—still only an implementation issue.)
Even if that was the bottleneck, a truly mind-boggling amount of effort went into making JS fast.
And because of its niche Lua was designed so it could be made relatively light and fast, and luajit is still a heroic amount of effort by Mike Pall.
I have no idea what informed the design of gdscript. But language design absolutely matters when it comes to the effort necessary to making a fast implementation (ignoring other issues such as exposed APIs / FFI, which can also severely hobble the effort).
In what respects is lua easier to implement performantly than js? I think it has most, if not all, of the same interesting dynamic characteristics. I am pretty sure it was originally designed as a configuration language—and hence not expected to be fast at all—and grew organically into its current role.
I think there are a good number of people who could produce something not much worse in quality than luajit given appropriate time (0.5–2 years) and interest/incentive. (I count myself in that number.) Especially considering that luajit already exists and has done a good amount of exploration of its design space (you could even just use its code as is…)
Strangeloop!
Strangeloop!
Strangeloop!
Strangeloop!
Not strangeloop :(
FWIW, the screencast community on YouTube seems to have universally adopted manim (created by 3blue1brown) as the tool of choice for programable animation in presentations. In fact, with some additional tooling, it’s possible to create an entire video programmatically with voiceovers. But yeah, if you’re just embedding videos in a powerpoint anyway, I’m sure manim is easier to use than animated svgs.
Or, if you’re not tied to powerpoint, you could always just use a 2D game engine which introduces a whole new world of animation, visualization, and interactivity. Godot’s UI framework isn’t terrible, and you could probably whip up a slideshow project in a couple of hours. Or if you want something lighter weight, Raylib + Raygui is all you really need. Assuming you’re crazy enough to want to write your slides in C :P
(jk there are Python bindings too)
Presentations using Godot have been done, but the project seems to be abandoned… https://github.com/GDQuest/godot-presentations
Manim seems like it would be fantastic if I was trying to do somewhat less depraved things
TIL (from this book’s chapter on Unix’s lousy documentation) that you can use
%cmdnameto foreground a process by name when it’s been backgrounded. e.g.:I’ve been using bash regularly for a decade and didn’t know that (and
man fgcertainly doesn’t tell you how to do this, instead helpfully referring you to the job spec defined in Base Definitions volume of POSIX.1‐2008, Section 3.204, Job Control Job ID which afaik isn’t available on a base Linux system and isn’t accessible viaman). I guess they proved the point :PI routinely ^Z and
fg(although I didn’t know aboutfg %name; I look atjobsand try to remember numbers), but I was under the impression it was a fairly obscure habit these days.Yes, it used to be relevant when all you had was a dumb terminal, and some BOFHs didn’t keep a helpful copy of GNU screen around and you had disk quotas so you didn’t have space to keep your own, so you only had your one VT220 thing to look at so managing jobs made sense.
I still use it all the time for switching between apps, whenever I don’t want to commit to opening an entire new window/tab/buffer/pane. It’s often faster and reduces clutter.
Though only recently I learned about fg %<number>, before that I only used fg <number>, which turned out to be a custom thing in bash. In some shells it also works with PID.
You can also refer to them by job index:
^Z
^Z
Yeah, that’s what I’ve always done in the past, but it requires an extra invocation of
jobsand/or some mental stack juggling which is less convenient.I’ve solved this problem by ditching the smartphone and switching to a nokia dumbphone instead. It provides a similar level of irritation with fewer interruptions. Smartphones make certain things convenient, but the only productive tasks they excel at are casual photography, providing navigation directions, and serving as portable music players. But mostly, they excel at being a source of constant distraction. I want a smartphone that’s a tool to help me get useful work done, but that smartphone simply doesn’t exist.
Pull based notifications are great. The FOMO is strong, but once you overcome it, you realize didn’t need push-based notifications in the first place.
I still have a Smartphone, but I’ve always had a default-deny policy for notifications. The only things that are allowed to send me notifications are Signal and Team, and Teams may do so only during the working day (my global notification settings in Teams also prevent any notifications going to my phone when I’m at my computer).
I have Sieve rules that file marketing emails in a marketing folder. I skim it once or twice a week to see if there’s anything interesting. The worst people for this are GOG.com. They often send messages where all of the content is images (no, I am not going to download trackers, so I will never see anything in the images) and they have time-limited discounts that have often expired by the time that I read the email. Their loss, I just don’t spend any money with them as a result of the emails.
Being overwhelmed by notifications always seems like a self-inflicted problem. Who, when prompted by a UI element that says ‘Hi, I want to annoy you’ says ‘Oh, great!’?
For me, it’s mostly apps I want or need some notifications from for some reason, but that insist on pushing marketing stuff or other bs, without a setting to just have the notification kinds I actually want. In theory, that’s against Apple‘s rules, in practice these are never enforced…
Software I use that makes me happy (I work mostly on Linux, and occasionally mess around on macOS and Windows stuff):
sqlitenumpyfor numerical stuff,raylibfor graphics programming,imguifor GUIs,streamlitfor fast python prototypingfzf,fd,rg,rsyncOne program that I love is Syncthing. Who would’ve guessed that the sentences “continuous peer-to-peer file synchronization” and “it just works” could ever appear together.
One that I certainly couldn’t recommend without caveats but nevertheless makes me very happy is the alternative Discord and Slack client Ripcord. It looks great and presents a remarkably better interface to these two proprietary services. Long may it live.
You mean like how if the admins of any of the “servers” you use find out you’re using this on discord, they’re required to narc on you and get your account banned? =(
https://github.com/mk-fg/reliable-discord-client-irc-daemon#more-info-on-third-party-client-blocking.
Syncthing is excellent. 3+ years, 4 machines (3 Mac, 1 FreeBSD), 120GB here. No hiccups. Love it. I’ve even built a SwiftBar plugin that works against the localhost API to show my machines and their sync states.
I personally using Resilio (formerly btsync) for same purpose.
That’s a pleasant surprise to hear. I used Syncthing quite heavily a long time ago (6-8 years?) and it worked great up until it didn’t, and then required a lot of manual screwing around or a full repo wipe to get a wedged state un-wedged. Not a common occurrance, but not an uncommon one either. It did what I needed but when I had the opportunity to ditch it and just use a NAS instead I had no regrets. Sounds like it might be worth taking another look at for wandering laptops and such.
:O just tried out ripcord, and it’s amazing. I could feel the latency difference vs the web app as soon as I popped it open.
I have exactly the opposite opinion. I’ve worked on projects that use submodules and ones that do almost all of the proposed alternatives to avoid using submodules. Submodules have some rough edges (mostly caused by the tools, like many other bits of git, having stupid defaults) but the alternatives have always caused more problems than they’ve solved. The one exception is using a proper module system: I’d favour something like vcpkg over sumbodules for third-party dependencies where I’m tracking releases, though having to maintain overlay ports for when I discover that there’s a fix I need that isn’t in the release is painful).
The benefits of submodules are:
insteadOfin your git config to redirect them all to your local mirror.For languages like Zig/Odin/C that don’t have package managers, my preferred workflow is to use submodules in my projects for precisely this reason. When I have a patch to make to one of my dependencies, I can test it inline in my project before pushing it to my fork for a PR to upstream into the original project.
The author mentioned subtrees which I’ve not used, but will have to try out. I think this workflow should be supported by them too.
Ok. That’s really cool. Surprised there’s not more excitement about this here. The real interesting thing is how this is all working so quickly, i.e. jumping to a specific timestamp and the hot reloading / incremental compiles. That has to be a lot of frame data under the hood.
This would all be great to have, even for a non-game app.
Yes! This. I really wonder how this works under the hood. I guess I can imagine running the entire game under a ptrace equivalent, and recording an execution trace of every change, storing only a diff, of sorts. But, I can’t quite see how you go back? I guess you undo every diff? But what if you free pointers? Does this not work with memory allocation?
The other way I think of it, is like call/cc in scheme. Essentially, building an efficient snapshot continuation tree, where all heap objects are forever live as they are referenced in continuations. But how would this not require immense memory, and, worse… allow you to debug memory allocation problems? One example shown was a null panicy thing. Their tooling works fine there.
Surely, there’s prior art for things like this that I could read about. I’ll have to investigate
From what I’ve pieced together of the youtube comments and this twitter thread, it looks like the replay is using a combination of state snapshotting and deterministic replay of inputs. Your point about freeing pointers is a good one, and it indicates that something lower level is going on.
I’m not familiar with most Windows tooling, so I’ll use the Linux equivalences for my guesses. Since it’s possible to change memory layouts and free/allocate memory, I’m assuming they’re either snapshotting the whole process memory (using something like the Windows equivalent of CRIU) or dumping a ton of introspection data in their custom compiler. To avoid snapshotting every frame, they seem to rely on replay of game inputs (mouse position/clicks). Still, I wonder how big the traces can get when you seem to be snapshotting large chunks of memory…
How they decide when to snapshot vs rely on input replay is unclear to me, but there seems to be a “loading” step when scrobbling through the replay (where the engine finds the nearest snapshot before and replays inputs to catch up) so it looks like the snapshot’s are relatively sparse. This is the biggest question for me, because if I knew what triggered a snapshot and what exactly was being snapshot, it might be a lot clearer what’s going on. In one of the comments on the Twitter thread, Allan says:
So perhaps its only snapshotting writes to non-stack memory? Again, I’m not sure if this is using some kind of dynamic instrumentation technique, but because the compiler stack is built from scratch, it would be pretty easy to add static instrumentation to each binary flagging each write that happens. This might be possible to bolt-on to other compiled languages with something like DynamoRIO which is a dynamic instrumentation toolkit that allows you to modify traces of code similar to a tracing JIT (albeit with a pretty high performance overhead, unfortunately…).
It’s also unclear what constraints there are on the code (other than it must be single threaded). If the game is written roughly as:
Then the
updatefunction is essentially pure, and that simplifies the problem a bit. This is the way I’ve attempted hot-reload in my game experiments in the past, but it’s still surprisingly hard to get right, so I’d really love a more in-depth writeup.I have a plan to read the technical paper on rr, though, I think there’s gotta be a lot more going on. Your point about the pure update function and recording inputs makes sense to me. They didn’t talk a lot about the language, but we did see lots of mutation happening, so I can’t imagine it’s just “write everything functionally, and you get the world.”
Thanks for all the ideas and thoughts!
I can’t edit my comment anymore, but realized there was a follow up comment from the original developer that explains some more details:
numpyand to a lesser extentmatplotlib, andrequests. Pandas is good because dataframes are great, not because the library is especially well designed :|. Alsocffi+hypothesismeans I can write property-based tests for any language that can expose an FFI interface.pwntools,scapy+ a lot of the stdlib makes it the best choice for infosec/ctfs/reverse engineering.@cImport+ all the C libraries out there. Technically not a library, but it enables the use of all the killer libraries in C with a more modern language without having to write FFI bindings by hand.That’s a very good way to put it. Gives neither too much nor too little credit to Pandas, better than my comments elsethread.
Lots of these design decision overlap with Odin, and Zig to a lesser extent. If this seems interesting to folks, I’d recommend checking them out as well. Odin has:
Zig has:
Interesting to see what features Jai includes though. I’m glad to see more languages pop up in this “C+” space. There’s a lot of small improvements you can make to C while still keeping the language lightweight.
Re. manual memory management: I kinda feel that, if we’re in a metaphorical epidemic of razor-slashings, then it would be good if language designers stopped coming out with new and exciting varieties of double-edged razor blades, and maybe considered product safety. I mean, even C++ has managed to wrap tape around one of the blades so you can hold it safely, most of the time, if you don’t slip.
I realize Jai is aimed at game dev, not something with bigger security concerns like OS kernels or banking systems, but given the well-known endemic problems in the game industry, it might be better to pay some attention to making development more reliable so teams don’t have to spend all their time in crunch-mode fixing bugs, rather than cowboying memory management to eke out another frame-per-second. (But then, I don’t work in games.)
That being said, there do seem to be some cool ideas. The “context” thing would be useful — it looks basically like Go’s ‘context’ package but with the context passed invisibly. I wonder, though, whether the ABI passes it as an invisible parameter to everything (which will take a toll on performance) or if it’s effectively a thread-local global (in which case you can pretty easily do the same thing in C/C++/Rust/etc.)
Eh, I feel like slices + bounds checking + maybe types + no (easy) pointer arithmetic gives you 90% of the memory safety that you need for problems with relatively simple memory-management schemes. Double-frees aren’t common in arena-allocated stuff like game code.
That being said, I’d be interested in seeing a memory-safe(r) language use a region-based or arena-based allocation scheme [1]. I wonder if that would be enough to convince more game devs to switch. C#’s structs are a huge help in reducing the amount of temporary short-lived allocations a program makes (and I regularly wish we had them in Java – garbage-free Java is not the most pleasant thing to write).
[1]: Although this might be a hard thing to combine with GC – I can’t have been the first person with this idea, but I can’t think of any languages that do this.
Yup. Computer performance is weird. See also: https://youtu.be/r-TLSBdHe1A
This is a large reason I’ve been drawn more to Zig and Python than Rust in my personal graphics/audio/gamedev related projects. I like to prototype numerical algorithms quickly, and numpy is killer for that. When I want control over memory (which absolutely matters in certain prototypes), I’ll drop to Zig. It provides better guardrails than C or C++, gives me access to all my favorite C libs through
@cImport, but is still much faster to prototype with than Rust. [1]Like the article mentions if you’re writing an OS kernel (or filesystem, or database, or a cryptography library, or a mission-critical scheduler, or any number of other multithreaded + safety + performance sensitive code [2]), Rust is a great choice. In established organizations, Rust also has an edge since larger companies are more focused on risk-mitigation, have larger teams, and benefit from stronger SDLC constraints to keep code consistent when dozens of programmers get their grubby paws on everything. But I can definitely see how a small startup that needs a bunch of short-term wins to survive would struggle with Rust. I’m sure there are startups building products where safety and performance are main features, and I bet Rust would provide an edge in some of those cases as well. But for a SaaS CRUD app with a handful of developers, it probably wouldn’t rank high on my list of languages to use.
[1]: I’d pick Rust even for my prototypes if worked on my problems that were multithreaded, but the safety guarantees of Zig are strong enough for the small, single-threaded programs I usually write. I’ve really enjoyed contributing a few small Rust patches to open source projects though, so I might also pick it if I wanted to make collaboration easy.
[2]: Although, it might even be tricky with some of these examples since they could require a decent chunk of unsafe code which is another sticking point for Rust.
I know this may be a simple thing for many but I wanted to share it just in case someone else struggles with similar problem, and I believe they have easier time finding it from here than stumble upon my blog :)
FYI, here’s a video I found helpful for getting an intuition for quaternions: https://www.youtube.com/watch?v=FRD0PgsY3pU
I’m going to be rating games for 32bitjam since I submitted a PS1-inspired action game for it built with Raylib/Python earlier this week. I’ll probably do a small writeup on my blog with lessons learned. Maybe I’ll clean up the music I wrote for the game and publish it on soundcloud too (even though it was rushed, I am starting to like aspects of it :P). I’m also planning on playing around with Zig 0.10 to see if I can setup web builds with Raylib for the next game jam I do.
For people interested in building games using Zig/Raylib with web output, I put together a template: https://github.com/charles-l/zig-raylib-template