1. 11
  1.  

  2. 1

    This benchmark is pointless and misguided. You are not likely to draw such a large number of lines in your game on a single canvas item, unless you are willingly abandoning benefits that the engine gives you for nearly no gain. All the real work is done by VisualServer, to which the calls are passed. Basically, all this benchmark measures is how fast can the language process sin/cos calls and call other code.

    And the tested options have no real difference in performance when tested for realistic conditions! I’ve decreased the number of lines drawn in each test, until the frame rate hit stable V-Sync(60 fps) on my (quite old and GPU limited) Thinkpad. Both GDScript and Rust options hit 600 lines, and only VisualScript lagged behind with 550 lines. The line count tested in the blog post was 10x bigger! 6000 lines is likely more lines than your screen has vertical ones!

    But I can say that tweaking with the numbers was a lot easier in GDScript and VisualScript than Rust. In those two I could just change the variable easily in node properties, while for Rust I had to change the code and wait for 4-5 seconds for it to compile.

    1. 3

      Always good to start a supposedly objective comment with such a personal note. Thanks for that :)

      My intention was to test performance of sending commands between the programming language and the engine api. this is why the amount of lines matters because increasing the load increases the measurability of the performance differences.

      Regarding value tweaking: the variables like the amount of lines is exposed as a variable you can tweak exactly like the ones in gdscript so no recompilation required.

      1. 3

        I’d love to see the same comparison with C++ and C#. C# is AoT-compiled with Mono, but is still a safe language and so I’d expect it to be slower than C++ or Rust. I’d expect Rust and C++ to be equivalent performance.

        I’ve not ever used Godot but it looks sufficiently interesting that I’m hoping to play with it over Christmas. I’d expect to always use GDScript for rapid prototyping and then move to either C# or C++ for anything that turns out to be a bottleneck after profiling and I’m curious about what the perf delta is between C# and Rust or C++ and whether it’s ever worth going all of the way to an unmanaged language.

        1. 2

          Given the rest of the thoughtful reply, I think the first sentence is entirely unintentional in its tone.

          **edit, I meant to ask, “Are you looking for feedback?” Because there was no submission comment, it is just a link.

          I do think you have an opportunity for a second edit of your benchmarking post. I think outlining just how much the number of draw calls is outside of a normal usage and that the other two solutions are totally workable is a good distinction. Benchmarking is hard and in that direction, we need to constantly apply hygienic practices.

          I also think it is important to go into the why and how one would measure it. Is it interpreter overhead? Memory allocation? Type checking? Or maybe the Rust code gets inlined, who knows?

          One thing I would have like to see is

          • change the number of loop iterations vs draw calls, what is the loop overhead, the draw call over head
          • how much allocation is going on inside each loop
          • how long does it take to get to the loop body
          • what if we reused the vec inside the loop body, drew the same line over and over again

          For bonus points, drawing a flame graph of each run would be amazing.

          Two folks I really look up for benchmarking and technical analysis are

          1. 1

            Always good to start a supposedly objective comment with such a personal note.

            I don’t understand what you mean. I said that this benchmark is pointless and misguided. It is pointless because it is measuring performance of code that is not likely to appear in real game code, therefore failing to provide any meaningful insights. It is misguided because what it is trying to measure is not indicative of real performance you are likely to see if such workload was actually used, as it does not measure the time it takes VisualServer to actually render the lines as the draw commands are queued until it’s time to draw.

            My intention was to test performance of sending commands between the programming language and the engine api.

            For that to avoid confusing people you should have chosen an example which is obviously doing just that, e.g. using engine’s print function. Currently, if remove the draw_line call, the performance ratios do not change between the languages, which suggests that what this benchmark is actually measuring is how fast can languages calculate sin/cos.

            Regarding value tweaking: the variables like the amount of lines is exposed as a variable you can tweak exactly like the ones in gdscript so no recompilation required.

            Oh, actually didn’t notice this because Rust has no default struct values, which I was looking for when trying to change the iteration count. Slightly annoying, but it is possible, so I do retract that statement.