1. 10
  1. 4

    First time submitting a story, the software I work on recently moved from C++ to Rust (in a Python extension) and this post goes thru a small PR moving some Python code into the Rust layer, and what changes need to be done (including the FFI layer).

    1. 2

      Thank you for writing and submitting this. It is well written: easy to follow and fairly succinct.

      I wonder if you could get a reasonable compromise between cpu and memory consumption by growing the list till it hits (say) 64 elements then flushing it? That way you’re amortizing the per-call ffi overhead by buffering, but your buffer doesn’t grow very large.

      1. 2

        I hadn’t thought of that, that’s a great idea!

        I’ll try it and report back, thanks!

        1. 2

          I implemented you suggestion, and the memory consumption is negligible now: https://github.com/dib-lab/sourmash/pull/840

          Runtime is slightly larger (77s vs 73s), but as I noted in the PR it’s the memory that will likely be the bottleneck in the future (as we grow to millions of signatures, instead of ~100k we have now).

          Thanks again!

          1. 2

            Sweet! Glad to hear it. <3

      2. 1

        Without context I have zero clues what sourmash is, a short intro or even just a link would improve the post.