1. 8

  2. 5

    The discussion in the linked rust bug thread w/ @burntsushi is informative.

    Probably already known by people who pay more attention to this area, but I was surprised hyperscan does so well. I had heard of it before and knew it was fast in a general sense, but because it’s described as specifically optimized for searching large numbers of regexes simultaneously, I assumed it wouldn’t necessarily outperform other engines on one-regex-at-a-time benchmarks.

    1. 5

      Hyperscan has a ton of very clever usages of SIMD, with many algorithms containing both SSE and AVX variants. Rust’s regex crate copied one of those algorithms (which is used in the OP when they enable SIMD), but Hyperscan is loaded with them. It’s really an impressive piece of work.

      There are other regex engines that would be interesting to look at in the sense of “they have somewhat unique implementation strategies.” lightgrep and icgrep in particular I know use some cool tricks, although given the current benchmark, I expect they might do poorly depending on their literal optimizations. The benchmark in the OP biases heavily in favor of regex engines with aggressive literal optimizations, but regex engines like the one found in icgrep specifically target more complex regular expressions involving Unicode character classes.