For those misled by the headline, the article does a deep dive into the ASP.NET Core entrant for the TechEmpower framework benchmark, and shows that it’s a (to be charitable) extremely contrived implementation which seems to have been written specifically to do well on the benchmark, rather than to show representative code or performance of ASP.NET Core.
A comment on reddit points out similar shenanigans in a top-performing Go submission.
I know “people are gaming the benchmarks” is not really news to many people at this point, but given just how much emphasis has been put on performance comparisons and specifically on the TechEmpower benchmark in things like the .NET release announcements and marketing, I do think it’s useful to have people shining a light on it.
For those misled by the headline, the article does a deep dive into the ASP.NET Core entrant for the TechEmpower framework benchmark, and shows that it’s a (to be charitable) extremely contrived implementation which seems to have been written specifically to do well on the benchmark, rather than to show representative code or performance of ASP.NET Core.
A comment on reddit points out similar shenanigans in a top-performing Go submission.
A Microsoft employee has also responded on Twitter.
I know “people are gaming the benchmarks” is not really news to many people at this point, but given just how much emphasis has been put on performance comparisons and specifically on the TechEmpower benchmark in things like the .NET release announcements and marketing, I do think it’s useful to have people shining a light on it.
Employee
Says the guy who code-golfed a plain tcp client and byte arrays until M$ rose up in the benchmarks.
Anyway I love a good deep debunking in the morning.
Check multiple queries. It’s much realistic results:
asp MVC ef pq 23k.
spring 15k.
asp mysql dapper 8k.
Laravel 3k or 6.5 w query builder
phoenix 3.8k.
Django 1.5k.
Better than spring but if you want to have table names in snake_case better to chose spring. If you need mysql - you better run from asp xD