But that only works for comms b/w procs on the same machine!

Did you find the PERL meeting more interesting though?

That’s why we call it “Perl” and never “PERL” :-)

That’s a worthy goal though.

The problem is that manuals do not tell you what to do, they just list warnings and do not supply any hints on how you should navigate those obstacles safely.

Was always my personal favorite router mod:

• Linksys WAG54G
• TP-Link TD-W8980

The TTL-232R-3V3-AJ cables were cheap to come by too.

The problem is these are all arguably over-defensive measures without the benefit of hindsight. In an “agile” setting, you won’t be able to justify writing code this defensively all the time. Even less so, since if you code like that, there won’t be any incidents to justify writing code like that :)

An exception could be if you managed to capture these patterns in abstract, reusable libraries, toolchains etc., then the up-front cost could be justifiable.

More like “replacement car exhaust is actually a blender in an exhaust-shaped can.”

Oh, no need to be like that, turn that frown upside down. :-)

It is a great gig replacing all that webscale cruft with 10 lines of shell and making it faster in the process too.

Usually it depresses the developers, but really they brought it upon themselves.

In my experience it is that the solutions are more complex than need be.

You’re right!

Actually I had planned to cover the change in scope of modern applications, but it was that much text, that I decided to write an own article about that topic ;)

There’s also nothing wrong with that.

…but…shiny. SQUIRREL!

Why hopefully? These numbers are embarrassing.

I don’t understand why anyone would be proud of these numbers.

product:([sku:3300?`5]; a:3300?0)
xsell:asc ([] sku:4600000?`5; cross_sku:4600000?`5; tstamp:.z.p-4600000?0)
\t select from xsell where not sku in exec sku from product
87

That’s msec; kdb is 100x faster than crate (on my macbook).

I suppose I should be at least happy that crate.io puts some actual benchmarks up when they say it’s “fast” so that we know that they mean not at all fast.

With ~4.6M rows, they’d be well served by vanilla PostgreSQL. Then again, I don’t know all of their requirements, perhaps they have need of ElasticSearch’s features (CrateDB is a SQL+management layer on top of ElasticSearch). Hopefully Gestalten.de is using this database for analytics and not important data storage.

Webshops have the problem that they are rarely written by yourself and come with their own share of issues. For example, a popular system is OXID. That means that you are often not so free to choose the database layer.

That said, you couldn’t pay me enough to use ElasticSearch as a primary datastore. A search index? Absolutely. Temporary log storage? Sure. Analytics? I suppose. ElasticSearch has measurably improved over the years, but once bitten, twice shy as the saying goes.

Elasticsearch themselves does not sell themselves this way.

ES is very popular in the shop scene, where the databases of your shop software are often a pain to work with and most of your frontend is search anyways. So, what happens is that they still use the shop software to store all articles, user data and transactions and sync that to Elasticsearch, with which they drive their full frontend. I’ve seen that in a quite a number of deployments and it works well.

Elasticsearch has good stability, just no guarantees. It’s perfectly fine to use it for something important, just be able to recreate it if it blows.

On the other hand, CrateDB may not be a good choice if you require:

• Strong (ACID) transactional consistency
• Highly normalized schemas with many tables and many joins

That’s perfectly fine if none of this is needed on that store.

Yea, I would have added some other labels with it, like kubernetes or monitoring/prometheus, but those don’t exist, so just put Erlang.

I vaguely recall it being less effort to simply open /dev/zero and use a private mmap()ing of that.

Of course if you are using this as an IPC between two processes you’ll have to use a regular file.

Take note of the difference in offset being used.

Mod isn’t super slow, but you can avoid mod entirely without the fancy page tricks by defining your buffer to be a power of 2. For example, a 4KiB buffer is 4096 = 2^12, so you can calculate the wrap-around with ( cur + len ) & 4095 without using mod.

You would still need two separate memcpy’s, and a branch for the wrap-around non wrap-around cases (which is normally not a big deal except when you’re racing against the highly optimized hardware cache in your MMU…)

Branches, conditionals such as if/switch statements, can cause performance problems so if you can structure things to avoid this sort of thing you can get a considerable bump in speed.

A lot of people look to software tricks to pull off speedups but this particular data structure can benefit directly from calling upon hardware baked into the CPU (virtual memory mapping).

Most of the time you have a 1:1 mapping of a 4kB continuous physical memory block to a single virtual 4kB page. This is not the only configuration though, you can have multiple virtual memory pages mapping back to the same physical memory block; most commonly seen as a way to save RAM when using shared libraries.

This 1:N mapping technique can also be used for a circular buffer.

So you get your software to ask the kernel to configure the MMU to duplicate the mapping of your buffer (page aligned and sized!) Immediately after the end of the initial allocation.

Now when you are at 100 bytes short of the end of your 4kB circular buffer and you need to write 200 bytes you can just memcpy()-like-a-boss and ignore the problem of having to split your writes into two parts. Meanwhile your offset incrementer remains simply:

offset = (offset + writelen) % 4096

So the speedup comes from:

• removing the conditionals necessary to handle writes that exceed the end of the buffer
• doing a single longer write, rather than two smaller ones

So it is not really that the CPU is handling this in hardware and so it is faster, the hardware is doing actually no more work than it was before. The performance comes from more a duck-lining-up excercise.

Modulo and division (usually one operation) are much slower than the other usual integer operations like addition and subtraction (which are the same thing), though I’m not sure I can explain why in detail. Fortunately for division by multiples of two, right shift >> and AND & can be used instead.

For why doing this with paging is so efficient, it is because the MMU (part of the CPU) does the translation between virtual and physical addresses directly in hardware. The kernel just has to set up the page tables to tell the MMU how it should do so.

group() is amazing! specially the groupCollapsed(). I considered them but then I thought I be opening the door to talk about count and all the other useful console functions so i focused only on .log(). Maybe next time :)

I could even live with out an adblocker, but for me it is the “I don’t care about cookies” extension.

MTCP does not solve the stalling flow problem (head of line blocking) due to packet loss and waiting on that retransmit to get through.

QUIC also has a 0 (zero) RTT cryptographically secure(ish) HTTP request mode; akin to the whole TCP 3 way handshake, SSL handshake and HTTP request rolled up into a single compact UDP packet so the request is serviced immediately by the server.

QUIC annoys me, because it’s a massive layering violation. It bakes in HTTP, when all I want is a better TCP – Something like CurveCP, which has zero roundtrips, and does encryption and authentication in one go.