From the comments (with no response by the author):
I might be missing something but couldn’t your issues with Octo on Node.JS be solved simply by using the Cluster module native to Node ? From what you describe as your immediate advantage for using Golang that sounds like exactly what Cluster does for Node.
Can anyone comment?
Maybe in the sense that having more processes would’ve reduced load on individual processes but their issues were with inconsistent timeouts under load which wouldn’t have “gone away” using Cluster. I don’t think that their use of Cluster would have been much more effective than just deploying more instances.
No matter how hard I tried, I couldn’t get Octo to timeout properly and I couldn’t reduce the slow down during request spikes.
but if you look at the docs for setTimeout it is clear that there are no guarantees about the timing of callbacks.
The callback will likely not be invoked in precisely delay milliseconds. Node.js makes no guarantees about the exact timing of when callbacks will fire, nor of their ordering.
In their case, it seems switching to any language with more strict timeout capability would’ve had the same effect. I believe Go’s After is like this.
In general canceling work in Node is really difficult, due to asynchronous callbacks - you generally can’t access a function in between when you call it and when a callback gets hit. Go makes this much easier, at least if you are using a context.Context, the downstream work gets canceled as well - all it has to do is listen for an event on the ctx.Done channel. Note contexts are coming to the database/sql package in Go 1.8.
I wrote more about this here: https://kev.inburke.com/kevin/logrole-api-client-speed/
To clarify; once something is on the queue for the node event loop, it’s going to get processed eventually.
You can cancel outstanding requests before that happens (e.g. by scheduling a ‘cancel’ operation for 200ms in the future), but that pushes the ‘cancel’ operation onto the back of the event queue after 200ms. If your server is heavily loaded, that event might not get to the front of the queue until 400ms has passed.
By default Node runs a single thread in a single process. The cluster module lets you run more processes. It depends on where the bottleneck was in their system and whether each machine had additional capacity that wasn’t being utilized. If a single process is already using all of the RAM, or the machine only has a single CPU, it might not help much.