My best guess about why OAuth took off while client certificates did not is that one can implement the client side of the former in JavaScript and run it in a browser, but probably not the latter.
I think it’s important to point out that this was a limitation of OAuth, but not a limitation of OAuth2, which is what most implementations use these days. I am assuming @jimdigriz is assuming OAuth2, and you OAuth 1.0. Are my assumptions correct?
*Edit—I am wrong here. I forgot that there’s a client secret possible for some OAuth2 flows as well.
This is a placebo of the worst order, a “security” library which is nothing of the sort. This will still leak your client secrets, it is logically impossible not to if you’re completing the OAuth flow from the client side.
I did read it, and I understand what it’s trying to do, and what it’s not doing is keeping your client secret a secret. It’s keeping it from other JavaScript, but anyone on your page can find the secret by popping up the network panel and watching the request go out. Try it yourself.
This library should come with a big bold header telling you that it does absolutely nothing at all to keep your client secret safe from an adversary who knows where the “view source” button is, and that it would be impossible to do so effectively.
Client side security, isn’t. This is a cold, hard fact.
If you put it on the server, then you don’t have to send it to the client! The whole point of the secret token is that you keep it on your server, behind your firewall, and then issue the request to obtain the exchange token there. NOT in code that you’ve sent to the client to do whatever they want with.
What does the client use to authenticate its-self? What is it that the client communicate to authenticate its-self. Your statement of ‘put the token on the server’ is nonsensical as the problem is authenticating a user-agent.
No one gives a damn what an untrusted client does with a token, it is authorised for whatever its allowed to do, no more, no less.
You also seem to be trying to add substance to your argument by making out this secret works like a root credential, which is an implementation detail and irrelevant.
Your arguments read as if you need to swot up on you AAA.
I don’t follow your line of questioning, but from the linked repo.
client_secret [optional and not recommended]: your application secret
SAP’s are considered a public (‘untrusted’) client as the secret would have to published making it no longer a secret and pointless
This is what you risk losing, not the access token. The application secret has to be stored somewhere and if there is no server-side component then its on the client, which like the documentation says is insecure.
Over in the 802.1X world, EAP-TLS (client certificates) is mostly a pain not at initial provisioning, but renewal of the certificates and the UX around that. Though a mostly awful standard, EAP-FAST tried to address this (including provisioning) and now things like TEAP are coming out the woodwork.
EAP-TLS of course is fixed by having irresponsibly long validity times on the certificates, in the hopes that the user tosses the device before the certificate expires.. And then you hope your user comes to you for getting a new certificate instead of reusing the one for their old device..
I was hoping for a real discussion of using Elixir at significant scale, alas. This is an intro article from an agency that, while employing a number of Elixir heavy hitters, mostly deals in agency-scale projects.
I was hoping for the same. I could not really find content about how to scale up a gen_server. If somebody has any good resources it would be great to see it here.
There’s no “scaling up” a genserver, a genserver is one process just like any other one on the VM. To scale past a bottleneck where one genserver can’t process its messages fast enough (a common Elixir/Erlang scaling problem), you need to figure out how to use more than one genserver to do the work. That means you can’t call the genservers by name anymore, but the process Registry is good for that
I’ve been using a pattern lately where, in the genserver’s init(args) function, I register the process under a name that is passed in the args parameter. Then I launch the genserver under a supervisor. If the process dies, its entry will be removed from the registry, and when the supervisor launches another genserver to replace it, that server will register itself automatically. I like it so far.
If your gen_server does not need a lot of state to start it up (or side steppable with persistent_term), it may be easier to use simple_one_for_one and spawn a new process for each unit of work and have the source of the request monitor the process to handle retries. This works especially well when the inputs are binaries as then a copy of that data is not produced so starting up becomes cheaper and GC is now just the process destruction.
Routing is difficult and sometimes it is best just to skip it altogether, especially when spawning is really really cheap.
timer:tc(fun() -> Self = self(), Pid = spawn(fun() -> Self ! ok end), receive ok -> ok end end).
Not tested, but IIRC if tou do not use the sandbox attribute and instead use document.origin = document.origin you should get a cross-origin iframe environment that your service worker should be able to intercept…as it is in the same origin.
The point is we have sensitive stuff like encryption keys in the outer context. So we need full separate process sandboxing to protect from Spectre et al.
Dependent of course on what I am doing, I tend to just have my handle_continue pass directly to handle_cast which lines things up for me to use handle_cast(hup, State) as a generic point to handle reloads throughout the lifetime of my processes.
Before continue was an option, I just used to just call gen_server:cast(?MODULE, hup) before my init function returned.
The problem with message to self is that it causes race condition. If anything send message after spawn but before such message then server can receive messages while in wrong state. handle_continue avoids that as it is ran just after the message is processed but before next loop call.
Because long running init will stop launching other processes in the supervision tree. So sometimes you want to continue launching while still prepping the process and queue all messages that will arrive in the meantime. This is where handle_continue becomes handy.
I think you need to go back and read my comments and bear in mind that the comments are about what the article is setting out to solve and not telling you what you should be doing.
The whole point of supervision is that it expects after your init returns your process is in a stable state. If you are using tricks (continue, gen_server:cast() from init, …) to speed up your startup time you have officially declared “I do not care about supervison” as your now out of bound long init task could still lead to a crash of that process.
Your point of a race condition is correct but is replacing it with an unstable supervision tree just to get a faster startup times something that is correct for everyone?
Either you think everyone should do this, or (more likely) you have not read or taken onboard:
“beforecontinue was an option…”
“Dependent of course on what I am doing…”
continue is a recent OTP feature (stated by yourself)
continue is not available in all generic servers (stated by the author)
So beating up on me to use continue when it may not be an option is not really fair, right?
Throwing out a stable supervision tree may be okay and manageable in your environment. The race condition you describe is correct but has zero impact on my life as either I arrange it so dropping those messages is okay (two lines in a gen_server) or alternatively expose ‘readiness’ to a load balancer (where startup times are often irrelevant) so making the race a non-issue.
I suspect others may have their own thoughts here, and I am interested in hearing from them. What I do not appreciate is being told to swap a race condition for a unstable supervision tree whilst being told to suck eggs.
The obvious case, to me, is starting up a process/service that is inherently flaky, e.g. a database connection. The “stable state” of the process is disconnected, and whenever it is disconnected it is attempting to establish a connection/reconnect. While in that disconnected/trying to connect state, it can still service requests from clients, by simply returning an error. The rest of the system can proceed with startup, even if the database service was temporarily unavailable, or otherwise flaky during boot.
This is especially important for resilience in the supervision tree. Using the example from above, if your process tries to establish a connection during init, and it fails, it will likely fail again immediately, probably triggering the max restarts of the supervisor and bringing down large swaths of the application (or in the worst case, the entire node). The same applies later on in the nodes life, should the database connection fail, and trigger a restart, if the connection cannot be immediately re-established, then it is very likely that the process will restart in a loop, trigger the max restarts, and take down a bunch of stuff it shouldn’t have impacted.
The idea of doing that fragile work post-init is not to boot faster, it is to boot stable, and to ensure that restarts during the application lifecycle aren’t prone to the crash loop problem I described above.
As database disconnects are ‘normal’ and expected (not exceptional) though this sounds like the stable process here describes a behaviour similar to a simple_one_for_one supervisor (where you do not care but if you did you would instead use one_for_one)?
I still need to fix my Erlang as I am still ‘broken’ in the manner that I prefer a SIGHUP-via-cast approach over process restarting as I am yet to see a simple way of handling multiple reload requests without a global lock; though I guess you could drain your message queue with a pattern for only reload requests and debounce the whole system that way before exiting?
As database disconnects are ‘normal’ and expected (not exceptional) though this sounds like the stable process here describes a behaviour similar to a simple_one_for_one supervisor
The example of a database connection was illustrative, not meant as the best example, as typically they are created on demand and so would not be restarted, or would run under a simple_one_for_one supervisor. However the point it was meant to demonstrate is that some processes/services that have external dependencies are by definition unreliable; as you point out, this is part of their “normal” behavioral profile, so it is critical that the process handles the flakiness gracefully. An alternative example might be an S3/RabbitMQ/etc. consumer/producer - the same rules apply, if you require init to block on successfully connecting to the service, it will bite you eventually, and adds fragility where it isn’t necessary.
I still need to fix my Erlang as I am still ‘broken’ in the manner that I prefer a SIGHUP-via-cast approach over process restarting as I am yet to see a simple way of handling multiple reload requests without a global lock
I’m not sure I follow, they are essentially orthagonal concerns right? You can have supervision/process restarts and provide the ability to do some kind of “soft restart” via a SIGHUP-like message sent to the process. The part I think I’m missing here is what you mean by handling multiple reload requests - for a single process the requests are all serialized anyway, so if they trigger an actual process restart, then only the first message matters. If you are doing some kind of soft restart, then you could “debounce” them by dropping duplicate restart requests that come within some window of time (no need to drain the mailbox with a selective receive). In general though, I would assume the SIGHUP-like signal you are sending is something triggered by an operator, not by the system, in which case handling that edge case seems like worrying about something that is unlikely to ever happen in practice. Maybe an example of where you are using this pattern would help illustrate the problem it solves though.
All valid points, in the most general case. For any kind of ‘production’ system which could handle unknown data or format variants, there are plenty of good libraries out there, and it would be irresponsible not to use them.
But… for disposable scripts to process known-boring CSV data, I think it’s usually fine to just read lines and split on commas. I do that a lot, and I’ve never regretted it. On the other hand, I’ve often regretted overcomplicating otherwise simple programs by adding unused generality.
I have no idea why tab-separated files are not more popular (supported everywhere, including Excel and Google Sheets), it solves a lot of problems around custom importers (ie. you can now even just use awk) as most datasets do not include tab or newlines in their values.
The greatest thing in pattern matching is that you can use previously matched numbers later, so if you have TLV encoded data then decoding it is a breeze, example:
I think that ASN.1 could benefit from revised version of ASN.2 that would remove obsolete cruft and uniform the naming. Maybe with some syntax cleanups. This would be useful as the ASN.1 representations are quite vast (for example JER and XER for those who may be interested in that) and I think that with proper RFC and promotion it could gain some traction again as a contender to ProtoBufs or similar data description languages.
I’d argue against having a new version. ASN.1 is used extensively in my line of work, telecom, and the lack of support in most libraries is daunting. Having a new revised standard would mean even more division. Right now the only parsers that can fully support ASN.1 syntax are all proprietary, none of the open source ones can decently parse advanced schemas. That’s in part because of a monopoly in who drives those standards to begin with as it can give an edge when new telecom standards choose to adopt specific encoding.
Having a new revised standard would mean even more division.
Partially I agree, however with “clean state” standard, that would remove much of accumulated cruft, it could be easier to implement new FLOSS libraries that would support it fully.
Additionally I meant to make it backwards compatible, so all existing encodings still would be possible to describe with new specification format.
…and the lack of support in most libraries is daunting
When ever I looked at protobuf everything seemed to just to C bindings to libprotobuf which means there is really only one implementation; which as a standard makes it pointless[1].
ASN.1, warts and all, at least there are multiple implementations to pick from. The Erlang ASN.1 implementation looks pretty comprehensive (unsurprising giving its history) though I have not had to use it in anger.
[1] msgpack is mostly awful, but it does at least have multiple implementations…for better or worse
Then please make your request more precise, I’m afraid this reply for now sounds to me as equal to “this counterexample doesn’t fit my narrative ergo it is not valid”. Please also try to avoid argument by vague ridcule.
edit: Umm, how about a Java implementation that I found like, on the first page of Google results? Ok, I suppose it is also authored by Google; so is it now bad that the protocol authors provide many implementations? If yes, why?
Marvelous, but should explicitly spell out in the guidelines that the project should be runnable from an empty VM inside 15 minutes on the readers workstation. Being able to run the project entirely offline locally is a must.
The number of projects that are a multi-day journeys in themselves to just start running or presented as a binary blob ‘holy VM’ with the dev environment inside is deeply depressing.
Worse is when people do not consider where the data for the project comes from without having to pull in a 50GB replica into the mix rather than just being able to build a suitable sample from production.
As someone who packages software, this is a major point to me. If your building quickstart doesn’t work from an empty vm without mentioning “containers” or “docker”, your get-started story needs work.
Comparing the TCP/IP stack to something that throws out congestion control, reordering and flow control is somewhat strange. Maybe I missed something but you may as well just be using UDP(lite).
I am more interested in the answer to is the “juice is worth the squeeze” compared to other less intrusive and specialist methods that are actually accessible to others.
I have not seen many, if any, articles that cover this. Do most then just in light finding their solution is slow just blame the kernel rather than looking and experimenting with the other readily available baked in options?
A tad unfair as equally this can be explained that your own priorities are not aligned with the rest of society.
Is Zoom meaningfully less secure than telephone conferencing, mobile phone calls, Skype/Teams, WebEx, …?
To answer that you need to decouple absolute security, the content of those meetings, the geographic location of the moving parts, who the participants are and what their other options were (ie. mobile phone).
Myself as a UK based grumpy sysadmin, I would be delighted to hear that someone in China is interested enough to listen to me waffle on technical stuff that is already in the public domain. Meanwhile a UK politician arguably would probably be better off using a landline from the local telco. Should Chinese citizens use non-local services such as WebEx or Skype though? Should the UK government use Cisco solutions, other than they make you think suicidal thoughts, but that is more a statement about UX rather than security.
I personally value Zoom’s non-end-to-end encryption as it means they can multiplex all the AV streams into a single stream that my mobile network can handle out in the middle of the sticks.
Zoom were just idiots blatantly caught mis-labelling and being stung by their handling of it all; time will tell if they actually fix all these problems. The real sob story here is everyone now being forced to use Teams, Skype, Slack and Google Hangouts instead. Is this a better outcome?
Trust Python developers to add another level of abstraction and call it a new programming paradigm.
Not threading/littering your code with IO calls is no different to recommending don’t write a ball of mud and a natural continuation of group similar things together.
Claiming IO-less is any different to just creating a class or bit banging via a function is ludicrous. You still have to deal with the problem, this has just moved it elsewhere and tends to let you kick the can down the road in postponing working on the harder part to your problem.
The rest of the world calls this message passing/actor model (Erlang, F#, …) or channels/coroutines (Go or libdill). Amusing is the irony that a Python developer would propose using callbacks/events as if JavaScript is not something they scoff at or a car crash we should have all learnt from.
I really don’t understand why developers when they see something fundamental to another language that helps them better their solution, the response is to try and crowbar it into their ‘one true language’ (which ain’t looking so one or true right now) rather than just embrace some languages are better in some problem spaces.
It is okay to blend languages together, I personally find this an approach that works well for me and the teams I work with. Maybe some would claim ‘we are a Python shop’ whilst ignoring what they are trying to introduce is something their team will struggle with and find themselves fighting with the language to use…unless of course we use another layer of abstraction.
Dude, that’s not an acceptable way to express advice or criticism. Its phrasing is very contemptuous, and amusingly it assumes that Python developers have the same culture of contempt, but aimed at JavaScript. You might be interested in reading Shaw’s short blog post on Contempt Culture?
I do assume it was just a slip-up, though – the rest of your posts aren’t like this at all. But, ya. You might want to edit or delete this one.
I was hoping to start a discussion around not my two cheap pot shots (50 of 250 words) at the Python community, but it seems that is what I have been given to work with.
The bulk, and overlooked, part of my comment was attempting to flag that the article is void of content but is describing as if it is a big deal. The article reads as if the ideas in there materialised straight out of the ether, which whilst tragic, given the demographic I consider this not wholly surprising.
When I use my tar brush, I probably should have thrown in the disclaimer, Python is as equally bad as Elixir, Go, Rust, Ruby, … and by the way all languages suck, but some suck less. I see those communities yet just putting a new coat of paint on everything they touch, and pretend it is the next best thing, but worse still is the passive acceptance of it all.
Sure, I could have dialled back the bluntness say by 30% but my statement is based on my experiences and that is important. Sorry it offends but I am equally offended by the acceptance and unchallenged stories and comments around these parts of “should be written in Rust”, “array languages, perl, … are terrible as it looks like line noise”, “protobuf is great, what is this asn1 cruft” and “using Zoom kills kittens”.
I’m disappointed that the focus ended up being on the 20% rather than the 80%, and again unfortunately around these parts it seems a prerequisite that the snark needs to be dialed to 0% otherwise it is deemed “unaccepted” and not worth engaging.
I hate all developers equally if that makes you feel better? :)
Probably would not be too much work to whack together a FUSE filesystem that uses a whitelist. The whitelist would provide an optional rewrite rule and return EPERM to anything not on it.
…someone would have to wade through the PRs for updates to the list though.
My best guess about why OAuth took off while client certificates did not is that one can implement the client side of the former in JavaScript and run it in a browser, but probably not the latter.
Every client implementation of OAuth which has no server-side component is leaking their secret key, which is a Bad Thing you are Not Supposed To Do.
With OAuth2, app secret keys should not exist for public clients (no matter if web or native, keys can be extracted either way).
The old way of implementing public clients was implicit grant; now it’s PKCE (RFC 7636).
I think it’s important to point out that this was a limitation of OAuth, but not a limitation of OAuth2, which is what most implementations use these days. I am assuming @jimdigriz is assuming OAuth2, and you OAuth 1.0. Are my assumptions correct?
*Edit—I am wrong here. I forgot that there’s a client secret possible for some OAuth2 flows as well.
Not all client implementations do https://gitlab.com/jimdigriz/oauth2-worker
This is a placebo of the worst order, a “security” library which is nothing of the sort. This will still leak your client secrets, it is logically impossible not to if you’re completing the OAuth flow from the client side.
So instead of walking through the methodology in there, you just decided to make wild statements.
Marvelous.
I did read it, and I understand what it’s trying to do, and what it’s not doing is keeping your client secret a secret. It’s keeping it from other JavaScript, but anyone on your page can find the secret by popping up the network panel and watching the request go out. Try it yourself.
This library should come with a big bold header telling you that it does absolutely nothing at all to keep your client secret safe from an adversary who knows where the “view source” button is, and that it would be impossible to do so effectively.
Client side security, isn’t. This is a cold, hard fact.
Pray, do tell, what server side componment would fix this attack vector?
If you put it on the server, then you don’t have to send it to the client! The whole point of the secret token is that you keep it on your server, behind your firewall, and then issue the request to obtain the exchange token there. NOT in code that you’ve sent to the client to do whatever they want with.
CLIENT SIDE CODE IS NOT SECURE.
What does the client use to authenticate its-self? What is it that the client communicate to authenticate its-self. Your statement of ‘put the token on the server’ is nonsensical as the problem is authenticating a user-agent.
No one gives a damn what an untrusted client does with a token, it is authorised for whatever its allowed to do, no more, no less.
You also seem to be trying to add substance to your argument by making out this secret works like a root credential, which is an implementation detail and irrelevant.
Your arguments read as if you need to swot up on you AAA.
I don’t follow your line of questioning, but from the linked repo.
This is what you risk losing, not the access token. The application secret has to be stored somewhere and if there is no server-side component then its on the client, which like the documentation says is insecure.
Do I understand correctly that you actually don’t care that an untrusted client has access (even limited) without you knowing it?
Why do you need oauth2 then?
God I pity those that have to work with you. Rockstar coder I assume?
Folks, let’s all try and be civil and charitable here.
Over in the 802.1X world, EAP-TLS (client certificates) is mostly a pain not at initial provisioning, but renewal of the certificates and the UX around that. Though a mostly awful standard, EAP-FAST tried to address this (including provisioning) and now things like TEAP are coming out the woodwork.
EAP-TLS of course is fixed by having irresponsibly long validity times on the certificates, in the hopes that the user tosses the device before the certificate expires.. And then you hope your user comes to you for getting a new certificate instead of reusing the one for their old device..
If they had used Coccinelle they could have automated the building of the patch to fix the bugs too :-)
I do like how the author did test this against a bunch of projects, file the bug reports and include a patch for them too, good work.
I was hoping for a real discussion of using Elixir at significant scale, alas. This is an intro article from an agency that, while employing a number of Elixir heavy hitters, mostly deals in agency-scale projects.
I was hoping for the same. I could not really find content about how to scale up a gen_server. If somebody has any good resources it would be great to see it here.
There’s no “scaling up” a genserver, a genserver is one process just like any other one on the VM. To scale past a bottleneck where one genserver can’t process its messages fast enough (a common Elixir/Erlang scaling problem), you need to figure out how to use more than one genserver to do the work. That means you can’t call the genservers by name anymore, but the process Registry is good for that
I’ve been using a pattern lately where, in the genserver’s
init(args)function, I register the process under a name that is passed in theargsparameter. Then I launch the genserver under a supervisor. If the process dies, its entry will be removed from the registry, and when the supervisor launches another genserver to replace it, that server will register itself automatically. I like it so far.You’ve just posted a very short introduction to the concept of how to scale up the use of gen_server(s, plural) 😸
If your
gen_serverdoes not need a lot of state to start it up (or side steppable withpersistent_term), it may be easier to usesimple_one_for_oneand spawn a new process for each unit of work and have the source of the request monitor the process to handle retries. This works especially well when the inputs are binaries as then a copy of that data is not produced so starting up becomes cheaper and GC is now just the process destruction.Routing is difficult and sometimes it is best just to skip it altogether, especially when spawning is really really cheap.
This seems to have broken our CI build.
Here would be the last place I expected anything to happen due to comodo expiry. Cool of you to share.
It wiped out Datadog which I think is fashionable around here…
Not tested, but IIRC if tou do not use the sandbox attribute and instead use
document.origin = document.originyou should get a cross-origin iframe environment that your service worker should be able to intercept…as it is in the same origin.…don’t ask how I know this ;-)
The point is we have sensitive stuff like encryption keys in the outer context. So we need full separate process sandboxing to protect from Spectre et al.
In new OTP releases there is
handle_continue/2for that and you do not need to utilise such dirty hacks.Thank-you, @hauleth. I added a note about this on the post.
Dependent of course on what I am doing, I tend to just have my
handle_continuepass directly tohandle_castwhich lines things up for me to usehandle_cast(hup, State)as a generic point to handle reloads throughout the lifetime of my processes.Before
continuewas an option, I just used to just callgen_server:cast(?MODULE, hup)before myinitfunction returned.The problem with message to self is that it causes race condition. If anything send message after spawn but before such message then server can receive messages while in wrong state.
handle_continueavoids that as it is ran just after the message is processed but before next loop call.Why would you want
initto return immediately but then not actually be ready to service requests and care about this race condition?Because long running
initwill stop launching other processes in the supervision tree. So sometimes you want to continue launching while still prepping the process and queue all messages that will arrive in the meantime. This is wherehandle_continuebecomes handy.I think you need to go back and read my comments and bear in mind that the comments are about what the article is setting out to solve and not telling you what you should be doing.
The whole point of supervision is that it expects after your
initreturns your process is in a stable state. If you are using tricks (continue,gen_server:cast()frominit, …) to speed up your startup time you have officially declared “I do not care about supervison” as your now out of bound long init task could still lead to a crash of that process.Your point of a race condition is correct but is replacing it with an unstable supervision tree just to get a faster startup times something that is correct for everyone?
Either you think everyone should do this, or (more likely) you have not read or taken onboard:
continuewas an option…”continueis a recent OTP feature (stated by yourself)continueis not available in all generic servers (stated by the author)So beating up on me to use
continuewhen it may not be an option is not really fair, right?Throwing out a stable supervision tree may be okay and manageable in your environment. The race condition you describe is correct but has zero impact on my life as either I arrange it so dropping those messages is okay (two lines in a
gen_server) or alternatively expose ‘readiness’ to a load balancer (where startup times are often irrelevant) so making the race a non-issue.I suspect others may have their own thoughts here, and I am interested in hearing from them. What I do not appreciate is being told to swap a race condition for a unstable supervision tree whilst being told to suck eggs.
The obvious case, to me, is starting up a process/service that is inherently flaky, e.g. a database connection. The “stable state” of the process is disconnected, and whenever it is disconnected it is attempting to establish a connection/reconnect. While in that disconnected/trying to connect state, it can still service requests from clients, by simply returning an error. The rest of the system can proceed with startup, even if the database service was temporarily unavailable, or otherwise flaky during boot.
This is especially important for resilience in the supervision tree. Using the example from above, if your process tries to establish a connection during init, and it fails, it will likely fail again immediately, probably triggering the max restarts of the supervisor and bringing down large swaths of the application (or in the worst case, the entire node). The same applies later on in the nodes life, should the database connection fail, and trigger a restart, if the connection cannot be immediately re-established, then it is very likely that the process will restart in a loop, trigger the max restarts, and take down a bunch of stuff it shouldn’t have impacted.
The idea of doing that fragile work post-init is not to boot faster, it is to boot stable, and to ensure that restarts during the application lifecycle aren’t prone to the crash loop problem I described above.
As database disconnects are ‘normal’ and expected (not exceptional) though this sounds like the stable process here describes a behaviour similar to a
simple_one_for_onesupervisor (where you do not care but if you did you would instead useone_for_one)?I still need to fix my Erlang as I am still ‘broken’ in the manner that I prefer a
SIGHUP-via-cast approach over process restarting as I am yet to see a simple way of handling multiple reload requests without a global lock; though I guess you could drain your message queue with a pattern for only reload requests and debounce the whole system that way before exiting?The example of a database connection was illustrative, not meant as the best example, as typically they are created on demand and so would not be restarted, or would run under a simple_one_for_one supervisor. However the point it was meant to demonstrate is that some processes/services that have external dependencies are by definition unreliable; as you point out, this is part of their “normal” behavioral profile, so it is critical that the process handles the flakiness gracefully. An alternative example might be an S3/RabbitMQ/etc. consumer/producer - the same rules apply, if you require init to block on successfully connecting to the service, it will bite you eventually, and adds fragility where it isn’t necessary.
I’m not sure I follow, they are essentially orthagonal concerns right? You can have supervision/process restarts and provide the ability to do some kind of “soft restart” via a SIGHUP-like message sent to the process. The part I think I’m missing here is what you mean by handling multiple reload requests - for a single process the requests are all serialized anyway, so if they trigger an actual process restart, then only the first message matters. If you are doing some kind of soft restart, then you could “debounce” them by dropping duplicate restart requests that come within some window of time (no need to drain the mailbox with a selective receive). In general though, I would assume the SIGHUP-like signal you are sending is something triggered by an operator, not by the system, in which case handling that edge case seems like worrying about something that is unlikely to ever happen in practice. Maybe an example of where you are using this pattern would help illustrate the problem it solves though.
All valid points, in the most general case. For any kind of ‘production’ system which could handle unknown data or format variants, there are plenty of good libraries out there, and it would be irresponsible not to use them.
But… for disposable scripts to process known-boring CSV data, I think it’s usually fine to just read lines and split on commas. I do that a lot, and I’ve never regretted it. On the other hand, I’ve often regretted overcomplicating otherwise simple programs by adding unused generality.
I have no idea why tab-separated files are not more popular (supported everywhere, including Excel and Google Sheets), it solves a lot of problems around custom importers (ie. you can now even just use
awk) as most datasets do not include tab or newlines in their values.There are a couple issues with tabs:
If you can choose the format then pipe (|) delimiters are often a good option.
Pipes are much rarer than newlines or tabs.
wot?! This is about data ingestion between tools, not manually editing a CSV file in a text editor.
CSV files are opened and edited manually all the time in real life.
By extension, I suppose ascii-separated values would be even better.
But, hey, network effect strikes again. I’m not gonna tell data producers how to format their products. When in Rome, etc.
I always quite liked MailHog
SMTP now lives on
localhost:1025and you have a webmail client on http://localhost:8025Me too! Compared to mailcatcher, it has more verbose logs and the http API is better for searching.
Just imagine what damage a well-coded computer virus could do! ;)
No worse than what well-coded software already inflicts? :-)
The greatest thing in pattern matching is that you can use previously matched numbers later, so if you have TLV encoded data then decoding it is a breeze, example:
Depending on your requirements, you have to be careful with this as it is not efficient, so double check with:
As per http://erlang.org/doc/efficiency_guide/binaryhandling.html you should punt the
X:Sizeinto a binary match done in the function.For most though it won’t matter.
I think that ASN.1 could benefit from revised version of ASN.2 that would remove obsolete cruft and uniform the naming. Maybe with some syntax cleanups. This would be useful as the ASN.1 representations are quite vast (for example JER and XER for those who may be interested in that) and I think that with proper RFC and promotion it could gain some traction again as a contender to ProtoBufs or similar data description languages.
I’d argue against having a new version. ASN.1 is used extensively in my line of work, telecom, and the lack of support in most libraries is daunting. Having a new revised standard would mean even more division. Right now the only parsers that can fully support ASN.1 syntax are all proprietary, none of the open source ones can decently parse advanced schemas. That’s in part because of a monopoly in who drives those standards to begin with as it can give an edge when new telecom standards choose to adopt specific encoding.
Partially I agree, however with “clean state” standard, that would remove much of accumulated cruft, it could be easier to implement new FLOSS libraries that would support it fully.
Additionally I meant to make it backwards compatible, so all existing encodings still would be possible to describe with new specification format.
When ever I looked at protobuf everything seemed to just to C bindings to
libprotobufwhich means there is really only one implementation; which as a standard makes it pointless[1].ASN.1, warts and all, at least there are multiple implementations to pick from. The Erlang ASN.1 implementation looks pretty comprehensive (unsurprising giving its history) though I have not had to use it in anger.
[1] msgpack is mostly awful, but it does at least have multiple implementations…for better or worse
I don’t think the Go protobuf library links to the C one? I don’t seem to see
Cin its import graph… so, I’d guess this counters your point?Go is a bit of cop out counterpoint by any stretch, the folks who invented both happen to interop is not really ‘stop the press’ territory.
Then please make your request more precise, I’m afraid this reply for now sounds to me as equal to “this counterexample doesn’t fit my narrative ergo it is not valid”. Please also try to avoid argument by vague ridcule.
edit: Umm, how about a Java implementation that I found like, on the first page of Google results? Ok, I suppose it is also authored by Google; so is it now bad that the protocol authors provide many implementations? If yes, why?
http://wondermark.com/735/
Marvelous, but should explicitly spell out in the guidelines that the project should be runnable from an empty VM inside 15 minutes on the readers workstation. Being able to run the project entirely offline locally is a must.
The number of projects that are a multi-day journeys in themselves to just start running or presented as a binary blob ‘holy VM’ with the dev environment inside is deeply depressing.
Worse is when people do not consider where the data for the project comes from without having to pull in a 50GB replica into the mix rather than just being able to build a suitable sample from production.
As someone who packages software, this is a major point to me. If your building quickstart doesn’t work from an empty vm without mentioning “containers” or “docker”, your get-started story needs work.
Does a TIS-100 count?
Comparing the TCP/IP stack to something that throws out congestion control, reordering and flow control is somewhat strange. Maybe I missed something but you may as well just be using UDP(lite).
More interesting would be comparing this to:
vmsplice,spliceandteeI am more interested in the answer to is the “juice is worth the squeeze” compared to other less intrusive and specialist methods that are actually accessible to others.
I have not seen many, if any, articles that cover this. Do most then just in light finding their solution is slow just blame the kernel rather than looking and experimenting with the other readily available baked in options?
It’s a sad indictment on the low priority society places on security that Zoom has a $42B market capitalization.
A tad unfair as equally this can be explained that your own priorities are not aligned with the rest of society.
Is Zoom meaningfully less secure than telephone conferencing, mobile phone calls, Skype/Teams, WebEx, …?
To answer that you need to decouple absolute security, the content of those meetings, the geographic location of the moving parts, who the participants are and what their other options were (ie. mobile phone).
Myself as a UK based grumpy sysadmin, I would be delighted to hear that someone in China is interested enough to listen to me waffle on technical stuff that is already in the public domain. Meanwhile a UK politician arguably would probably be better off using a landline from the local telco. Should Chinese citizens use non-local services such as WebEx or Skype though? Should the UK government use Cisco solutions, other than they make you think suicidal thoughts, but that is more a statement about UX rather than security.
I personally value Zoom’s non-end-to-end encryption as it means they can multiplex all the AV streams into a single stream that my mobile network can handle out in the middle of the sticks.
Zoom were just idiots blatantly caught mis-labelling and being stung by their handling of it all; time will tell if they actually fix all these problems. The real sob story here is everyone now being forced to use Teams, Skype, Slack and Google Hangouts instead. Is this a better outcome?
Trust Python developers to add another level of abstraction and call it a new programming paradigm.
Not threading/littering your code with IO calls is no different to recommending don’t write a ball of mud and a natural continuation of group similar things together.
Claiming IO-less is any different to just creating a class or bit banging via a function is ludicrous. You still have to deal with the problem, this has just moved it elsewhere and tends to let you kick the can down the road in postponing working on the harder part to your problem.
The rest of the world calls this message passing/actor model (Erlang, F#, …) or channels/coroutines (Go or
libdill). Amusing is the irony that a Python developer would propose using callbacks/events as if JavaScript is not something they scoff at or a car crash we should have all learnt from.I really don’t understand why developers when they see something fundamental to another language that helps them better their solution, the response is to try and crowbar it into their ‘one true language’ (which ain’t looking so one or true right now) rather than just embrace some languages are better in some problem spaces.
It is okay to blend languages together, I personally find this an approach that works well for me and the teams I work with. Maybe some would claim ‘we are a Python shop’ whilst ignoring what they are trying to introduce is something their team will struggle with and find themselves fighting with the language to use…unless of course we use another layer of abstraction.
Dude, that’s not an acceptable way to express advice or criticism. Its phrasing is very contemptuous, and amusingly it assumes that Python developers have the same culture of contempt, but aimed at JavaScript. You might be interested in reading Shaw’s short blog post on Contempt Culture?
I do assume it was just a slip-up, though – the rest of your posts aren’t like this at all. But, ya. You might want to edit or delete this one.
I was hoping to start a discussion around not my two cheap pot shots (50 of 250 words) at the Python community, but it seems that is what I have been given to work with.
The bulk, and overlooked, part of my comment was attempting to flag that the article is void of content but is describing as if it is a big deal. The article reads as if the ideas in there materialised straight out of the ether, which whilst tragic, given the demographic I consider this not wholly surprising.
When I use my tar brush, I probably should have thrown in the disclaimer, Python is as equally bad as Elixir, Go, Rust, Ruby, … and by the way all languages suck, but some suck less. I see those communities yet just putting a new coat of paint on everything they touch, and pretend it is the next best thing, but worse still is the passive acceptance of it all.
Sure, I could have dialled back the bluntness say by 30% but my statement is based on my experiences and that is important. Sorry it offends but I am equally offended by the acceptance and unchallenged stories and comments around these parts of “should be written in Rust”, “array languages, perl, … are terrible as it looks like line noise”, “protobuf is great, what is this asn1 cruft” and “using Zoom kills kittens”.
In the past when I have tried to challenge a different view, @tedu amusingly and correctly summarised that lobste.rs literally considers me the Stasi. Other times I sigh when I see zero love for using OAuth2 implicit grants and refresh tokens safely in SPAs whilst nonsense like an IP subnet calculator bubble up; though I’m willing to accept that this is all pure bitterness on my part. :)
I’m disappointed that the focus ended up being on the 20% rather than the 80%, and again unfortunately around these parts it seems a prerequisite that the snark needs to be dialed to 0% otherwise it is deemed “unaccepted” and not worth engaging.
I hate all developers equally if that makes you feel better? :)
This website somehow doesn’t render on iOS.
I suspect it would be better to contact the author rather than posting on some aggregation site.
Worked fine on iOS for me.
Probably would not be too much work to whack together a FUSE filesystem that uses a whitelist. The whitelist would provide an optional rewrite rule and return EPERM to anything not on it.
…someone would have to wade through the PRs for updates to the list though.
https://lore.kernel.org/linux-arm-kernel/20200401065636.1751-1-ardb@kernel.org/