Threads for quobit
-
So… helix:vim::zee:emacs?
-
-
-
-
-
I took that as more of a dig at xi-editor, but I’ve known about that longer than I’ve known about zed.
-
-
-
IMO emacs is about the introspection and customization more than the keybindings (which is why I use evil-mode :) ).
It’s definitely interesting that both helix and zee are terminal editors. I think that prevents you from doing a lot of ‘cool’ things like widgets with arbitrary graphics or embedding a markdown preview, but I think the only ‘reasonable’ choices for cross-platform are either Qt or Electron. And if you want people to be able to make widgets easily without having to learn a bespoke scripting system, you’re basically stuck with Electron. :/
-
-
I suggest to the author, as I suggest to anybody who finds themselves imagining that everyone around them (except for themselves) must be pathologically stupid, that they suppose that all of the things they decry are motivated by sensible, relatable concerns, which nevertheless might not be the same concerns as are primary to the author, or which might be realized via methods which nevertheless might not be the ones that the author prefers or knows.
-
Devil’s advocate even though I think your approach is the right default one:
Does this mean it’s impossible for large groups of people to do “crazy” things? Empirically, historically, this seems false. How then do you distinguish between those cases and cases where you simply lack context?
“motivated by sensible, relatable concerns”
What if the relatable concern is wanting to feel cool, and use something fresh and new, or a thing that Google uses and people talk about at conferences? I’m not just being glib and dismissive. But if that is the impulse – and I think it drives deeper than most people admit – you don’t necessarily get sensible. Or… there’s plain old not knowing about simpler ways.
People don’t have to be dumb to partake of sub-optimal trends.
-
Does this mean it’s impossible for large groups of people to do “crazy” things? Empirically, historically, this seems false. How then do you distinguish between those cases and cases where you simply lack context?
I normally try creating a Fermi Estimate for the problem. If it differs substantially from the scope of work then I assume there’s some missing context.
-
I normally try creating a Fermi Estimate for the problem. If it differs substantially from the scope of work then I assume there’s some missing context.
For example, how would this work to answer a question like “Are far too many companies using Angular when something simpler could have solved their problem better and saved many engineering hours?”
-
I don’t think it would work for that kind of question; I use this approach for projects at work where there’s usually a missing context.
-
-
-
-
-
-
-
By Betteridge’s law, no.
-
-
-
I don’t think Betteridge’s Law applies to blogs like this especially since this is obviously rhetorical, while Betteridge’s “Law” (which it’s really not) applies to possibly actionable (or at least evaluable) headlines about factual conditions (rather than rhetorical questions about possible future).
-
-
Maybe this quote from Knuth (1985) could add something to the discussion:
I suppose the name of our discipline isn’t of vital importance, since we will go on doing what we are doing no matter what it is called; after all, other disciplines like Mathematics and Chemistry are no longer related very strongly to the etymology of their names.
I found it closing a chapter named “What’s in a name?” in the book “The Science of Computing: Shaping a discipline” from Matti Tedre (2015), which is very interesting since it takes an historical journey on the subject.
Some articles from Peter Denning could be also of interest here. In fact both worked together in a more recent book (also related to this): “Computational thinking” (2019).
-
-
-
Maybe this one is easier to learn:
-
-
-
I think Łukasz Langa, Python core developer, has some serious comments about the benchmark setup: https://twitter.com/llanga/status/1271719778324025349?s=19
-
Thanks for linking this. A bit of rebuttal from me:
-
As I stated in the article, I did try 4 async workers. Performance was worse than with 5 workers (though not hugely). I don’t have a potted explanation for this I’m afraid - I can only say for sure that using 4 async workers instead of 5 did not change the results for the better in asyncland. (Note: I tried varying the worker numbers for all frameworks individually, not as a collective).
-
I take the point about running the whole thing on one machine. It would be better if I hadn’t of course. It seems unlikely that doing so would change the result since load on the other components was so low. I would be keen to read of any benchmark results using such a multi-machine setup, particularly any that find favourably for async, as I don’t know of any. I would add for anyone hoping to replicate my results (as friend of mine did): it takes a lot of time. It’s not enough in my opinion to just throw up these servers in naive manner, you need to make a good faith effort to tune and add infrastructure to improve performance. For example, when I ran the async servers without a connection pool they broke everything (including themselves).
-
Beyond my own results, there is a chunky body of extant “sysadmin lore” that says: async is problematic under load. I reference a few of the publicly available reports in my article: from Etsy; claims from inside a ridesharing startup; etc. I have also had negative private experiences too (prior to asyncio). The SQLAlchemy author wrote several years ago about this problem and kindly appeared in the HN thread to repeat his claims. The Flask author alluded to unfavourable private benchmarks, presumably from his workplace. The list goes on (including in other language communities).
-
Hi.
The point about not scaling above ~4 workers on 4 vCPUs has little to do about 4 vs 5 workers. It’s about being able to saturate your CPU cores with much fewer processes compared to sync workers.
You could at least acknowledge in your post that sync frameworks achieve on par performance by using more memory. Hard to do an exact apples to apples comparison but the idea stands: async frameworks allow much denser resource usage.
The reason why running your database with your Python process is not a realistic case goes beyond the operational problems with it (no high availability, no seamless scaling, hard upgrades and backups). The problem is that it unrealistically minimizes latency between the services. It doesn’t take much for the sync case advantage to go away as soon as you put the database on a separate box.
That separation would also allow for cheaper scaling: you can run just a few micro instances with little memory and a single vCPU and async workers will be perfectly happy with that.
Finally, appealing to authority and “sysadmin lore” should be out of scope for a benchmark that tries to be objective. For every Etsy I can give you Facebook that moved entirely to an async request model, including Instagram which is using Python 3. And Nginx which you’re using yourself in your benchmark was a big upgrade over Apache largely because of its single-threaded async model vs. a pre-fork server.
You also need to be careful whose authority you’re appealing to. Quoting Nathaniel J. Smith point out deficiencies of asyncio loses its intended strength when you add that he is such a strong proponent of asynchronous programming that he created his own framework. That framework, Trio, is a fantastic research environment and already has informed evolution of asyncio and I’m sure will keep doing so. That’s the point: Nathaniel’s posts aren’t saying “stop using async programming”. They are saying “here’s how we can make it better”.
-
The memory point is fine - for sure less memory is used. How important that is depends on deployment, as traditionally memory usage is not a huge problem for webservers. I contend: not very important for most people.
I don’t accept that the implication that I need to build a HA postgres cluster with backups and replication chains and whatnot in order to test. That would just raise the goalposts so high that it would just be a huge amount of effort and cost for anyone to construct a benchmark. If you’re aware of a cache of publicly available benchmarks that met your exacting criteria in this respect, referencing them would be great.
Going to the harder nut of that - the lower latency via running on the same machine - I am doubtful about how much it matters. Adding more blocking IO operations is simply not going to help because (as I stated elsewhere on this page) IO model just does not seem relevant to throughput for “embarassingly parallel” tasks like webservers. The fact that UWSGI is native code is the biggest determinant of throughput. For response times of course, doing something else while waiting actually seems to hurt - async workloads don’t get scheduled as fairly as the kernel scheduler does for processes.
Nginx using async is fine - everyone seems to think that nginx works ok and the Python community did not have to rewrite a large portion of their ecosystem in order to switch from apache2 to nginx.
On the subject of syadmin lore - I’m afraid that I don’t agree that it is out of scope! I’m not bound by intergalactic law only to consider my own evidence and I think it’s probably a good idea to consider outside evidence as well as what I have available to myself - after all it’s not as though I will have many opportunities to replicate multi-year programmes of software engineering in a cleanroom environment.
Thanks for taking the time to find me on a different medium in order to respond.
-
-
-
I mean you really shouldn’t go past the title here.
The claim that sync code somehow would be faster is absurd in its own righ unless your program has absolute 0 IO wait the async overhead will always be lower than benefits.
The only really argument here would be this increased code complexity increases likelihood of faults.-
The claim that sync code somehow would be faster is absurd in its own righ unless your program has absolute 0 IO wait the async overhead will always be lower than benefits.
Maybe true in python, I don’t know. Demonstrably untrue for high-throughput work on servers with high core counts due to the locking overhead.
-
And yet it is faster and I try hard to explain why in the body of the article (which of course I recommend strongly as the author of it :)). To briefly recap:
- IO model is irrelevant as OS scheduled multi-processing solves the problem of embarrassingly parallel workloads blocking on IO
- Use of native code matters a great deal and is otherwise the dominant factor
-
And yet it is faster
To me it seems like really digging for minute edge cases.
Async code, especially in python, is about implicitly eliminating wait. Meaning I can deploy my app anywhere, on any machine, in any way and it will always choose to optimally manage IO wait time.
-
-
-
Greg Wilson is very active in CSEd and also recommend his very helpful ‘Ten simple rules’ series: http://third-bit.com/10rules/
-
Maybe we should defend the 1x engineer: http://1x.engineer
-
This whole thing seems like nonsense. I thought a 10x engineer was defined as one who delivers 10x the value (and a myth, according to the internet)… and a lot of the things on that 1x list delivers value. Over the last couple days, I’ve seen “10x engineer” get defined (IMO redefined) as “jerk” (which now very much exists), and now 1x engineer - instead of just being “average skill” - is being redefined as “not a jerk” just to contrast with the alleged 10x.
And it is now a useless term completely divorced from reality. A lot of extraordinarily productive programmers are team players. A lot of average programmers are arrogant jerks.
-
Yeah, 10x was originally about 10x the value. It’s better to keep that definition since Nx programmers actually exist. They’re also rare and often problematic enough that we can continue critiquing companies whose HR focuses on them.
-
-
I love the retro styling on that.
-
-
I notice that there are a lot of overlapping traits between 1x and 10x engineers.
-
-
I just have a python two-liner that picks four random words from
/usr/share/dict/wordsand downcases them. This is sufficiently entropic for my purposes:print(' '.join([w.strip().lower() for w in random.sample(list(open('/usr/share/dict/words')), 4)]))-
You can also do this with Bash directly from the terminal:
for i inseq 1 10; do nice_dogs=shuf -n 5 /usr/share/dict/words | tr ‘\n’ ‘-’&& echo $nice_dogs$RANDOM; doneExample output:
- aggregations-Tahitian-Biden’s-laundries-lagniappe-32369
- aridity’s-fortification’s-Teri’s-surfboard’s-stinted-12072
- wick-homophone’s-Leander-triteness’s-Hamlin-7182
- Seneca’s-flags-ideogram’s-Yosemite’s-meter’s-28483
- beryllium-rubdowns-showdown-replaceable-Siamese-22326
- inaugural’s-fan-echelon’s-Devi’s-nightie-3720
- extortion’s-coolies-highfalutin-reconcilable-spotlight’s-24242
- Hatsheput-secrete-angioplasty-snacks-ruggedness’s-9776
- bordering-turds-binomial’s-conclusively-glimpse’s-25920
- odder-buzzes-hypotenuses-theocracy’s-sportier-16552
-
Like in https://xkcd.com/936/ ?
-
Someone I can’t recall coined the quip that while password policies are stuck in 70s mainframe land, the technological sophistication of attackers use the latest machine learning and statistical analysis tools.
The XKCD panel is no longer secure advice, since attackers learned how to aggressively mutate and statistically analyze large password datasets. On massive data breaches they are above 95-98% recovery rate from hashed passwords.
The state of the art is if it is a combination of human meaningful words and some additonal mutation and a few random characters on top, it’s already within reach of the people reversing hashes. (Which is the threat model, credential stuffing, not someone trying to brute-force a login).
This is why security researchers are pushing for password managers that generate non-human-meaningful 18+ character random passwords per service.
-
Can you explain this more? How does statistical analysis help with the entropy of words more than with the entropy of characters? Remember (I’m sure you do, but for onlookers) than in the XKCD panel we’re not mistakenly counting each letter as entropically meaningful but only the words vs the size of a reasonable dictionary. So a dictionary attack is the assumed vector, I’m curious what new statistical tools improve on this attack.
-
I also would like to see this explained. Regarding password managers and credential stuffing: using a 6-word passphrase (the recommended length for current required entropy levels) doesn’t mean you can keep using the same passphrase for every service. You still need a unique correct-horse-battery-staple-defection-epilogue passphrase for each service. But when you need to type a password from your password manager and can’t use autotype, it’s easier to correctly type a 6-word passphrase than PIXROU8i+00((AJM4s$$.
-
A couple of things worth mentioning here:
- if you’re generating passwords based on a strongly random source, of course it doesn’t matter if you’re using the randomness to generate random characters or select random words out of a large dictionary, overall entropy matters.
- Contrary to the small-print text in the xkcd comic, offline cracking attacks are the ones to worry about, not online guesses against a service.
- How much entropy is enough? That depends on the way a particular service is storing the passwords. If you’re using a password manager then it’s a mute point, you might as well use a strong 20+ character autogenerated password per service. If you’re using a memorable passphrase without a password manager though, the lowest common denominator hashing/salting method across all services the password is shared with is the one you want to protect against. It’s unlikely that you’ll ever get hard confirmation from any service about their password storage procedures, especially from services that are at the most risk from data breaches (the ones not too much on top of things). Recent versions of GPU-based password cracking benchmarks range from 100 Gigahashes/s per GPU to a couple thousand for more hardened password-storage methods. If you can attack at 100Gh/s instead of a thousand guesses per second, the correct-battery-horse-staple example would take 175 seconds to find, assuming you know the dictionary it was generated from.
- This was all assuming that people use strong randomness as a source of their passwords. Most people still don’t do that and password cracking tools learned long-ago to account for all the clever tricks anyone could ever think of to generate passwords from various dictionary or other methods (and therefor vastly narrow the search space).
- The biggest problem with the xkcd comic therefor is to suggest that you can ever remember passwords. I guess it’s more important that you use a different non-related password per service than the individual entropy levels of the passwords, but to do 1. you already need a password manager so no point in using weak passwords. I agree with the narrow point though that using a long passphrase composed from common words might be easier to type from a phone.
-
I should have added – you need to use a sufficiently long, memorable passphrase for your password manager, so there is always one passphrase that has to be memorable. You will probably have to memorize your workstation/laptop passphrase, too, unless you want to keep looking it up on another device (your phone), since the local password manager is not available while you are logged out/screen locked. And it’s likely to be worthwhile to have a memorable passphrase for some of your most used services, such as email. So that’s maybe three or four passphrases that you need/want to memorize, and which should therefore be diceware-or-similar.
For everything else, I don’t really care whether I’m using a diceware passphrase or a random-characters passphrase, since it’s in my password manager. But even then, if the password manager offers the option to generate diceware passphrases, I will use those, because they are easier to type and visually verify.
-
-
-
-
If you somehow learn that github user ken has password p3ssw4rd, try that username and password on every site you’ve heard of, stupidly. Most people reuse passwords, see? So you stuff the github credentials you learned into facebook’s login form, linkedin’s, every service’s login form.
-
Data breaches are now so common and wide-ranging (just check the billions of records in https://haveibeenpwned.com) that if you’re not using a password manager with individualized password for each service, then the likelihood is very high that the few passwords people inevitably reuse across many services has been already part of a data breach.
So nefarious people just take the data dumps with the cracked passwords (or email, password combinations) and just try to login to other services with the same username/password combination. With a quite high success rate.
To combat this there are two actions people can take:
- use a different password per service (only really feasible with a password manager)
- use a strongly random, long password that resists offline cracking (only really feasible with a password manager)
-
Thanks! I am familiar with the concept, but the specific term was unknown to me.
I personally use a password manager, and I think one should probably be integrated in services like Google or Apple IDs. Perhaps banks can include a subscription for one as part of the fee for having an account - it would probably help a lot with fraud, so could be a net positive for them.
-
-
-
-
-
Maybe even shorter: jot -rcs ‘’ 20 33 126
:)
-
-
out of the current top 10 distros:
Debian is the only one that has “jot” available, and its called “athena-jot”:
https://packages.debian.org/stretch/athena-jot
so this suggestion is not helpful.
-
Didn’t see the Linux tag, my bad.
-
-
-
-
-
-
Mark Guzdial is one of the most active researchers in Computer Science Education. Anyone interested can see his blog at: https://computinged.wordpress.com/
-
Love the gentle fun poking at Kenneeth Reitz :)
-
-
-performancebecause she isn’t benchmarking code, but people! -
-
It seems BSD is dying (2002) for almost 20 years…
Remember the hilarious Jason Dixon’s presentations BSD is Dying (2007) and BSD is still dying (2009).
-
adamo
5 years ago
|
link
| on:
Microsoft Considers Adding Python as an Official Scripting Language to Excel
I seem to remember there was a company that sold a spreadsheet to python converter. Can’t recall their name right now.
-
-
Hmm..
I think he misses one thing though.
There are also expert learners out there.
That video is nice for a novice teacher teaching mediocre learners at novice or competent level.
The click click click through bullet point as he reads drives me nuts. Where are the slides? Why can’t I run this video at 1.5x speed? This is tooooo slow for me.
I’m not sure he is as expert as he claims… I have seen papers that indicate the 7 plus or minus 2 thing is pretty debunked.
The one useful take away that was new? The idea of TDD’d via starting with the exercises.
-
Slides are here: http://third-bit.com/lesson-design/
I don’t get when he claims to be an expert. Are you sure he does?
-
-
Bear in mind this is a proposal and is not part of the roadmap. You can read it on a sprint report:
Another interesting vision statement was about using Rust in Mercurial. Most people agreed that Mercurial would benefit from porting its native C code in Rust, essentially for security reasons and hopefully to gain a bit of performance and maintainability. More radical ideas were also proposed such as making the hg executable a Rust program (thus embedding a Python interpreter along with its standard library) or reimplementing commands in Rust (which would pose problems with respect to the Python extension system). Later on, Facebook presented mononoke, a promising Mercurial server implemented in Rust that is expected to scale better with respect to high committing rates.
-
Also from release notes: “Python 2 is no longer installed by default. Python 3 has been updated to 3.6.”
-
This is a step in the right direction. I’ve been using Python 3 for all my projects in the last few months and I have to say, outside of whatever performance differences there might be (haven’t noticed them yet) the transition has been smooth, and things like MyPy are great.
-
-
-
quobit
5 years ago
|
link
| on:
How the Catalan government uses IPFS to sidestep Spain's legal block
Also they’re using Telegram channels and groups as well as GitHub web hosting to organize all activities of civil disobedience.
-
tldr: book promotion
-
I suppose, but the books being “promoted” are each over 30 years old. The Psychology of Computer Programming was originally published in 1971. Becoming a Technical Leader was published in 1986.
-
-
-
No, it hasn’t. The page is there in preparation for the release but it has not yet actually been released.
It hasn’t been properly announced but sets/packages are already available on the mirrors e.g. https://fastly.cdn.openbsd.org/pub/OpenBSD/7.1/
I’d argue if it’s not announced it’s not released. Also sets and packages could in theory still be overwritten.
I’d argue further if the release date on the page itself doesn’t yet exist, it hasn’t been released.
They just actually released! :)
Including an announcement from The de Raadt:
https://marc.info/?l=openbsd-announce&m=165054715122282&w=2
Maybe vermaden knew more?
Haha. Whelp, I’ll go back to my corner.
It’s all in the commits…
Received email from Theo (from the announce maillist).
Therefore I consider it officially released, now.
I assumed that if OpenBSD Webzine states that - then its released - sorry, my bad :)