Threads for retronav

  1. 2

    This is a good solution. I don’t think storing IP addresses would provide much benefit except for checking on device logins/sessions.

    1. 2

      Maybe some kind of rate limiting protection ?

    1. 1

      This is a good article. Zerodha still have a large user base with this philosophy of theirs, which can be taken as an example of “people will end up using your service if it brings value to their life”.

      1. 3

        Defense hasn’t loaded yet. Did it load for you?

        1. 24

          I suspect that’s why it has the satire tag.

          1. 11

            It will not load if your faith in the future of single-page applications isn’t strong enough. ;)

            1. 6

              This article requires at least 8gb of available RAM to load.

              1. 3

                I was able to interact with the components of the page that had loaded, so that’s a plus.

              1. 2

                I was trying to do something sort of like this as done in the post, and type guards seem to come very handy for filtering and finding in arrays. Going to use more of type guards from now.

                1. 4

                  User agents are a mess, I agree. Up to the point that it was said that three digit browser numbers would break UA parsing in some libraries. This looks like a decent first step towards standardizing UA strings.

                  1. 2

                    Dumb question; but why can’t we do something about GIL if it hurts parallelism? Maybe option to remove/disable it? I think it must’ve been done somewhere.

                    1. 14

                      One reason it is hard technologically is because at the moment: any operation that involves only a single Python bytecode op, OR any call into a C extension which doesn’t release the GIL or re-enter the Python interpreter, is atomic. (Re-entering the Python interpreter may release the GIL.)

                      This means all kinds of things are atomic operations in Python. Like dict reads/writes and list.append(), either of which may call malloc or realloc in the middle.

                      You can write many data race-y programs in Python that have well-defined (messy, but still well defined) semantics. I think nobody in the world has an idea of how much code there might be in the wild that (possibly accidentally) abuses this. So making data races be undefined behaviour would be quite a large backwards compatibility break, in my opinion.

                      You don’t want to “just” slap a mutex on every object because then the lock/unlock calls world kill performance.

                      I believe the PyPy developers are/were looking at shipping an STM implementation and the GILectomy fork involves a lot of cleverness of which I can remember no details.

                      1. 6

                        There have been (more than) a few experiments to remove the GIL in the past 20 years. To my knowledge they end up performing worse or being less safe.

                        There’s a new PEP to get a more granular GIL.

                        1. 11

                          There is an exciting new approach by Sam Gross (https://github.com/colesbury) who has made an extremely good NOGIL version of Python 3.9 (https://github.com/colesbury/nogil) It performs almost without any overhead on my 24 core MacPro test machine.

                          It is a sensational piece of work, especially as you mentions there have been so many other experiments. I know Sam has been approached by ThePSF. I am crossing my fingers and hope they will merge his code.

                          1. 9

                            I’ve been struggling with a Python performance issue today that I suspected might relate to the GIL.

                            Your comment here inspired me to try running my code against that nogil fork… and it worked! It fixed my problem! I’m stunned at how far along it is.

                            Details here: https://simonwillison.net/2022/Apr/29/nogil/

                          2. 6

                            They tend to perform worse on single threaded workloads. Probably not all, but I’m quite sure that several attempts, even rather naive ones, produced multi-threaded speed ups, but at the cost of being slower when running on a single thread.

                            Even ideas that succeeded to improve multi thread performance got shot down because the core team believes this (slower single core for fast multi core) is not an acceptable trade off

                            1. 4

                              IIRC the position was taken fairly early on by Guido that proposals to remove the GIL would not be accepted if they imposed slowdowns on single threaded Python on the order of… i think a cutoff of about 5% or 10% might have been suggested?

                              1. 1

                                That’s kind of what I remember too.

                          3. 4

                            There are experiments underway, e.g. https://lukasz.langa.pl/5d044f91-49c1-4170-aed1-62b6763e6ad0/, and there have been previous attempts that failed.

                            1. 3

                              Because alegedly, the gain in safety is greater than that of efficiency of concurrency.

                              It is a reliable, albeit heavy handed, way of ensuring simple threaded code generally works without headaches. But yes, it does so by eroding the gains of multithreading to the point of questioning if it should exist at all. Arguably.

                              Some async libraries mimic the threading API while resoursing to lower level async primitives. Eventlet and gevent come to mind.

                              1. 2

                                No, it’s about performance and a little bit about compatibility.

                                Most Python programs are single-threaded, and removing the GIL would not cause most of those to want to become multi-threaded, since their average Python program’s workload is not something that benefits from being multi-threaded. And basically every GIL removal attempt has caused performance regression for single-threaded Python programs. This has been declared unacceptable.

                                Secondarily, there would be a compatibility issue for things which relied on the GIL and can’t handle having the acquire/release turned into no-ops, but the performance issue is the big one.

                                1. 2

                                  And basically every GIL removal attempt has caused performance regression for single-threaded Python programs. This has been declared unacceptable.

                                  Why does this happen?

                                  1. 5

                                    Most of the time when a GIL removal slows down single-threaded code, it’s because of the GC. Right now Python has a reference-counting GC that relies on the GIL to make incref/decref effectively atomic. Without a GIL they would have to be replaced by more cumbersome actually-atomic operations, and those operations would have to be used all the time, even in single-threaded programs.

                                    Swapping for another form of GC is also difficult because of the amount of existing extension code in C that already is built for the current reference-counting Python GC.

                              2. 2

                                Because significant portions of the Python ecosystem are built with a GIL in mind, and would probably break the moment that GIL is removed. You’d essentially end up with another case of Python 2 vs Python 3, except now it’s a lot more difficult to change/debug everything.

                                1. 2

                                  A heavy-handed approach is to use multiprocessing instead of multithreading. Then each subprocess gets its own independent GIL, although that creates a new problem of communicating across process boundaries.

                                1. 1

                                  I think I’ve read a couple of such scenarios where user’s functions get DDOSed and is handed a huge bill. They get in contact with AWS and they forgive the bill. Does this still happen, or they stopped paying heed?