Threads for aufziehvogel

  1. 18

    It’s a great pity to see somebody I really admire allow himself to be dazzled by this technology’s ability to mimic competence by pattern matching. An autocomplete with the entire corpus of stackoverflow will come up with some brilliant answers for some things, some subtly wrong answers for others, some absolutely wrong answers for yet others and utterly garbled output for more.

    The whole issue is that you need to be able to differentiate between all of them. And again, it is all entirely based on you stealing the work of others with an advanced search tool which has absolutely no idea as to the veracity or quality of what you’re stealing.

    I’m sorry Stevey, this is the next crypto. And it’s funny he mentions the whole not sticking with amazon thing because he was skeptical or k8s or whatever, because surely that argument equally applies to crypto? It’s survivorship bias, you regret not betting big on the things that turned out to work, then decide because this seems like a crazy claim like some other things that succeeded, this one must be true.

    The good thing with LLM type solutions is that you can go and try and see for yourself how wrong it gets it. Picking and choosing some lisp example that happens to work is exactly how this stuff gets you.

    I genuinely find this whole LLM craze extremely depressing. No waiting for the results to come in, no considering what the limits of such technology might be (yes LLMs will be VERY useful for some things, just not all). It’s just like the self driving nonsense. If you take time to think about how these algorithms work (essentially pattern matching with better results at higher data density) it becomes really obvious that it won’t work well for fields that require precise answers and which quickly get novel (driving, coding).

    It’s the emperor’s new clothes, and god’s sake Stevey I thought you were wiser than this (this is really about wisdom more so than intelligence, many smart people I know have been sucked in).

    I think this whole phenomenon is well described by Filip at https://blog.piekniewski.info/2023/02/07/ai-psychosis/

    1. 5

      A point I haven’t seen discussed yet is that, right now the stuff seems great because the majority of the content on the internet is training content: writing by people. But in a year or two when the internet is flooded with ChatGPT crap, the training data will be full of ChatGPT crap too. The algorithm can only do so much to compensate for the fact that soon it will be metaphorically eating its own shit.

      1. 3

        IMO there’s one very big difference between the radiology and self-driving stuff, and what we have now. Radiology was big companies making shrinkedwrapped products that they sold to hospitals, ChatGPT is an AI anybody can try to use for anything they want.

        So to finish this essay on a bit more positive note, here are some professions which in my opinion may actually get displaced with existing tech: … Pharmacist. It is absolutely possible today to build a machine that will automatically dose and pack custom prescriptions. A drug consultation could be done remotely via a video call before releasing the product to the customer.

        Okay this is totally insane.

        1. 7

          All it takes is a (few) high-profile case(s) where someone got killed by an LLM that got the dosage wrong (grams instead of milligrams seems like a likely mistake) and let’s see how quickly politicians will outlaw this kind of tech for serious use.

          1. 1

            I think the author does not refer to LLM in this section. It’s introduced with (highlight by me):

            here are some professions which in my opinion may actually get displaced with existing tech

            If he was refering to LLMs, I would have expected “with LLMs” or “with the new tech” or at least “with the existing tech” (even if existing was a bit weird to refer to something new). But written like this to me this is a reference to a broad spectrum of technology.

            I understand it so that he means existing technology in general, so in most cases probably something more tailored to the use case (e.g. bureaucrat: web application, cashier: self-service cashiers, bank clerks: online banking). But all of this already exists to more or less extent in different parts of the world.

      1. 7

        My main concern is two fold for ML products:

        1. Code interviews are going to be a bigger mine field for the interviewer.

        2. Developers will become code janitors full-time, developers part time. As in the project/product manager will generate a half working program and the developers are there to fix it.

        1. 6

          Re. 2: that transformation has already largely taken place. I started coding in 1987, professionally in 2000.

          When I’m coding in 2023 most of the time I’m gluing together preexisting libraries, and dealing with bugs in other people’s code. That was not at all the case when I started out.

          1. 1

            Gluing together pre-existing libraries was what OOP advocates like Brad Cox were promising in the ‘80s for most software engineers (with a smaller group writing those components). Brad’s vision was always that end users, not professional programmers, would do a lot of the final assembly.

            1. 1

              That’s the promise/premise of “low code/no code” as well. The public-facing rationale is shortening the lead between customer wishes and software solution, the hidden one is lessening the reliance on scarce, expensive software developers.

          2. 1

            Code interviews are going to be a bigger mine field for the interviewer.

            Just last week I put a few of our interview questions into ChatGPT and it solved most of them fairly reasonably. I received a slightly misguided answer that would have been okay for a live coding interview. The questions were all basic to intermediate SQL questions. Basically a series of stakeholder questions with a some stub data to query.

            We decided that we couldn’t let people do them as a take home. There was too much risk that someone could fake it. Even a screen sharing session might result in some trickery on a second screen.

            1. 3

              I think the solution is interviews that ask for broad understanding.

              I personally have always been interviewed in a quite open discussion (leet code interviews are very uncommon in Germany), only one out of six interviews in my career gave me a take home assignment. As a senior, when I myself was asked to join an interview I asked mostly questions that (imo) should show whether a candidate understands general concepts, e.g. “We have problem X, which technologies do you think could solve them?”, “You mentioned Y and Z, what advantages/disadvantages does each of Y and Z have in this use case?”.

              At least I guess that it would be hard to hear the question, type the question into ChatGPT, wait for and understand the answer and respond with the answer with a small enough latency that it would not seem weird in the interview.

              1. 1

                Just last week I put a few of our interview questions into ChatGPT and it solved most of them fairly reasonably. I received a slightly misguided answer that would have been okay for a live coding interview. The questions were all basic to intermediate SQL questions. Basically a series of stakeholder questions with a some stub data to query.

                We had someone scam us for a week at my last employer (I never interviewed the candidate), but they refused to turn on their camera and started giving different responses.

                So now does the interviewer need to keep in mind things like that, but I also need to ensure that the person isn’t using ChatGPT and CodePilot on their end to deliver answers to my problems. Maybe this will give rise to killing working remotely. “Sorry, I need you to come into our office for the interview.”

              2. 1

                Perhaps it would promote the habit of designing better-constrained interfaces to rein in the complexity, and force more effort into writing comprehensive unit tests.

              1. 2

                Plus, I’ve now heard often enough that it’s basically impossible to get in contact with a human person at Google if you don’t fit into their standardized processes. So good luck if you’re locked out of your account, even if you could proof that you’re the owner of the business (which at least in my jurisdiction is quite easy, because you have a business registration document, for some business types you’re also listed in a public registry).

                1. 18

                  Seems like an awful lot of angst for something people pay about $30 a year for! Maybe repurpose that to look into the ethics of your bank, telco, credit card company, ISP, grocery store, or something else you put hundreds of times as much money into?

                  (Also, lobste.rs’ TLD, .rs, is Russia. Not to get all political but I hear Russia’s done some bad things over the years?)

                  1. 40

                    .rs is Serbia (Republika Srbija), Russia would be .ru. Going back to the 90’s Serbia was also doing bad things, but currently to my knowledge the state Serbia only has some small conflicts with Kosovo from time to time.

                    I agree with the general comment though, that also Lobsters is using a ccTLD from a completely different state.

                    1. 1

                      Damn, you’re right of course. Somehow I got them mixed up.

                    2. 7

                      All of those things are much harder to avoid than .io domains are

                    1. 7

                      I’m a little puzzled. I thought the storage was actually encrypted on these things, and the existence of this bug seems to strongly suggest otherwise unless I’ve severely misunderstood. If swapping out an attacker controlled SIM can get you access to the device storage, it’s not encrypted, right? Is everything here a lie?

                      1. 3

                        After accepting my finger, it got stuck on a weird “Pixel is starting…” message, and stayed there until I rebooted it again.

                        After rebooting the phone, putting in the incorrect PIN 3 times, entering the PUK, and choosing a new PIN, I got to the same “Pixel is starting…” state.

                        I thought the same thing until I saw these snippets. I believe the “Pixel is starting…” screen is it decrypting the phone using your pin (and failing in this case).

                        1. 3

                          To my knowledge an Android phone is encrypted (if you have encryption enabled) when shut off. On boot, you decrypt it using a pin or password.

                          After the decryption after boot the lock screen is just a simple lock screen. It prevents somebody from accessing your data through the GUI, but the decryption key is loaded somewhere and a dedicated attacker might be able to get the data off a running phone.

                          There is also a small difference between the two lock screens. The first lock screen (which decrypts the device) has a small additional message telling you to unlock the phone to use all features (translated it from my language, probably other words on native English devices). The lock screens afterwards do not show this message.

                          I’m really bad at mobile phones though, so my understanding might be wrong. That’s how I understood it when I researched android device encryption.

                          1. 5

                            To my knowledge an Android phone is encrypted (if you have encryption enabled) when shut off. On boot, you decrypt it using a pin or password.

                            For a while now android uses file-based encryption and not full-disk encryption. This means that on boot there is no longer a point where you need to type the password to continue booting. Android’s file-based encryption allows the phone to boot all the way to the lockscreen. However at this point user data is still all encrypted.
                            After the user types their pin correctly (the first time after boot) user data is decrypted.
                            And yes you’d be correct that after this point the user data is decrypted and the lockscreen now just acts as a lockscreen.

                            but the decryption key is loaded somewhere and a dedicated attacker might be able to get the data off a running phone.

                            That’s not entirely correct, at least not for modern phones with dedicated security chips, like the Pixel’s Titan M. The decryption key is ‘stored’ in the Titan M - its very much protected in there. I say ‘stored’ in quotes because its technically a lot more complicated than that (Key Encryption Keys, Weaver tokens, etc).

                            1. 2

                              The key is stored, there but the data is not. Which is what the commenter above said that the attacker could get.

                              1. 1

                                Oh, I see.

                              2. 1

                                So, is the thought here that inserting the new SIM and resetting its PIN then resulting in a “unlock encrypted user volume” functionality?

                                1. 1

                                  I honestly have no idea. In fact I’m surprised doing anything with the SIM affects the encryption system like this.

                              3. 1

                                I was assuming the physical SIM swap involved a reboot. Maybe that was too generous an assumption.

                                1. 3

                                  The video clearly shows doing the SIM swap whilst powered on.

                                  1. 1

                                    I didn’t doubt that. But I thought swapping it would reboot from a cold state, not hold any decryption keys in memory.

                              4. 1

                                That’s how I first interpreted this too, but in the demo video you can see that they never turn the phone off.

                                It’s still a pretty useful bug. If someone steals/seizes your phone you don’t have time to turn it off, and you probably don’t carry it around powered off.

                              1. 5

                                Prioritize real-world validation.

                                Real world validation i.e. integration tests are the most effective way to assert properties of the system from the perspective of an outside user. This is incredibly important, but it’s orthogonal to another incredibly important dimension of systems, which is internal coherence and maintainability.

                                Human beings have a more or less fixed cognitive capacity. The most effective way to leverage that limited capacity towards complex systems is via abstractions, which are models that encapsulate and reduce complexity.

                                Tests are the most effective way to define and enforce layers of abstraction. If all you have is integration tests, then your system has effectively a single layer of abstraction, which is the customer facing interface. Whatever abstractions happen to exist within that system are unenforced and therefore basically incidental.

                                Unit tests — in the broad sense — are the best way to assert abstractions internally. And that’s essential for any system of meaningful complexity. If you can’t understand a piece of something without also needing to understand the thing as a whole, then that thing can’t grow beyond a very limited size. You gotta break it down. Unit tests are how you do that.

                                1. 1

                                  Thanks for writing this comment. The point that pushing to production is more important than testing in the article also took me off.

                                  I want to add that pushing to production only works as a test if your customers tell you what’s wrong. If they just silently quit or cancel the subscription you not only do not recognize the error, you also lost customers. And if they are so kind to complain that something does not work, you still are on the way to lose them if more mistakes happen.

                                  1. 1

                                    I want to add that pushing to production only works as a test if your customers tell you what’s wrong.

                                    Pushing to production — “test in prod” — is definitely only viable if your observability infrastructure can identify and surface errors before they become business-critical problems. Otherwise, yeah, it’s bonkers.

                                1. 3

                                  I really like AWS for personal computing projects because it’s almost impossible to go above the free tier for that. 400,000 gb-seconds of lambda compute a month? Psssh like I’m ever gonna go above 10,000

                                  1. 3

                                    Funny, I hated them for personal projects because they don’t support spending limits. I’m afraid of configuring something wrongly (even though I am careful) and even with a spending alarm going up to more than I want to pay before I can react to the alarm.

                                    1. 1

                                      Funnily enough you can DIY this feature using.. (surprise) Lambda.

                                  1. 3

                                    I am not a fan of AWS, but they do have their use cases. Startups usually do not have many savings available, so paying upfront for own servers can be too expensive. Startups often do not know how many customers they will have the next month and how the software requirements evolve. One reason for AWS over some cheaper competitors can be GPU servers. I have not done a market check, but e.g. Hetzner (where I host my private projects) afaik currently does not offer GPU.

                                    Personal experience is: New big customer wants to do a test run, company needs a few more servers quickly. Will they sign a contract? Who knows, but for a few weeks they need to see that we can handle their load. The other way round: Code improvements led to much faster processing, so fewer servers are needed. This is a constant up and down I saw in a startup company.

                                    I totally agree on the point that you should focus on application simplicity. With the myriad of services that AWS offers one can easily pick too many. And then the application becomes more and more difficult to understand.

                                    And the title says “Probably”, that sounds right. I agree with the statement that one should evaluate whether AWS is the right fit. Not just use it, because everyone uses it.

                                    1. 1

                                      One thing I found very interesting when I tested the “small” model with Croatian: I was feeding it a Croatian news show and it was producing words like “Harvatski”. To me this seems to show that the model did not really understand the Croatian language, otherwise it would have known that the word clearly must be “Hrvatski”. It seems to have a basic understanding of what Croatian words look like or something like that and produces words from that. At least that’s what I estimate.

                                      Another example, it wrote “sasmo bolnici”. That indeed was the pronunciation in the audio, but of course it should be “sad smo (u) bolnici”.

                                      On the other hand, I am wondering how it’s supposed to translate to English if it lacks understanding of the language.

                                      1. 1

                                        Have now also tested the “large” model on one Croatian series episode. After about 18 hours it’s at 17 minutes, so about 1 hour per minute up to now.

                                        The quality looks so much better than with the other models! Have only checked two minutes, I hope it stays like this. If it’s consistently at this quality, I can use it for my goal to learn vocabulary before watching Croatian series. Some mistakes, but good enough for me.

                                      1. 1

                                        Automated preview environments

                                        Has anyone working at a big tech, any thoughts to share on this? I always feel like what I could achieve in a nice way on my personal project (or some startup) is simply utterly impossible at $JOB because we’re stuck with a normal Jenkins CI/CD solution and bitbucket PR system.

                                        1. 2

                                          I’m not a big fan of SPA architecture at all, but the one thing I will say for it is that it makes preview environments SO much easier: if your application’s UI is pure HTML and JavaScript pulling from production web APIs you can turn on Netlify previews and they just work.

                                          … for client-side code at least. Server-side code remains much harder to do previews for, though Heroku has had this sussed for about a decade now.

                                          The hardest bit is when there’s a database involved, since you need a mechanism that can provision a fresh database for each PR - not cheap or easy.

                                          I think this is one of those things though that it massively less expensive if you build it early on. Having preview environments on a young project is hopefully only a day or so of tinkering - and then you can incrementally improve them as the system grows more complicated.

                                          Adding them to a large existing application with a lot of moving parts is a whole lot harder.

                                          1. 1

                                            I always feel like what I could achieve in a nice way on my personal project (or some startup) is simply utterly impossible at $JOB

                                            Not at $BIGTECH, but I feel the same way. I can see it not being too difficult if you’ve got a monolith, but once you have multiple teams deploying their own services on different it becomes difficult / tedious / expensive to get a complete environment up for every PR. Probably less useful too.

                                            For a web frontend it might still be practical to deploy a preview pointing to the production backend, but for stuff further down the stack I’d recommend deploying behind a feature flag and toggling that on for yourself / your organisation only. And once you have that you might just want to use feature flags for the frontend too, for the consistency it provides and ability to slowly roll out the feature to new cohorts of customers—not to mention the ability to instantly roll back.

                                            1. 1

                                              We were thinking about using Jenkins parametrized builds to allow devs to create preview environments from PRs on-demand. But on a first closer look it didn’t seem to be as easy as expected with the parametrized builds and I didn’t spend enough time to dig into Jenkins.

                                              1. 1

                                                I just finished migrating a few internal tools in order for them to be used from the Jenkins agent, which previously we could’ve ran from the terminal ourselves (compliance reasons…) and played a lot with parametrized builds myself.

                                                The major pain-point, to me, is that you’re writing groovy without knowing wether or not it is going to work out, feedback look is tedious.

                                                Let me know if you find yourself knee-deep in parametrized builds somewhere down the line!

                                            1. 4

                                              I like this. it reads to me like a take on vertical slicing, which I wish was better known amongst engineers. One of these days I should try something like Elephant Carpaccio with my team at work.

                                              1. 2

                                                Thanks for the kind words, and for the links. I knew there must have been some writing about this previously, so I’ll have to read through that to understand how they compare.

                                                I’ve used this approach with a few different teams in the past to good effect, though never explained in quite so much detail. Sprints or milestones can both be organized around stripes very easily. That’s part of what I like about the strategy - it applies equally as well to team development as it does to individual.

                                                1. 3

                                                  Another term that may come up is a “spike”, which is a vertical slice that is as narrow as possible. You might make a prototype that shows off only one feature, for example.

                                                2. 1

                                                  I feel one really has to be careful with this approach. I read the “Elephant Carpaccio” more thoroughly than the “vertical slicing”, but both seem to be about slicing down work in an Agile environment in tickets. The “Elephant Carpaccio” takes this to an extreme with tickets that take only 15 minutes to implement.

                                                  Of course, it all depends on where you’re coming from (how big your job tasks are at the moment) and what experience you have. A more experienced engineer will be happy with a big chunk and wants to do the slicing on their own. A less experienced engineer might need smaller chunks. That’s why I got so hung up on the 15 minutes per ticket, because that’s extreme in my opinion.

                                                  In my opinion this approach can - depending on who writes the tickets - easily lead to complete micromanagement of engineers where any real software engineering is completely overlooked. I have a picture in my head where product management defines in extremely small steps “Give us A”, “Give us B”, “Give us C” and at some point the engineer starts to recognize “If they had given me the clear picture in advance, I could have designed the software completely different according to best practices.”

                                                  1. 1

                                                    It’s true that this could be abused; but then again, so can many things. If you team isn’t trusted to deliver independently, and the attitude from management is to micro-manage rather than to support independence, then that’s not something you can paper over.

                                                    The really fine grained examples are mostly there for training purposes. Like most things, you can start taking bigger steps as you get more confident.

                                                    I’m mostly interested because when you have a pull request (say) that forms a complete increment of observable functionality, then it’s a a lot easier to review. Not just in terms of say, will this work, or is this maintanable, but will this work well in context.

                                                    For example, at work we’ve a convention that if an object isn’t found when reading it, we’ll return a nil from our data access layer instead of an error. So, I managed to produce a pull request which would crash instead of yielding an error, if I tried to read a non-existent object. Thankfully, my team-mate spotted that one almost immediately. On the other hand, if I’d submitted seperate pull requests for say, the data storage layer, and our request handlers, it would have been harder to spot.

                                                1. 6

                                                  I feel like this article engages in some muddy thinking around programming tools like IDEs versus programming languages.

                                                  As @student mentions below, adoption is a key factor in any programming language’s success, and being open source or at least free as in beer is definitely an accelerant.

                                                  1. 4

                                                    Adoption indeed is key. I worked in an automotive company with a software that contained a proprietary language (quite simplistic aimed towards mechanical engineers etc). The software was used to visualize and analyze signals from engine/vehicle measurements.

                                                    You could easily see that its standard library developed over time, so it definitely was not perfect. But in my opinion the language and the library contained some nice features to work with the kind of data that engineers had to analyze with the tool. Nonetheless, engineers started to demand Python support in the application, so the software had to ship a Python interpreter and add Python integration.

                                                    1. 4

                                                      Being open source under a permissive license is table stakes these days. A programming language implementation is the core dependency for any software project. No company is going to build a new product on top of a dependency from a startup without a second source, being open source gives you a second source for free (any other company can be one if upstream moves in a bad direction, from the perspective of their users). Non-permissive licenses are problematic because they require legal review (sure, it’s fine to use the compiler, but is it fine to use dev containers that ship the compiler? Is there any liability from distributing the compiler to third-party hosted CI? What requirements exist for any customers who want to build extensions in the same language? What happens if you want to bundle part of the compiler to provide an extension mechanism? Do these problems get easier or worse if your product is SaaS?)

                                                    1. 2

                                                      I have no experience in machine learning, but for a long time I have wanted to add subtitles to foreign videos from Youtube. Thus, I gave Whisper a try. But to me it feels horribly slow on CPU. It has been running on the default model (I think small) for about 10 minutes to detect less than 1 minute. Have to say I am a bit disappointed that you probably also need GPUs for inference nowadays.

                                                      Does somebody have experience with the speed on GPU?

                                                      1. 3

                                                        It transcribed a 1:00:31 YouTube video[1] (with what looks to be >95% accuracy) in 29:18 on my 3080 using the medium model. Using the small model took 15:55 (looks like mostly punctuation difference from medium and a few things like “laced” being wrongly transcribed as “least”), and base was 7:44 (looks about the same transcription as small.)

                                                        [1] https://www.youtube.com/watch?v=ffRmE69pQhM

                                                        1. 2

                                                          My 24 core CPU seemed close to 1x on small, probably a bit worse. So depends on your CPU a bit.

                                                          GPU in server that doesn’t cost all my money would be nice.

                                                        1. 12

                                                          One speculation I heard about Whisper is that it was developed to generate text training data to train GPT-4. GPT-3 used Common Crawl, and there is no larger text dataset in existence. OpenAI is running out of text, and Whisper is the answer.

                                                          This explains some strange features of Whisper. There is a mode to translate to English. Input is 30 seconds chunks, and there is no way to stream. Both are strange for a speech recognition system, but make perfect sense for a text training data generator.

                                                          1. 1

                                                            Do you want to say they want to collect more textual data from existing audio data? But is there really so much more audio/video data than text data?

                                                            1. 2

                                                              Do you want to say they want to collect more textual data from existing audio data? But is there really so much more audio/video data than text data?

                                                              Not the parent commenter, but yes, that’s what they appear to do.

                                                              But is there really so much more audio/video data than text data?

                                                              The amount of non-English audio data certainly is bigger than English textual data.

                                                          1. 1

                                                            Do I see this right that this sort of attack would not be possible with TOTP? I have TOTP as 2FA mechanism with my AWS account and there is no push notification involved. Don’t see how somebody could spam me into accepting them as a login device for AWS.

                                                            Is a push-based 2FA more secure in normal operation or why would I want to use it?

                                                            1. 2

                                                              I have a lot of thoughts about this, but see here for them because I don’t want to type them all out again.

                                                              1. 2

                                                                TOTP can be brute forced without the account owner knowing.

                                                                Push notifications from a vendor like duo is part of the incident Uber experienced.

                                                                For those wishing to keep push notifications, look into “Number matching”, Active Directory + Microsoft Authenticator and Duo have “number matching” where the account owner enters digits on screen into their device. While this solution makes it harder, I think it is still capable of being phished.

                                                                I collaborated with cadey on the post they linked next to this comment, consider reading push 2FA considered harmful.

                                                                1. 1

                                                                  I don’t think the brute-force argument against TOTP is that convincing. If you aren’t counting failed logins and locking accounts after X failures, then it’s a problem with your implementation not the concept as a whole.

                                                                  I still agree with your conclusions though. TOTP fails because it sucks to use, and webauthn looks to be a better alternative.

                                                                  1. 1

                                                                    If you aren’t counting failed logins and locking accounts after X failures, then it’s a problem with your implementation

                                                                    I completely agree with this. But either through ignorance or misconfiguration or no such feature at all, this check may not be performed on some SSO deployments or independent services and that is where it is vulnerable to brute force.

                                                              1. 2

                                                                I think this is the write-up where the author explains how he developed it: https://debugged.it/blog/ssh-direct-to-docker/

                                                                Have not read it yet, but looks interesting, because I was curious how you implement a SSH-server replacement.

                                                                1. 1

                                                                  golang’s ssh package supports that. It is easy to implement a new sshd alternative.

                                                                  1. 1

                                                                    Here is a tiny implementation https://github.com/ContainerSSH/MiniContainerSSH

                                                                    It’s a super barebones implementation of ContainerSSH to demonstrate how it works

                                                                  1. 5

                                                                    You should really use something like Go where it’s statically compiled into one binary and you don’t have to be a pip wizard to fix weird problems when they happen (not if, when)

                                                                    1. 14

                                                                      I dunno, all this stuff is weird to me since I know that people have been building sysadmin-y scripts and tools and daemons and other things in Python for literal decades and it’s basically just in the past couple years that people seem to have decided that none of it ever happened or worked. And weirdly there’s an inverse relationship between the quality of Python’s packaging ecosystem – which used to be horrible and now is pretty darned good – and the loudness of the complaints or the strenuousness of the people insisting Python packaging is unfit for any purpose.

                                                                      Like, I remember vividly what it was like to try to package and distribute Python in 2008. It was awful! Django literally maintained a “no external dependencies” policy for years because of how awful it was. And now it’s easy! It’s as if all the complaints that ought to have been happening back then have somehow been timeskipped forward to now.

                                                                      1. 7

                                                                        Maybe it’s even the problem that packaging became simpler and people start to include more external packages into their code.

                                                                        When I think about my Python experience at work (with too many badly tracked dependencies) I immediately want to avoid Python for any simple tooling, even though it is my primary language. But on the other hand, if I just use stdlib Python it’s so good for small scripts: Copy to my server, execute, fix bugs on the server, copy back to local PC.

                                                                        1. 3

                                                                          Jevon’s paradox but for dependencies.

                                                                          I think part of the issue is that packaging ecosystems keep evolving. Once Python had nothing, but it was okay, because everything had nothing. Then Python got pip and PyPI, which was a big step up! But around the same time Node came out, and while it was a mess with unpinned and nested dependencies, the concept of a node_modules folder was a big step forward and eventually they started pinning all the dependencies and it became pretty easy to use. (Of course, since it was easy to use, people made crap like leftpad and iseven.) Now compared to Go modules or Cargo, Python has fallen pretty badly behind.

                                                                          It’s interesting to imagine what will happen next to raise the bar again. Probably something inspired by Nix?

                                                                          1. 1

                                                                            I have similar experiences. However, I realised that I really fret if I have to package and share any python code if it’s more than a single file (some of this is rooted in how we do things at work). For sharing a single file, even with several dependencies, I have no fears, as I usually wrap it with Nix, just as described here: https://notes.yukiisbo.red/posts/2021/07/Spice_up_with_Nix_Scripts.html

                                                                          2. 4

                                                                            It’s cool in some circles to hate on what’s popular.

                                                                            1. 1

                                                                              And weirdly there’s an inverse relationship between the quality of Python’s packaging ecosystem – which used to be horrible and now is pretty darned good – and the loudness of the complaints or the strenuousness of the people insisting Python packaging is unfit for any purpose.

                                                                              That’s probably an indication of how much easier it got for other languages. It seems like Go and Rust are defining the standard language tooling. Even if python got a lot friendlier it’s not as friendly as those languages.

                                                                              Or maybe it’s an indication of pythons rise in popularity. When you are a niche language you get dedicated developers who care to learn the whole process. When you are popular everyone has to interact with you even if they didn’t take the time to really learn it.

                                                                              1. 3

                                                                                Go and Rust are also kind of cheating, because they both, for the moment, are static-linking-only. Which means you only have to worry about having local copies of dependencies in a build directory as you compile. As soon as you introduce dynamic linking, you have to make decisions about where to look for your dynamically-linked dependencies, and you’re back into the thing people specifically are complaining most about in Python, including in this thread (the need for venv or a similar construct to isolate potentially-conflicting sets of dependencies from each other).

                                                                                Rust at least seems open to the idea of eventually supporting dynamic linking, but it requires committing to an ABI and solving a lot of hard problems that Cargo currently doesn’t have to deal with. Go, I don’t know.

                                                                            2. 11

                                                                              Or, just maybe, we should use well-supported languages we’re good at and that we enjoy.

                                                                              My experience with Python tooling in regards to embedded development is good, especially because in the world of embedded development where you might need to rapidly iterate on how your tooling works you can dive into the source and modify it without much stress. It’s inane to me the author hasn’t heard of poetry or pipenv or such yet, maybe a couple minutes of Google would save him weeks yearly.

                                                                            1. 26

                                                                              Anyone who pays attention to copyright and software licensing will quickly discover that most of the open-source world is totally clueless on the subject, and small businesses are even worse. The only places that consistently get this right are big companies that can pay a small army of IP lawyers to review any dependency on third-party code.

                                                                              Example 1: Someone on a Slack I visit was asking for advice about their co-worker’s use of https://gitlab.com/kokizzu/gokil, which is written in what could be called “non-idiomatic” Go. I brought up that besides the code style issues, that library had no license and it would therefore be inadvisable for this company to be building their product on it. Their response was:

                                                                              it’s not like we have IP lawyers who are going ot enforce a clean-room rewrite or anything

                                                                              I’d rather remove it on its merits, than on a technicality.

                                                                              Example 2: Last week the project Objective Smalltalk was posted on HN (comments). One poster noted that the code was not open source:

                                                                              https://github.com/mpw/Objective-Smalltalk/blob/master/Stsh/Classes/MPWStsh.m#L6

                                                                              // Copyright 2006 Marcel Weiher. All rights reserved

                                                                              The author of the project replied:

                                                                              mpweiher | https://news.ycombinator.com/item?id=32450011

                                                                              Not sure why you think the presence of copyright makes something [ed: not] open-source. All open-source licenses depend on copyright, otherwise the code would be in the public domain.


                                                                              Anyway, to this post’s question of “Can someone please tell me what the fuck is going on with licensing because I am losing my goddamn mind” – it’s not you, it’s them. They’ve internalized the YouTube meme of “no copyright intended”. The only thing you can do about it is write tooling to discover this stuff automatically, and try to contribute to projects like Debian that will at least take a copyright bug report seriously.

                                                                              1. 8

                                                                                Your second callout is wrong. The chap you quote has a good point: copyright and licensing are not the same thing. They are interrelated but not the same. In order to define the license for something you have to be the copyright holder in the first place.

                                                                                1. 33

                                                                                  This is the same mistake as mpweiher made. The problem is not the copyright notice, it’s the “all rights reserved” written afterward.

                                                                                  1. 8

                                                                                    Okay fair enough. It wasn’t clear what part your were criticizing but yes that last phrase is more about licensing than copyright. I stand corrected.

                                                                                    1. 6

                                                                                      (and as far as I can tell not even a “all code here licensed under XYZ” statement elsewhere, unless I missed it?)

                                                                                  2. 3

                                                                                    law enforcement is as clueless: Paris cops tell tourists it’s forbidden to take photos of the Eiffel tower due to copyright (e.g. during events).

                                                                                    Taking the picture is what your eyes do, too – you just may no make it publicly available.

                                                                                    1. 5

                                                                                      Taking a picture vs. looking at something with your eye is (in my opinion) a difference with regards to copyright - at least in Germany. I do not know about France. § 15 UrhG says that the author has the sole permission to create copies; § 16 UrhG defines a copy as a transfer of the work to a medium that is designed to replay the work.

                                                                                      When you take a picture of a work you transfer it to a medium that is designed to replay it, e.g. on a computer.

                                                                                      § 15 lists different rights that only the author has (he can transfer them to someone else, e.g. selling these permissions): copying, distributing, exhibiting, publishing, broadcasting, and some more. Publishing is only one of many rights.

                                                                                      Copying, though, in Germany usually has an exception clause for private use. So, for private use it is usually allowed, which in effect would make your statement true in Germany (again, I do not know about France). But I think I read that it’s the same with the Eiffel tower illumination: For private use you are allowed to photograph it.

                                                                                      There are also other related exceptions. I think again in Germany, we have the exception that everything that is permanently visible from the street can be freely photographed and even distributed and published (Freedom of panorama). France seems to also have this exception in some form since 2016, but seemingly the illumination is not covered by it. I think in Germany freedom of panorama was heavily discussed for some temporary arts installations like the Wrapped Reichstag by Christo.

                                                                                      All in all, copyright is a very interesting topic :-)

                                                                                      1. 1

                                                                                        interesting indeed, §15, Abs 1, Satz 1 UrhG [1] says “Der Urheber hat das ausschließliche Recht, sein Werk in körperlicher Form …” – physical form!

                                                                                        And according §21 a non-physical reproduction is ok for private use.

                                                                                        [1] https://www.gesetze-im-internet.de/urhg/

                                                                                      2. 3

                                                                                        Your mention of the Eiffel tower made me curious.

                                                                                        This is what the official website says: “The Eiffel Tower’s lighting and sparkling lights are protected by copyright, so professional use of images of the Eiffel Tower at night require prior authorization and may be subject to a fee. Professionals should therefore contact the Eiffel Tower’s management company to learn about conditions for using the images depending on the case.” [source]

                                                                                        Pretty weird, IMHO. No idea if it’s been tried in court though.

                                                                                        1. 3

                                                                                          IMO the “professional use” is the important part.

                                                                                          1. 2

                                                                                            Indeed – good point. Would still be interesting to know if this reasoning would hold up in court.

                                                                                            1. 2

                                                                                              The saying goes “In court and on the high seas, you are in God’s hands.”

                                                                                    1. 20

                                                                                      I long left web development. Funny to see that this Tailwind thing seems to rediscover approaches we already overcame 10-20 years ago in web development when we went from hard-coded styles to CSS classes. And now Tailwind goes a step backwards and uses almost-hardcoded styles again disguised as CSS classes.

                                                                                      I also love this quote, which according to the article comes from Tailwind:

                                                                                      You have to think up class names all the time — nothing will slow you down or drain your energy like coming up with a class name for something that doesn’t deserve to be named.

                                                                                      Yeah, naming is hard, let’s stop naming things so nobody who reads the code later understands our intentions :)

                                                                                      1. 10

                                                                                        Yeah, naming is hard, let’s stop naming things so nobody who reads the code later understands our intentions :)

                                                                                        This is a straw man of the argument, which is not that you should avoid all names, but that you should avoid names when they aren’t needed. It is the difference between (part of) McIlroy’s famous UNIX 1-liner:

                                                                                        tr A-Z a-z |
                                                                                        sort |
                                                                                        uniq -c |
                                                                                        sort -rn |
                                                                                        sed ${1}q
                                                                                        

                                                                                        and something like this (pseudocode):

                                                                                        lowercased = lowercase(input)
                                                                                        sorted = sort(lowercased)
                                                                                        counts = uniq_counts(sorted)
                                                                                        sorted_down_by_count = sortByCount(counts)
                                                                                        topN = topN(sorted_down_by_count, n)
                                                                                        

                                                                                        The only thing we care about naming here is the final result: “the top n most frequent words”. Naming the intermediary calculations within this context is just noise, and makes the code less readable, not more.

                                                                                        1. 9

                                                                                          The only thing we care about naming here is the final result: “the top n most frequent words”. Naming the intermediary calculations within this context is just noise, and makes the code less readable, not more.

                                                                                          Someday, in one, two, twenty years, somebody will go through that code to debug a problem with locale-aware sorting and they will thank you for having named the intermediate steps. Or they will curse you having named only the result.

                                                                                          1. 5

                                                                                            they will thank you for having named the intermediate steps

                                                                                            They won’t, because the intermediary names provide no new information.

                                                                                            1. 9

                                                                                              the intermediary names provide no new information.

                                                                                              I would agree with this if the intermediate commands used descriptive long options, like sort --reverse --numeric-sort. In that case, yeah, sorted_down_by_count doesn’t add much. But compared to sort -rn and having to guess what those short options do or needing to peek at the manpage, i’d take a meaningful name like sorted_down_by_count any day.

                                                                                              Short options are nice for exploration on the command-line. But for actual scripts written onto files that other people might need to read, the descriptive naming of long options should always be preferred IMO.

                                                                                              1. 4

                                                                                                Yes I agree with this point. I was just using the McIlroy command verbatim. But long options for scripts all the way.

                                                                                              2. 3

                                                                                                They absolutely do provide new information

                                                                                                Firstly, you don’t have to mentally translate tr A-Z a-z into ‘start by lowercasing the input’ every time you read the code: you directly read ‘lowercase the input’. So the name provides information that is new when you haven’t mentally performed the translation step yet. Which you then don’t have to, because you already have the name.

                                                                                                Secondly, the name communicates the intent: the ‘sorted’ variable is intended to hold a sorted list. If some refactoring or other code shuffle results in that variable being used before ‘sort’ is called, that’s a clear red flag to the reader that something went wrong. So the name provides additional information that the first version does not have at all. You could use documentation/comments instead of names, but not the plain McIlroy version.

                                                                                                I’ll take the second version over the first one for anything that isn’t a throwaway command I construct just to get something done. I’m not a shell and I don’t like being forced to play one.

                                                                                                1. 4

                                                                                                  I used to think like this too, and the argument sounds reasonable, but it leads to an overly verbose style of programming which hinders readability for anyone fluent in the language.

                                                                                                  The only situation where the argument holds water is if you expect your maintainers to be unfamiliar or barely familiar with the language. Sometimes this does happen, and turning your code into a mini language tutorial can be appropriate then.

                                                                                                  Generally, there are two kinds of errors you can make (in code or prose): you can under-explain, or you can over-explain. It seems to me the position you are arguing assumes that over-explaining is not really an error, or is only a minor one. But this is not the case. Habitual over-explaining wastes a lot of time and greatly dampens the efficiency of communication. You want to get it just right.

                                                                                                  Firstly, you don’t have to mentally translate tr A-Z a-z I’m not a shell and I don’t like being forced to play one.

                                                                                                  Maybe you aren’t that familiar with bash? Because if you are, it is faster to read the original version. You immediately recognize the familiar names and what they mean.

                                                                                                  In fact, the new names introduce new cognitive load, because I must now load up the bespoke lexicon a random programmer into short term memory, mapping between the new terms and their definitions, which I already knew. And no matter how good you are at naming, your “self explanatory” names will never be as good as the terms someone already knows by heart.

                                                                                                  1. 2

                                                                                                    I used to think like this too, and the argument sounds reasonable, but it leads to an overly verbose style of programming which hinders readability for anyone fluent in the language.

                                                                                                    This sentiment is often stated, but I don’t think it’s true.

                                                                                                    People read/consume entire words at a time. For example, “–reverse” may be more verbose than “-r”, but it requires the same cognitive effort for experienced veterans. (And, obviously, a lot more cognitive effort for newbies.)

                                                                                                    This is why APL isn’t popular, and it’s also why glyph-based written languages are, in my opinion, objectively inferior.

                                                                                                    1. 3

                                                                                                      This is why APL isn’t popular, and it’s also why glyph-based written languages are, in my opinion, objectively inferior.

                                                                                                      As a counterpoint, and one of the best demonstrations of what I’m arguing, I encourage you to see whether the source or rendered output of Latex is easier for you to read:

                                                                                                      https://www.mathjax.org/#demo

                                                                                                      In the case of APL, the confounding variable is that acquiring fluency takes a very long time. But as someone fluent in J, and somewhat familiar with APL, I can attest that it can become quite readable. Aaron Hsu has some good videos on this.

                                                                                                      For example, “–reverse” may be more verbose than “-r”,

                                                                                                      As I said in my other response, I am in favor of long form flags, but it is a fair question to ask me why, as my argument could imply that short ones are better, so long as they are familiar. I think that, with flags, that so long as they are familiar is just too rarely true – there are too many of them, across too many tools. It’s too hard to make assumptions about familiarity, as opposed to “knowing what sort or tr does”, which is safe to assume of a bash programmer.

                                                                                                      There is a lot of judgment involved in getting your “density” right, and I am not arguing to never have intermediate names, far from it. Only that there are plenty of times where doing so is a mistake.

                                                                                                      Steve Yegge’s “Portrait of a n00b” is relevant here too:

                                                                                                      The question is, what do you do when the two groups (vets and n00bs) need to share code?

                                                                                                      I’ve heard (and even made) the argument that you should write for the lowest common denominator of programmers. If you write code that newer programmers can’t understand, then you’re hurting everyone’s productivity and chances for success, or so the argument goes.

                                                                                                      However, I can now finally also see things from the veteran point of view. A programmer with a high tolerance for compression is actually hindered by a screenful of storytelling. Why? Because in order to understand a code base you need to be able to pack as much of it as possible into your head. If it’s a complicated algorithm, a veteran programmer wants to see the whole thing on the screen, which means reducing the number of blank lines and inline comments – especially comments that simply reiterate what the code is doing. This is exactly the opposite of what a n00b programmer wants. n00bs want to focus on one statement or expression at a time, moving all the code around it out of view so they can concentrate, fer cryin’ out loud.

                                                                                                      1. 2

                                                                                                        There is a lot of judgment involved in getting your “density” right

                                                                                                        +1

                                                                                                        And there’s probably no “one true density” because people are different, even people with the same level of experience (e.g., veterans in computer programming language X).

                                                                                                        My real goal here is simply to express the idea that, for example, “–reverse” does not hurt, or slow down, veterans, when reading – their brains gulp down the entire word “reverse” in a single “gulp” - at least, based on what I’ve read about how the human brain works and reads.

                                                                                                        1. 1

                                                                                                          I encourage you to see whether the source or rendered output of Latex is easier for you to read:

                                                                                                          I’d argue that rendering LATeX as glyphs is exactly the same as rendering tr A-Z a-z as lowercase.

                                                                                                          If you’re the one who wrote the LATeX and revised it umpteen times, you don’t need to look at the rendered version: you just read the LATeX. The rendered version is for other people to read. And those are not people unfamiliar with LATeX or noobs. It’s also yourself a few months later: then you prefer the glyphs over the LATeX.

                                                                                                          1. 1

                                                                                                            What you’re missing is that the names typically won’t have the familiarity of well-known mathematical symbols. Sure, in the case of “lowercase”, it does. But the point is that what you typically get is programmers inventing strained words for ideas that don’t map to well-known concepts or anything in the domain – because they’re naming “in between” stuff. The result is a name soup that clouds rather than illuminates.

                                                                                                            Another analogy:

                                                                                                            The phrase “jump in place” will usefully convey to you a sequence of body positions and the associated motion.

                                                                                                            But if I am inventing some weird new dance move that you’ve never seen, which is used only once, then naming it is a waste of time. The demonstration of the move is the most efficient way to communicate it.

                                                                                                      2. 1

                                                                                                        Because if you are, it is faster to read the original version. You immediately recognize the familiar names and what they mean.

                                                                                                        I have my doubts about that, primarily because that’s just not how it works for me.

                                                                                                        It only happens like you describe when it’s a piece of code I have recently seen and I already know what it says. Then the pattern tr A-Z a-z is just a place locator for where I am in the code. I don’t need to interpret it, because 9/10 times I don’t care for it. I already know the code is correct, because I’ve recently seen it.

                                                                                                        If it’s new code or it’s been a few months, then I need to doublecheck. What happens in my head is tr, OK, A-Z, OK, a-z, OK, we’re lowercasing everything. Every time. It may look like tr A-Z a-z at a glance, but I have to check. It looks a lot like tr a-z A-Z, tr A a, tr AZ az, etc. There are too many small variants that change the meaning. ‘Lowercase’ does not have that problem and has the advantages I described before. The word carries more information than the command.

                                                                                                        1. 1

                                                                                                          Now the conversation is getting interesting, because I agree with a lot of what you just said but still, for most use cases, prefer the original code.

                                                                                                          because that’s just not how it works for me What happens in my head is tr, OK, A-Z, OK, a-z, OK…. ‘Lowercase’ does not have that problem

                                                                                                          This is a fair point, but I have two counterpoints:

                                                                                                          1. The “checking” you describe is really fast, like a second or less. Granted I have written a lot of bash, but fluency is one of my assumptions.
                                                                                                          2. This is perhaps the more important point…. You either:
                                                                                                            1. Have to “trust” that the name “lowercase” is correct (just as you’d trust that it’s A-Z a-z and not a-z A-Z when scanning without “checking”)
                                                                                                            2. Or have to read the definition anyway.

                                                                                                          So in either case, you are choosing your reading mode: “grok gist and trust details” or “verify code” (mode typically depends on use case – eg, if you are debugging you need to verify). If you are “verifying” then the advantage of “lowercase” doesn’t help you (you still have to verify), and if you are “grokking the gist” it also doesn’t help you, because all you care about is that the entire phrase “gets words sorted down by frequency of use”. It only helps when you are unfamiliar enough with the language that, say, it saves you having to read documentation to figure out what a function does.

                                                                                                          There’s more…. I chose the example because it was a brief and convenient way to get my point across, but it’s actually not great at demonstrating the worst parts of this style of intermediate naming….

                                                                                                          1. The names I chose here are pretty clear, which is “lucky” in a way, because the intermediate objects map to familiar ideas with well-known descriptions. In general, the intermediate objects of your data transformations don’t, and you end up groping to name things that don’t really have “names” (“user input with names validated but addresses still unvalidated”, etc).
                                                                                                          2. If you do this anywhere but in a tight local scope (eg, within a small function), the names pollute the larger scope, and a reader must verify that “this is just a temporary name used to describe a step in a data validation” and not “a variable definition that we use again later.”

                                                                                                          Again, I’m not against this style when it’s really needed to clarify something confusing… but only against the idea that “all familiar language constructs / short code idioms are inherently more confusing than a named variable.” That view, to me, is almost akin to code like the following, where every line has a redundant comment, which I’m guessing you do not like:

                                                                                                          // Gets the array without the first element
                                                                                                          const tail = arr.slice(1)
                                                                                                          
                                                                                                2. 4

                                                                                                  Yes, and as the article mentions, CSS lets you avoid names when they’re not needed.

                                                                                                3. 7

                                                                                                  if you’re talking about moving away from hard-coded style attributes, this isn’t really a fair comparison at all

                                                                                                  There are 2 main differences as I see it

                                                                                                  1 - tailwind classes are reusable & concise both as utility classes & via components

                                                                                                  2 - you can’t do :hover or :active states among other more complicated css features in style attributes

                                                                                                  I think like everything its hard to judge at a distance, need to really try it out to see if its actually problematic.

                                                                                                  For me, I used it for a few years & then I stopped but I’m seriously considering coming back because it tends to be better css than I write. I think its really great if you really know css, but for me it was valuable to spend a lot of time in plain css to understand it better.

                                                                                                  1. 5

                                                                                                    Web development was really different 10-20 years ago. I would argue that Tailwind isn’t really the same approach that we overcame but rather a familiar approach in a different context (component-based frameworks and libraries).

                                                                                                    1. 2

                                                                                                      !

                                                                                                      Yes. I was working on a series of posts about “naming things” this summer, and in the last two posts (https://t-ravis.com/post/doc/the_gizmos_role_in_markup/, https://t-ravis.com/post/doc/the_trouble_with_anonymous_gizmos/) I was bumping up against this and briefly included Tailwind in one of my drafts.

                                                                                                      I (temporarily) backed down from mentioning it when I caught a glimpse of truth underlying the separation of concerns part of the argument. I decided to take some time to reflect on whether it’s a logical reaction to (and makes lemonade of) a cluster of ways that HTML+CSS muddle separation of concerns (which I was/am already planning to write about).

                                                                                                    1. 3

                                                                                                      I think I already heard something similar years ago. If I remember correctly, back then the suspicion was that sending a link via Skype could lead to it being indexed by a crawler.

                                                                                                      From a provider perspective it’s probably a way to index the deep web.

                                                                                                      From a user perspective it’s an intrusion into privacy. But then again, MS is a US company and I have no trust in US companies (and US government) whatsoever.

                                                                                                      From a security perspective the problem is the HTTP GET login link. Even when I did web development (which is lots of years ago) I learnt that sensitive information should never be included in URLs.