Threads for bwr

  1. 3

    Benchmarks are often viewed with suspicion, for good reason. But I’m always grateful when someone posts a fairly detailed description of what they did and why, and what they saw as a result. I don’t like that posts like this mostly get comments like “Did you try X? Seems like you’re not really being fair if you didn’t!”, especially when X is, like, not using Python to interface with SQLite. The point wasn’t really to be fair, the point was to make an informed and realistic decision. When you want to make a decision like this, for what is ultimately a relatively small project, it doesn’t make sense to check every possible X.

    Anyway I’m glad we get to read about the results, and I hope people aren’t discouraged from posting things like this, or worse, bothering to measure performance at all.

    1. 2

      Sure. I think testing e.g. raw SQLite vs. raw Postgres wouldn’t be helpful because in this case the goal was to improve end-to-end performance, and the final app was written in Python with SQLAlchemy, so the speed of single threaded SQLite Python or whatever is a factor in the final analysis. But I wonder about indexes just because indexes so totally dominate database performance that it’s hard to compare SQLite vs Postgres before you compare SQLite vs SQLite…

    1. 2

      I don’t understand why this hasn’t been solved by browser manufacturers. Couldn’t there be a browser API for requesting cookie permissions? Then:

      • The browser manufacturers would be less likely to use dark patterns because their incentives are aligned more with users than with sites
      • The browser could enforce the user’s selection automatically without relying on the site to have special infrastructure in place
      • You could give the browser a policy to apply to every web site, and never have to deal with this again.

      I know this by itself wouldn’t solve the problem, since at first web sites would still (I assume) be required to use banners by law. But if the technology is widespread in browsers, the law can eventually be amended. “If you build it, they will come”.

      1. 9

        Couldn’t there be a browser API for requesting cookie permissions?

        You’ve fallen into the trap of the ad companies. You do not need consent to store cookies on a client device. That is the lie that they are pushing by calling these things ‘cookie banners’. If you require explicit user action to store cookies then:

        • You block some legitimate things. For example, DuckDuckGo stores your user preferences in a cookie. This contains the same text string that you can append to the query URL. It is not unique to you in any way (assuming a sufficiently large anonymity set).
        • You don’t block non-cookie forms of tracking.

        That’s precisely the response that the ad vendors want: it harms everyone doing non-evil things and causes backlash against privacy regulations, and it doesn’t stop them from tracking you using any of the other forms of fingerprinting available to them.

        1. 2

          I understand these criticisms and they’re valid, but I don’t currently see a realistic alternative that would be better - do you know of one?

          1. 3

            The current state seems pretty good to me: sites that spy on their users have to pop up something that annoys their users. Sites that don’t spy on their users don’t. This provides an incentive for people to move. Half the time I see a consent pop-up, my reaction is to just close the tab and go somewhere else for the information. If anything, I’d like to see browser manufacturers (at least, the ones that aren’t Google, since it’s counter to their interests) make these pop ups more annoying.

            If Google did not have a massive conflict of interest here, the obvious thing to do would be to penalise sites with consent pop-ups in their search rankings. Other search engines could do that and drive traffic to sites that didn’t spy on their visitors.

            The fallout from the recent court case that ruled that some of these ‘dark pattern’ consent pop-ups did not constitute informed consent is likely to be interesting because it means that just sticking an obnoxious banner that drives people to hit ‘I consent to whatever you want because I don’t understand what’s going on’ is not, in fact, the CYA move, it’s a way of increasing your liability.

            1. 3

              Make tracking completely illegal.

          2. 6

            I like this solution from a technical standpoint but I’m not at all convinced that the interest of browser manufacturers align with those of the users, not the sites. That’s true of Mozilla, to some degree, and maybe of Apple if we’re willing to stretch it a little. But the incentives of Google, who makes the dominating browser today, are pretty much aligned with those of the websites, not those of the users. Real web privacy would make Google pretty much bankrupt.

          1. 3

            I have recently been getting a lot of value out of using xonsh for things like this. It’s about as convenient as writing things in bash, but also about as powerful as writing things in python. The main tradeoffs are that it’s slower than either, and you have to deal with python package and environment management, but for many one-off jobs I don’t care about those things very much.

            1. 2

              Company: SecureDNA

              Company site: https://securedna.org

              Position(s): We are seeking a small number of developers and test engineers. Can you write code which is correct, secure, and performant? Are you full of devious ideas for how to break and compromise everything you touch? We want you.

              Location: 100% remote (primarily US and Europe, but also some China)

              Description: The SecureDNA project seeks to enable fast, reliable, cheap, automated screening of DNA synthesis orders, to prevent both careless and malicious users from synthesizing potentially-hazardous DNA sequences anywhere in the world. Our work is intended to prevent major pandemics, and we believe this project is potentially very important.

              We are an international, academic/commercial partnership project, including researchers from MIT, Tsinghua University, Aarhus University, Shanghai Jiao Tong University, and other world-leading academic institutions, in partnership with major international DNA synthesis providers. To learn more, visit our website or read our technical whitepaper.

              Tech stack: Rust, with some Python

              Compensation: Competitive salary, flexible scheduling

              Contact: jobs+lobsters at securedna.org

              1. 3

                One possibility, if all you’re interested in is longevity, is Coq. It’s a year older than Haskell (making it nearly 33), and for nearly all of that time it has had many fancy features that Haskell still lacks. It’s widely used in some mathematical research communities, and I believe its popularity has steadily grown. The primary downside is that it doesn’t do I/O by default (though there are now ways of getting around that, and e.g. extracting OCaml programs from your Coq code).

                1. 2

                  Been there, done that; Coq is great for many reasons but the language I’m looking for needs to be somewhat performant and i/o capable. I still love Coq regardless.

                1. 2

                  I think there are some good ideas here, but they’re all also in the decimal64 type described in IEEE 754, and decimal64 lacks many flaws that DEC64 introduces.

                  In particular, because nearly all user interfaces use decimal to display numbers, binary floating point representations result in many avoidable surprises for users. Thus, if you’re forced by performance constraints to use a fixed-size numeric type rather than an arbitrary precision library, following the law of least surprise dictates using a decimal floating point representation where possible.

                  1. 2

                    This is intriguing, but the firehouse-strength description kind of knocked me to the ground, This earlier description I found was a bit more approachable, until it lost me when defining True and False. Any hints for dummies who only know dummy languages like C++ and Forth?

                    1. 3

                      The basic idea here is that, when you want to represent some data in lambda calculus, you’ve only got one option for how to represent it: It’s gotta be a function, because that’s the only tool the lambda calculus gives you.

                      You want to choose your functions so that, whatever operations you wanted to perform on your data, you can instead do by function application. So when they say that True := λx.λy.x; False := λx.λy.y, what they’re really saying is “These functions give us enough information that we can distinguish one from the other, which is the main thing we need from a single bit: For example, if we know some value has got to be a bit, we can figure out which one it is by calling it as a function on two distinct values, and seeing which of those values we get back.” (note that this procedure is basically how you’d implement if/then/else, given that you’ve defined True/False this way)

                      1. 2

                        Imagine you have a list (or stack) of things and pretend that λ is pronounced chomp. Even though lambda in reality is simply just a function that always accepts exactly one argument (e.g. function (x) { ... } we’re going to compare it to eating food just for fun.

                        Now let’s start off simple with the identity function. Here’s how a Krivine machine works. Say our list of things is λ0 x y z. We see the lambda so we chomp the x. It’s eaten into a closure where it’s stored at slot 0. Then we get 0 y z. We evaluate the 0. We pull x out of the closure and we arrive at x y z.

                        One cool thing worth noting about the iconoclastic definitions in SectorLambda is that the if statement is defined as the identity function if = λ 0. Since the behavior of if is kind of integral to way this language operates. Most people actually don’t even bother including if in their lambda expressions for that reason. Another way if could be defined is λλλ [[2 1] 0] which is kind of like a longer version of the identity function, but that’s a better exercise to think about once you’re done reading this comment, especially if you mess around with the lambda.com -v program.

                        So let’s move on to something more complicated than the identity function. Consider false which is λλ0 by our definition. This expression chomps twice, so if our stream of things is λλ0 x y z then we chomp and the stream becomes λ0 y z and 0=x is tucked away in a closure. Then we chomp again and the stream becomes 0 z and 0=y is tucked away in a new closure, which means we have to reindex or push down 0=x to 1=x. Now we need to evaluate 0 z. Since 0 in de Bruijn indexing references the argument passed to the nearest enclosing function, we resurrect y so the stream becomes y z.

                        So what did λλ0 accomplish? In summary, it transformed the stream of things that follows from x y z to being y z. It acted as a selector or predicate. It dropped the x and kept the y. So if false=λλ0, then surely true would be λλ1. True selects the first of the two things that follow in the stream, so it would transform x y z into x z.

                        In any case, this selection behavior is the basis for logic the language. It lets us divide and conquer and that’s really all you need to start building your coding empire.

                        One thing you may be wondering is, why are the definitions of true and false backwards compared to C++? You’d expect false would come before true. One example of what makes this convention useful is the EOF behavior. For example, the NIL terminator is also defined as false. In the monadic I/O model, what NIL does is it skips ahead in the list of terms. In order to do that, it needs to hop over things until it eventually reaches an undefined term. That’s the point at which VM knows it needs to perform a read i/o operation, in order to “gro” the input list. It also allows us to determine program exit. It’s defined as a NIL selector skipping past the program image.

                        1. 2

                          Thanks very much! I like the Pac-Man analogy. It’s reminding me of concatenative languages, except here it sounds like you’ve got two stacks — one is the “list of things” being chomped & barfed, the other is the list of saved closures?

                          Mind-bending in a fun way. I reckon I can look up “Krivine machine” for more info when I find the time for a good rabbit-hole dive…

                          1. 2

                            Pac-Man that’s perfect. Yes two lists. Plus lists grow off each item in the two lists. There’s also a free list.

                      1. 5

                        This is not mutation. Mutation would be s[3]=‘d’.

                        1. 4

                          It is a mutation. If I have a list l, and I call l.append("a"), of course I’ve mutated the list. This is directly analogous, because the resulting string is not a newly-allocated string, but has the same identity as it did before. Semantically the difference can’t usually matter, since python knows there’s only one reference. But operationally, the string has been modified.

                          1. 5

                            There’s too much fighting around an assumption of “Python forbids this ever happening” for what really ought to be read as “Python may internally do this from time to time as it sees fit, but you the programmer cannot explicitly rely on Python doing it or exposing a predictable way to do it”.

                            1. 3

                              A mutable object is one whose contents can be changed after creation.

                              Here we have a new string that is being created and assigned to the variable. The fact that the memory location is being reused sometimes is quite different from the string object being mutable.

                              Only if we were able to create a string object then change parts of it without reassigning the variable would the string be mutable.

                              s="1234"
                              s[1]="9"
                              

                              Here s is not being reassigned, it is being mutated. You can’t do this with Python strings.

                              1. 3

                                The problem is nothing to do with immutability, it is around assumptions on the meaning of identity. Two distinct objects may have the same identity in Python if their lifetimes overlap. This is no different from the following in C:

                                const char *str1 = someString();
                                uintptr_t id1 = (intptr_t)str1;
                                free(str1);
                                const char *str2 = someOtherString();
                                uintptr_t id2 = (intptr_t)str2;
                                

                                It is perfectly acceptable for id1 and id2 to have the same values, because the lifetime of str1 ended and so its identity (memory location, in C) may be recycled. This is a necessary property for any language that has a notion of explicit identity and a finite set of values (e.g. a 32-bit or 64-bit integer) used to represent them. Without it, you would exhaust your identity space.

                                Languages such as Java and C# avoid directly exposing object identity for precisely this reason. Instead, they provide a hash-code value, which is guaranteed not to change while the object is life but is not guaranteed to be unique (though it is expected to have a low probability of collisions), and an identity comparison (which requires two live objects as arguments and so doesn’t have to worry about stale identities).

                                I’m not really a fan of languages allowing identity to be captured. Even the Java-style (Smalltalk-style?) hash code adds some overhead with copying GC, because it requires you to capture the original hash code when you copy an object (if it has been accessed).

                          1. 14

                            I don’t recommend every programmer read SICP, but for me personally it was one of the most insight-dense books I’ve ever read. If someone asks me for my ten favorite books on programming (or even my twenty favorite books on any topic), it’s gonna be on the list.

                            I think the author is wrong that it’s poorly written, because you can’t evaluate well-writtenness without considering the intended purpose. e.g. the best and most-recommended advanced math books are typically quite hard to read even for a math student, and downright unpleasant for those unfamiliar with what one might call “the effective way to learn math”. They wouldn’t be improved by making them appear “well-edited”.

                            That said, I wouldn’t recommend Rudin to someone looking for “a good math book”. And I don’t think programmers should read SICP unless they’re interested in a particular kind of engagement with the theory and practice of building software.

                            1. 13

                              I think the author is wrong that it’s poorly written, because you can’t evaluate well-writtenness without considering the intended purpose. e.g. the best and most-recommended advanced math books are typically quite hard to read even for a math student, and downright unpleasant for those unfamiliar with what one might call “the effective way to learn math”. They wouldn’t be improved by making them appear “well-edited”.

                              They’d be improved quite a bit, IMO. Writing well, teaching well, and doing math well are three extremely different skills. I’ve seen people who are good at all three explain advanced math topics, and they’re miles ahead of even highly-recommended advanced math books. But they had to spend a lot of time actively improving their teaching and writing, which isn’t something that’s as incentivized by academia as doing math well is.

                              1. 2

                                I get it. I got mired down in SICP and never finished, but for me The Little LISPer was that kind of insightful and dare I say even inspiring experience.

                                People are different, brains are different, and we all bring very different things to the table when it comes to books.

                              1. 1

                                Is this basically saying “you can use software transactional memory to build an API for mutexes?”

                                1. 2

                                  From what I understand, i think it’s more saying that we can use some similar techniques used in wait-free data structures (ie: have other scheduled threads help out a blocked, unscheduled thread to unblock others). I’d imagine that there are some commonalities with the implementation techniques, but I can’t really say for sure. (I mostly just think it’s neat).

                                1. 23

                                  The fsf is more interested in ideological purity than helping users solve their problems and do so with free software

                                  1. 14

                                    I think that’s kind of the point. If you want to benefit from the fruits of the FOSS community while also using the odd non-free app, firmware/drivers or even OS, then there is a wealth of information and tools available to you. Many Linux users are happy operating their computers this way and they certainly don’t need FSF’s help.

                                    FSF’s main purpose is being the irritatingly uncompromising voice. I like this article because it shows that they’re not managing to do that either - due to historical exceptions that probably made sense at the time, they’re now trapped in a weird middle ground where they’re neither pragmatic nor ensuring the freedom they really wanted. Hopefully they can modernise without making concessions to pragmatism or they will cease to be relevant at all.

                                    1. 10

                                      You say ‘ideological purity’; another might say ‘morality.’ I assure that you have some inviolable moral precepts; so does the FSF.

                                      1. 14

                                        Another might say “black-and-white thinking,” which as I grow older, I find to be harder and harder to maintain. The world is a mess of shades of gray.

                                        1. 17

                                          My take is that the FSF’s positions congealed in the 1980s and they (well, RMS) has lacked the nimbleness of thought to keep up with the changing world.

                                        2. 6

                                          What is the moral argument for closed-source blobs being totally fine so long as you don’t have to upload them from software, though?

                                          1. 1

                                            It kind of makes sense for things that talk to a remote service: it means they can’t force you to take a firmware update you don’t want by saying “firmware versions older than X will not be supported”. But it’s still not really a good argument.

                                        3. 2

                                          The thing about ideological purity is that it’s quite attractive to social movements. Totalizing worldviews are simple, and they give their adherents a feeling of righteousness (deserved or not) that can be a great recruitment tool, and also a great motivator for getting things done.

                                          1. 6

                                            A truly totalizing worldview is the opposite of simple. Take for example the ideology of the market, which subsumes and totalizes all social relations under the measurement of how productive labor investments are in exchange as money. The web of justifications for this naturally self-expanding totality are as varied and complicated as modern history itself

                                            1. 3

                                              Can you explain more about what you mean here? I think you’re disagreeing with the parent comment, but I don’t understand how what you say is incompatible with it.

                                              I.e. “the ideology of the market” is simple, at least as I understand it (“Trade lets people get what they want”). Its instantiation and justification is complicated. The same seems true in many cases (Communism: “The means of production should be owned by everyone”; Free Software: “Software must preserve these 3 particular freedoms”; Nationalism: “Our country is the best country”; Futuramism: “All Glory To The Hypnotoad”). The claim is that this core is usually simple, and easy to put a lot of emotion into. There are almost always a lot of contortions that go into dealing with objections, and into putting that core into practice. But people are drawn in by the central idea, and by how emotionally appealing it is, which puts a lot of pressure on the central figures to be extremely devoted to the simple core, at least in their outward behavior.

                                          2. 1

                                            On of my observances is that the FSFE on the other hand is very pragmatic and levelheaded - which makes them far more effective in my view.

                                          1. 2

                                            Rather than attempting to estimate the entropy of a scheme, wouldn’t it be better to just define a scheme that represents randomness directly in the encoding? If you target N bits of randomness, then you will not fall into these sorts of traps if you directly encode those bits in a symmetric number scheme, such as base16, base32, or diceware (base ~12.92).

                                            The math is sort of interesting, but I’m not sure why anyone would in practice want to do anything other than directly encoding randomness.

                                            1. 1

                                              I became interested in this because I was trying to develop highly memorable passwords. For example, you might want to choose a random grammatical English sentence as your passphrase. A natural way to do this might be to choose a random parse tree of a given size, but if you do that you’ll have some duplicated sentences (with ambiguous parses).

                                              1. 2

                                                There are also the EFF word lists, which are designed to be memorable, real, and chosen by dice roll. I particularly like the list where each word can be uniquely identified by its first three letters: https://www.eff.org/deeplinks/2016/07/new-wordlists-random-passphrases

                                                1. 1

                                                  I’ve created rpass for this. It generates random mnemonics, ie. rpass 128 yields:

                                                  juthor kezrem xurvup kindit puxpem vaszun bok

                                                  1. 1

                                                    Is it really English if the majority of the words are made up?

                                                    1. 1

                                                      You’re right, isn’t English, but “juthor” and “krezrem”, and, “puxpem” are pretty memorable non-words IMO.

                                              1. 13

                                                [T]he pragmatic reality is that FOSS developers live in a material world, a market economy, and need to make money somehow.

                                                Suppose I replace “FOSS developers” with “musicians”. (We can pick other trades, but music is the one I know.) Then there is a standard fragment of advice: get a dayjob.

                                                More importantly, musicians already have dual-licensing collectives, called record labels; artists signed under the label can easily sample each others’ music, but general usage requires paying the record label a licensing fee. It is well-documented that record labels are aggressive litigants, that they deal with organized criminals, that they abuse artists who sign over their copyrights, and that they lobby governments to improve their legal standing at the expense of independent artists and the public. Do we really want such entities representing us in the Free Software world?

                                                1. 3

                                                  I fear what you describe a bit, and that’s why I advise to avoid fully commercial and stick with non-profits. However there’s also a difference in that such projects would be dual licensed. Open Source users do not have to go through any collectives. For them each given software project is a normal (A)GPL library that they use as they used to. Wrangling with collectives/labels and their aggressive nature is just a day as usual for business. You use IP, you pay for it. Same as you charge others for using the IP you’ve built on top of IP you paid for.

                                                  1. 4

                                                    The problematic moment is when the collective aggressively pursues exclusive rights on the non AGPL distribution.

                                                    1. 3

                                                      Very true. Similar to gaming platforms (hi Epic!) and music and video streaming platforms. Part of the reason why I’d like coops to be non-profits. But that’s how markets work, and that’s how people pay and get paid in commercial settings.

                                                      1. 1

                                                        I don’t think being a non-profit helps that much. It’s easy to siphon off money to partners and get kick backs on the side.

                                                        I think at least some of the initial orgs should let the devs elect some oversight.

                                                        1. 1

                                                          Possibly true. “some oversight” seems like a good idea, but also can be undermined and/or corrupted. Part of the reason why Open Source is so awesome is that it avoids some of it altogether. But the whole idea is to let Open Source stay Open Source, yet participate in the commercial world with all the benefits and drawbacks it brings.

                                                  2. 3

                                                    I wonder if a partial solution here is for the collective to be owned jointly by the developers. That way its incentives can at least align with their interests, though this doesn’t limit other downsides of record labels.

                                                    1. 1

                                                      Coops are nice for this.

                                                      I’d join a coop and help out if there was enough money to go around to help me keep afloat. Damn, I would be willing to help found an European one - language permitting.

                                                    2. 1

                                                      This was the first analogy I thought of, too, but the music & art “industry” has gotten a bit better since those days. Ko-fi and Bandcamp are the first examples that leap to mind, and I’m sure there are others.

                                                      Honestly, a co-op that manages licenses and litigation for you sounds like a good idea. I would join one.

                                                    1. 11

                                                      The only “utility” for a cryptocurrency (outside criminal transactions and financial frauds) is what someone else will pay for it and anything to pretend a possible real-word utility exists to help find new suckers.

                                                      It’s also useful for making transactions that are strictly legal under the laws of the state, but that the people running the common payment processors don’t want you to make (possibly due to some amount of informal, off-the-record coercion from the government, or possibly on their own volition). I’ve personally used cryptocurrency transactions to send money to people who were being informally blacklisted by certain parts of the financial system.

                                                      In any case, I don’t think that the utility of being able to make criminal transactions is worthy of casual dismissal. There are lots of states in the world, with lots of laws regulating financial activity, either for its own sake or in order to accomplish some other state goal, and I don’t trust that all such laws are actually good.

                                                      It’s true that you can use cryptocurrency to commit fraud; but you can also do that with electronic fiat currency, or physical cash or gold or Ea-Nasir’s famous copper ingots. That’s not an argument against using cryptocurrency specifically.

                                                      1. 31

                                                        In any case, I don’t think that the utility of being able to make criminal transactions is worthy of casual dismissal.

                                                        I do. People bring this up as a defense of web3, but it’s libertarian insanity. Government is a thing we do together. If government is broken, that’s the problem. You don’t fix the problem by having people flout the law. That’s just adding a second problem instead of fixing the first!

                                                        It’s like saying there’s a tiger in my house, but it’s okay because all the windows are smashed. Tigers are extremely bad, and you should not tolerate them! Broken glass is also very bad (not as bad as tigers, but quite bad!), and while yes, the tiger does make having a broken window seem useful, it’s not actually helping in any way. Also, the tiger can just jump out the window and eat you anyway! (In the analogy, a bad government can still force you to not do crypto transactions by use of force. Crypto only mitigates against bad but lazy governments.)

                                                        At the end of the day, there’s no alternative to society. Lone human beings die. At the very least you need a decade of training to learn how to survive in the woods alone, and then you need unoccupied woods! If your society is bad, you can try to change it or you can join a different society, but you can’t just ignore it and think you’ve solved things.

                                                        1. 9

                                                          Government is a thing we do together. If government is broken, that’s the problem.

                                                          I agree, but sadly, this is a version of reality exclusive to relatively-well-functioning democracies. Lots of people don’t live in those. That said, cryptocurrency probably isn’t necessarily going to help those people much either.

                                                          1. 8

                                                            Government is a thing we do together everywhere. The reason democracy is good is that it has a mechanism built in to acknowledge this reality, but even in e.g. North Korea, the system doesn’t work without apparatchiks who do whatever Kim Jong-un tells them to do. In every case, government is based on the consent of the governed. In any event, crypto in NK just helps the state evade global currency controls, but does not help the man on the street who is not able to afford or allowed to have an internet connected computer.

                                                            1. 2

                                                              In every case, government is based on the consent of the governed.

                                                              This is true for some definition of “consent”, but I don’t think it’s a very useful definition. Indeed, in repressive regimes the choice is often “live your life as well as you can under the circumstances”, or “fight the system and almost certainly die or be imprisoned”. I don’t think it’s at all fair to claim that NK governs “by consent of its people” when this is the situation for most of those people.

                                                              There is of course always a question of whether a given technology is helpful or harmful to those who might want to change the government, and it’s not clear to me which way this goes with respect to decentralized currency and apps - it seems highly contingent on facts about each regime and its citizens.

                                                              1. 4

                                                                It’s a semantic argument about the meaning of “consent”. Certainly, in dictatorships where the Schelling point changes, the dictator ends up hung in a town square. If you want to use a different word for that than a withdraw of “consent” fine, but no government works without 80%+ of people going with the flow.

                                                          2. 3

                                                            I’m on my phone so I am not going to be able to give as good a response as I’d like but I believe you are making a unfair assumption about the means of fixing government (namely you assume it is always good and possible to fix government using government) however I think that in certain (extreme!) situations it is necessary to essentially “rewrite it” furthermore I believe we just happen to be living in such an extreme time so I am sympathetic to the “crime must be possible” viewpoint because we certainly have not made a sound+complete system of law and our governance is failing to adjust to various changes (mostly internet right now but global warming is also an elephant in the room).

                                                            1. 14

                                                              namely you assume it is always good and possible to fix government using government

                                                              I am not assuming that.

                                                              furthermore I believe we just happen to be living in such an extreme time

                                                              LOL, not if you live in the United States or the EU. Not even close.

                                                              mostly internet right now but global warming is also an elephant in the room

                                                              Climate change is an international coordination problem. Making it easier to evade national rules does not help with international coordination. The opposite, in fact.

                                                              1. 1

                                                                Our premise is not compatible, I’d be curious how you define coordination because from how you use it I read it almost as a synonym for “benevolent dictator” - one concept that may nudge your premise closer to mine is “evolving to extinction” - the “high modernism” of monoculture agriculture + GMO + pesticide / antibiotics are examples of these hyper-efficient but very brittle optimizations that can blow up in out face if the climate shifts a bit (or any of our current assumptions gets invalidated).

                                                                Web3 is nonsense (or at least the way it is marketed) and PoS/PoW as a consensus mechanism are also not a solution to the real problems but that doesn’t mean that you are right about these other points you make; evading national rules will likely become a necessity in some places (arguably already has).

                                                                Anyway all your responses are just catchy soundbites I don’t see any real point being made.

                                                                Flouting national law is exactly going to be a very important thing to do if we want the international coordination required to react to climate problems (fwict).

                                                                1. 2

                                                                  Are you proposing wildcat geoengineering? I mean, if you think it’s come to that, wow, but I don’t think so, and to the extent we do climate hacking, it should be with international treaties and as much democracy as possible because we all share the same planet.

                                                                  1. 1

                                                                    No I am suggesting that the economic system is the driver of pollution and that by rights it shouldn’t be considered legitimate, countries that could perfectly well stand on their own are in ruins due to economic policies that citizen should be able to subvert or circumvent in order to organize into actually functioning communities and economies for dealing with the actual problems.

                                                                    1. 1

                                                                      I see. I agree that capitalism is problematic and needs to be constrained because the core assumption of endless growth is not healthy. OTOH, the Soviets literally erased the Aral Sea by mismanaging their economy, so I don’t want make the mistake of assuming that something else is going to work just because it purports to be different. It’s not a good situation when there’s one huge, seemingly impossible problem, and then someone says, Oh, to solve that problem we just need to solve this other huge, seemingly impossible problem.

                                                            2. 2

                                                              In the analogy, a bad government can still force you to not do crypto transactions by use of force.

                                                              Offering up that choice is exactly the point. Eroding a bad government’s ability to masquerade as a good one is a means of social change.

                                                            3. 9

                                                              That’s not an argument against using cryptocurrency specifically.

                                                              So what we’re saying isn’t that “cryptocurrency can be used to buy illegal drugs and therefore crypto is bad”. We’re saying “one of the few use cases of cryptocurrency is to buy illegal drugs”.

                                                              Cryptocurrency has other problems (efficiency, climate impact, cost, volatility), some of which are outlined in the article.

                                                              If the only point in cryptocurrency’s favor (scoring some tabs on the DL) is getting misinterpreted as an argument against it, then that’s saying something about how awful cryptocurrency is overall.

                                                              1. 6

                                                                And I’m saying that the ability to buy “illegal drugs” is actually a really important use case that itself justifies the use of cryptocurrencies - especially when you extend the category to any type of transactions that is illegal or informally restricted by payment processors.

                                                                My point was about fraud though - I don’t care that it is possible to defraud people using cryptocurrency, because it has always been possible to defraud people with non-cryptocurrencies. Mitigating the possibility of fraud is something that has been a consideration for every economic transaction we’ve ever done in our lives, going back 4000 years to the Ea-Nasir tablet. Cryptocurrencies allow both new types of fraud and new ways to mitigate fraud; they are not fundamentally a fraud-prone medium of exchange distinct from other ways of implementing money.

                                                                1. 12

                                                                  We’re still talking past each other.

                                                                  You don’t need to convince me that enabling restricted transactions is a pro (although your defensiveness around it tells me that it’s a kind of iffy pro).

                                                                  Instead, look at the enormity of what’s more unambiguously in “con” column.

                                                                  We who hate cryptocurrency are arguing against a world computer weaker than a Raspberry Pi, insane energy waste and electronics waste..

                                                                  That’s what tech dorks urgently need to have a conversation about.

                                                                  The web3 hype as a whole with its weak and flaky tech is a scam to suck bagholders into crypto.

                                                                  The article is about web3, about wack stuff like Reddit tying upvotes and karma into proof of work blockchain tokens etc. That kind of kooky BS is what we need to put a kibosh to because it’ll eat the Earth.

                                                                  The tech stack sucks and is fraudulently oversold.

                                                                  That’s a bigger problem than getting the best dope stash on the cinder.

                                                                  If someone says “the only use case (outside of foo) is bad, and there’s these thirty thousand other horrible showstopping problems” there’s no point in trying to convince them that “foo is good”. They said outside of foo,

                                                                  Like, my foot is pretty good and important but If someone said “The only reason (outside of you getting to keep your foot) that you should take this new medicine is that we’ll make money, because the drawbacks include your head falling off and your heart stopping and your tongue turning to ash”. I don’t wanna hear about how good it is to have a foot, I’m not ready to lose my head over it!

                                                                  1. 7

                                                                    We who hate cryptocurrency are arguing against a world computer weaker than a Raspberry Pi, insane energy waste and electronics waste..

                                                                    It doesn’t bother me that the Ethereum blockchain is a weaker computer than a RasPi in some sense, any more so than it bothers me that the RasPi is weaker in some sense than a commercial server rack. These are different constructs designed for different tasks, and there’s no reason why I can’t use all of them at different times for different purposes. The point of the Ethereum blockchain is that it’s a censorship-resistant global smart contract platform, which is not a thing an individual RasPi can do. It doesn’t matter that the Ethereum blockchain would be bad at doing things I might do with my RasPi, like running my home automation system. Of course I’m not arguing that it would be bad if the Ethereum blockchain had higher computational throughput, or that Ethereum specifically is the best smart contract platform.

                                                                    I also don’t care that some people think the existence of blockchains constitutes energy and hardware waste. Whether or not you view the use of some resource as waste depends on your judgment of the ends to which those resources are put, which is the actual thing we disagree about. It takes a lot of energy and raw materials to manufacture Raspberry Pi’s, but I wouldn’t call that waste because RasPi’s are a product people are willing to pay money for to accomplish goals, just like blockchain computations.

                                                                    That’s what tech dorks urgently need to have a conversation about.

                                                                    As a tech dork (I prefer the term “professional computer programmer”), I deliberately decided to seek software developer jobs at companies working on cryptocurrency-adjacent technologies, because I think this technology is important and I want to work on making it better. If other technologists think that cryptocurrency is bad and that I shouldn’t do this, then I do so in defiance of those people. If other people want to impose political changes in polities I live in to try to make it harder to legally work with cryptocurrencies, I will fight them on the political level.

                                                                    The article is about web3, about wack stuff like Reddit tying upvotes and karma into proof of work blockchain tokens etc. That kind of kooky BS is what we need to put a kibosh to because it’ll eat the Earth.

                                                                    I don’t think it is a bad idea for a Reddit-like communication platform to be implemented using smart contracts, although I am skeptical of the motivations of Reddit as a company in doing so. A lot of the problems with Reddit are precisely that a single, rather-small private firm controls the canonical database holding the upvotes and karma and so forth, and can mess with them in arbitrary ways, including ways that the userbase disapproves of. A smart-contract based solution removes the single point of control, which is worth the computational efficiency hit.

                                                                    See also here for interesting ideas about practical applications of blockchains to create better versions of existing internet technologies (in this particular case, user logins).

                                                                    The tech stack sucks and is fraudulently oversold.

                                                                    I don’t think it is actually true that the tech stack is fraudulently oversold, and that it “sucks” only in the sense that the technology is in its infancy right now. This is the same point of view that someone could reasonably have taken about any number of particular technological innovations in computing, in their early days.

                                                                    1. 16

                                                                      I also don’t care that some people think the existence of blockchains constitutes energy and hardware waste. Whether or not you view the use of some resource as waste depends on your judgment of the ends to which those resources are put

                                                                      Unlike any other human endeavor, proof-of-work is exceptionally inefficient and wasteful, and it’s wasteful by design: if humanity comes up with a more efficient process to mine, the difficulty factor increases to meet that ten minute dream time, immediately wasting all that newfound efficency. If everyone were mining on like a hand-cranked soroban they’d get the same rate as if everyone were mining on a 1.21 gigawatt beowulf cluster of skyscrapers-sized hypercubes. All efforts of making it more efficient is inherently wasted (but because of prisoner’s dilemma, miners are still incentivized to pour more and more juice in).

                                                                      A product people are willing to pay money for to accomplish goals, just like blockchain computations.

                                                                      A transaction that doesn’t adequately account for the cost in externalities (such as planets being wrecked up). That’s why the “it’s none of your business what we wanna spend resources on” doesn’t apply. It’s one world, we all need to take care of it.

                                                                      If other people want to impose political changes in polities I live in to try to make it harder to legally work with cryptocurrencies

                                                                      OMG yes that’s what I want! Specifically proof of work needs to be banned.

                                                                      I don’t think it is a bad idea for a Reddit-like communication platform to be implemented using smart contracts

                                                                      What I’m specifically arguing against is the proposed implementation which is going to suck and which is going to be a layer-2 app on the original proof-of-work iteration of Etherium. For the third time: I’m not saying that the promised benefits are bad: if someone would’ve told me a few years ago that Reddit was going federated and decentralized like email or Jabber I would’ve been overjoyed. It’s burning down the world to do it that I don’t quite think is worth it. I don’t wanna have the smartest contracts on the cinder.

                                                                      A smart-contract based solution removes the single point of control, which is worth the computational efficiency hit.

                                                                      We’re not talking about a bounded efficiency trade-off that we could reason about, do LCA on, or try to mitigate or offset. Instead, it’s a runaway inefficency spiral. No programmer would willingly leave memory leaks in their code.

                                                                      1. 10

                                                                        For the third time: I’m not saying that the promised benefits are bad: if someone would’ve told me a few years ago that Reddit was going federated and decentralized like email or Jabber I would’ve been overjoyed. It’s burning down the world to do it that I don’t quite think is worth it. I don’t wanna have the smartest contracts on the cinder.

                                                                        Notably, Lemmy already provides a federated and decentralized Reddit-alike, without any Web3 nonsense — it uses ActivityPub for federation. Obviously, that means trusting your local admin to some extent, and other site admins to a lesser extent, but I don’t really think “trustlessness” as an ideal or as a practical matter is a good thing, and certainly not worth the costs of cryptocurrency. I don’t need to have the most censorship-resistant shitposting site on the cinder, either.

                                                                        1. 9

                                                                          For that matter, I don’t trust the crypto admins either. I mean, you mine shit these days in pools? There, I have to give trust. Just like in the article. We don’t remove anything technologically, we only add a lot of complexity (and burn the planet as a bonus) and I still have to trust someone.

                                                                  2. 6

                                                                    And I’m saying that the ability to buy “illegal drugs” is actually a really important use case that itself justifies the use of cryptocurrencies - especially when you extend the category to any type of transactions that is illegal or informally restricted by payment processors.

                                                                    Buying illegal drugs (and ethically-produced artisanal porn) are good, but blockchains are so bad that even being able to use them to buy illegal drugs and pay a fursuited femboy to read “State and Revolution” to me in a throaty whisper when PayPal won’t let me pay them still isn’t enough to justify their use. Like ~snan said, I don’t want the best drug and porn stash on the cinder.

                                                                    Obviously, this mainly applies to proof-of-work, which is an existential threat to human civilization; proof-of-stake has other, separate, intolerable problems.

                                                                    1. 5

                                                                      I don’t care that it is possible to defraud people using cryptocurrency, because it has always been possible to defraud people with non-cryptocurrencies

                                                                      The big selling point for centralised payment processing systems is that it’s possible to reverse fraud. If someone empties my bank account, my bank has a mechanism for reversing the transaction. This has, itself, been used as a mechanism for fraud where someone sends you a million dollars and asks you to send them a hundred thousand as a processing fee, you send them a hundred thousand and at the end of the window to reverse the transaction, they pull back the million and unless you notice immediately you’re too late to pull back the hundred thousand. Aside from cases like that, it’s very useful. I can spend money on my credit card knowing that, if the seller doesn’t actually deliver the goods or if they’re defective, then I can reverse the transaction. With a crypto currency, the only way that I can reverse a transaction is if 50% of the participants agree. In theory, you can implement this kind of thing with smart contracts, but you then need a trusted arbiter who will decide whether the transaction should be reverted. At this point, you’ve just reinvented a centralised system (that arbiter could just be a bank) but with all of the inefficiencies of cryptocurrencies.

                                                                      Most of the regulations surrounding consumer banking exist because they address problems that impact consumers in an unregulated banking system (not all, some exist to promote lock in, as happens in any system susceptible to regulatory capture). Any system that wants to replace the banking system and be useful to normal people will need to provide the same protections.

                                                                      For drug buying, it’s effectively an incredibly inefficient way of implementing unbacked IOUs. It may be useful in removing the Mafia (who traditionally provide backing for IOUs used for black-market transactions) from the equation.

                                                                      1. 1

                                                                        In any money transfer system where it’s possible for some party to reverse a fraudulent transaction, it’s possible for that same party to fraudulently declare a legitimate transaction fraudulent, and commit fraud in the other direction. It’s sometimes useful to be able to engage in a monetary transaction where all parties to the transaction know it is impossible to reverse (e.g. if one or both of those parties does not trust the centralized bank system). For other cases, it is possible to build arbitrarily complex trusted arbiter systems with smart contracts, including systems that haven’t yet been tried in traditional banking.

                                                                        For drug buying, it’s effectively an incredibly inefficient way of implementing unbacked IOUs. It may be useful in removing the Mafia (who traditionally provide backing for IOUs used for black-market transactions) from the equation.

                                                                        “drug buying” needs to be understood as synechdoche for a large class of transactions, not all of which are actually illegal, just opposed to the political and moral sensibilities of private payment processing firms. But in any case, what’s a more efficient way of implementing unbacked IOUs? The ability to facilitate monetary transactions between parties that don’t trust each other is exactly the technological innovation that blockchains make possible; they can serve as a replacement for informal mafia violence backing black- or grey-market IOUs for the exactly the same reason they can serve as a replacement for the (usually) implicit state violence backing fiat currencies.

                                                                        1. 6

                                                                          In any money transfer system where it’s possible for some party to reverse a fraudulent transaction, it’s possible for that same party to fraudulently declare a legitimate transaction fraudulent, and commit fraud in the other direction.

                                                                          Fraud is rare in existing centralized systems due to many externalities, and recovering from it is possible and normal. Fraud is categorically more common in decentralized systems due in large part to the lack of those externalities, and recovering from it is extremely difficult at best and usually literally impossible.

                                                                          Pointing out that fraud exists in both models and other similar equivocations are not convincing arguments, because the fundamental point being made is about degree, not mere existence.

                                                                          It’s sometimes useful to be able to engage in a monetary transaction where all parties to the transaction know it is impossible to reverse

                                                                          “Sometimes” is doing a lot of heavy lifting in this sentence. In fact these situations are extraordinarily rare.

                                                                          For other cases, it is possible to build arbitrarily complex trusted arbiter systems with smart contracts, including systems that haven’t yet been tried in traditional banking.

                                                                          It certainly is. And when you burn through enough iterations of those things, stomp out all of the bugs and fraud and graft and incentives for malicious actors, build in protections for users to make the systems humane, what you end up with will be an inefficient clone of the current monetary system. And a whole new generation of cynics fed up with its idiosyncrasies, chomping at the bit to reinvent it all over again.

                                                                          The ability to regulate a system to constrain it is actually good and important and necessary if that system will serve humanity. A system that can’t be regulated is actually a bad system!

                                                                  3. 2

                                                                    Also the quote applies to state fiat currency just as much as to cryptocurrency.

                                                                  1. 9

                                                                    The black market would have paid much more for these exploits than the chump change he was given as bug-bounty by most of the companies (if at all).

                                                                    Given we all know how much money is wasted in big corporations everywhere, they should drastically increase their bounties if they really want to motivate people to become and remain white hats.

                                                                    1. 5

                                                                      I agree with you that bounties should be much higher, but I think there’s also substantially higher risk involved with the black hat world, so the bounties shouldn’t actually need to have perfect parity.

                                                                      There’s also the risk of “cobra effect” if your bounties get too high, where your employees or vendors are eventually incentivized to secretly collaborate with “researchers” to introduce and “find” security flaws.

                                                                      1. 3

                                                                        Is it illegal or unlawful to sell computer exploits?

                                                                        1. 7

                                                                          IANAL, but Yes-ish. Even if found not guilty, it is immensely powerful to silence and intimate security researchers by reaching for broadly worded laws like the Computer Fraud and Abuse Act (CFAA) or DMCA. There are some intimidating laws in various jurisdictions (Germany has the “Hacker paragraph” which seems worse than CFAA imho)..

                                                                          For things to be research-friendly, bug bounty programs typically have to promise not to sue explicitly. E.g., https://blog.mozilla.org/security/2018/08/01/safe-harbor-for-security-bug-bounty-participants/

                                                                    1. 17

                                                                      I have yet to hide any tag on Lobsters, but if there were a tag for these stupid OOP vs FP hot takes I would hide them in an instant.

                                                                      1. 16

                                                                        I’d generalize that to every article “Uncle Bob” writes.

                                                                        1. 3

                                                                          But you would miss the discussions on Lobsters, which are often better than the OP ;)

                                                                          1. 2

                                                                            Same; I think I’d hide even an OOP tag by itself, never mind FP.

                                                                          1. 8

                                                                            https://rmmh.github.io/abbrase/ walks Markov chains. Memorable and doesn’t lose entropy.

                                                                            1. 3

                                                                              That method would still need 12-13 words to get the 127bits of entropy. But I like that it’s helping with her total length of the final password without losing entropy.

                                                                              1. 4

                                                                                I’m curious where her 128-bit requirement came from. That’s a typical minimum these days for symmetric crypto keys, but that assumes a lot about the threat model and the attacks that will be mounted against the crypto. A password has to handle two threat models:

                                                                                • Online attacks. If someone has access to the authentication interface for the thing that you’re trying to password protect, can they guess your password?
                                                                                • Offline attacks. If someone leaks the password database, can they generate your password from the salted hash?

                                                                                For online attacks, 50 bits of entropy seems like plenty. If you limit logins to one per second with a given username, then it would take 2^49 seconds on average to guess a 50-bit password. That’s almost 20 million years, so it’s far more than you’d need for this threat model.

                                                                                For offline attacks, assume hash is properly salted and so you have to attack each password independently. Most password hashing algorithms are designed to be slow (and difficult to implement as fixed-function circuits). If you assume that the person has been incompetent and used SHA256, then this starts to be a bit worrying. A single GPU will crack a 50-bit password in around 10 days on average (using the 622 million SHA256 hashes per second from Some Guy On the Internet). If they’re using Argon2, this is a lot better. It looks as if GPU hash rates Argon2 (which is designed to not get much speedup from GPUs or cheap fixed-function silicon or FPGA) are still under 20,000/second. That means that we’re looking at about 8-10 years of GPU time per password for offline attacks. Of course, these attacks are intrinsically parallel and so if you wanted crack a single specific password then you’d be able to just throw more GPUs at it. If you’re using a cloud provider, one GPU for a year and 365 GPUs for a day cost the same, so you might as well throw a load of them at the problem. Using Azure’s spot pricing, cracking a single 50-bit Argon2-hashed password would cost a bit under $10K.

                                                                                Next comes the security economics bit: If it would cost $10K to crack your password, what is the value of the stuff that it would gain access to? If it’s a lot more than $10K, then you should worry about your 50-bit password if offline attacks are in your threat model. Typically; however, they are not. For high-value targets, the password is provided to some isolated hardware / software that stores the real encryption key. For example, most of my day-to-day work use is protected by a 6-digit PIN that is used to authenticate me to the TPM. That PIN is completely useless if you steal it unless you also steal the TPM. Once I’ve logged in, the TPM will provide RSA signatures with a key that the TPM is designed to make it impossible to extract. To impersonate me to a remote system, you’d need to extract that key, not my password. At that point, it’s significantly more secure than the OS kernel and so any sensible attacker would look for a browser or OS exploit to compromise the endpoint. Adding more entropy to my password would not make life any harder for an attacker.

                                                                                All of that said, I really like the abbrase model. It’s a shame that it can’t actually be used in most places as-is, because so many things require you to enter a password with a mix of upper-case, lower-case, numbers and symbols. I suppose you could just add a A1; on the end of every password…

                                                                              2. 1

                                                                                That’s interesting! I think you could actually have a system that has some of the advantages of both of these methods, if you made an alternative diceware list with a way of uniquely shortening each word. i.e. one reason diceware is nice is that you can use it without relying on a prng. I suppose you could also make a version of this app that takes any sequence of triplets and mnemonizes them, which would be another good way to do it if you don’t mind typing your password into an app once (which I usually don’t mind too much)

                                                                              1. 3

                                                                                It’s a good idea but I think 13 words is still pretty crazy. It’s not only a lot to remember, you also need to type a lot. Also, as OP described, even in the best case scenario where people use a password manager, they still have to remember at least two to three passwords.

                                                                                On a side note: I wasn’t able to open the link for the entropy calculation method.

                                                                                1. 2

                                                                                  It’s definitely a tradeoff; you can of course use the same trick for shorter passwords.

                                                                                  I changed the link to point at dropbox instead of overleaf; hopefully that works better.

                                                                                  1. 2

                                                                                    How often do you type your password? Once a day? Once an hour? I think that that helps definite what ‘too long’ is.

                                                                                    If you have a 13 word password that you only type once a month when you restart your computer, and a 6 word password to unlock the second lock on your gpg key every 4 hours, it’s different than a 10 character password you have to retype every 10 minutes.

                                                                                    1. 1

                                                                                      I see were you want to go but the catch is that the passwords you type the least would then also the hardest to remember. Not a good combination.

                                                                                      1. 1

                                                                                        That’s where the tradeoff between memorability and shortness comes in.

                                                                                        Passwords I type very frequently are generated with about 5 bits per character by excluding pairs on a qwerty keyboard that are hard to type together, passwords that I type less frequently are much longer diceware type which are easier to memorize with infrequent use. Passwords I don’t type at all are just 100 bits or so of base64

                                                                                  1. 2

                                                                                    I’m normally impressed by the Rust team, but from my position of ignorance this seems like a surprisingly bad solution to this problem. It sounds like at least some people were planning the whole time to release 1.52 without fixing the bugs that were exposed by this check (just, without the check in place). Maybe they didn’t know about the miscompilation?

                                                                                    The chosen solution, to totally disable a feature that’s crucial for usability, seems pretty extreme - surely there are less extreme workarounds? e.g. you could presumably simply clean the cache when you see this error.

                                                                                    I do really appreciate the helpfulness of this blog post in describing the problem, and I’m probably missing some important details that make this the best way to deal with the situation.

                                                                                    1. 21

                                                                                      Rust is a lot about correctness, and we’d rather prefer to keep such a critical feature off rather than having it broken. We didn’t know about this bug until last Thursday and it’s actually present in versions before Rust 1.52.0.

                                                                                      Even if rare and sporadic, this is most likely an old bug. This means that the recommendation for all users is to disable incremental compilation, even on older Rust versions if they want to make sure they don’t hit this bug.

                                                                                      The new version was released to avoid users from downgrading to avoid the ICE, which would just hide a miscompilation - leading potentially to other problems that we would need to trace down to this bug.

                                                                                      Incremental compilation also isn’t quite as crucial in Rust: it only happens on a per-crate level, so it’s not like you need to rebuild everything. It’s annoying, but this gives us time to write a proper fix.

                                                                                      I would expect a 1.52.2 soon once engineers had a good look at this issue and validated solutions.

                                                                                      1. 2

                                                                                        surely there are less extreme workarounds?

                                                                                        Add RUSTC_FORCE_INCREMENTAL=1 to your .profile and move on ? Hopefully you don’t ship anything critical this way. The new default just prevents any kind of miscompilations by giving you the safe route by default. That way nothing goes sideways at some possibly critical application due to a known bug. Which might do real world damage.

                                                                                      1. 47

                                                                                        I’m so tired of rehashing this. Pointing out that SemVer is not 100% infallible guarantee, or that major versions don’t always cause major breakage adds nothing new.

                                                                                        Lots of projects have a Changelog file where they document major changes, but nobody argues that reading changelogs would hurt you, because it may not contain all tiniest changes, or mention changes that would discourage people from upgrading, staying on insecure versions forever, etc.

                                                                                        SemVer is just a machine-readable version of documentation of breaking changes.

                                                                                        1. 23

                                                                                          Yes, and the article tries to succinctly sum up what value can be derived from that and what fallacies await. I’d have to lie to have ever seen it summed up thought that lens in one place.

                                                                                          I’m sorry it’s too derivative to your taste, but when the cryptography fire was raging, I was wishing for that article to exist so I can just paste it instead of extensive elaborations in the comments section.

                                                                                          1. 11

                                                                                            I thought the same thing initially, but it could also be coming from the perspective of using Rust frequently, which is strongly and statically typed. (I don’t actually know how frequently you use it; just an assumption.)

                                                                                            A static/strong type system gives programmers a nice boundary for enforcing SemVer. You mostly just have to look at function signatures and make sure your project still builds. That’s the basic promise of the type system. If it builds, you’re likely using it as intended.

                                                                                            As the author said, with something like Python, the boundary is more fuzzy. Imagine you write a function in python intended to work on lists, and somebody passes in a numpy array. There’s a good chance it will work. Until one day you decide to add a little extra functionality that still works on lists, but unintentionally (and silently) breaks the function working with arrays.

                                                                                            That’s a super normal Python problem to have. And it would break SemVer. And it probably happens all the time (though I don’t know this).

                                                                                            So maybe for weakly/dynamically typed languages, SemVer could do more harm than good if it really is unintentionally broken frequently.

                                                                                            1. 8

                                                                                              That’s all very true!

                                                                                              Additionally what I’m trying to convey (not very successfully it seems) is that the reliance on that property is bad – even in Rust! Because any release can break your code even just by introducing a bug – no matter what the version number says. Thus you have to treat all versions as breaking. Given the discussions around pyca/cryptography this is clearly not common knowledge.

                                                                                              The fact that this is much more common in dynamic languages as you’ve outlined is just the topping.

                                                                                              I really don’t know what I’ve done wrong to warrant that OP comment + upvotes except probably hitting some sore point/over-satiation with these topics in the cryptography fallout. That’s a bummer but I guess nothing I can do about it. 🧘

                                                                                              1. 7

                                                                                                Car analogy time: You should treat cars as dangerous all the time. You can’t rely on seatbelts and airbags to save you. Should cars get rid of seatbelts?

                                                                                                The fact that SemVer isn’t 100% right all the time is not a reason for switching to YOLO versioning.

                                                                                                1. 3

                                                                                                  Except that SemVer is not a seatbelt, but – as I try to explain in the post – a sign saying “drive carefully”. It’s a valuable thing to be told, but you still have to take further measures to ensure safety and plan for the case when there’s a sign saying “drive recklessly”. That’s all that post is saying and nothing more.

                                                                                                  1. 2

                                                                                                    Seatbelts reduce the chance of death. Reading a changelog reduces the chance of a bad patch. Trusting semver does not reduce the chance of an incompatible break.

                                                                                                    1. 6

                                                                                                      I really don’t get why there’s so much resistance to documenting known-breaking changes.

                                                                                                      1. 3

                                                                                                        I really don’t get why there’s so much resistance to documenting known-breaking changes.

                                                                                                        I mean you could just…like…read the article instead of guessing what’s inside. Since the beginning you’ve been pretending the article’s saying what it absolutely isn’t. Killing one straw man after another, causing people to skip reading because they think it’s another screech of same-old.

                                                                                                        I’m trying really hard to not attribute any bad faith to it but it’s getting increasingly harder and harder so I’m giving up.

                                                                                                        Don’t bother responding, I’m done with you. Have a good life.

                                                                                                        1. -1

                                                                                                          mean you could just…like…read the article instead

                                                                                                          So where in that article do you say why people don’t want to document known breaking changes ?

                                                                                                          Offtopic: That was really hard to read. Too many fat prints and

                                                                                                          quotes

                                                                                                          with some links in between. It just destroyed my reading flow.

                                                                                                          I also think the title “will not save you” is obviously telling everything about why people are just not reading it. It’s already starting with a big “it doesn’t work”, so why should I expect it to be in favor of it ?

                                                                                                          1. 4

                                                                                                            So where in that article do you say why people don’t want to document known breaking changes ?

                                                                                                            Well, the pyca/cryptography team documented that they were rewriting in Rust far in advance of actually shipping it, and initially shipped it as optional. People who relied on the package, including distro package maintainers, just flat-out ignored it right up until it broke their builds because they weren’t set up to handle the Rust part.

                                                                                                            So there’s no need for anyone else to cover that with respect to the cryptography fight. The change was documented and communicated, and the people who later decided to throw a fit over it were just flat-out not paying attention.

                                                                                                            And nothing in SemVer would require incrementing major for the Rust rewrite, because it didn’t change public API of the module. Which the article does point out:

                                                                                                            Funny enough, a change in the build system that doesn’t affect the public interface wouldn’t warrant a major bump in SemVer – particularly if it breaks platforms that were never supported by the authors – but let’s leave that aside.

                                                                                                            Hopefully the above, which contains three paragraphs written by me, and only two short quotes, was not too awful on you to read.

                                                                                                            1. 1

                                                                                                              Thanks, your summary is making a good point, and yes the original blogpost was hard to read, I did not intend this to be a troll.

                                                                                                              And nothing in SemVer would require incrementing major for the Rust rewrite

                                                                                                              Technically yes, practically I know that many rust crates do not increment the minimum required rust compiler version until a major version. So fair enough, semver in its core isn’t enough.

                                                                                                  2. 3

                                                                                                    AFAIU, I think the OP comment may be trying to say that they agree with and in fact embrace the following sentence from your article:

                                                                                                    Because that’s all SemVer is: a TL;DR of the changelog.

                                                                                                    In particular, as far as I can remember, trying to find and browse a changelog was basically the only sensible thing one could do when trying to upgrade a dependency before SemVer became popular (plus keeps fingers crossed and run the tests). With the main time waster being trying to even locate and make sense of the changelog, with basically every project showing it elsewhere, if at all. (Actually, I seem to remember that finding any kind of changelog was already a big baseline plus mark for a project’s impression of quality). As such, having a hugely popular semi-standard convention for a tl;dr of the changelog is something I believe many people do find super valuable. They know enough to never fully trust it, similarly as they’d know to never fully trust a changelog. Having enough experience with changelogs and/or SemVer, they however do now see substantial value in SemVer as a huge time saver, esp. compared to what they had to do before.

                                                                                                    Interestingly, there’s a bot called “dependabot” on GitHub. I’ve seen it used b a team, and what it does is track version changes in dependencies, and generate a summary changelog of commits since last version. Which seems to more or less support what I wrote above IMO.

                                                                                                    (Please note that personally I still found your article super interesting, and nicely naming some phenomena that I only vaguely felt before. Including the one I expressed in this post.)

                                                                                                    1. 2

                                                                                                      I think there is something a bit wrong about the blanket statement that others shouldn’t rely on semver. I suspect that for many projects, trying one’s best to use the API as envisioned by the author, and relying on semver, will in practice provide you with bugfixes and performance improvements for free, while never causing any major problems.

                                                                                                      I like the parts of this blog post that are pointing out the problems here, but I think it goes way too far in saying that I “need to” follow your prescribed steps. Some of my projects are done for my own enjoyment and offered for free, and it really rubs me the wrong way when anyone tells me how I “should” do them.

                                                                                                      [edited to add: I didn’t upvote the top level comment, but I did feel frustrated by reading your post]

                                                                                                      1. 1

                                                                                                        I’m not sure to respond to that. The premise of the article it that people are making demands, claiming it will have a certain effect. My clearly stated goal is to dissect those claims, so people stop making those demands. Your use case is obviously very different so I have no interest to tell you to do anything. Why am I frustrating you and how could I have avoided it?

                                                                                                        1. 3

                                                                                                          My negative reaction was mostly to the section “Taking Responsibility”, which felt to me like it veered a bit into moralizing (especially the sentence “In practice that means that you need to be pro-active, regardless of the version schemes of your dependencies:”). On rereading it more carefully/charitably, I don’t think you intended to say that everyone must do it this way regardless of the tradeoffs, but that is how I read it the first time through.

                                                                                                    2. 9

                                                                                                      Type systems simply don’t do this. Here’s a list of examples where Haskell’s type system fails and I’m sure that you can produce a similar list for Rust.

                                                                                                      By using words like “likely” and “mostly”, you are sketching a sort of pragmatic argument, where type systems work well enough to substitute for informal measures, like semantic versioning, that we might rely on the type system entirely. However, type systems are formal objects and cannot admit such fuzzy properties as “it mostly works” without clarification. Further, we usually expect type-checking algorithms to not be heuristics; we expect them to always work, and for any caveats to be enumerated as explicit preconditions.

                                                                                                      1. 2

                                                                                                        Also there were crate releases where a breaking change wasn’t catched because no tests verified that FooBar stayed Sync/Send.

                                                                                                        1. 1

                                                                                                          All I meant is that languages with strong type systems make it easier to correctly enforce semver than languages without them. It’s all a matter of degree. I’m not saying that languages like Rust and Haskell can guarantee semver correctness.

                                                                                                          But the type system does make it easier to stay compliant because the public API of a library falls under the consideration of semver, and a large part of a public API is the types it can accept and the type it returns.

                                                                                                          I’m definitely not claiming that type systems prevent all bugs and that we can “rely entirely on the type system”. I’m also not claiming that type systems can even guarantee that we’re using a public API as intended.

                                                                                                          But they can at least make sure we’re passing the right types, which is a major source of bugs in dynamically typed languages. And those bugs are a prominent example of why OP argues that SemVer doesn’t work—accidental changes in the public API due to accepting subtly different types.