1. 16

    The article is written by Thomas Bushnell who worked on hurd and worked closely with RMS. What I’m glad this point so clearly put, was that the issue with RMS was not just about that particular email chain. It brought the issue into a larger picture. It takes someone who knows the guy to see that since most people don’t have that kind of experience over such a long period of time.

    1. 9

      It takes someone who knows the guy to see that

      I mean, Selam G. did a pretty comprehensive post with references and quotes from women harassed by RMS over the years without ever knowing RMS. I’d argue that the women in question didn’t know RMS either (which is part of why he should not have been propositioning them).

      When someone is being abusive we usually just need to listen to the victims and observe reality.

      1. 13

        I find it very hard to trust in information provided alongside clear and obvious lies about a quote, and I am suspicious of the author’s motives for misrepresenting such a quote.

        1. 4

          Did you read the article ? What part wad misrepresented ?

          1. 3

            One example out of a few from the peice, but an example with the most relevancy, a misquotation that was used by the medias (social and mainstream) to coerce Stallman out of his role.

            What Stallman wrote:

            We can imagine many scenarios, but the most plausible scenario is that she presented herself to him as entirely willing.

            What Selam G. wrote in her piece:

            …and then he says that an enslaved child could, somehow, be “entirely willing”.

            This is, undoubtedly, intentionally malicious.

            1. 1

              The author could also just have poor reading comprehension, I guess.

              1.  

                Why should we listen to an author with such poor reading comprehension? Both situations make her lose a lot of credibility.

        2. 4

          Yes! listening to the victims is the most important part. I like Selam G.’s followup post which has quotes from victims.

          1. 4
            Today, I found out that Stallman had issued a rather weak and confusing apology to the CSAIL mailing list:
            
                “I want to respond to the misleading media coverage of messages I
                posted about Marvin Minsky’s association with Jeffrey Epstein. The
                coverage totally mischaracterised my statements.
            
                Headlines say that I defended Epstein. Nothing could be further from
                the truth. I’ve called him a “serial rapist”, and said he deserved to
                be imprisoned. But many people now believe I defended him — and
                other inaccurate claims — and feel a real hurt because of what they
                believe I said.
            
                I’m sorry for that hurt. I wish I could have prevented the
                misunderstanding.”
            

            I’m not sure what is weak or confusing about that apology.

            1. 7

              I think it’s because even while apologising he manages to say (paraphrasing) “I’m hurt by what you said I said” and “you didn’t understand me”. In fact, now I read it again he doesn’t actually apologise for anything except for being hurt.

              1. 5

                From what I read:

                He defended Marvin Minsky as potentially not a rapist because he might not have known the truth, but he said Epstein definitely was a rapist. He is sorry he couldn’t make that more clear.

                RMS said from Marvin Minsky’s perspective it was possibly a rich guys a prostitute harem of 18 year olds. Still gross, but to RMS the injustice was calling him a willing rapist, not a creep, with total certainty.

            2. 4

              The quotes in the article you linked seem to show him as a lonely sad guy who sleeps in his office. Could you elaborate on the assaults he did?

              To me when he allegedly said he would kill himself if he didn’t get a date, that is a sign of someone who is very lonely and needs help, not necessarily a predator. Sure it should be addressed somehow, but the author of your articles has very little empathy for other people ‘not like her’. It just sounds like a bad situation for everyone.

        1. 45

          RustTL;DR: It’s a very impressive language with a clear vision and priorities, but the user interface needs a lot of work. Also, the collection library is much better than Scala’s.

          • Generics with <>. It’s 2017 by now, we know it’s a bad idea. One of the reasons why the language suffers from abominations like the “turbofish” operator ::<>.

          • Strings don’t offer indexing, because it doesn’t make sense for UTF-8. Correct! But Strings offer slicing … WAT?

          • Misuse of [] for indexed access. Having both () and [] doing roughly the same thing, especially since [] can be used to do arbitrary things, doesn’t make sense. Pick one, use the other for generics.

          • Inconsistent naming. str and String, Path and PathBuf etc.

          • :: vs. . is kind of unnecessary.

          • Mandatory semicola, but with some exceptions in arbitrary places: struct Foo; vs. struct Foo {}

          • Arbitrary abbreviations all over the place. It’s 2017, your computer won’t run out of memory just because your compiler’s symbol table stores Buffer instead of Buf.

          • Can someone decide on a casing rule for types, please, instead of mixing lowercase and uppercase names? Some types being “primitive” is an incredibly poor excuse.

          • Also, having both CamelCase and methods_with_underscores?

          • Library stutter: std::option::Option, std::result::Result, std::default::Default

          • iter(), iter_mut(), into_iter() … decide prefix or postfix style and stick with it.

          • Coercions do too many things. For instance, they are the default way to convert i32 to i64, instead of just using methods.

          • Also, converting numbers is still broken. For instance, f32 to i32 might result in either an undefined value or undefined behavior. (Forgotten which one it is.)

          • Bitcasting integers to floats is unsafe, because the bits could be a signaling NaN, causing the CPU to raise an FP exception if not disabled.

          • Forward and backward annotations: #[foo] struct Foo {} vs struct Foo { #![foo] }. Also /// for normal documentation, //! for module level documentation. Documentation already uses Markdown, so maybe just let people drop a markdown file in the module dir? That would make documentation much more accessible when browsing through GitHub repositories.

          • Also, documentation can cause compiler errors … that’s especially fun if you just commented a piece of code for testing/prototyping.

          • Type alias misuse: In e.g. io crate: type Result<T> = Result<T, io::Error> … just call it IoResult.

          • Macros are not very good. They are over-used due to the fact that Rust lacks varargs and abused due to the fact that they require special syntax at call-site (some_macro!()).

          • Pattern matching in macros is also weird. x binds some match to a name in “normal” pattern matching, but matches on a literal “x” in “macro pattern matching”.

          • println! and format! are very disappointing given that they use macros.

          • Compiler errors … ugh. So many things. Pet peeve: “Compilation failed due to 2 errors” … 87 compiler errors printed before that.

          1. 7
            • Library stutter: std::option::Option, std::result::Result, std::default::Default
            • Type alias misuse: In e.g. io crate: type Result<T> = Result<T, io::Error> … just call it IoResult.

            How ya gonna square that circle?

            1. 2

              I think std::io::IoResult would be fine – it would solve the issue of having vastly different Results flying around, while not having single-use namespaces that are only used by one type.

              1. 1

                The general pattern is to import Io instead. When doing this, IoResult would be jarring.

                use std::io;
                
                fn my_fun() -> io::Result<T> {
                
                }
                
            2. 13

              It’s 2017,

              I have some news for you, @soc.

              1. 3

                Haha, good catch. Now you see how old this list is. :-)

                The only thing I got to delete since then was “get rid of extern crate”.

              2. 3

                What’s your preferred alternative to generics with <>?

                1. 6

                  [], as it was in Rust before it was changed for “familiarity”.

                  Unlike <>, [] has a track of not being horribly broken in every language that tried to use it.

                  1. 5

                    How is <> broken?

                    1. 16

                      It complicates parsing due to shift and comparison operators.

                      1. 2

                        Ah, yeah, that makes sense.

                      2. 19

                        Pretty much no language has ever managed to parse <> without making the language worse. The flaws are inherent in its design, as a compiler author you can only pick where you place the badness; either:

                        • Add additional syntax to disambiguate (like ::<> vs. <> in Rust).
                        • Have weird syntax to disambiguate (like instance.<Foo>method(arg1, arg2) in Java).
                        • Read a potentially unlimited amount of tokens during parsing, then go back and fix the parse tree (like in C#).
                        • etc.

                        In comparison, here are the issues with using [] for generics:

                        • None.

                        For newly created languages (unlike C++, which had to shoehorn templates/generics into the existing C syntax) it’s a completely unnecessary, self-inflicted wound to use <> for generics.

                        More words here: Why is [] better than <> for generic types?

                        1. 2

                          Those are good reasons to not use <>, but as a Haskeller I personally find either style somewhat noisy. I’d rather just write something like Option Int. Parentheses can be used for grouping if needed, just like with ordinary expressions.

                          1. 2

                            Haskell feels like it is in the same category as D, they both just kicked the can a tiny bit further down the road:

                            Both need (), except for a limited special-case.

                            1. -1

                              I don’t see how Haskell kicked the can down the road. The unit type is useful in any language. Rust has a unit type () just like Haskell. Scala has it too.

                              I’m not sure what “special case” you are referring to.

                              1. 2

                                The fact that you still need () for grouping types in generics as soon as you leave the realm of toy examples – just as it is in D.

                                (Not sure what’s the comment on the unit type is about…)

                                1. 4

                                  Ah, I understand what you’re saying now. But that’s already true for expressions at the value level in most programming languages, so personally I find it cleaner to use the same grouping mechanism for types (which are also a form of expressions). This is especially applicable in dependently typed languages where terms and types are actually part of the same language and can be freely mixed.

                                  However, I can also appreciate your argument for languages with a clear syntactic distinction between value expressions and type expressions.

                    2. 1

                      D’s use of !() works pretty well. It emphasizes that compile-time parameters aren’t all that crazy different than ordinary runtime parameters.

                      1. 1

                        I prefer my type arguments to be cleanly separated from value arguments (languages that fuse them excepted).

                        I find D’s approach slightly ugly, especially the special-cases added to it.

                        1. 1

                          I prefer my type arguments to be cleanly separated from value arguments

                          Well, in D they aren’t type vs value arguments, since you can pass values (and symbol aliases) as compile-time arguments as well. That’s part of why I like it using such similar syntax, since it isn’t as restricted as typical type generics.

                          I find D’s approach slightly ugly, especially the special-cases added to it.

                          The one special case is you can exclude the parenthesis for a single-token CT argument list and actually I thought I’d hate it when it was first proposed and I voted against it… but now that it is there and I used it, I actually like it a lot.

                          Sure does lead to a lot first timer questions on the help forums though… it certainly isn’t like any other language I know of.

                    3. 2

                      Also, converting numbers is still broken. For instance, f32 to i32 might result in either an undefined value or undefined behavior. (Forgotten which one it is.)

                      Yeah, I kind of feel the same way. Even with try_from() dealing with number conversions is a pain in Rust.

                      1. 1

                        You saved me a lot of typing. 100% agree.

                        1. 1

                          Thanks! I’d love to know the reason why someone else voted it down as “troll” – not because I’m salty, but because I’m genuinely interested.

                        2. 1

                          2 pains I have with Rust right now:

                          • I would like to be able to connect to a database (Teradata specifically)
                          • I want to launch a subprocess with other than the default 3 stdio descriptors (e.g. exec $CMD $FD<>$PIPE in sh)
                          1. 2

                            I know it’s technically unsafe and that might preclude it from your use, but does CommandExt::pre_exec not fit your bill?

                            1. 1

                              That could work. I’m still new to Rust so I haven’t fully explored the stdlib.

                          2. 1

                            Ahahaha, this is a great list. I’m curious about a couple things though, since you’ve obviously put a lot of thought into it…

                            Bitcasting integers to floats is unsafe, because the bits could be a signaling NaN, causing the CPU to raise an FP exception if not disabled.

                            The docs for f32::from_bits() and such talk about precisely this, but I considering the misdesign of signaling NaN’s really don’t see how it could possibly be made better. Any ideas?

                            …They are over-used due to the fact that Rust lacks varargs…

                            What little experience I have with programming language design makes me feel like varargs are a hard problem to deal with in a type-safe language, at least if you want to allow different types for the args (instead of, say, forcing them all to be what Rust would call &dyn Display or something). Do you know of any language which does it Right?

                            1. 1

                              The docs for f32::from_bits() and such talk about precisely this, but I considering the misdesign of signaling NaN’s really don’t see how it could possibly be made better. Any ideas?

                              Rust could have disabled the trapping of signaling NaN’s on start up, but I think Rust fell into the same design mistake of C:

                              Scared of making the use-case of the 0.01% (people who want signaling NaN’s to trap) harder to achieve, they made life worse for the 99.99%.

                              varargs are a hard problem to deal …

                              Agreed, it’s safe to say that language designers hate them. :-)

                              … at least if you want to allow different types for the args

                              I think this is only partially the reason. You can still have only same-typed varargs at runtime, but allow recovering the individual types of the arguments in macro calls – which is exactly the case for format! and friends.

                              Do you know of any language which does it Right?

                              I think in the case of format strings, focusing on varargs is the wrong approach. If you imagine how you want an ideal API to look like, you probably want to interpolate things directly inside the string, never having to go through the indirection of some vararg method.

                              Instead of having the formatting parameters in one place, and the to-be-interpolated values in a different one, like in …

                              let carl = "Carl"
                              let num = 1.234567;
                              format!("{}'s number is {:.*}, rounded a bit", carl, 2, num)
                              // -> "Carl's num is 1.23, rounded a bit"
                              

                              … wouldn’t it be much nicer to write (this is Scala):

                              val carl = "Carl"
                              val num = 1.234567
                              f"$carl's num is $num%.2f, rounded a bit"
                              // -> "Carl's num is 1.23, rounded a bit"
                              
                              1. 1

                                Julia has nice string interpolation too. I honestly don’t understand why more programming languages don’t have it. Does everyone just forget how useful it is in bash when they come to design their language?

                          1. 59

                            With Python I dislike everything related to packaging.

                            1. 11

                              One of my buddies at work was absolutely roasting pip for weeks. He ran across poetry and was like “why isn’t this a first-class citizen like pip is?”

                              1.  

                                I’ve had a really good experience with pex - a single executable like a JAR. Worked everytime, except with some fancy python c extensions, but worth a try

                                1. 1

                                  I’ll try that next time! I used pipenv last time I had to work on a python project but didn’t love it.

                                2. 4

                                  +1. On the up side, I think the community knows this is a giant pile of brown stinky stuff, and various solutions have been proposed, but the core team hasn’t had the resources to address this in a whollistic way as of yet.

                                  There are several levels to this as well - there’s the micro - creating and publishing a package sucks, and the macro - the nature of this suck means that Python programs are unnecessarily difficult to install and deploy in the wild. Again, see above about various community solutions.

                                  1. 1

                                    You might find this project (it’s like Rust’s cargo but for Python) interesting: https://github.com/David-OConnor/pyflow

                                  1. 2

                                    Python:

                                    • module system, I hate that it is tied to the file that contains the code. I would like to declare the namespace of a file at the top and distributes classes/functions within the same namespace across files.
                                    • Side-effectful laziness that was made the default on python 3.

                                    Functional programming, Love it in principle and toy examples. So elegant

                                    • Option/either types for error handling are not really better than exceptions. Sure you kind of get reminded by the compiler about error paths. But: lots of results types are quickly turned into option types that don’t contain valuable information for debugging anymore and I found myself wishing for exceptions again.
                                    • I miss classes, especially for parametrizing behaviour.
                                    1. 1

                                      Maybe you know, but Python modules can exist in different files if they are all in the same directory. It is annoyingly complicated to work out, though.

                                      1. 1

                                        I guess you are talking about the pattern where in the __init__.py you import from all those other files so classes can be imported from the same module qualifier.

                                        The problem with this is that the classes / functions still report / identify with the module path they were defined in. So if you define a class in mymodule/_foo.py and use imports it as from mymodule import Foo, then it is till identified as mymodule._foo.Foo.

                                    1. 10

                                      Shame that Huawei is government spyware; this seems really pretty and a good option for people.

                                      1. 12

                                        A hardware teardown would be interesting. Note that many people (even security aware ones) are still using laptops from another Chinese vendor Lenovo. I would not know what would make Huawei such a different case … though that argument can be taken two ways. Should we trust Huawei more, or Lenovo less?

                                        1. 7

                                          Should we trust Huawei more, or Lenovo less?

                                          I’ve long since lost count of the number of times Lenovo’s been caught distributing spyware or firmware backdoors (I think I tuned out after the third instance).

                                          You shouldn’t trust them less only because you shouldn’t trust them at all.

                                          1. 2

                                            Lenovo’s known spyware has all been at the OS level, right? Nothing that would survive a fresh reinstall?

                                            1. 7

                                              I believe there was a BIOS level one where chkdsk.exe was replaced from a copy in ROM. Dependent on Windows but still scary as hell.

                                              Edit: https://www.theregister.co.uk/2015/08/12/lenovo_firmware_nasty/

                                              1. 3

                                                I believe at least one was in the BIOS vendor region that Windows automatically reinstalls (naively expecting the mechanism to be used to provide critical hardware drivers and not, yknow, spyware), and thus persisted across “clean” reinstalls.

                                                Linux doesn’t voluntarily install unknown shit from ROM, but it gives me zero faith in Lenovo’s EFI to not be backdoored

                                                1. 1

                                                  I think this mechanism was intended for “anti-theft” software. Windows gets drivers from Windows Update, all critical drivers are built-in anyway.

                                            2. 9

                                              I don’t think that any mass product thing is not spyware. World changed, everything is spying on you: hardware, software, sites or whatever else that has internet access. There is no way to escape this shit. You can just try some handmade notebooks like MNT Reform or Purism Librem 5 smartphone, use trusted Linux, TOR over VPN and of course refuse to use services from big companies like Google. But you still won’t get 100% guarantee that they didn’t track you by some suspicious fingerprint.

                                            3. 15

                                              Can we just write away everything Huawei makes as spyware? Should people assume that Intel processors and the Windows OS are all full of US-gov backdoors? I know there’s a bit more evidence against Huawei, but a blanket dismissal may not be the good approach.

                                              1. 8

                                                what is the evidence against huawei?

                                                1. 1

                                                  In all honesty, I hadn’t done all that much research before writing this comment. I suppose the ambient FUD about Huawei got to me.

                                                  The most I can find is here. There was an accusation of a hacking attempt in India. I don’t know of anything else that’s close to a smoking gun.

                                                  1. 1

                                                    i don’t blame you; it’s the impression that U.S. sources give.

                                                  2. 1

                                                    This video about supply chain security mentions some interesting supply chain attacks that could happen anywhere. Bunnie:studios also gives an interesting overview of the open source casualty of the trade war as companies are coerced to stop doing business with Huawei.

                                                    Yet here in the UK we have the Huawei Cyber Security Evaluation Centre (HCSEC) which is reported on annually (2015, 2016, 2019), so I’m not convinced Huawei is any worse than any other manufacturer…

                                                  3. 9

                                                    Intel gets most of its direction outside the government. Huawei is basically controlled by China. If the NSA made computers, I wouldn’t trust them either. Thankfully that’s the closest our system gets to complete authoritarian control.

                                                    1. 5

                                                      They don’t have to make computers, they have TAO which has a history of supply chain attacks

                                                      1. 8

                                                        where do you get the perception that huawei is controlled by the chinese government, but intel is not controlled by the u.s. government?

                                                        1. 3

                                                          I don’t think that “controlled” is the right word for intel, but rather incentivised to cooperate when they want backdoors or stuff like that.

                                                          1. 2

                                                            conversely, is there evidence that huawei is “controlled” rather than “incentivized”?

                                                            1. 5

                                                              I wish I had a go-to explainer for this but I suggest you google around regarding how close Chinese companies work with the Party. I find it intriguing that people think our own conceptions of private property and how corporations work can be transplanted to a Chinese context with little to no caution. Read about their economic reforms under Deng Xiaoping to decrease the inferential gap a bit

                                                              1. 1

                                                                but you do have to be careful and look at quantitative measures because you can expect american sources to give a skewed impression

                                                                1. 1

                                                                  What quantitative measures? I’m certainly not suggesting they read about the actual economics behind the reform (aside from a surface layer of motivating factors), but the social implications of what it even means to be a huge Chinese corporation.

                                                                  1. 1

                                                                    the technique i would use to guard against bias would be to see if any claim about china could also be made about the U.S. from an inverted perspective. for example, chinese companies share user data with their government, and U.S. companies share data with their government.

                                                                    this comparison could be refined by looking at how much data about the public is collected and shared, but obviously that data is not available. i admit that no available quantitative measures come to mind, so i suppose the upshot is that it’s hard to draw conclusions because so little is known.

                                                                    1. 1

                                                                      For the record, I do think that caution in the other direction is needed: I get annoyed when Americans decide to have their 10547th Reddit thread bashing the Chinese social credit system as if their own vast network of private aggregators wasn’t just as capable. However there are clear qualitative differences in how these two countries operate. China is actually, for real way more oppressive, and it’s not like the SCS shouldn’t be bashed.

                                                                      1. 1

                                                                        depends on the criteria you use. you could just as well say the U.S. is more oppressive because our incarceration rate is over 5 times that of china. or if you include the world population in those who you care about being oppressed.

                                                    2. 6

                                                      Considering how much computer hardware is manufactured in China, maybe it’s naive to think only finished end-user products have “government spyware” embedded in them.

                                                      1. 5

                                                        is there evidence or is this an a priori assumption that also applies to american companies?

                                                        1. 1

                                                          There is adequate evidence as I understand it.

                                                          American companies exist outside The Party. Chinese “companies” have less freedom.

                                                          We are spoiled in America by what’s closest to freedom the world has ever seen. We have free speech, for example, and I don’t know of other countries that don’t criminalize various speech for whatever reasons. We can’t imagine what government control looks like.

                                                          1. 3

                                                            We are spoiled in America by what’s closest to freedom the world has ever seen.

                                                            The freedom to enter crippling debt over medical care. The freedom from equality of opportunity. The freedom from justice if you are poor or brown. The freedom to vote for one of two capital-approved candidates in most elections. The freedom to have your vote count more in some states than others.

                                                            I’ll take my chances elsewhere.

                                                            1. 2

                                                              The freedom to enter crippling debt over medical care. The freedom from equality of opportunity. The freedom from justice if you are poor or brown. The freedom to vote for one of two capital-approved candidates in most elections. The freedom to have your vote count more in some states than others.

                                                              These things are Not Good. Compared with the Chinese government’s neo-Gulag though, there is no question at all that the US is indescribably more free.

                                                              1. 2

                                                                The quoted statement is not “The USA has more freedom than China”

                                                            2. 1

                                                              There is adequate evidence as I understand it.

                                                              can you say what the evidence is? or you just have a general understanding that there is some evidence somewhere?

                                                          2. 2

                                                            I guess we should completely forget that spyware thing and accept that all governments do that. Think Snowden.

                                                            On the other hand we should understand China follows a completely different paradigm than western societies. Confucianism, where the whole society is integrated from family to business and government. Yes, it’s autocracy vs. democracy as we know.

                                                            1. 6

                                                              Hot take: ALL existing societies are oligarchies. Literally all of them. Representative “democracy” is not actually that democratic.

                                                              1. 8

                                                                There’s also always moisture in the air. Literally always. “Rain” is not actually that wet.

                                                                1. 5

                                                                  Maybe, but at least representative democracy holds up free speech in many places. E.g. I can openly and loudle criticize the government or companies in a Western European country without repercussions. Try the same in China.

                                                                  1. 4

                                                                    So, they’re all the same. Your views might be controversial over there to authorities. Even your alias given they’re a surveillance state. Since all countries are the same, how did yours treat you in the re-education center? And what steps are you taking to see news media and send your files through your country’s national firewall controlling what you see and hear?

                                                                    1. 3

                                                                      I did not say they’re all the same in all aspects. I said they’re all controlled by the rich and powerful, by the owner class.

                                                                      Of course the US is more invested in the appearance of freedom. Western governments are smart enough to allow criticism that doesn’t have much impact. They use subtler, stealthier tactics (e.g. inserting feds into organizations and entrapping members) to disrupt activism, but they still do it.

                                                                      1. 2

                                                                        Now, I agree with all that. :)

                                                              1. 2

                                                                I think that’s a reasonable rational and I wonder if DoH is going to end up being OS supported at some point.

                                                                1. 4

                                                                  Absolutely not! Why the hell would you want to centralise something that was decentralised since before Al Gore invented the internet?

                                                                  1. 5

                                                                    What? How would providing a DoH at an OS level centralize anything more than providing dns over tcp?

                                                                    Edit: It occurs to me that perhaps you thought I meant dns over https (DoH) as is implemented by firefox, ie with cloudflare being the defacto resolver. What I meant was that I wonder if DoH might come to be provided as a an alternative to or super set of normal OS DNS support with some sort of resolver discovery.

                                                                    1. 2

                                                                      Maybe cnst is talking about CAs.

                                                                      1. 1

                                                                        DoH/DoT don’t inherently require CAs. The OS could offer an interface like “set IP address and expected certificate in resolv.conf”, for example. (but, IMO, concerns about CAs are silly. Everything in userspace WILL use CAs, why would an OS take a hard stance against CAs?)

                                                                  2. 2

                                                                    I’m still not convinced that we need DoH in the OS. What does DoH gives us that DoT doesn’t?

                                                                    1. -1

                                                                      What does DoH gives us that DoT doesn’t?

                                                                      Transport encryption.

                                                                      1. 3

                                                                        What does the T in dot stand for?

                                                                        1. 1

                                                                          TCP

                                                                          1. 6

                                                                            No, it’s TLS.

                                                                            1. 1

                                                                              Is it? My bad.

                                                                                1. 2

                                                                                  Conventional DNS is a UDP protocol ;)

                                                                                  1. 5

                                                                                    Primarily UDP, but TCP if the response it too large and EDNS is not supported; also for zone transfers.

                                                                    1. 1

                                                                      Interesting, but is the reason this works different from the reason ye olde manual loop unrolling works?

                                                                      1. 6

                                                                        Loop unrolling doesn’t always give you this benefit.

                                                                        This technique is about rewriting an algorithm to be branchless and to separate memory accesses that are dependent on each other with independent work (here by interleaving iterations). Doing both allows the processor to request lots of memory locations in parallel.

                                                                        Unrolling reduces the number of branches but doesn’t necessarily interleave independent memory accesses.

                                                                      1. 1

                                                                        Does anyone use SOLID? Are there any useful apps for it?

                                                                        1. 0

                                                                          I am disappointed in the Python Software Foundation for once again leaving PyPy out in the cold. It has been a pattern for over a decade that CPython’s developers shortsightedly ignore alternative Python implementations, and now they have taken the extraordinary step of forcefully deprecating the dialect of Python used by the PyPy team.

                                                                          I’ve reached out to the Python Software Foundation repeatedly through official and semi-official channels, but never gotten a good clarification on the matter. PSF’s continued endorsement of a pile of C over a pile of Python is hypocritical, and now their attempt to control an ecosystem that is beyond them is ignorant; this ridiculous situation should not ever have happened, but continues to happen because of poor communication.

                                                                          1. 24

                                                                            From the linked post

                                                                            […] in 2008, we announced that we would sunset Python 2 in 2015, and asked people to upgrade before then. Some did, but many did not. So, in 2014, we extended that sunset till 2020

                                                                            (my emphasis)

                                                                            So the PyPy team has had between 12 and 5 years to move to Python 3, depending on how seriously they took the deprecation message. Or am I missing some internal Python wrangling here?

                                                                            1. 2

                                                                              Yeah, there’s two things you’re missing. First, PyPy’s toolchain, RPython, requires a Python 2.7 interpreter in order to bootstrap. These days, PyPy is capable of providing that bootstrap interpreter itself. Rewriting RPython would require a massive engineering effort, and there is not funding for such a venture. The latest position of the PyPy/RPython team is in a 2016 blog post:

                                                                              PyPy’s own position is that PyPy will support Python 2.7 forever—the RPython language in which PyPy is written is a subset of 2.7, and we have no plan to upgrade that.

                                                                              Second, it is well-known community folklore that GvR doesn’t really like PyPy or Jython or any other alternative to the reference interpreter. He would like for Python to be unified under a single holistic banner, and alternative interpreters present a threat. I was able to recently get him to clarify his position on the GitHub issue for the linked post:

                                                                              An endorsement of PyPy as an alternative to porting to Python 3 will not be coming from the PSF – we wish PyPy the best of luck with their own Python 3 transition, but we don’t think they are a good alternative to dropping Python 2 support.

                                                                              They would like to exterminate Python 2, and by extension, snuff out any ecosystems which aren’t porting forward to the latest and greatest PEPs offered by the CPython 3.x branches.

                                                                              1. 10

                                                                                Sounds like you want a hard fork. Why not do that?

                                                                                1. 0

                                                                                  Python 3 already was a hard fork of Python 2. The two languages are clearly distinct; they are mutually incompatible in syntax and semantics. Why must the Python 3 folks now destroy the last of Python 2?

                                                                                  1. 5

                                                                                    They do not “destroy the last of Python 2”, they are just saying that they have no will in maintaining it any longer. Would you blame them?

                                                                                    1. 0

                                                                                      That’s quite the opposite of what’s actually happening; they are saying that they want alternative Python 2 implementations, which are still going strong and aren’t dying anytime soon, to also go away. I do blame them, yes.

                                                                                      1. 9

                                                                                        Speaking as someone who maintains Python packages, I understand their position.

                                                                                        If the message were “one of many Python 2 interpreters is going to be unmaintained, but that’s no big deal and you can still get a maintained one to keep running all your Python 2 code”, well, that would mean people coming to me, and other package maintainers, and asking to have third-party packages maintained with Python 2 compatibility for as long as there’s a supported interpreter.

                                                                                        And I have no interest in that. Some aspects of 2/3 compatibility are not that hard, sure, but others can be a bit complex. And I’d be permanently held back from using most new features of modern Python, because those features aren’t going to be backported into a supported Python 2 interpreter (as I understand it, PyPy would never have interest in this; their desire is for Python 2.7 to remain supported but feature-frozen).

                                                                                        If PyPy wants to internally maintain the interpreter they use to bootstrap, I don’t care one way or another. But if PyPy wants that to also turn into broad advertisement of a supported Python 2 interpreter for general use, I hope they’d consider the effect it will have on other people.

                                                                                        1. 1

                                                                                          Sure. As somebody who maintains Python 2 codebases, I have felt the constricting effects of more and more packages no longer offering Python 2 versions, forcing folks to commit to increasingly-outdated vendored code.

                                                                                          PyPy has broadly advertised a general-purpose Python interpreter for sixteen years. An example release announcement from last year advertises “almost a drop-in replacement for CPython”. It’s fun to watch the effect that this already existing system has on folks; this entire thread has been folks having effects because of PyPy. PyPy exists, and that disrupts the picture of the ecosystem painted by PSF.

                                                                                          1. 3

                                                                                            I’m aware of PyPy’s existence.

                                                                                            There are a few things here that your comments have equivocated, and that equivocation leads to difficulty.

                                                                                            1. The CPython interpreters, for various versions of the Python programming language
                                                                                            2. The end-user deliverable interpreters of the PyPy project, for various versions of the Python programming language
                                                                                            3. The interpreter used by the PyPy project to bootstrap RPython
                                                                                            4. The abstract concept of the Python programming language, as embodied in a prose specification

                                                                                            Items 1 and 4 are copyrighted by the Python Software Foundation, which also owns the trademark for the Python programming language. Items 2 and 3 are under the control of the holders of PyPy’s copyright (a long list of individuals and other entities).

                                                                                            PyPy could, if the project chose, sunset its end-user-deliverable Python 2 interpreter on 2020-01-01, refuse to accept bugs from end users who wish to continue running software that is compatible only with Python 2, and maintain a Python 2 interpreter solely for the purpose of bootstrapping RPython.

                                                                                            Or, PyPy can publicly maintain an interpreter for Python 2 past 2020-01-01, including accepting bug reports from end users who wish to continue running software that is compatible only with Python 2.

                                                                                            You seem to strongly prefer the latter option. The only way in which your stated list of enemies (Guido, the CPython core team, the PSF) could interfere with that is by asking that the interpreter not be named “Python” (since the PSF owns the trademark). But the interpreter is already not named “Python”, so that’s moot.

                                                                                            But by doing so, PyPy would introduce a maintenance burden, or at least pressure to take on a maintenance burden, for other projects which wish to drop Python 2 and make use of new features only available in Python 3, since PyPy’s continued offering and support of an end-user-deliverable Python 2 interpreter would extend the life of software that is only compatible with Python 2.

                                                                                            This is the point I was making above, as a way of reminding you that while you’re complaining about the impact of the CPython/PSF messaging on you, you’re neglecting the impact of your messaging on others.

                                                                                            Also, I do wonder whether PyPy is really prepared to take on the burden of maintaining a high-profile Python 2 interpreter over potentially very long time scales, given the resource constraints which already have prevented being able to move it to bootstrap from Python 3.

                                                                                            Finally:

                                                                                            I have felt the constricting effects of more and more packages no longer offering Python 2 versions, forcing folks to commit to increasingly-outdated vendored code.

                                                                                            The alternative is to force the maintainers to continue supporting software they don’t want to support. For my own projects, I’m supporting Python 2 until 2020-01-01, and then I’m done and will enjoy getting to finally start using features of Python 3. While I have some sympathy for people who are still immovably on Python 2, I don’t feel any obligation, nearly 12 years after Python 3 was released, to continue maintaining things for them for free.

                                                                                        2. -5

                                                                                          They will wave their totalitarian hand over the open source and free software they have released with the right for anyone to do with as they please and say “magically, we will destroy your ability to yourself do as you please with this software we don’t want anymore” and then we will all weep our crocodile tears as open source python has been cancelled.

                                                                                          Rest in peace, Python

                                                                                          Idk when - 2019


                                                                                          We will never forget, we will never forgive

                                                                                      2. 3

                                                                                        Because they no longer wish to volunteer their time and effort keeping your software running for you.

                                                                                        Want to keep python 2 alive? Step up and do it.

                                                                                        1. 1

                                                                                          I already contribute to PyPy and am hoping to switch nixpkgs to PyPy instead of CPython as the default Python 2 interpreter.

                                                                                          Why must PSF “sunset” our community?

                                                                                          1. 5

                                                                                            Because they don’t want to release updates for you. You just have to do it yourself now, as you had before.

                                                                                    2. 16

                                                                                      What do you propose they do then? Extend Python 2 support forever and let Python 2 slow down Python 3 development for all time?

                                                                                      What do you suggest they do about the real problem that it’s confusing for new programmers and users, and annoying for experienced programmers and users, that there are two Python dialects? Should Python programmers just forever have to decide between limiting themselves to writing code compatible with both Python 2 and Python 3, or writing code which a lot of users can’t run?

                                                                                      I get the feeling that you think they should’ve just made backwards-compatible improvements to Python 2 rather than breaking it with a major release of the language, and maybe that would’ve been better, but that’s not what they did. Therefore, I’m curious about what you would want them to do given the current situation, not what they should have done a decade ago.

                                                                                      1. 1

                                                                                        What they should do now, given the current situation, is admit that PyPy is a reasonable implementation of Python 2, and admit that CPython’s demise does not doom the rest of the Python 2 ecosystem.

                                                                                        New Python programmers already must contend with the fact that the PSF publishes two manuals, one for Python 2 and one for Python 3, which look extremely similar, overlap greatly, and come up simultaneously in search results.

                                                                                      2. 7

                                                                                        “First, PyPy’s toolchain, RPython, requires a Python 2.7 interpreter in order to bootstrap.”

                                                                                        That’s them choosing and forever staying on a specific dependency. We generally don’t force suppliers to support anyone doing that unless they’re customers paying for backward compatibility. Even then suppliers might not find it worth their time and effort.

                                                                                        Let’s say we address the dependency. Is it really that difficult for Python programmers to rewrite one Python program in the newer version of Python? That versus the effort to maintain and polish Python itself for just that dependency?

                                                                                        Seems more fair for the project that wants the dependency to be the one reworking it. Also fairer division of labor given other option is an externality.

                                                                                        1. 1

                                                                                          “Supplier” is a bad term. The PyPy team is the RPython team, and they supply their own Python 2 implementation.

                                                                                          Is it really that difficult for Python programmers to rewrite one Python program in the newer version of Python?

                                                                                          Yes. Additionally, you clearly don’t know either dialect of Python, and you clearly have not done such a port. I encourage you to do so! You will learn a lot. You will mostly learn that Python 2 to Python 3 is not automatic, not trivial, and can take months at a minimum. For RPython, I would say that it is infeasible and not worth attempting unless somebody is paying.

                                                                                          1. 5

                                                                                            ““Supplier” is a bad term. The PyPy team is the RPython team, and they supply their own Python 2 implementation.”

                                                                                            Your original claim was about the Python Foundation, which you said backed CPython, was sunsetting support for Python 2 after telling everyone they’d do it for a long time. They were putting all their effort into Python 3. You then said this was a problem since another team, PyPy team, had a dependency on Python 2, had no intention to change that due to one component, did their own implementation of Python 2, and would stay on it.

                                                                                            That means this is an externality for PyPy, costing Python Foundation, where some other group is expected to support a language and its associated ecosystem just because of their dependency choices and porting preferences. I don’t see anything wrong with the other group saying, “No, we and our foundation have moved on in what we think are better directions. Anyone wanting to support the legacy tech can feel free. It’s on them since it’s not benefiting us.” Just going by what facts you’ve given here.

                                                                                            “you clearly don’t know either dialect of Python”

                                                                                            I used one here and there a long time ago. Don’t know either currently. I’ve been reading about Python 2 maybe getting ditched for Python 3 in these forums for a long time. So, I started asking some questions.

                                                                                            “You will mostly learn that Python 2 to Python 3 is not automatic, not trivial, and can take months at a minimum. “

                                                                                            Likewise, supporting an entire language, its libraries, security fixes, etc so some other people don’t have to port code is probably manual, less trivial than one component, and will drag on for years to decades. I’ve seen lots of companies port Python to Go. A Python to different kind of Python port of one component by Python experts is probably easier than maintaining a whole implementation of Python and its libraries. Just a guess.

                                                                                            If PSF is doing it for PSF, then by all means maybe they should maintain a whole platform. It’s different if it’s a lot of work, an externality for someone else, and they refuse to do a smaller set of work on their end. Again, just going by what you’ve described. If the facts are different, then this analysis would be wrong.

                                                                                            1. 0

                                                                                              You are equating the support from PSF of Python 2 with the existence of any Python 2 interpreter. If, tomorrow, all CPython 2.x installations stopped working, PyPy users would nonetheless still be able to execute pip or another pure-Python package manager and download Python 2 packages from PSF-operated infrastructure. This is an example of the sort of non-CPython-specific support for the entire Python community under PSF’s control.

                                                                                              The PyPy team can, and will, and must, support their entire toolchain, including RPython, PyPy, and a copy of the Python parts of CPython’s standard library.

                                                                                              Please ensure that your facts include the corporate employers of various CPython core developers, as well as GvR’s unyielding shadowy influence. Many folks simply wanted Python 2 to become Python 3, a fantasy paid for with developer-hours.

                                                                                              1. 4

                                                                                                “The PyPy team can, and will, and must, support their entire toolchain, including RPython, PyPy, and a copy of the Python parts of CPython’s standard library.”

                                                                                                That’s on them is my point. If they want PSF support, they need to give them a reason that works to their liking. If not, ignore them with a marketing strategy and/or foundation of their own that supports their goals. That’s how these things work.

                                                                                                “Please ensure that your facts include the corporate employers of various CPython core developers, as well as GvR’s unyielding shadowy influence.”

                                                                                                Sure. If corporations and GvR are in control, then CPython development needs to continue to reflect what they want out of pragmatism or expect lots of failure. Anyone not liking the decisions of those corporations and GvR should fork Python totally to support their vision with their own resources with no hate or aggravation toward the others acting in their own self interests. Maybe even out-compete them with their own language extensions and/or higher-performance implementations [1] on top of better marketing strategy and funding outreach. The kinds of things developers and organizers did for older versions of Python to get them where they were.

                                                                                                Edit: [1] Adding that Nim and Julia are sort of doing that already even though I meant an improved version of Python. Nim has Python-like syntax with extra features and higher-performance during runtime. Its getting some Python users. Python is entrenched in scientific computing. Julia is grabbing some of those people with alternative design, focus on higher performance, and focus on easier integration with C libraries while still keeping Python ones. They’re both getting users with Julia getting a lot of them.

                                                                                                1. 1

                                                                                                  This isn’t about PSF support, although if we’re going to talk about PSF support, it’s notable that the total sum investment of the PSF in PyPy is $10,000. I can guess at the salaries of the typical CPython core developer, and they’re a little better than five figures. It was only a year or two ago that PSF even opened up their grant program to allowing PyPy, Jython, or other non-CPython-based projects to apply for grant money.

                                                                                                  This is about the words at the top of the original post. To remind:

                                                                                                  We are volunteers who make and take care of the Python programming language. We have decided that January 1, 2020, will be the day that we sunset Python 2.

                                                                                                  These words purport to speak for PyPy’s team and any other Python 2 team. These words claim that Python 2 will be removed from our hands next year. Can you grok this? Do you understand why it might be a bit of a problem that the group that sits on the pile of money and trademarks has unilaterally decided that entire branches of the family tree must be pruned?

                                                                                                  Not Nim, but indeed Julia, as well as my own Monte, Hy, Quill, Lever, and likely others, all came from the realization over half a decade ago that the PSF was disowning Python 2 and would only acknowledge Python 3 as the one true Python. If we’re not going to keep the name, why keep the semantics? We could improve.

                                                                                                  1. 2

                                                                                                    These words purport to speak for PyPy’s team and any other Python 2 team

                                                                                                    The original post was published on python.org, which has an SSL cert published by the Python Software Foundation. It’s published by the PSF and speaks for them.

                                                                                                    You are making weird semantic shifts between “Python the language”, “Python the implementation”, and “Python the trademark” all over the place.

                                                                                                    It’s clear you (and maybe the rest of PyPy) have no love for Guido van Rossum, and maybe not the PSF. But the fact is that they own the trademark to Python. Trademarks exist in part to prevent customers from confusing one product for another. That way, no-one will be confused when the code they get from the PSF (Python 3) won’t run on PyPy (Python 2).

                                                                                                    That said, I’m sure no-one would object to say a product called “PyPy - a version of the Python language and runtime based on Python version 2.7”.

                                                                                                    From what I’ve read in this thread, it seems that PyPy wants to have their cake and eat it - they want full and continued support from the PSF for Python 2, while at the same time being entirely separate from them and not have to follow the PSF in any other matter. I really don’t see why the PSF has to go along with this arrangement.

                                                                                        2. 5

                                                                                          Second, it is well-known community folklore that GvR doesn’t really like PyPy or Jython or any other alternative to the reference interpreter. He would like for Python to be unified under a single holistic banner, and alternative interpreters present a threat. I was able to recently get him to clarify his position on the GitHub issue for the linked post:

                                                                                          This is a sticky wicket, and it’s by no means unique to Python. Witness the difficulty non standard Ruby, Perl etc. implementations have had keeping up with the reference implementation.

                                                                                          In the end, a most language communities end up, as Guido/Python did, deciding to focus on one implementation and say “This is THE reference implementation of the language”.

                                                                                          Doing anything else requires a massive outlay of man hours that just don’t exist in communities like Python - you’d need something like the Jave TCK.

                                                                                          1. 7

                                                                                            He would like for Python to be unified under a single holistic banner, and alternative interpreters present a threat. I was able to recently get him to clarify his position on the GitHub issue for the linked post:

                                                                                            This is not a fair reading of what Guido actually said in that issue. He is talking about PyPy being an alternative to Python 2, and being disappointed they’re not implementing Python 3. He didn’t mention anything about Python being “unified under a single holistic banner” or PyPy being a “threat”

                                                                                            1. 1

                                                                                              It’s totally understandable that you dislike my opinion. This is a synthetic opinion of GvR’s desires that comes not just from this post, but from years of interactions with him, including a particularly memorable dinner from a PyCon in Montreal a few years ago.

                                                                                              PyPy is not just an alternative to Python 2; there is a Python 3 branch. Indeed, the main PyPy page offers 2.7.13, 3.5.3, and 3.6.x versions of PyPy. I can’t imagine that his disappointment is that PyPy doesn’t have Python 3 support, but more likely that PyPy exists at all and mars the otherwise-singular transition between languages.

                                                                                              PyPy has never, to my knowledge, cracked 5% of all Python usage. The only measurements I’ve seen have come up at 1-2%. It has always been marginalized due to the desires of CPython core developers. We sometimes like to imagine a world where Python interpreters were fast, but this is apparently not that world.

                                                                                              1. 6

                                                                                                Can you cite conversations online where CPython core developers are displaying this sentiment?

                                                                                                For me, peak-PyPy and excitement around alternative implementations happened in the 2008-2012 window. What I mean by this, is that everyone was excited about alternative implementations driving performance forward at a time when folks weren’t so worried about Python 3 compatibility, and Python itself was just getting into a growth curve. I’d cite Dave Beazley’s PyCon Keynote at PyCon 2012 as an example of how PyPy wasn’t sidelined to the degree you argue it is.

                                                                                                Fast forward a few years and folks are in the trough of disillusionment, to steal a phrase from tech analysts. There were lots of (invalid, but) broken expectations in the community. Unladen Swallow, Jython, IronPython, and PyPy (to name just a few) hadn’t taken the world by storm and we were still trying to remove the GIL without breaking C extension compatibility. There were a lot of good intentions about getting and staying compatible with the Python mainstream - but this didn’t happen, and the alternative implementations got behind. PyPy’s history of Python 3 funding has some relevant detail.

                                                                                                It’s great that PyPy has caught up, but there is some way to go as far as being a strategic choice for folks. Even with CPython’s pretty obvious deficiencies it’s been able to carry a community forward to a place that’s very different from 2008. PyPy can make the argument that it’s a viable and supported Python 2 that could even get better, but very few people want this, even if they have to support Python 2. Python developers for the most part want to move to Python 3 because it’s finally a compelling option. In my mind, it’d be better if PyPy adapted to the current environment and attempted to overtake CPython as the platform of choice for Python 3 instead of fighting old battles for a diminishing platform.

                                                                                                1. 1

                                                                                                  Sure, you’re part of one arm of the massive galaxy of Python. However, you should know that there is another arm, where:

                                                                                                  • C extensions are shunned; speed comes from PyPy
                                                                                                  • Conda? Ha, no, we use real package managers, like Nix
                                                                                                  • asyncore? asyncio? No, we use Twisted

                                                                                                  There is a clear difference between Unladen, Pyjion, Pyston, etc. vs PyPy: PyPy is a true community project, not a corporate timewaster. If that makes it less strategic, so be it; the clear difference is that PyPy ships, and has been shipping for years. In 2008, it was already shipping, already fast, and the PyPy position was that CPython folks should keep CPython as the reference interpreter, but encourage PyPy and Jython experimentation, deprecate the C API, and encourage platform-neutral portable Python. This position of ours is over a decade old.

                                                                                                2. 3

                                                                                                  “Indeed, the main PyPy page offers 2.7.13, 3.5.3, and 3.6.x versions of PyPy.”

                                                                                                  They’ve done a lot of work. They deserve more visibility. I’ve always mentioned them to people interested in Python implementations.

                                                                                                  “PyPy has never, to my knowledge, cracked 5% of all Python usage. The only measurements I’ve seen have come up at 1-2%. It has always been marginalized due to the desires of CPython core developers.”

                                                                                                  The desires of a project’s developers rarely determine what massive amounts of people in the real world do. Especially across many disciplines that we see Python in. That’s marketing, what default binaries are available in package managers, what people in charge prefer, what gets the most press, word of mouth pushing each person’s favored version, etc. These things drive language adoption on large scales.

                                                                                                  The core Python team has a good brand and product they got bundled in most Linux distro’s by default with almost anything people read about Python pointing them in that direction. That’s why most use something like that vs PyPy. The PyPy team would have to step up big time on all the non-technical things that help achieve market success. Especially working with major distro’s to bundle them instead of reference Python. I don’t know what they’ve done on any of that since I just occasionally looked at the project being impressed at the technical work they put in.

                                                                                                  1. 2

                                                                                                    I need to quote a PyPy developer. They contribute quite a bit of code, but crucially rely on funding, either from community awards or from paid consulting, to make their living. Their reaction upon reading GvR’s words earlier:

                                                                                                    it seems like they’re trying to actively hide the fact that there is and will be a supported python 2 version

                                                                                                    As for distro involvement, all of the big distros (Debian, Fedora, Gentoo, OpenSUSE) have a PyPy package of some sort. It’s second-class in many distros, because the core packages of the system are written specifically for CPython.

                                                                                                    Many many PyCon hallway discussions have been expended over this. I know that, as an outsider, it seems like this is a matter of marketing or some other intangible/bullshit matter of business, but inside the Python ecosystem, there has grown to be a vast divide between those of us focused on speed in Python and speed beside Python. We former have always been willing to accept being second-class to some extent, but here we are asked to abide a terrible distortion of reality where Python 2 and Python 3 are equivocated in order to fulfill the prophecy and consolidate power over both languages.

                                                                                                    1. 1

                                                                                                      I am enjoying your explanations, and lament your suffering, and if I had a clue, I might also have an opinion. But I just want to point out that I think when you grabbed the word “equivocated,” you were really reaching for “equated.” The way you have it reads as tho both the languages are being “made into mistakes.”

                                                                                                      1. 1

                                                                                                        If you meant it in the sense of “hide the truth,” then that should be something being done by people. “To equivocate” cannot take a direct object. CPython partisans may equivocate about Python versions, but the versions may not be equivocated by somebody (only truth in general can “be equivocated,” and to so state is tautological to the point of ungrammaticality) nor may they themselves equivocate (unless it’s anthropomorphism, which makes sense, e.g. Python 2 equivocates about strings being Unicode or bytes).

                                                                                              2. 2

                                                                                                Thanks for taking the time to reply. I can see from the language you’re using is that this is not just a technical matter, there are deep personal disagreements at play on both sides which makes this transition so hard.

                                                                                                I just have two followup questions related to the future of PyPy

                                                                                                1. would not the the new features of Python 3 make it easier to implement an alternative Python runtime to compete with CPython?
                                                                                                2. if not, it sounds as if the PyPy project is the perfect team to take over maintenance of Python 2, effectively forking the language and leaving a path forward for those to do not want to update to Python 3. The lack of funding would be alleviated by the support of those people or organizations who wish to continue to use Python 2.
                                                                                                1. 2

                                                                                                  To your first question: No; see for yourself. Some features are neutral, some features are harmful. When it comes to RPython, the language is already restricted compared to Python 2, because the whole program is subject to type inference. Python 3 doesn’t offer any features which support type inference. (Annotations don’t help type inference more than actual code!) Some features, like ordered dictionaries by default, were first developed in PyPy and then copied by CPython later.

                                                                                                  There is exactly one feature worth borrowing from Python 3, f-strings, but don’t bother. Python 3 gained them around the same time that ECMAScript gained template literals. Of course, in the original languages Scheme and E where the technique was pioneered, these templating strings can be used not just as builders of values, but also as patterns which destruct values. And in E, it is possible to customize how the value is built, by writing a custom builder. However, in Python 3, f-strings cannot be used as patterns, nor can custom builders be used. I asked somebody involved in drafting that PEP why the feature wasn’t lifted in toto, and they replied that nobody else in the room of CPython core developers had understood why it would be useful.

                                                                                                  To your second bullet point, yes, there is a possible world where PyPy jumps from less than 5% usage this year to over 50% next year for Python 2. I don’t know whether that world is going to happen, because I believe that the PSF will put in effort to prevent Python 2 ecosystems from flourishing without their permission.

                                                                                            2. 3

                                                                                              But funnily enough I expect that we’re likely to see tools to analyze codebases for safety to port to PyPy for this reason; and one-and-done solutions for adapting c-based packages to work with PyPy.

                                                                                              That or blog posts from the interested parties explaining why that’s so infeasible that they’re finding CPython 2.7 security.

                                                                                              1. 1

                                                                                                PyPy is a compliant implementation of Python. There are documented differences from CPython, but in general, you should be able to write pure Python and have it work with PyPy.

                                                                                                The main barrier to PyPy adoption is the continued existence of CPython’s C API.

                                                                                                1. 3

                                                                                                  More or less all scientific and data science users of python use libraries that depend on the CPython C API.

                                                                                                  For them, PyPy is not a useful alternative.

                                                                                                  Besides that, as they say in the announcement, many open source projects will stop supporting python 2. They’ll do that because it is work to support python 2 and they prefer to use or be compatible with 3.

                                                                                                  PyPy has reasonable support for python 3 and the py2.7 required for bootstrapping can be maintained or replaced - it’s not like it needs any bug or security fixes for this purpose.

                                                                                                  1. 1

                                                                                                    Right. Which is why you’d want to analyse your codebase for both use of c-based libraries and the weird dumb things you would do hit those documented differences.

                                                                                              1. 4

                                                                                                I’m waiting on an async prompt (not sure if this is or isn’t, but seeing as they’re not bragging about it I’m assuming it’s not). I like having git info (and such) in my shell, but I invariably get into some enormous repository and suddenly my prompt takes 10 seconds to display because it’s stuck running git status or similar.

                                                                                                Would dispatching a process but allowing it to keep printing to the console (via, I suppose, curses?) even work?

                                                                                                1. 4

                                                                                                  Async fork here: https://github.com/maximbaz/spaceship-prompt

                                                                                                  Not tried it myself but it sounds good. I’m using fish for the moment as it just seems to be speedy enough everywhere.

                                                                                                  1. 3

                                                                                                    For the time being, there’s no support for async prompts directly baked into starship. That said, it would be possible to write an async prompt wrapper for starship in zsh using zsh-async or in fish using fish-async-prompt, since prompt modules can be rendered individually with starship module <module_name>.

                                                                                                    We have plans to write async wrappers for shells, though it’s not high on our priority list right now. Contributions are welcome! :)

                                                                                                    1. 2

                                                                                                      zsh’s vcs-info library works pretty fast for me.

                                                                                                      I’ve only had to have workarounds on ssh mounts.

                                                                                                    1. 1

                                                                                                      A federated, confidential instant messaging service that’s not a huge PITA like XMPP would be nice.

                                                                                                      I think we can do better than Trust on First Use, though. E.g. keybase chat does better.

                                                                                                      A really simple algorithm to make identities persistent is to just encrypt a master key for each user on the client with a user-known passphrase and then store it on the server. And/or you can share keys between devices (matrix do both of these, keybase does the latter).

                                                                                                      Slide 36 talks about keeping metadata private. If they can’t mine the metadata for value, then what’s the business case for something like WhatsApp? (Honest question)

                                                                                                      1. 20

                                                                                                        Sad :-( I still think Mercurial far better meets the needs of most people, and that the chief reasons for git’s popularity are that Linus Torvalds wrote it, GitHub, and that Linus Torvalds wrote it.

                                                                                                        That said, I did end up switching from BitBucket/mercurial to GitHub/git a few years ago, simply because it’s the more pragmatical thing to do and I was tired of paying the “mercurial penalty” in missed patches and the like. I wrote a thing about it a few ago: https://arp242.net/git-hg.html

                                                                                                        1. 6

                                                                                                          Why do you think hg is better for most people? I honestly find it vastly more complex to use.

                                                                                                          1. 15

                                                                                                            The hg cli is light years ahead of git in terms of intuitiveness.

                                                                                                            1. 6

                                                                                                              I’d say it’s years behind ;)

                                                                                                              1. 10

                                                                                                                How long have you been using Mercurial? I find most people who dislike Mercurial’s UI, are mainly coming from years of experience with Git. I disliked Mercurial at first as well, but after a few years of forced usage it clicked. Now I appreciate how simple and well composed it is and get frustrated whenever I need to look up some arcane Git flag on StackOverflow.

                                                                                                                In general, I’d say you need several years experience with both Git and Mercurial before you can draw a fair comparison.

                                                                                                                1. 3

                                                                                                                  I used mercurial for about 2 years before using git.

                                                                                                                  1. 3

                                                                                                                    Sorry if my post came across a bit accusatory (not my intent). In that case I guess to each their own :).

                                                                                                                  2. 3

                                                                                                                    but after a few years of forced usage it clicked.

                                                                                                                    I’m pretty sure that git clicked for me in a much shorter timeframe.

                                                                                                                    1. 1

                                                                                                                      Me too, but I know many (otherwise perfectly competent engineers) 5-10 years in who still don’t get it and aren’t likely to.

                                                                                                                  3. 9

                                                                                                                    I’m going to strongly disagree. I’ve used git intensively and I find Mercurial to be a well-designed delight. I’ve run across features that Mercurial supports flawlessly, with a nice UI, and Git requires a hacky filter-branch that takes hours to run and doesn’t even behave correctly.

                                                                                                                    IMO, a lot of the badness in projects is down to Git badness. it doesn’t scale and people feel compelled to break things down into tiny sub-projects.

                                                                                                                    The only reason Git is winning anything is GitHub’s support of it.

                                                                                                                    1. 3

                                                                                                                      The only reason Git is winning anything is GitHub’s support of it.

                                                                                                                      Why then was github ever used in the first place? Kind of a strange proposition.

                                                                                                                      1. 1

                                                                                                                        Network effect of the social network is pretty important.

                                                                                                                        1. 1

                                                                                                                          Why would there ever be a network effect in the first place if git was so bad that github was the only reason to use it. I get that the argument technically holds but it seems very unlikely.

                                                                                                                2. 8

                                                                                                                  You find mercurial more complex to use than git? That’s an… unusual view, to say the least. The usual recitation of benefits goes something like this

                                                                                                                  • Orthogonal functionality in hg mostly has orthogonal commands (compare git commit, which does a half-dozen essentially unrelated different things).
                                                                                                                  • hg has a somewhat more uniform CLI (compare git branch -a, git remote -v, git stash list).
                                                                                                                  • hg either lacks or hides a bunch of purportedly-inessential and potentially confusing git functionality (off the top of my head, partial commits aren’t baked into the flow a la git’s index/staging area; and rebasing and history rewriting are hidden behind an extension).

                                                                                                                  I personally prefer git, but not because I think it’s easier or simpler; I’m more familiar with it, and I find many of those purportedly-inessential functions to be merely purportedly, not actually, inessential.

                                                                                                                  1. 5

                                                                                                                    One more thing I like about mercurial that the default set of commands is enough for >90% of people, and that everything else is “hidden” in extensions. This is a very different approach than git’s “kitchen-sink” approach, which gives people 170 commands (vs. Mercurial’s 50, most of which also have much fewer options/switches than git).

                                                                                                                    Git very much feels like “bloatware” compared to Mercurial.

                                                                                                                    1. 3

                                                                                                                      I used git for many years, and then mercurial (at FB) ever since we switched over. The cli interface for mercurial is definitely more sensible, crecord is delightful, and overall it was fine. But I was never able to build a mental model of how mercurial actually worked. git has a terrible interface, but it’s actually really simple underneath.

                                                                                                                      1. 1

                                                                                                                        I didn’t think that underneath they were different enough to matter much. What differences do you mean? I guess there’s git’s remote tracking stuff. Generally, it seems like they differ in how to refer to and track commits and topological branches, locally and remotely. (IMHO, neither has great mechanisms for all the things I want to do.) Mercurial is slightly more complex with the manifest, git is more complex with the staging area that feels absolutely critical until you don’t have it (by using hg), at which time you wonder why anyone bothers with it. I’m a heavier hg user than git user, but that’s about all I can come up with.

                                                                                                                      2. 2

                                                                                                                        You find mercurial more complex to use than git?

                                                                                                                        I actually found – in a professional shop – mercurial far more complex to use. Now, the fact is that mercurials core – vanilla hg is IMHO absolutely without doubt vastly superior to git. Git keeps trying to make the porcelain less painful (including a release just a bit ago) – but I still think it is ages behind.

                                                                                                                        The problem is – I never used vanilla mercurial in a professional environment. Not once. It was always mercurial++ (we used $X extension and $Y extension and do it like $Z) which meant even if I knew hg, I felt painfully inexperienced because I didn’t know mq, share, attic, collapse, evolve, and more… not to mention both the bigger shops I worked with using mercurial has completely custom workflow extensions. I suspect part of this was just the ease of writing mercurial extensions, and part of it was wanting to fall into a flow they knew (mq, collapse). But, regardless of how we got there, at each place I effectively felt like I had to relearn how to use the version control system entirely.

                                                                                                                        As opposed to git, wherein I can just drop in and work from day one. It might be less clean, it might be more finicky and enable things like history rewriting by default. But at the end of the day, the day I start, I know how to generally function.

                                                                                                                        I am curious how Mercurial would have faired if instead of shipping default extensions you had to turn on – if they had just baked a little more functionality, to try to cover the 80% of what most shops wanted (not needed, I think most could have gotten by with what vanilla mercurial had) – if the shop to shop transition would have been easier.

                                                                                                                        1. 2

                                                                                                                          mq, I think, is responsible for many of the “mercurial is too complicated” complaints people have. Evolve, if it ever stabilized and ships with core hg would really enable some killer capabilities. Sadly for social and technical reasons it’s perpetually in beta.

                                                                                                                        2. 1

                                                                                                                          whoa, no index? Admittedly I didnt really use index as intended for several years, but now its an important part of my workflow.

                                                                                                                          1. 1

                                                                                                                            In Mercurial, commits are so much easier to make and manipulate (split, fold, move), that you don’t miss the index. The index in git is just a limited special cased “commit”.

                                                                                                                            1. 3

                                                                                                                              The index in git is just a limited special cased “commit”.

                                                                                                                              I disagree.

                                                                                                                              The index is a useful way to say “these lines of code are ready to go”. If you are making a big commit, it can be helpful to add changes in logical blocks to the index as you go. Then the diff is not polluted with stuff you know is already fine to commit.

                                                                                                                              You might say, “why not just make those changes their own commits, instead of trying to do one big commit?” That’s a valid question if you are talking about a 200 line commit or similar, but sometimes the “big” commit is only 50 lines. Instead of making a bunch of one line or few line commits, its helpful to “git add” small chunks, then commit at the end.

                                                                                                                              1. 0

                                                                                                                                You can as well amend to a commit instead of adding to the index.

                                                                                                                                1. 3

                                                                                                                                  True, but all thats doing is bastardizing the commit process. If you are committing a one line change, just to rebase minutes or hours later, thats not a commit.

                                                                                                                                  Rebase to me is for commits that were intended to be commits, but later I decided it would be better to squash or change the history. The index is for changes that are never meant to be a full commit on their own.

                                                                                                                                  1. 1

                                                                                                                                    Having a distinction between draft and published phases in mercurial I think makes it easier to rewrite WIP work. There’s also a number of UI affordances for it. I don’t miss the index using mercurial. There’s also academic user interface research that shows the index is a big conceptual barrier for new users.

                                                                                                                                    1. 1

                                                                                                                                      There’s also academic user interface research that shows the index is a big conceptual barrier for new users.

                                                                                                                                      this isnt really a valid point in my opinion. some concepts are just difficult. if some goal can be achieved in a simpler way i am on board, but I am not a fan of removing useful features because they are hard to understand.

                                                                                                                                      1. 1

                                                                                                                                        But the point is the index is hard to understand and unnecessary.

                                                                                                                                        There’s no need to have a “commit process”. Just commit whatever you want and rewrite/amend it for as long as you want. As long as your commits are drafts, this is fine.

                                                                                                                                        Is the problem the word “commit”? Does it sound too much like commitment?

                                                                                                                                        There’s no need to have two separate ways to record changes, an index, and a commit, each with different degrees of commitments. This is multiplying entities beyond necessity.

                                                                                                                                        1. 1

                                                                                                                                          That’s your opinion. The index is quite useful to me. I’d rather make a proper commit once it’s ready, not hack together a bunch of one line commits after the fact.

                                                                                                                                          1. 2

                                                                                                                                            The index is a commit. Why have two separate ways of storing the same sort of thing?

                                                                                                                                            Also, it’s not my opinion that it’s hard to understand and unnecessary; it’s the result of usability studies:

                                                                                                                                            https://spderosso.github.io/oopsla16.pdf

                                                                                                                                            You’re also not “hacking together” anything after the fact. There’s no more hacking together after the fact whether you use git amend (hypothetically) or git add. Both of those mean, “record additional changes”.

                                                                                                                                            1. 0

                                                                                                                                              It seems you have a fundamental misunderstanding of the difference between add and commit. Commit requires a commit message.

                                                                                                                                              1. 1

                                                                                                                                                This isn’t a useful distinction. You can also create commits with empty commit messages in both git and Mercurial.

                                                                                                                                                With both git and Mercurial you can also amend commit messages after the fact. The index in git could well be implemented as a commit with an empty commit message that you keep amending and you wouldn’t notice the difference at all.

                                                                                                                                                1. 1

                                                                                                                                                  you keep amending and you wouldn’t notice the difference at all.

                                                                                                                                                  yeah, you would. again it seems that you either dont know git, or havent used it in some time. when you amend a commit, you are prompted to amend the message as well. another facet that doesnt exist with git add, because add doesnt involve a message.

                                                                                                                                                  if you wish to contort git internals to suit your agenda thats fine, but git add has perfectly valid use cases.

                                                                                                                                                  1. 0

                                                                                                                                                    you are prompted to amend the message as well.

                                                                                                                                                    This is UI clutter unrelated to the underlying concepts. You can get around that with wrappers and aliases. I spoke of a hypothetical git amend above that could be an alias that avoids prompting for a commit message.

                                                                                                                                                    Don’t git users like to say how the UI is incidental? That once you understand the data structures, everything else is easy? The UI seems to have locked you into believing the index is a fundamentally necessary concept, but it’s not. It’s an artifact of the UI.

                                                                                                                                                    1. 1

                                                                                                                                                      The UI seems to have locked you into believing the index is a fundamentally necessary concept, but it’s not.

                                                                                                                                                      Nothing has locked me into believing its a necessary concept. Its not necessary. In fact, for about 7 years I didnt use the index in any meaningful way.

                                                                                                                                                      I think what you are missing is that Im not compelled to use it because its the default workflow, I am compelled to use it because its useful. It helps me accomplish work more smoothly than I did previously, when I would just make a bunch of tiny commits because I didnt understand the point of the index, as you still dont.

                                                                                                                                                      The argument could be made to move the index into an option, like somehow make commit only the default workflow. Im not sure what that would look like with Git, but I dont think its a good idea. It would just encourage people to make a bunch of smaller commits with meaningless commit messages.

                                                                                                                                                2. 1

                                                                                                                                                  You have a set of things you want to accomplish. With git, you have N+1 concepts/features/tools to work with. With hg, you have N (because you drop the index). That means you have to expand your usage of the remaining N.

                                                                                                                                                  Specifically, since you no longer have this extra index concept, you now expand commits to cover the scenarios you need. Normally, you’d make an initial commit and then amend a piece at a time (probably with the interactive curses hunk selector, which is awesome.) If you’re unsure about some pieces, or you have multiple things going on that you’d like to end up in separate commits, you can always make a series of microcommits and then selectively collapse them later. (In practice, it’s even easier than this, because of the absorb extension. But never mind that.)

                                                                                                                                                  Yes, those microcommits need commit messages. They don’t need to be good ones, because they’re temporary until you squash them out of existence. I usually use a one word tag to specify which of the separate final commits they belong to. (If you don’t have separate final commits, you may as well amend, in which case no messages are needed.)

                                                                                                                                                  …or on the other hand, maybe mercurial ends up with N+1 concepts too, because phases really help in keeping things separate. As I understand it, one reason git users love the index is because it keeps rapidly changing, work in progress stuff separate from mostly set in stone commits. Phases perform the same purpose, but more flexibly, and the concepts are more orthogonal so they compose better. In my opinion.

                                                                                                                        3. 6

                                                                                                                          I never particularly liked git and find it unintuitive, too.

                                                                                                                          I wouldn’t consider myself a git poweruser. But whenever I had to work with alternatives I got the feeling that they’re just inferior versions of git. Yeah, maybe the usage was a bit more intuitive, but all of them seemed to lack things that I’d consider really basic (bisecting - hg has that, but e.g. svn has not - and shallow copying - not avaible in hg - are examples what I often miss).

                                                                                                                          1. 3

                                                                                                                            Mercurial was actually my first DVCS, and like you I ended up switching to git not out of a sense that it was technically better, just more pragmatic. For me, the change is more of a mixed bag, though. It is definitely the case that Mercurial’s UI is worlds better, and revsets in particular are an amazing feature that I sorely miss, but when I made the switch I found that the way git handles branches was much more intuitive to me than Mercurial’s branch/bookmark system, and that the affordances around selectively editing commit histories were very much worth the risk in terms of being able to manage the narrative of a project’s history in a way that makes it more legible to others. Ultimately, I found that git’s advantages outweighed its downsides for my use case, since learning its UI idiosyncrasies was a one-time cost and since managing branches is a much more common occurrence for me than using revsets. That said, I think this is a really unfortunate development.

                                                                                                                            1. 2

                                                                                                                              I occasionally convert people’s git repos to hg for my use. Stubborn like that.

                                                                                                                            1. 1

                                                                                                                              I’m squarely in that group that simply avoids using Ncurses. I find Ncurses to be a baroque API, from what I know, and as with many UNIX APIs it seems so often I learn of some new quality and can’t tell if it’s a joke or not, at first.

                                                                                                                              My advice is to simply use ECMA-48; every terminal emulator you’re likely to come across supports a subset and it’s easy to use, being free from ’‘color pairs’’ and other nonsense. The only issue is sending different sequences to terminals that don’t support the subset being used, or finding a common subset, but there is a point where this ceases to be reasonable and one adopts a ’‘If you’re terminal doesn’t even support this, I don’t care if it works.’’ attitude.

                                                                                                                              Writing a library in most any language that sends ECMA-48 codes is simple, so you can work in whichever language you’re actually using instead of binding with a decrepit C library. It’s also important to stress that people only really use Ncurses and whatnot still because the graphical APIs of UNIX are far worse to cope with.

                                                                                                                              Now, to @Hales :

                                                                                                                              If there’s is one lesson that we should have learned from the past few decades of software development it’s that you should not assume that current fashions, expectations or trends in programming are in any way better than prior ones. Most of them are just as bad (in different ways), and this is only properly realised years later.

                                                                                                                              If one digs deep enough into the history of computing, one lears that what’s modern is distinctly worse than prior systems. There were easy graphical systems on computers decades ago, but UNIX is still stuck sending what it thinks are characters to what it thinks is a typewriter and using X11 lets you pick your poison between working with a directly (Wayland is coming any year now, right?) or using a gargantuan library that abstracts over everything. It’s a mess, I think, don’t you agree?

                                                                                                                              Also, @Hales , you’re note on UTF-8 reminded me of something I found amusing from the Suckless mailing list, when I subscribed to it trying to get proper parsing of ISO 8613-6 extended color codes into st. Simply put, UTF-8 is trivially transparent in the same way color codes are transparent, in that there are many cases where invariants are violated and programs misbehave, commonly expressed as misrendering. There was an issue with the cols program, that shining example of the UNIX philosophy: it didn’t properly handle colors in columnating its output; to properly handle this, the tool would need to be made aware of color codes and how to understand them, as it would otherwise assume each would render as a single character would, ruining the output; the solution, according to one in the mailing list, was that colors are bloat and you don’t need them.

                                                                                                                              Don’t you agree that’s amusing?

                                                                                                                              1. 4

                                                                                                                                I’m squarely in that group that simply avoids using Ncurses. I find Ncurses to be a baroque API, from what I know, and as with many UNIX APIs it seems so often I learn of some new quality and can’t tell if it’s a joke or not, at first.

                                                                                                                                I agree with you. 100%. HOWEVER. Ncurses comes with pretty much every mainstream unix, making it a very attractive target for applications. I think that ncurses’ limitations are holding back innovation of TUI interfaces. If every unix comes with an adequate TUI widgeting library, this encourages quality TUI widget-based systems.

                                                                                                                                If one digs deep enough into the history of computing, one learns that what’s modern is distinctly worse than prior systems. There were easy graphical systems on computers decades ago, but UNIX is still stuck sending what it thinks are characters to what it thinks is a typewriter and using X11 lets you pick your poison between working with a directly (Wayland is coming any year now, right?) or using a gargantuan library that abstracts over everything. It’s a mess, I think, don’t you agree?

                                                                                                                                It’s important to understand things in context. At the time it came out, X was very useful and its complexity was every bit justified. Each bit of bloat was entirely justifiable when added, it’s just in hindsight, when looking at the whole, that it appears bloated. Yes, xorg is a mess. (Incidentally, wayland is also a mess and I hope that it doesn’t come at all. Much prefer arcan.) However, I think it’s a mistake to say that in the dawn of computing they understood simple, sensible interfaces and nowadays it’s all just pointless abstractions and bloat. Sure, Unix, BeOS, and C were all clean and well-made; while javascript, c++, and windows are all currently a mess. So what? There’s still significant amount of innovation and progress being made in the modern world of software and to ignore it would be a mistake.

                                                                                                                                [cols/utf-8/colour/alignment-related stuff]

                                                                                                                                Yes. The entire concept of a TTY is a mess. I am going to work on replacing that. Other people are working on replacing that. (The solution, obviously, is to make the code that generates the text and the code that displays the text work on the same level of abstraction.) That doesn’t change the fact that we’re stuck with ttys for the time being; is it wrong to try to improve our experience with them until we can really leave them?

                                                                                                                                1. 3

                                                                                                                                  However, I think it’s a mistake to say that in the dawn of computing they understood simple, sensible interfaces and nowadays it’s all just pointless abstractions and bloat

                                                                                                                                  The stuff we’ve kept from the dawn of computing is basically all either ‘simple, sensible interfaces’, or ‘backwards compatible monstrosity’. Those are, after all, the two reasons to keep something - either because it’s useful as a building block, or because it’s part of something useful.

                                                                                                                                  1. 3

                                                                                                                                    The entire concept of a TTY is a mess. I am going to work on replacing that. Other people are working on replacing that. (The solution, obviously, is to make the code that generates the text and the code that displays the text work on the same level of abstraction.) That doesn’t change the fact that we’re stuck with ttys for the time being; is it wrong to try to improve our experience with them until we can really leave them?

                                                                                                                                    Have you looked at the TUI client API for Arcan? and the way we deal with colour specifically? If not, in short:

                                                                                                                                    1. Static set of semantic colour labels with some compile-time default palette.
                                                                                                                                    2. Clients resolve on-draw the labels into R8G8B8(FG, BG).
                                                                                                                                    3. Channel values are set as bg/fg attributes to the appropriate cells in the grid.
                                                                                                                                    4. Server/WM side has commands for runtime replacing the colour values stored at some label index.
                                                                                                                                    1. 1

                                                                                                                                      Do you allow the client to set arbitrary 24bit colours to the grid?

                                                                                                                                      Does the TUI API work with text-based but not-fixed-width interfaces (e.g. emacs, mlterm)?

                                                                                                                                      Thank you for posting, I hadn’t heard about arcan until today but have just read a chunk of your blog with interest :)

                                                                                                                                      1. 2

                                                                                                                                        colors: both arbitrary fg/bg (r8g8b8) and semantic labels to resolve (such as “give me fg/bg pair for alert text”) shaped text: yes (being reworked at the moment to account for server- side rendering), but ligatures, shaped is there as a ‘per-window’ attribute for now, testing out per line.

                                                                                                                                        1. 1

                                                                                                                                          Thanks for the reply.

                                                                                                                                          I think I would want to be able to use different fonts (or variations on a font) for different syntax highlighting groups in my editor. This looks quite nice in emacs and in code listings in latex. Perhaps you consider this to be an advanced use where the application should just handle their own frame buffer, though.

                                                                                                                                          While I have your ear, what’s the latency like in the durden terminal and is minimising latency through the arcan system a priority?

                                                                                                                                          1. 2

                                                                                                                                            in principle, multiple fonts (even per line) is possible, and that’s how emojii works now, one primary font for the normal glyphs and when there is a miss on lookup, a secondary is used. There is an artificial limit, that’ll loosen over time. Right now, the TUI client is actually handling its own framebuffer, and we are trying to move away from that, which can be seen in the last round of commits. The difficulty comes from shaped rendering of ligatures, where both sides need to agree on the fonts and transformations used; doing it wrong creates round-trips (no-no over the network), as the mouse-selection coordinate translation needs to know where the cursor position actually became.

                                                                                                                                            Most of this work is towards latency reduction, removal of terminal protocols fixes synchronization, moving rendering server-side allows direct-to-scanout buffer racing-the-beam rasterization, or at least, entirely on-gpu for non-fullscreen cases.

                                                                                                                                            1. 1

                                                                                                                                              The difficulty comes from shaped rendering of ligatures, where both sides need to agree on the fonts and transformations used;

                                                                                                                                              When you say “both sides” do you mean the client on e.g a remote server and a “TUI text packing buffer” renderer on e.g. a laptop?

                                                                                                                                              Sounds like you could just send the fonts (or just the sizes of glyphs and ligature rules) to the server for each font you intend to use and be done with no round trips. Then you just run the same version of harfbuzz or whatever on each side and you should get the same result. And obviously the server can cache the font information so you’re not sending it often (though I imagine just the sizes and ligatures could be encoded real small for most fonts).

                                                                                                                                              Do you have any plan about what to do about the RTT between me pressing a key on my laptop, that key going through arcan, the network, remote arcan and eventually into e.g. terminal vim and then me getting the response back? I feel like mosh’s algorithm where they make local predictions about what keypresses will be shown is a good idea.

                                                                                                                                              Most of this work is towards latency reduction, removal of terminal protocols fixes synchronization, moving rendering server-side allows direct-to-scanout buffer racing-the-beam rasterization, or at least, entirely on-gpu for non-fullscreen cases.

                                                                                                                                              Sounds exciting! I don’t know what you mean by “moving rendering server-side”, though. Is the server here the arcan server on my laptop? And moving the rendering into arcan means you can do the rendering more efficiently?

                                                                                                                                              Is arcan expected to offer an advantage in performance in the windowed case compared to e.g. alacritty on X11? Or is the benefit more just that anything that uses TUI will be GPU accelerated transparently whereas that’s more of a pain in X11?

                                                                                                                                              1. 1

                                                                                                                                                Right now (=master) I am tracking the fonts being sent to the client on the server side, so both sides can calculate kerning options and width, figure out sequence to font-glyph id etc. The downsides are two: 1. the increased wind-up bandwidth requirement when you use the arcan-net proxy for network transparency, 2. the client dependency on freetype/harfbuzz.

                                                                                                                                                1. can be mitigated through caching and stream cancellation if there is a client side match already. Only solutions for 2. seems to be a roundtrip for the shaping offsets.

                                                                                                                                                My first plan for the RTT is type-ahead (local echo in ye olde terminal speak), implemented on the WM level (anchored to the last known cursor position, etc.) so that it can be enabled for other uses as well, such as input-binning/padding for known-networked windows where side channel analysis (1 key, 1 packet kind of a deal) is a risk.

                                                                                                                                                Sounds exciting! I don’t know what you mean by “moving rendering server-side”, though. Is the server here the arcan server on my laptop? And moving the rendering into arcan means you can do the rendering more efficiently?

                                                                                                                                                Is arcan expected to offer an advantage in performance in the windowed case compared to e.g. alacritty on X11? Or is the benefit more just that anything that uses TUI will be GPU accelerated transparently whereas that’s more of a pain in X11?

                                                                                                                                                Both performance and memory gains. Since the actual drawing is being deferred to the composition stage, windows that are partially occluded or clipped against the screen would only have its visible area actually being processed - while alacritty has to both render into an offscreen buffer (that is double buffered) that then may get composed. So whereas alacritty would have to pay for (glyph atlas texture, vertex buffer, front-buffer, back-buffer) on a per pixel basis in every stage, the cost here will only be the shared atlas for all clients (gpu mem + cache benefits), rest would be a ~12b / cell + vertex buffer.

                                                                                                                                  2. 2

                                                                                                                                    Wayland is coming any year now, right?

                                                                                                                                    It has been here for a while. GNOME and KDE support it natively, a few popular distros ship with Wayland enabled by default. Firefox is a native wayland app. What makes you think it’s “any year now” ?

                                                                                                                                    1. 1

                                                                                                                                      :D

                                                                                                                                      Colours are not the only invisible control codes that I’d expect cols to have to handle. Alas I can’t see a “simple” solution to this. You pretty much have three options:

                                                                                                                                      1. Treat everything as 1 char wide (simplest option)
                                                                                                                                      2. Use a full unicode library to determine your character widths (most “correct”, but much bigger)
                                                                                                                                      3. Whitelist known ANSI characters and assume everything else is zero-width (allows colour support, but upsets international users with you being ‘unfair’)

                                                                                                                                      Of the crappy options I can see: #1 does seem the most like something suckless devs would like.

                                                                                                                                      Suckless makes some great stuff, but some of their projects are a bit too minimal for me. Take for example their terminal emulator st:

                                                                                                                                      ## How do I scroll back up?
                                                                                                                                      
                                                                                                                                      Using a terminal multiplexer.
                                                                                                                                      
                                                                                                                                      * `st -e tmux` using C-b [
                                                                                                                                      * `st -e screen` using C-a ESC
                                                                                                                                      

                                                                                                                                      I love my scrollwheel, whether it’s a real one or two-fingers on my cheap little laptop’s touchpad. That and being able to use Shift+PageUp/Down. I guess everyone draws the line somewhere differently, and I have more things about my current term (urxvt) that I could moan about.

                                                                                                                                      or using a gargantuan library that abstracts over everything. It’s a mess, I think, don’t you agree?

                                                                                                                                      I don’t have any raw X11 experience. I’ve primarily used SDL, which yes indeed abstracts that away for me.

                                                                                                                                      I’m not completely convinced that wayland is going to be the answer: from everything I read it seems to be solving some problems but creating entirely new ones.

                                                                                                                                      From a user perspective however: Xorg is wonderful. It just works. You have lots of choice for DE, WM, compositor (with and without effects), etc. The open source graphics drivers all seem to have a TearFree option, which seems to work really well. I’d love to see latency tested & trimmed, but apart from that the idea of having to change a major piece of software I use scares me. I don’t want to give up my nice stack for something that people tell me is more better (or more “modern”).

                                                                                                                                      1. 2

                                                                                                                                        You forgot one:

                                                                                                                                        1. Interpret control codes as 0-width, and assume each UTF-8 codepoint is 1 char wide (composing characters and some alphabets will break this, but if you need this, then option 2 is your best bet).

                                                                                                                                        Interpreting ECMA-48 for this isn’t that bad, I have Lua code that does such [1]. And a half-assed approach would be to assume all of C0 (0-31, 127) [3] as 0-width, all of C1 (127-169) as 0-width, with special handling for CSI (155) and ESC (27). For ESC, just suck up the next character (except for ‘[’) and for CSI (155 or ESC followed by a ‘[’) just keep sucking up characters until a character from 64 (@) to 126 (~) is read, and that will catch most of ECMA-48 (at least, the parts that are used most often).

                                                                                                                                        [1] It uses LPEG. I also have a version that’s specific to UTF-8 [2] but this one is a bit easier to understand in my opinion.

                                                                                                                                        [2] https://github.com/spc476/LPeg-Parsers/blob/master/utf8/control.lua

                                                                                                                                        [3] 127 isn’t technically in C0 or C1, but I just lump it in with C0.

                                                                                                                                      2. 1

                                                                                                                                        If one digs deep enough into the history of computing, one learns that what’s modern is distinctly worse than prior systems.

                                                                                                                                        Flying on a modern airliner is also distinctly worse than passenger jet travel in the 1950’s. But I can still afford to go anywhere in the world in a day.

                                                                                                                                        1. 1

                                                                                                                                          Worse in experience? I am almost certainly safer flying on a modern passenger jet now than in the 1950s.

                                                                                                                                          1. 1

                                                                                                                                            https://i.huffpost.com/gen/1840056/thumbs/o-56021695-570.jpg

                                                                                                                                            Enough said. :-P

                                                                                                                                            (I am aware this is not a conclusive statement.)

                                                                                                                                            1. 1

                                                                                                                                              Heh, you can still get that (actually, much better) on international business class. Hmm, I’m curious if the cost is comparable between 1950’s tickets like those in your picture (inflation adjusted) and international business class today…

                                                                                                                                      1. 5

                                                                                                                                        Well. I propose making pipes multi-channel & segmented, and using that to write metadata/attributes without inline escaping. stdout.channel("color").write("red"); stdout.write("Hello"). Drag Unix kicking and screaming into the Year of the Justifiably Defensive Lobster!

                                                                                                                                        1. 8

                                                                                                                                          Having inline escape codes might not seem pretty, but it’s good for the same reason UTF-8 is good:

                                                                                                                                          • anything will accept it, even if it doesn’t understand colour.
                                                                                                                                          • programmers can put a lot less effort in, especially initially, if they don’t care about all of the features.

                                                                                                                                          Or: don’t add more dimensions to a problem that can already be serialised into 1D, if you start solving problems by adding dimensions then you will end up with a hypercube.

                                                                                                                                          Drag Unix kicking and screaming into the Year of the Justifiably Defensive Lobster!

                                                                                                                                          Slippery sidenote: I’m not a fan of using the word ‘modern’ to describe software in a good light, or saying something is bad because it’s “not modern”.

                                                                                                                                          If there’s is one lesson that we should have learned from the past few decades of software development it’s that you should not assume that current fashions, expectations or trends in programming are in any way better than prior ones. Most of them are just as bad (in different ways), and this is only properly realised years later.

                                                                                                                                          EDIT: If you’re writing something to be useful, then stick to tried and true development & design practices. Don’t equate “modern” to mean “tried and true”.

                                                                                                                                          1. 3

                                                                                                                                            I’m not a fan of using the word ‘modern’ to describe software in a good light

                                                                                                                                            Same. I roll my eyes whenever I read the phrase “modern programming language”.

                                                                                                                                            As for UTF-8: it can have just as graceful a fallback. Suppose we want to port ls -l. We’ll start by writing the same bytes as before, but they’ll be annotated by channel. For -rw-rw-r--, we could write that to mode. The next field is the number of hard links. But first comes a space, and there might be multiple spaces to pad out the field. They can be written into tab. Eventually we’ll reach a filename that we might want to color, so we’ll write the whole escape code to escape, and when we get to the \n, we’ll write it to “record-separator”.

                                                                                                                                            This gets us some slight benefit. You can use ≈jq instead of cut/sed, and feel less nervous about filenames with funny characters. There isn’t a whole lot of benefit until further incremental steps are taken:

                                                                                                                                            1. Annotate existing output.
                                                                                                                                            2. Write extra data to channels that are hidden by default. The writer must know if the read end is naive, or only interested in particular subset of channels.
                                                                                                                                            3. Separate data from presentation. A library can handle rendering to the naive format.
                                                                                                                                            4. The shell & terminal become type-aware. Their capabilities increase over time. Short filenames hide a full path, and your SSH client sticks the hostname in front. The output of ls doubles as a filemanager. wget & wget no longer makes a mess because each command requests a separate output pane. You can run find ~/Pictures/ | grep cat, pass the output to ImageMagick to make a collage, display it inline, and right click -> rerun on input change. Running program --help attaches a typed grammar for tab-completion. The shell is merely twice as terrifying as a spreadsheet.

                                                                                                                                            Each step can be reached from the previous. No worlds need to be rewritten. All that’s required is implementing a few syscalls and kickstarting some coreutils.

                                                                                                                                            1. 1

                                                                                                                                              nushell has thought about some of this a bit: https://book.nushell.sh/en/working_with_tables

                                                                                                                                          2. 3

                                                                                                                                            If you’re modernizing, why not just drop curses for drawing and just use a graphics library?

                                                                                                                                            1. 2

                                                                                                                                              How would you read this multi-channel pipe? Color channels must stay synchronized with data but it should be possible to read data without reading color channels. I’m afraid that this is a dilemma.

                                                                                                                                              1. 1

                                                                                                                                                “Channels” might be a bad name. When reading, the *buf is filled just like the usual read, but there is a second buffer that contains channel span information. Perhaps “labels” would be a better name. A high-level wrapper for this syscall in Rust might return a Vec<(ChannelName, Vec<u8>)>.

                                                                                                                                            1. 6

                                                                                                                                              I use an 8 year old Thinkpad X220 every day and the only upgrades are a solid state drive, linux and some extra RAM (which I need because some of my datasets are obnoxiously large and I keep too many tabs open).

                                                                                                                                              It would be real nice to have a better screen, though.

                                                                                                                                              1. 4

                                                                                                                                                If you are comfortable with soldering, I highly recommend this mod to add support for a 1080p display. I did this mod about 9 months ago to my X230 and I’m so glad I did.

                                                                                                                                                1. 1

                                                                                                                                                  I’ve let my X220 get so bashed up that I should probably buy a new one to do the mod in.

                                                                                                                                                  I suppose I could put the mod in this and then take it out again later if this laptop ever dies, but I’m kinda bad at soldering, so I’m not sure that’s likely ;)

                                                                                                                                                  1. 2

                                                                                                                                                    The last time I looked (~1 year ago) there were still a surprising number of X220 and X230 laptops for sale on ebay some as cheap as $100. I picked up an X230 to use for spare parts.

                                                                                                                                                    The mod was a little tricky to solder, especially getting the solder to sink down the through-holes to hit the pins sticking out of the motherboard. I ended up re-applying solder 3 times to finally get good connections there. The previous attempts seemed to work but then the display would cut out after using it for a few hours, or moving it, etc. Other than that, it has been solid ever since (I run Linux on it, not windows, so I cannot speak to the windows experience..)

                                                                                                                                                    1. 2

                                                                                                                                                      As if I use Windows ;)

                                                                                                                                                      Know about the ebay sales. Thanks for the insight on the soldering experience, though. Might come in handy.

                                                                                                                                                      1. 2

                                                                                                                                                        Heh, I didn’t check your profile before replying. Sorry for insinuating you might use windows :P

                                                                                                                                                  2. 1

                                                                                                                                                    Thank you for posting this. A while back I had only seen the mod for the X220 and there was uncertainty about whether something similar would emerge for the X230. Finally I might be free of the only part of this laptop that I don’t care for…

                                                                                                                                                  3. 3

                                                                                                                                                    Thinkpad w530 here, great machine. Similar, SSD, add ram, linux. The Thinkpad driver for linux situation has alway been great.

                                                                                                                                                    1. 2

                                                                                                                                                      Adding an SSD makes a lot of sense, since the bottleneck for performance is rarely the CPU & usually the disk.

                                                                                                                                                    2. 3

                                                                                                                                                      Pretty much the same. 6 year old X230, no SSD, Linux, and extra RAM. The only thing I miss is a better screen. And I frequently use a much less powerful laptop than this one when I want to save some weight (an ASUS eeePC 1015).

                                                                                                                                                      1. 2

                                                                                                                                                        My T420 has been great so far. They’re about $230-250 on eBay. Supports most OS’s.

                                                                                                                                                      1. 3

                                                                                                                                                        Wish the proportional variants were available for inspection somewhere (the web page or the specimen). I’ve been enjoying using another “designed for code” proportional font in emacs lately, and it’s really surprisingly good. The variable width makes it much more humane, but it’s not too variable, so it doesn’t look weird in a code buffer like, say, DejaVu Sans would.

                                                                                                                                                        1. 6

                                                                                                                                                          Here’s a look: https://imgur.com/a/hSeO38h

                                                                                                                                                          1. 1

                                                                                                                                                            Ta. I actually just went ahead and installed it to try it. On my (Windows) system it looks a lot more narrow than your sample, but I’m giving it a shot for today.

                                                                                                                                                            1. 1

                                                                                                                                                              The proportional l looks too thin to me. I use Dina because I need a pixel font for sharp rendering

                                                                                                                                                              1. 1

                                                                                                                                                                Completely unrelated but this is a beautiful Emacs theme. It’s so hard to find a nice light-background one.

                                                                                                                                                                1. 1

                                                                                                                                                                  Looks like Solarized Light to me!

                                                                                                                                                                  1. 1

                                                                                                                                                                    This is doom-one-light and it’s a great theme indeed. There’s also a dark variant.

                                                                                                                                                              1. 5

                                                                                                                                                                Do folks using ligatures not incur the cognitive cost of wondering what two (or more) keys were pressed to create a given ligature? Especially when reading other peoples’ code. This is a nice customizable font, but I’m set in my ways w/ source code pro.

                                                                                                                                                                1. 5

                                                                                                                                                                  Iosevka has the term variant for people who don’t want ligatures.

                                                                                                                                                                  Agree on Source Code Pro, an amazing font.

                                                                                                                                                                  1. 4

                                                                                                                                                                    Not really–the ligatures are just stylized combinations of the keys in most cases. >=, <=, !=, and == all become one syntactic element when reading, and in practice I have no trouble typing them. And they are especially nice in closures and lambdas.

                                                                                                                                                                    1. 1

                                                                                                                                                                      As an Emacs user, it sounds like everything prettify-symbols-mode does. Is there any real benefit, giving that one seems to give up dynamic reprogrammability, that ligatures have to offer?

                                                                                                                                                                      1. 2

                                                                                                                                                                        For reference, vim has something similar called conceal that people use for rendering latex in unicode and so on.

                                                                                                                                                                        1. 2

                                                                                                                                                                          I haven’t used prettify-symbols-mode specifically, so this may not apply, but when I tried similar things in the past, the problem was that it was generally replacing two or three characters with one character, meaning that either it messed with alignment, or it had to try and do weird things like surrounding the symbol with two half-width spaces.

                                                                                                                                                                          Personally, I use the Hasklig font, and find that the ligatures look good, but it’s also unobtrusive enough that I just set it as my terminal font (I use Konsole, which supports ligatures) and use it everywhere, rather than trying to only use it with certain files (When I’m using a GUI editor or IDE, I usually use Hasklig as my font there too).

                                                                                                                                                                          1. 1

                                                                                                                                                                            I hadn’t run across prettify-symbols-mode, thanks for mentioning it! I’ll take a look at it.

                                                                                                                                                                            1. 1

                                                                                                                                                                              giving that one seems to give up dynamic reprogrammability

                                                                                                                                                                              Can you expand on this? As I understand it, nothing is changing in the actual textual content when using ligatures, so I’m not sure what you’re referring to.

                                                                                                                                                                              Is it just in comparison to prettify-symbols-mode where you create your own mappings?

                                                                                                                                                                              1. 2

                                                                                                                                                                                My understanding is that you can’t (easily?) turn specific mappings or as you say add ones by manipulating a variable (i.e. dynamically reprogramming), but it’s kind of “set” in the font.

                                                                                                                                                                                1. 2

                                                                                                                                                                                  Ah, I see, thank you.

                                                                                                                                                                                  I had never considered changing ligature mappings dynamically, though the fact that this release has different ligature sets for different languages presents a usecase I hadn’t thought of before.

                                                                                                                                                                                  I wonder if doing these mappings in opentype rendering vs in the “editorspace” so to speak would make a difference performance-wise, sort of like what’s happened recently with bringing linum-mode into C.

                                                                                                                                                                            2. 1

                                                                                                                                                                              But what if say you’re reading some code in OCaml with <> or Fortran with .NE. and /= While those would probably not be ligatured, if they were it could be super confusing. I agree that there’s a certain elegance to the look, the ergonomics of not knowing what was typed to create something means you could conceivably half erase something and not know how to retype it.

                                                                                                                                                                            3. 1

                                                                                                                                                                              Ligatures in code listings sound like a nightmare to me.

                                                                                                                                                                              Is it just a display thing? I.e. my code has “>=” while I see ‘≥’ in the editor? [1]

                                                                                                                                                                              1. 2

                                                                                                                                                                                That’s right. In editors like Sublime or VSCode, you can turn off ligatures if you don’t like them; in the case of Iosevka, you can also download a version that does not have ligatures (the term variant).

                                                                                                                                                                                1. 2

                                                                                                                                                                                  Yeah it’s just that, your code still lives on disk as >= but the font does a ligature like it might do with fine or flare (the fi and fl merging) in a wysiwyg word processor. The difference is fine and flare still preserve the 2 characters visually, many of these ligatures toss them completely and sometimes go from 2 fixed width chars to 1 which makes alignment a bit strange as well. Though in some cases it could be much nicer.

                                                                                                                                                                              1. 3

                                                                                                                                                                                I agree that replacing built in array methods probably warrants profiling, but you should also be maintaining constant vigilance against writing code that will obviously be slow to begin with.

                                                                                                                                                                                1. 7

                                                                                                                                                                                  I think constant vigilance is falling into the same premature optimisation trap. Back when I taught CS we had undergrads wasting their time on macro (mostly data structure choices) and micro-optimisations (preincrement instead of postincrement and other weird tricks from C) before their code or algorithms were even correct. Most often, these optimisations had little to no benefit.

                                                                                                                                                                                  Make the code correct. Then 99% of the time it will be fast enough anyway. For the remainder, profile it and see if there are quick and obvious improvements. If there aren’t obvious improvements you probably need to go and examine your algorithm and maybe read the literature.

                                                                                                                                                                                  If you’re in a domain where it pays to care about performance then you’ll get to know what works well with experience.

                                                                                                                                                                                  1. 2

                                                                                                                                                                                    I generally think you’re right, but also that macro optimizations like data structure choices can be considered part of “correctness”. Not that you should agonize over small differences in best and worst case performance, but that you should be choosing data structures with generally appropriate performance characteristics.

                                                                                                                                                                                    1. 2

                                                                                                                                                                                      I think a fair amount of accepted wisdom about data structures tends not to take processor pre-fetching or performance characteristics of the relevant language into account. Often the right data structure for the job is not obvious and if you just use vectors, sets and maps as appropriate you’ll usually be fine.

                                                                                                                                                                                      Obviously you should think about what you’re going to write before you write it, but people get very excited about these details that I would wager make very little difference in most cases. Where it does matter you’ll pick that up in the design if you’re experienced enough or in review or evaluation if you’re not.

                                                                                                                                                                                      E.g. if you do a lot of data science you’ll probably pick up that e.g. sparse arrays and vectors can be very useful; or if you’re searching a lot of strings you might read up on tries or DFA theory, but focussing on these issues early is usually going to be self-indulgent.

                                                                                                                                                                                1. 2

                                                                                                                                                                                  funny, 5 out of the 25 entries currently listed are reposts on Lobsters by @calvin

                                                                                                                                                                                  1. 4

                                                                                                                                                                                    @calvin likely copies things from a lot of sites. He’s a curator for Lobsters.

                                                                                                                                                                                    On a related note, I’ve noticed that the growth of the site has led to HN-like submissions getting more attention than what did in the past. Things are changing. So, I’ve been posting more of those, too, since it’s what they want to read. It’s also too busy with such submissions to ensure the CompSci stuff would even be seen. I’ve been posting it on days with less going on.

                                                                                                                                                                                    1. 4

                                                                                                                                                                                      I do cross-post from Reddit/HN, but only if:

                                                                                                                                                                                      1. The article fits the scope of the site. HN has a lot of tech business stuff that’s just on the edge and outright non-tech related stories.

                                                                                                                                                                                      2. It has enough “meat on the bone” to be interesting. I do try to consider the fact that not everyone here is at the same skill level in everything though, so more introductory stuff could be submitted if I think it fits Lobsters’ quality standards.

                                                                                                                                                                                      3. It’s not a blatant advertisement. Vendor blogs are OK if they’re talking about something technical and not trying to sell you on things (At most, a little quip saying “btw if you liked this please buy our stuff”.). Sometimes my filters get fouled though, out of confusion or just mental exhaustion.

                                                                                                                                                                                      Of course, I also have an RSS reader…

                                                                                                                                                                                      1. 2

                                                                                                                                                                                        I’ve noticed that the growth of the site has led to HN-like submissions getting more attention than what did in the past

                                                                                                                                                                                        From the July stats we can see that the links in common are ~2% of the total links on HN[1], while they’re ~60% on Lobste.rs.

                                                                                                                                                                                        Lobste.rs is way less aggressive in outright removing stuff. There’s quite a lot of links on Lobste.rs with negative karma (like this one). That link has a 0.34 comment-to-score ratio on HN.

                                                                                                                                                                                        The scale of the two sites is way different, so it’s natural that HN will have some “outsized” influence on the stuff that ‘s posted here. It’s up to the community to ensure it’s the stuff they want.

                                                                                                                                                                                        [1] note that I don’t (yet) weed out links that can be flagged “dead” after submission, unless they’re part of a pair.

                                                                                                                                                                                        1. 1

                                                                                                                                                                                          And I don’t just mean in common: I mean the kinds of content that appeals to folks on HN more. Lobsters had a lot of deeper tech stories in the past that weren’t as mainstream. Even when here, the HN-like content often covers them up.

                                                                                                                                                                                          1. 2

                                                                                                                                                                                            I browse using the /newest link, because there’s seldom more content than maybe half of that gets replaced daily. I find plenty of interesting content, and I flag the inappropriate stuff.

                                                                                                                                                                                            As mentioned in my blog post, that’s essentially impossible to do on Hacker News. Discovery is at the mercy of the front page.

                                                                                                                                                                                            So increasing size and popularity does carry risks. But I don’t want to exclude people from the stuff posted here.

                                                                                                                                                                                            1. 2

                                                                                                                                                                                              Yeah, I might just have to stop relying on the front page. I think most folks use it, though. So, I’ve shifted my submitted content from CompSci to more HN-like stuff with some CompSci to fit the audience’s preferences. Well, my schedules have kept me off the site, too.

                                                                                                                                                                                              And I’m also not trying to exclude anyone: just talking about patterns of behavior they’re all doing. I’m letting the site evolve into whatever it will become.

                                                                                                                                                                                              @pushcx would you run a query to see how many requests/views we get on front page, recent, and new? I’m curious what the audience ratio is for those. I expect some will be scripts that might run many times a day. Might be ways to reduce that like unique I.P.‘s or account ID’s.

                                                                                                                                                                                              1. 3

                                                                                                                                                                                                So, I’ve shifted my submitted content from CompSci to more HN-like stuff with some CompSci to fit the audience’s preferences. Well, my schedules have kept me off the site, too.

                                                                                                                                                                                                For what it is worth, I have joined lobsters specifically, to exclude from my frequent consumption, the HN’s 10%-stories (this is my ‘tag’ for stories that are only tangentially related to CS, Maths, Physics, Material Engineering, Biology), and their associated comments.

                                                                                                                                                                                                On the other hand, if people are more interested in social/economic/political/moral impacts and positioning of technology+sciences and its leaders, HN is probably a much better place. I am interested in those as well, but I prefer to consume those at much less frequent intervals.

                                                                                                                                                                                                Also, the ‘I have built a business over the weekend’ – kind of stories, often frequent on HN, are inspirational.. but to a point :-).

                                                                                                                                                                                                1. 2

                                                                                                                                                                                                  Lobsters was mostly like that when I joined it. That’s a smaller part of the audience and/or submissions now.

                                                                                                                                                                                                2. 3

                                                                                                                                                                                                  Why not just submit the kind of content that interests you most rather than trying to fit your beliefs about other’s preferences?

                                                                                                                                                                                                  1. 1

                                                                                                                                                                                                    I do. I just also want it to delight and help more people. There were quite a few people who used that sort of work when they saw it. If it gets covered up regularly, then there’s no point in submitting it.

                                                                                                                                                                                                    I have thought about reviving the idea of a feed of that stuff folks can subscribe to or post here if they wanted. Then, just tell the target audience about it. Pretty swamped by main job currently. Barely time to do my research at all. Got 61 tabs open on my phone I gotta go through. Off today so will do.

                                                                                                                                                                                                    1. 2

                                                                                                                                                                                                      Thanks for the response.

                                                                                                                                                                                                      A twitter feed would be pretty low set-up time and you can easily publish an RSS feed from it (https://wp.me/p9vpb5-43)

                                                                                                                                                                                        2. 1

                                                                                                                                                                                          @calvin submitted 59 entries last month (data collection started on 5 Jul 2019 so the entire month is not covered).

                                                                                                                                                                                        1. 20

                                                                                                                                                                                          And yet I have worked as a DevOps engineer whose job was orchestrating spreadsheets in a brokerage. I had a script that would spin specific versions of windows, install specific versions of excel on them, copy the last days spreadsheets into clean folders then start executing them in a specific order.

                                                                                                                                                                                          Once that was done everything was nuked and the results were saved to a network drive for the next day and other data pipelines.

                                                                                                                                                                                          Someone had cost the brokerage a few tens of millions by running the sheets in the wrong order which was why they were now locked down with no one having access to the ones that were used for trades any more.

                                                                                                                                                                                          This was not the only, or worst, example of excel insanity in industry I’ve been privy to.

                                                                                                                                                                                          There is a reason why we don’t let people build lego bridges for civil engineering and then put them in production. That we let it happen in software engineering in the name of ‘usability’ says a lot about how immature we are. Especially since spreadsheets have already killed thousands in Europe: https://www.washingtonpost.com/news/wonk/wp/2013/04/16/is-the-best-evidence-for-austerity-based-on-an-excel-spreadsheet-error/

                                                                                                                                                                                          1. 21

                                                                                                                                                                                            Are those issues because spreadsheets are conceptually bad or because we haven’t bothered to make better infrastructure around spreadsheets?

                                                                                                                                                                                            1. 5

                                                                                                                                                                                              This is an important point. There’s no fundamental reason modern VCS and deployment techniques cannot be applied to the spreadsheet concept. Thanks to software patents, you might have a tough time building a startup around the concept, though.

                                                                                                                                                                                              1. 7

                                                                                                                                                                                                Licenses.

                                                                                                                                                                                                What I was doing at the brokerage was pretty clearly not allowed by the licences of the software. No one would ever hear about it because we system was completely isolated and couldn’t phone home.

                                                                                                                                                                                                If I turned this into a startup I’d be sued to hell and back before you can say mvp. Patents are so far down the list of blockers for this I wouldn’t even think about them.

                                                                                                                                                                                                1. 2

                                                                                                                                                                                                  Okay, but that’s for a specific instance of spreadsheet software (excel) and actual spreadsheet code. I’m talking about applying modern software engineering best-practices (e.g. VCS and dependency management) to the spreadsheet concept in general.

                                                                                                                                                                                                  1. 3

                                                                                                                                                                                                    You might as well be applying modern software engineering to forth. No one uses it in business and there is no money in it from the hobbyists. If you’re not bit compatible with xls files you might as well not exist.

                                                                                                                                                                                                    1. 1

                                                                                                                                                                                                      You could provide function compatibility with excel. Macro compatibility would be much harder, but perhaps not necessary in order to gain some significant mindshare.

                                                                                                                                                                                                      Combined with a shared editing mode, this could be a great startup idea. Except for the risk of getting sued out of existence by Microsoft and Google.

                                                                                                                                                                                              2. 2

                                                                                                                                                                                                There’s constantly startups about replacing spreadsheets with something similar but better. I haven’t seen any of them make it. The moderator of Hacker News, Dan Gackle, even did one for YC called Skysheets. I don’t know any details about it. There was one that combined spreadsheets with a database that kept everything in sync instead of scattered around various PC’s. Lots of clever ideas. I think an interesting study would be a survey of all the failed companies to figure out what’s actually going on.

                                                                                                                                                                                                Meanwhile, I think it’s a situation combining a massively-popular tool, its usability, the herd mentality that brings new people into groups’ existing habits, and especially its format that was designed for lock-in with ridiculous switching costs. A bunch of Excel licenses with plumbing built around it is way easier to justify than getting off Excel in an already Excel-heavy industry with lots to lose in a failed transition with managers who have seen failed transitions eliminate jobs and bonuses.

                                                                                                                                                                                                1. 1

                                                                                                                                                                                                  Yeah, I read the GP comment and I thought “start-up opportunity”. Spreadsheets / Excel aren’t going away. Maybe better infrastructure for managing them is really what people would want / be able to use.

                                                                                                                                                                                                  1. 2

                                                                                                                                                                                                    Airtable.

                                                                                                                                                                                                    1. 3

                                                                                                                                                                                                      This seems like a replacement for Excel? I guess I did say “Spreadsheets” but I really meant “infrastructure around Excel”.

                                                                                                                                                                                                2. 14

                                                                                                                                                                                                  And if those spreadsheets had instead been glorious Haskell programs it would have been impossible to run them in the wrong order?

                                                                                                                                                                                                  1. 1

                                                                                                                                                                                                    Yes.

                                                                                                                                                                                                    1. 0

                                                                                                                                                                                                      Or better yet, Idris, where it is trivial to prove mathematically that everything is happening in the correct order, by encoding state machines in types which then don’t let your programs transition incorrectly:D

                                                                                                                                                                                                      1. 4

                                                                                                                                                                                                        How does idris prevent you from running other programs out of order?

                                                                                                                                                                                                        1. 5

                                                                                                                                                                                                          By wrapping calls to those programs in an interface / type class of its own, and then controlling calls to that interface. The interface is stateful, that’s how it ensures correct protocols.

                                                                                                                                                                                                          1. 1

                                                                                                                                                                                                            I think the issue was probably that the running of the spreadsheets was done manually and/or the semantics of excel changed between windows/excel versions. In this context, a reasonably reliable bash script or makefile + statically compiled binaries would probably have been sufficient. No fancy tech required.

                                                                                                                                                                                                      2. 1

                                                                                                                                                                                                        Totally noob on spread sheets. How does one use the result of one spreadsheet on another? Is VBA magic involved?

                                                                                                                                                                                                        1. 4

                                                                                                                                                                                                          In excel you can refer to data in another file by including its filename and the sheet within the file you want to reference. The complexity progresses something like:

                                                                                                                                                                                                          Same sheet: =A3

                                                                                                                                                                                                          Same file: =!sheet3:A3

                                                                                                                                                                                                          Same filesystem: =[other.xslx]Sheet3:A3

                                                                                                                                                                                                          1. 1

                                                                                                                                                                                                            Ah thanks!

                                                                                                                                                                                                            One more question: In the case of OP, how does running it out of order work? Wouldn’t it have referred to empty cells and thrown an error? or does excel make up random values if you access a non-value cell?

                                                                                                                                                                                                            1. 3

                                                                                                                                                                                                              Daily updates got propagated through. A simple example were some conversion rates grabbed online.