1. 3

    Hmm…isn’t this really just a bug in the test program?

    It just steps by a constant amount regardless of the number of iterations, so greater number of iterations increases the range of inputs to way beyond what is reasonable for sin()/cos() and triggering the range reduction.

    Easy fix: divide whatever your step size by the number of iterations.

    1. 2
      1. How about detaching a thread that then simply blocks?
      let location = try userLocation()
      let conditions = try weatherConditions(for: location)
      let (imageData, response) = try URLSession.shared.data(from: conditions.imageURL)
      if (response as? HTTPURLResponse)?.statusCode == 200 {
          let image = UIImage(data: imageData)
      } else {
          // failed
      }
      

      The OS already has all these facilities, not sure why we have to recreate them from scratch in user space.

      1. Both Combine (Rx,…) and async/await map dataflow onto familiar and seemingly “easy” call/return structures. However, the more you make the code look like it is call/return, the further away you get from what the code actually does, making understanding such code more and more difficult.
      1. 3

        Apple points out in a Swift concurrency WWDC talk that wasting threads can have a much bigger impact on devices like iPhones. Having 100k worker threads on a modern Linux server isn’t a big deal at all. But on a RAM-constrained device trying to use as little energy as possible that’s not a good idea.

        Consider an app that needs to download 200 small files in the background (the example from the video linked above). Blocking in threads, that’s 100 MB of thread stacks alone, not to mention the OS-level data structures and other overhead. On a server that’s negligible. On a brand new 2021 iPhone with 4 GB of RAM that’s 1/40 of physical memory. 1/40 sounds small, but users run dozens of apps at a time. 1/40 of RAM can be 1/4 to 1/2 your entire memory budget for your app. Not a good use of resources.

        Update: both replies mention thread stacks are virtual memory, and likely won’t use the full 512 KB allocated for them. Which is a good point. Nevertheless, the async model has proven repeatedly to use less RAM and have lower overhead than a threaded model in multiple applications, most famously nginx vs Apache. Personally I think async/await has more utility on an iPhone than in 99% of web app servers.

        1. 2

          Thread stacks are demand paged.

          But even if they weren’t, userspace threads are a cleaner abstraction. Async and await is manual thread scheduling.

          1. 2

            That Apple statement, like a lot of what Apple says on performance and concurrency, is at best misleading.

            First, iPhones are veritable supercomputers with amazing CPUs and tons of memory. The C10K problem was coined in 1999, computers then had something like 32-128MB of RAM, Intel would sell you CPUs between 750MHz and 1GHz. And the C10K problem was considered something for the most extreme highly loaded servers. How are you going to get 10K connections on an iPhone, let alone 100K? No client-side workloads reasonably produce 100K worker threads. Wunderlist, for example, used a maximum of 3-4 threads, except for the times when colleagues put in some GCD, at which point that number would balloon to 20-40. Second, Apple technologies such as GCD actually produce way more threads than just having a few worker threads that block. Third, those technologies also perform significantly worse, with more overhead, than just having a few worker threads that block. We will see how async/await does in this context.

            For your specific example, downloading 200 (small) files simultaneously is a bad idea, as your connection will max out way before that, more around 10 simultaneous requests. So you’re not downloading any faster, you are just making each download take 20 times longer. Meaning you increase the load on the client and on the server and add a very high risk of timeouts. Really, really bad idea. If async/await makes such a scenario easier to accomplish and thus more likely to actually happen, that would be a good case for not having it.

            Not sure where you are getting your 100MB of thread stacks from. While threads get 512K of stack space, that is virtual memory, so just address space. Real memory is not allocated until actually needed, and it would be a very weird program that would come even close to maxing out the stack space (with deep recursion/nested call stacks + lots of local data on those deeply nested stacks) on all these I/O threads.

            And of course iOS has special APIs for doing downloads even while your app is suspended, so you should probably be using those if you’re worried about background performance.

            1. 2

              This may be true, but it doesn’t have to leak into our code. Golang has been managing just fine with m:n green threads.

              1. 1

                I do wonder why Apple didn’t design Swift that way. Maybe there are some Obj-C interop issues? I’d love a primary source on the topic.

                1. 2

                  Why it was designed this way is a good question, but Objective-C interop is not the reason.

                  NeXTstep used cthreads,a user-level threads package.

          1. 21

            Hmm…doesn’t surprise. I only had a very brief encounter with a part of the Racktet core team, but it was…memorable.

            I attended a Racketfest out of curiosity when it was held in my city, with one of the core team in attendance. His presentation was OK, but his behavior during another presentation truly outlandish. He kept interrupting the presenter and telling him how he was wrong and everything he was saying BS. Admittedly the thesis was a bit questionable, but it was still interesting. And if you really, really want to make such a comment, do it in the Q&A. Once. Definitely not interrupting the presentation. And most definitely not multiple times.

            OK, so maybe a one-off. Nope.

            Same person was a visitor at my institute a little later. People presented their stuff. One person kept trying to tell him that if he only let him continue with his presentation, it would explain the things he wasn’t getting. Nope. Kept stopping, saying the terms used were wrong (they weren’t) and refused to let the presenter continue.

            At some point, he bluntly said: you are just PhD students, and I will be on the program committees of the conferences where you need to get your papers published, so you better adapt to how I see the world. Pure power play.

            Never seen anything like it.

            And I personally find what I’ve seen/heard of Linus, for example, absolutely OK. And RMS to me seems a bit weird, but that’s it.

            1. 15

              Didn’t amazon do this with The Decree:

              1. All teams will henceforth expose their data and functionality through service interfaces.
              2. Teams must communicate with each other through these interfaces.
              3. There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network.
              4. It doesn’t matter what technology they use. HTTP, Corba, Pubsub, custom protocols — doesn’t matter.
              5. All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.
              6. Anyone who doesn’t do this will be fired.
              7. Thank you; have a nice day!
              1. 6

                Rereading this now, it’s not clear to me why a read of a data store is not a service interface. How do you draw the line?

                1. 3

                  It’s a lot easier to version your API than it is to be stuck with a schema that’s now incorrect due to changing business logic.

                  1. 4

                    Only if the business logic uses the schema of the data store directly. But I’ve seen use cases where the business logic separates the internal schema from an external “materialized view” schema.

                    And the quote above says, “it doesn’t matter what technology you use.” If you can use HTTP, well reading from an S3 bucket is HTTP. So this is one of those slippery cases where somebody can provide a “versioned API” that is utter crap and still not solve any of the problems that the decree seems to have been trying to solve.

                    I guess what I’m saying is: Never under-estimate the ability of humans to follow the letter of the law while violating its spirit. The key here is some way to be deliberate about externally visible changes. Everything else is window dressing.

                  2. 1

                    The same way reading a field isn’t a method call.

                    1. 2

                      a) I think you mean “reading a field isn’t part of the interface”? Many languages support public fields, which are considered part of the interface.

                      b) Creating a getter method and calling the public interface done is exactly the scenario I was thinking about when I wrote my comment. What does insisting on methods accomplish?

                      c) Reading a value in an S3 bucket requires an HTTP call. Is it more like reading a field or a method call?

                      1. 2

                        Maybe it is assumed that AWS/Amazon employees at the time were capable of understanding the nuances when provided with the vision. It is not too much of a stretch to rely on a mostly homogenous engineering culture when making such a plan.

                  3. 4

                    How so?

                    They just decreed a certain architectural style, nothing about actually supporting this (and other styles!) by providing first class support.

                    1. 4

                      Thank you; have a nice day!

                      OR ELSE

                    1. 2

                      This is a great summary of things to be aware of with C++ ABIs. It’s unfortunate enough that some libraries go for the other extreme of header only. For some prior art here, you might also be interested in COM, which achieved API stability with vtables very well. I really wish there was some more expressive calling convention so we didn’t have to shove C++2x/Rust binaries through the very thin C straw.

                      1. 2

                        I’m honestly really surprised they’re not just doing COM, straight-up. Nano-COM/XPCOM are lightweight, well-understood, have solid existing tooling, run on all relevant OSes, and map cleanly to JS as-is. Hell, the interface model used heavily in TypeScript even maps very well to COM interface usage (by convergent evolution, not intentionally). That’d be a lot easier than going through the whole stabilize-the-ABI hell C++ always dies on.

                        1. 3

                          Hey, Mozilla got pretty good mileage out of COM and JS!

                          1. 2

                            I wonder if the background of the Facebook developers that have been working on the C++ React Native core is a factor. I imagine there are many C++ developers these days that just aren’t familiar with COM. My guess is that many Googlers and ex-Googlers fall into that category, since Google tends to use monolithic statically linked binaries (even Chromium is mostly one giant binary), with no COM or other ABI boundary internally.

                            Is Nano-COM an actual thing, e.g. an open-source project, or just an informal name for a minimal subset of COM? And is it practical to use XPCOM outside of the huge Gecko codebase and build environment? If Nano-COM isn’t an actual package and XPCOM is too tied to Gecko, then maybe another factor is that people don’t have a straightforward way of introducing a COM-ish thing into a cross-platform project. I, for one, would be afraid to just copy from the Windows SDK headers, for legal reasons. But then, with everything that Microsoft has open-sourced, I suppose there’s a COM variant under the MIT license somewhere now.

                          2. 1

                            That’s basically Objective-C, which is essentially “COM with language support”. And incidentally, the COM/Objective-C interim in d’OLE was stellar.

                          1. 16

                            Great post. I think it’s really important to separate out the additive effect (how much does a person accomplish on their own) and the multiplicative effect (how much do they change the contributions from others). In general, junior developers should have a reasonable additive effect (they get work done) but may have a slightly negative multiplicative effect (everyone they work with needs to mentor them a bit and so they reduce everyone else’s productivity by a few percent). Over time they additive effect goes up and the multiplicative effect at least reaches 1 (they are doing code reviews and other things that make everyone else more productive and offset any time spent by others doing the same for them).

                            A more senior developer may have a large additive effect (they’re really productive at writing code) but their real value is from their large multiplicative effect. Someone who makes everyone else on the team 10-20% more productive is incredibly vastly more valuable than someone who gets twice as much work done on their own as the average. A really good dev lead can achieve far more than a 20% multiplicative improvement by teaching, preventing people from wasting time implementing things that won’t work or won’t be maintainable, by having the ten-minute bug-hunting conversations that make people look in the right place, and so on.

                            If you’re really lucky, you’ll find junior devs with a high multiplicative effect who will do a load of glue activities (writing docs, talking to everyone and making sure everyone has the same mental model of the design, asking useful clarifying questions in meetings, cleaning up the build system, improving test coverage, keeping CI happy). If you find one of these folks, do whatever you can to keep them. They’re going to end up being the real rockstars: the dev leads who make everyone on their team twice as productive as they would otherwise be.

                            The folks that make everyone else less productive? They’d need a really big additive effect to make up for the impact of their multiplicative effect. It may be that someone is making everyone on the team 50% less effective but still being a net positive contribution but it’s spectacularly unlikely. On a team with ten other people, their additive effect needs to be five times the average just to reach break-even point. This is only really likely if the rest of your team is completely useless. In this case it’s unlikely that there’s a good forward path that involves keeping your existing team.

                            1. 2

                              Interesting analysis, but I think it’s missing a couple of points and gets some of the numbers sufficiently wrong as to invalidate the results.

                              You wrote:

                              five times the average just to reach break-even point. This is only really likely if the rest of your team is completely useless

                              Let’s see what the research says:

                              Study after study shows that the very best designers produce structures that are faster, smaller, simpler, cleaner, and produced with less effort. [12] The differences between the great and the average approach an order of magnitude.

                              No Silver Bullet – Fred Brooks

                              Let that sink in: the difference between the best designers and the average approach an order of magnitude. Not the difference between the best and the worst (“completely useless”). That means the outcome you described as spectacularly unlikely is actually spectacularly likely, in fact it is the norm (if we are actually dealing with someone who is brilliant).

                              In fact, it means a single very good designer can completely replace that team of 10 you mentioned, and that’s when doing the simplistic thing of just summing individual output, which we know isn’t true, team productivity significantly lags the sum of individual productivities (see The Mythical Man Month). Making the brilliant designer just 10% more effective would be worth a complete engineering salary, which is why Brooks advocated for “Chief Programmer Teams”, a concept that is entirely ordinary in other fields such as law or medicine.

                              I probably still wouldn’t want to have “Bob” around, because I don’t think the Bob described fits the 10x designer described by Brooks.

                            1. 24

                              Dunno…given the description of actual behavior rather than pejorative value judgements, I would much rather have Alice on my team than people who criticise her, and would find it more likely that they are the jerks (or bullies, to be more precise).

                              In my experience, workplaces that frown on Alices are nice, but not kind, and actually have way more underhandedness and backstabbing.

                              1. 26

                                I think the problem here is isolating people and their behaviour. I enjoy blunt and direct people around me. I even encourage them, it gives me clarity. I even prefer them, I don’t need to decode them. Sometimes, people see me with a person that I enjoy working with and are like “they treat you hard” and I have to tell them that this is how I want it. Fast. Clear. Cut me off if you got my point and agree or disagree. I’m okay with people being grumpy on a day or - given that I run projects of global scope - people just being tired and through that sometimes not take the mental load of choosing every word carefully. I would still expect those people to treat other people differently and not approach me this way right away.

                                The blog post does not talk about sources of Alices behaviour. For example, Alice may take this approach because her feedback is never addressed, which is quite different category of a jerk than someone who just makes that their persona and feels good in it.

                                My problem with niceness is: it’s highly cultural. It’s a category that is immediately exclusive e.g. to foreigners. A lot of people find different cultures behaving “nice” actually odd and weird and are insecure around them. For someone who’s third-language English, it may be a struggle to fulfill those requirements. I’ve definitely seen writing of e.g. people writing in English as a foreign language being framed as jerky just because it was direct and to the point and omitted some amount of fluff. When I approached those people, the reason was simple: they already spent more time than others on formulating their point right, the “niceness” would even incur more cost to them.

                                There are jerks out there. But we need to be careful that perceived jerks also exist.

                                I also agree with your underhandedness and backstabbing reading. “nice” cultures tend to be conflict-avoidant and conflict always finds its way.

                                It’s okay to formulate sharp boundaries around the edges (e.g. using peoples identity to attack their points), but inside of those boundaries, communication is a group exercise and curiosity and willingness to understand other people needs to apply, especially in a more and more interconnected world.

                                1. 7

                                  I think the crux is that there’s an entire “taxonomy of jerks” one could make; Alice doesn’t sound too bad, and Bob sounds like a right cunt. But in between those two there’s a whole lot of other jerks.

                                  Take the “this is one true only correct way to do it, anyone who disagrees is an idiot, and I will keep forcing my views through ad nauseam even when the entire team has already decided against them”-kind of jerk. These are “selfless jerks” in a way, but can be super toxic and demotivating if not dealt with properly by management. These people don’t even need to be abrasive as such (although they can be) and can be very polite, they just keep starting the same tired old discussions all the time.

                                  They’re not like Bob’s “selfish jerk”, and genuinely think they’re doing it in the good for the company.

                                  I once called out such an jerk as an “asshole” in a GitHub comment when he reverted my commit without discussion; that commit fixed some parts of our documentation toolchain. The change was a trivial one: it changed a lower-case letter to a capital one one in a Go import path, which is what we used everywhere else. Perhaps unfortunate that the capital was used, but fixing that would be a lot of work and inconvenience on people’s local dev machines (not all of whom are dedicated Go devs) so after discussing this at length we decided to just stick with the capital.

                                  Yet he sneaked it in anyway, in spite of the previous consensus of just a few weeks prior, while simply calling the toolchain “broken” and reverted my fix (a toolchain I spent quite some time on outside of my regular work duties I might add, and it’s wasn’t broken; Go import paths are just case-sensitive and it didn’t account for two import paths differing only in case, dealing with that is actually an entire can of worms I did already look in to and discussed with him when we talked about changing this company-wide).

                                  From the outset I looked like the “perceived jerk”, as you put it, as I swore at him in frustration while he was perfectly polite, but that ignores all the context that we had already discussed this before, came to an agreement, that he decided to just ignore this, and kept forcing his opinion through, and that this was the umpteenth subject on which this happened. HR didn’t see it that way though, “because it’s inappropriate to call people assholes”. True I suppose, but … yeah. And trust me, “asshole” was the filtered polite response and was subtle compared to what I had typed (bit didn’t send) originally.

                                  It’s almost devious and underhanded in a way, especially the taking offence at the “asshole” part and harping on about that while refusing to discus your own toxic “polite” behaviour. Having to deal with this kind of bullshit for years was a major reason I got burned out at that job, and I’m from the Netherlands which is probably one of the most direct no-nonsense cultures there is.

                                  My point is: politeness is not unimportant, but often overrated, especially in a context where you all know each other and know perfectly well that they’re just the abrasive sort but are basically just decent folk (it’s different on a public forum like Lobsters and such).


                                  Adding to that, I’ve definitely been a “perceived jerk”. Communicating over text is hard; my culture is very direct and also kind of sweary, and even as a skilled non-native speaker at times it can be difficult to fully grasp the subtleties of how something is perceived by native speakers. In all the COVID craze of “remote work is the dog’s bollocks and the future” I think people are forgetting about how hard all of this is, and why I’m skeptical that remote work really is the future (but that’s a different discussion…) It’s probably also a major reason for a lot of Open Source drama.

                                  I’ve had quite a few people tell me “I thought you were a jerk until I met you in person”, a major reason I always went out of my way to meet new people face-to-face for a pint at the pub (not always easy, since we had people world-wide). I tried very hard to change this, and I think I’ve mostly succeeded at it, but it was a lot of active effort that took years.

                                  1. 2

                                    I also fully agree with you there. I have worked as a moderator for years and before I engage e.g. in the whole thing I wrote above I do an assessment whether it’s worth figuring out. Someone jumping on a board, registering an account and blowing off steam with the first comment? => kick But someone going on a long and unfair rant after 12 months of participation? Let’s check out what happened, rather than taking all he said at face value.

                                    While I’d love to give every person the benefit of doubt and a long conversation, the day has 24 hours and this assessment must be done. But usually, if that assessment must be done, we’re talking about 1-2h chatting, which is manageable.

                                    Adding to that, I’ve definitely been a “perceived jerk”. Communicating over text is hard; my culture is very direct and also kind of sweary, and even as a skilled non-native speaker at times it can be difficult to fully grasp the subtleties of how something is perceived by native speakers. In all the COVID craze of “remote work is the dog’s bollocks and the future” I think people are forgetting about how hard all of this is, and why I’m skeptical that remote work really is the future (but that’s a different discussion…) It’s probably also a major reason for a lot of Open Source drama.

                                    I have a bit of a handle on that, interestingly because I got trained in international relationships and marketing. The trick is as easy as it is hard to consistently implement: Always voice your feelings clearly. “I am frustrated that…”. “I am happy that”. Cut tons of slack and assume good faith.

                                    “Lack of training” is also a hard problem in open source.

                                    I’ve had quite a few people tell me “I thought you were a jerk until I met you in person”, a major reason I always went out of my way to meet new people face-to-face for a pint at the pub (not always easy, since we had people world-wide). I tried very hard to change this, and I think I’ve mostly succeeded at it, but it was a lot of active effort that took years.

                                    Same :).

                                    1. 2

                                      I have a bit of a handle on that, interestingly because I got trained in international relationships and marketing. The trick is as easy as it is hard to consistently implement: Always voice your feelings clearly. “I am frustrated that…”. “I am happy that”. Cut tons of slack and assume good faith.

                                      I’ve independently discovered the same trick as well. I also wish people would do this more in political/social justice discussions and the like, e.g. “I feel [X] by [Y] because [Z]” rather than “[X] is [Y]!”, but okay, let’s not side-track this too much 🙃

                                      I wish people would give more feedback on his type of stuff in general. I once heard through a co-worker “hey, Bob has a real problem with you, I don’t know why”, “oh wow, I had no idea”. I followed up on this later by just asking him and it turned it was some really simple stuff like PR review comments “just do [X]?” being taken as passive-aggressive. I can (now) see how it was taken like that, but that wasn’t my intention at all. So I explained that, apologized, slightly adjusted the phrasing of my comments, and we actually became quite good friends after this (still are). But if I hadn’t been told this while drunk at 3am in some pub by another coworker then … yeah.

                                      Granted, not everyone responds well to this kind of feedback; but if you don’t try you’ll never know, and the potential benefits can be huge, especially in the workplace where you’re “stuck” with the same people for hours every day. I can close a website if I don’t like it; bit harder to do that with your job.

                                      I found that communicating with native English speakers in general (much) harder than communicating with other non-native proficient speakers. Maybe we should just go back to Latin so everyone’s a non-native speaker.

                                2. 8

                                  In my experience, workplaces that frown on Alices are nice, but not kind, and actually have way more underhandedness and backstabbing.

                                  Extreme examples of this from the real-world,

                                  The leadership style in Elm is extremely aggressive and authoritarian.

                                  By that I do not mean impolite or rude. It is almost always very civil. But still ultimately aggressive and controlling. — Luke Plant

                                  … once enough people are piling on to complain or tell you what’s wrong with what you did, you’re going to feel attacked - even if every single comment is worded politely and respectfully. — StackExchange

                                  1. 3

                                    Yeah, if someone (or the news) tells me what to think about someone, without telling (or far better SHOWING) me exactly what they did, I tend to just reserve judgement. Even then, it’s easy to make someone look like the jerk by omission of important information.

                                  1. 3

                                    I actually chuckled. This is seriously a self aware wolf moment. This guy is so very, very close to realizing how to fix the problem but is skipping probably the most important step.

                                    He mentioned single-core performance at least 5 times in the article but completely left out multi-core performance. Even the Moto E, the low end phone of 2020, has 8 cores to play with. Granted, some of them are going to be efficiency/low performance cores but 8 cores, nonetheless. Utilize them. WebWorkers exist. Please use them. Here’s a library that makes it really easy to use them as well.

                                    ComLink

                                    Here’s a video that probably not enough people have watched.

                                    The main thread is overworked and underpaid

                                    1. 7

                                      The article claims the main performance cost is in DOM manipulation and Workers do not have access to the DOM.

                                      1. 1

                                        if you’re referring to this:

                                        Browsers and JavaScript engines continue to offload more work from the main thread but the main thread is still where the majority of the execution time is spent. React-based apps are tied to the DOM and only the main thread can touch the DOM therefore React-based apps are tied to single-core performance.

                                        That’s pretty weak. Any javascript application that modifies the DOM is tied to the DOM. It doesn’t mean the logic is tied to the DOM. If it is then at least in react’s case it means that developers thought rendering then re-rendering then rendering again was a good application of user’s computing resources.

                                        I haven’t seen their code and I don’t know what kinds of constraints they’re being forced to program under but react isn’t their bottleneck. Wasteful logic is.

                                        1. 2

                                          The author’s point is that a top of the line iPhone can mask this “wasteful logic”. Unless developers test their websites on other, less expensive, devices they may not realize that they need to implement some of your suggested fixes to achieve acceptable performance.

                                          1. 1

                                            You’re right. I missed the point when I read into how he was framing the problem. Excuse me.

                                      2. 3
                                        1. iPhones also have many cores, so that’s not going to bridge the gap.

                                        2. From TFA: “Browsers and JavaScript engines continue to offload more work from the main thread but the main thread is still where the majority of the execution time is spent.”

                                        3. See also: Amdahl’s Law

                                        1. 1

                                          Gonna fight you on all of these points because they’re a bunch of malarkey.

                                          iPhones also have many cores, so that’s not going to bridge the gap.

                                          If you shift the entire performance window up then everyone benefits.

                                          From TFA: “Browsers and JavaScript engines continue to offload more work from the main thread but the main thread is still where the majority of the execution time is spent.”

                                          This shouldn’t be the case. If it is then people are screwing around and running computations in render() when everything should be handled before that. Async components should alleviate this and react suspense should help a bit this but right now I use Redux Saga to move any significant computation to a webworker. React should only be hit when you’re hydrating and diffing. React is not your bottleneck. If anything it should have a near constant overhead for each operation. You should also note that the exact quote you chose does not mention react but all of javascript. Come on.

                                          See also: Amdahl’s Law

                                          I did. Did you see how much performance you gain by going to 8 identical cores? It’s 6x. Would you consider that to be better than only having 1x performance? I would.

                                          1. 1

                                            Hmm..if you’re going to call what I write “malarky”, it would help if you actually had a point. You do not.

                                            If you shift the entire performance window up then everyone benefits.

                                            Yep, that’s what I said. If everyone benefits, it doesn’t close the gap. You seem to be arguing against something that nobody said.

                                            Amdahl’s law … 8 identical cores? 6x speedup

                                            Er, you seem to not understand Amdahl’s Law, because it is parameterised, and does not yield a number without that parameter, which is the portion of the work that is parallelizable. So saying Amdahl’s law says you get a speedup of 6x from 8 cores is not just wrong, it is non-sensical.

                                            Second, you now write “8 identical cores”. I think we already covered that phones do not have 8 high performance cores, but at most something like 4/4 high/efficiency cores.

                                            Finally, even with an exceedingly rare near perfectly parallelisable talks that kind of speedup compared to a non-parallel implementation is exceedingly rare, because parallelising has overhead and also on a phone other resources such as memory bandwidth typically can’t handle many cores going full tilt.

                                            but the main thread is still where the majority of the execution time is spent

                                            This shouldn’t be the case … React …

                                            The article doesn’t talk about what you think should be the case, but about what is the case, and it’s not exclusively about React.

                                      1. 1

                                        I was just reading K&R C the other day. It seems in the first version C, declarations were optional. Object model in C is surprisingly elegant. If data and methods are separate then they can be evolved separately - only C allows this. On a trivial note, copy paste is better than Inheritance because the copied code can evolve separately instead of changing every time the base class changes.

                                        In terms of generality,

                                        Pointers > Lexical Scope

                                        Function Pointers > Closures, Virtual Methods

                                        Gotos > Exceptions

                                        Arrays, Structs > Objects

                                        Co-routines > Monads

                                        C with namespaces, pattern matching, garbage collection, generics, nested functions and defer is the C++ that I wish had happened. Go is good but I miss the syntax of C. I recently came across Pike scripting language which looks surprisingly clean.

                                        1. 6

                                          It seems in the first version C, declarations were optional.

                                          Yup, which sucked. It combined the lack of compiler checks of a dynamic language, with the data-corruption bugs of native code. For instance, what happens when you pass a long as the third argument, to a function whose implementation takes an int for that parameter? 😱

                                          Object model in C is surprisingly elegant. If data and methods are separate then they can be evolved separately - only C allows this.

                                          Maybe I’m unsure what you’re getting at, but many languages including Objective-C, Swift and Rust allow methods to be declared separately from the data, including adding more methods afterwards, even in separate binaries.

                                          copy paste is better than Inheritance because the copied code can evolve separately instead of changing every time the base class changes.

                                          But it’s worse than inheritance because, when you fix a bug in the copied code, you have to remember to also fix it every place it was pasted. I had a terrible time of this in an earlier job where I maintained a codebase written by an unrepentant copy/paster. This is the kind of nightmare that led to the DRY principle.

                                          1. 2

                                            For instance, what happens when you pass a long as the third argument, to a function whose implementation takes an int for that parameter? 😱

                                            Usually nothing, or rather, exactly what you would want 😀. Last I checked, K&R C requires function parameters to be converted to the largest matching integral type, so long and int get passed the same way. All floating point parameters get passed as double. In fact, I remember when ANSI-C came out that one of the consequences was that you could now have actual float parameters. Pointers are the same size anyway, no struct by value parameters.

                                            It still wasn’t all roses: messing up argument order or forgetting a parameter. Oops. So function prototypes: 👍😎

                                            #include <stdio.h>
                                            
                                            int a( a, b )
                                            int a;
                                            int b;
                                            {
                                                 return a+b;
                                            }
                                            
                                            
                                            int main()
                                            {
                                                long c=12;
                                                int b=3;
                                                printf("%d\n",a(c,b));
                                            }
                                            
                                            [/tmp]cc -Wall hi.c
                                            [/tmp]./a.out 
                                            15
                                            
                                            1. 2

                                              Usually nothing, or rather, exactly what you would want 😀.

                                              Except, of course, when the sizes differed.

                                              1. 1

                                                No. The sizes do differ in the example. Once again: arguments are passed (and received) as the largest matching integral type.

                                                I changed the printf() of the example to show this:

                                                	printf("sizeof int: %ld sizeof long: %ld result: %d\n",sizeof b, sizeof c,a(c,b));
                                                

                                                Result:

                                                 sizeof int: 4 sizeof long: 8 result: 15
                                                
                                              2. 1

                                                A lot of this is assuming arguments passed in registers. Passing on the stack can result in complete nonsense as you could have misaligned the stack, or simply not made a large enough frame.

                                              3. 1

                                                I don’t mean copy paste everything, use functions for DRY ofcourse … just to get the effect of inheritance copy paste is better. Inheritance, far from the notions of biology or taxonomy is similar to a lawyer contract that states all changes of A will be available to B just like land inheritance. Every time some maintainer changes a class in React, Angular, Ruby, Java, C++, Rust, Python frameworks and libraries everyone has to change their code. If for every release of a framework you have to rewrite your entire code, calling that code reuse is wrong and fraudulent. If we add any method, rename any method, change any implementation of any method that is not a trivial fix; we should create a new class instead of asking millions of developers to change their code.

                                                when you fix a bug in the copied code, you have to remember to also fix it every place it was pasted.

                                                If instead we used copy paste, there would be no inheritance hierarchy but just flattened code if that makes sense and you can modify it without affecting other developers. If we want to add new functionality to an existing class we should use something like plugins/delegation/mixins but never modify the base class … but absolutely no one uses or understands this pattern and everyone prefers to diddle with the base class.

                                                In C such massive rewrites won’t happen, because everything is manually wired instead of automatically being inherited. You can always define new methods without bothering if you are breaking someone’s precious interface. You can always nest structs and cast them to reuse code written for the previous struct. Combined with judicious use of function pointers and vtables you will never need to group data and code in classes.

                                                1. 6

                                                  Every time some maintainer changes a class in React, Angular, Ruby, Java, C++, Rust, Python frameworks and libraries everyone has to change their code.

                                                  That is simply not true. There are a lot of changes you can make to a class without requiring changes in subclasses. As a large-scale example, macOS and iOS frameworks (Objective-C and Swift) change in every single OS update, and the Apple engineers are very careful not to make changes that require client code to change, since end users expect that an OS update will not break their apps. This includes changes to classes that are ubiquitously subclassed by apps, like NSView, NSDocument, UIViewController, etc. I could say exactly the same thing about .NET or other Windows system libraries that use OOP.

                                                  I’m sure that in many open source projects the maintainers are sloppy about preserving source compatibility (let alone binary), because their ‘customers’ are themselves developers, so it’s easy to say “it’s easier to change the signature of this method and tell people to update their code”. But that’s more laziness (or “move fast and break stuff”) than a defining feature of inheritance.

                                                  In C such massive rewrites won’t happen

                                                  Yes, because everyone’s terrified of touching the code for fear of breaking stuff. I’ve used code like that.

                                                  1. 1

                                                    That is simply not true.

                                                    How ?

                                                    In C you would just create a new function and rightfully touching working code except for bug fixes is taboo. I can probably point to kernel drivers that use C vtables that haven’t been touched in 10 years. If you want to create an extensible function, use a function pointer. How many times has the sort function been reused ?

                                                    OO programmers claim that the average joe can write reusable code by simply using classes. If even the most well paid, professional programmers can’t write reusable code and writing OO code requires high training then we shouldn’t lie about OO being for the average programmer. Even if you hire highly trained programmers, code reuse is fragile requiring constant vigilance of the base classes and interfaces. Why bother with fragile base classes at all ?

                                                    Technically you can avoid this problem by never touching the base class and always adding new classes and interfaces. I think classes should have a version suffix but I don’t think it will be a popular idea and requires too much discipline. OO programmers on average prefer adding a fly method to a fish class as a quick fix to creating a bird class and thats just a disaster waiting to happen.

                                                    1. 5

                                                      I don’t understand why you posted that link. Apple release notes describe new features, and sometimes deprecations of APIs that they plan to remove in a year or two. They apply only to developers, of course; compiled apps continue to work unchanged.

                                                      OO is not trivial, but it’s much better than resorting to flat procedural APIs. Zillions of developers use it on iOS, Mac, .NET, and other platforms.

                                                      1. 1

                                                        My conclusion - OO is fragile and needs constant rewrites by developers who use OO code and procedural apis are resilient.

                                                        1. 9

                                                          Your conclusion is not supported by evidence. Look at a big, widely used, C library, such as ICU or libavcodec. You will have API deprecations and removals. Both of these projects do it nicely so you have foo2(), foo3() and so on. In OO APIs, the same thing happens, you add new methods and deprecate the old ones over time. For things like glib or gtk, the churn is even more pronounced.

                                                          OO covers a variety of different implementation strategies. C++ is a thin wrapper around C: with the exception of exceptions, everything in C++ can be translated to C (in the case of templates, a lot more C) and so C++ objects are exactly like C structs. If a C/C++ struct is exposed in a header then you can’t add or remove fields without affecting consumers because in both languages a struct can be embedded in another and the size and offsets are compiled into the binary.

                                                          In C, you use the opaque pointers idiom to avoid this. In C++ you use the pImpl pattern, where you have a public class and a pointer to an implementation. Both of these require an extra indirection. You can also avoid this in C++ by making the constructor for your class private and having factory methods. If you do this, then only removing fields modifies your ABI, because nothing outside of your library can allocate it. This lets you put fast paths in the header that directly access fields, without imposing an ABI / API contract that prevents adding fields.

                                                          In C++, virtual methods are looked up by vtable offset, so you can’t remove virtual functions and you can’t add virtual functions if your class is subclassed. You also can’t change the signature of any existing virtual methods. You can; however, add non-virtual methods because these do not take place in dynamic dispatch and so are exactly the same as C functions that take the object pointer as the first parameter.

                                                          In a more rigid discipline, such as COM, the object model doesn’t allow directly exposing fields and freezes interfaces after creation. This is how most OO APIs are exposed on Windows and we (Microsoft) have been able to maintain source and binary compatibility with programs using these APIs for almost three decades.

                                                          In Objective-C, fields (instance variables) are looked up via an indirection layer. Roughly speaking, for each field there’s a global variable that tells you its offset. If you declare a field as having package visibility then the offset variable is not exposed from your library and so can’t be named. Methods are looked up via a dynamic dispatch mechanism that doesn’t use fixed vtable offsets and so you are able to add both fields and methods without changing your downstream ABI. This is also true for anything that uses JIT or install-time compilation (Java, .NET).

                                                          You raise the problem of behaviour being automatically inherited, but this is an issue related to the underlying problem, not with the OO framing. If you are just consuming types from a library then this isn’t an issue. If you are providing types to a library (e.g. a way of representing a string that’s efficient for your use or a new kind of control in a GUI, for example), then the library will need to perform operations on that type. A new version of the library may need to perform more operations on that type. If your code doesn’t provide them, then it needs to provide some kind of default. In C, you’d do this with a struct containing callback function pointers that carried its size (or a version cookie) in the first field, so that you could dispatch to some generic code in your functions if the library consumer didn’t provide an implementation. If you’re writing in an OO language then you’ll just provide a default implementation in the superclass.

                                                          Oh, and you don’t say what kernel you’re referring to. I can point to code in Linux that’s needed to be rewritten between minor revisions of the kernel because a C API changed. I can point to C++ code in the XNU kernel that hasn’t changed since the first macOS release when it was rewritten from Objective-C to C++. Good software engineering is hard. OO is not a magic bullet but going back to ‘70s-style designs doesn’t avoid the problems unless you’re also willing to avoid writing anything beyond the complexity of things that were possible in the ’70s. Software is now a lot more complex than it was back then. The Version 6 UNIX release was only about 83KLoC: individual components of clang are larger than that today.

                                                          1. 0

                                                            Your conclusion is not supported by evidence.

                                                            It absolutely is. Please reuse code from an earlier version of any framework released in the last 50 years. OO was sold as the magic bullet that will solve all reuse and software engineering problems.

                                                            Do you think homeopathy is medicine just because people dress up and play the role of doctors doing science ?

                                                            How many times has the sort function been reused by using function pointers ? Washing machines don’t make clothes dirtier than the clothes you put in.

                                                            Both of these projects do it nicely so you have foo2(), foo3() and so on.

                                                            If they are doing it that way, then thats the way to go. Function signatures are the only stable interface you need. Don’t use fragile interfaces, classes and force developers to rewrite every time a new framework is released because someone renamed a method.

                                                            For the rest of your arguments, why even bother with someone else’s vtables when you can build your own, trivially.

                                                            My point is simply this - How is rewriting code, code reuse ?

                                                            1. 5

                                                              It absolutely is. Please reuse code from an earlier version of any framework released in the last 50 years.

                                                              This is what Windows and Mac OS programmers do every day. My experience with COM is the Windows APIs built on it have great API/ABI stability.

                                                              1. 1

                                                                I don’t know much about COM but if it provides API/ABI stability then that’s great and thats what I am complaining about here. It seems to be an IPC of sorts, how would it compare to REST which can be implemented on top of basic functions ?

                                                                1. 5

                                                                  COM is a language-agnostic ABI. for exposing object oriented interfaces. It has been used to provide stable ABIs for object oriented interfaces for around 30 years to Windows APIs. It is not an IPC mechanism, it is a binary representation. It is a strong counter example to your claim that OO APIs cannot be made stable (and one that I mentioned already in the other thread).

                                                                  1. 4

                                                                    I’m not sure about the IPC parts (there is a degree of “hosting”); however, DCOM provides RPC with COM.

                                                                2. 6

                                                                  It absolutely is. Please reuse code from an earlier version of any framework released in the last 50 years. OO was sold as the magic bullet that will solve all reuse and software engineering problems.

                                                                  I’ve reused code written in C, C++, and Objective-C over multiple decades. Of these, Objective-C is by a very large margin the one that caused the fewest problems. Your argument is ‘OO was oversold, so let’s use the approach that was used back when people found the problems that motivated the introduction of OO’.

                                                                  How many times has the sort function been reused by using function pointers ? Washing machines don’t make clothes dirtier than the clothes you put in.

                                                                  I don’t know what this means. Are you trying to claim that C standard library qsort is the pinnacle of API design? It provides a compare function, but not a swap function so if your structures require any kind of copying between a byte-by-byte copy then it’s a problem. How do you reuse C’s qsort with a data type that isn’t a contiguous buffer? With C++‘s std::sort (which doesn’t use function pointers), you can sort any data structure that supports random access iteration.

                                                                  If they are doing it that way, then thats the way to go. Function signatures are the only stable interface you need.

                                                                  That’s true, if your library is producing types but not consuming them. If code in your library needs to call into code provided by library consumers, then this is not the case. Purely procedural C interfaces are easy to keep backwards compatible if they are not doing very much. The zlib interface, for example, is pretty trivial: consume a buffer, produce a buffer. The more complex a library is, the harder it is to maintain a stable API. OO gives you some tools that help, but it doesn’t solve the problem magically.

                                                                  Don’t use fragile interfaces, classes and force developers to rewrite every time a new framework is released because someone renamed a method.

                                                                  Absolutely none of that is intrinsic to OO. If you rename a C struct field or a function, people will need to rewrite their code. The set of things that you can break without breaking compatibility is strictly larger in an OO language than in a purely procedural language.

                                                                  For the rest of your arguments, why even bother with someone else’s vtables when you can build your own, trivially.

                                                                  Why use any language feature when you can just roll your own in macro assembly?

                                                                  • Compilers are aware of the semantics and so can perform better optimisations.
                                                                  • Compilers are aware of the semantics and so can give better error messages.
                                                                  • Compilers are aware of the semantics and so can do better type checking.
                                                                  • Consistency across implementations: C library X and C library Y use different idioms for vtables (e.g. compare ICU and glib: two completely different vtable models). Library users need to learn each one, increasing their cognitive burden. Any two libraries in the same OO language will use the same dispatch mechanism.

                                                                  My point is simply this - How is rewriting code, code reuse ?

                                                                  Far better in OO languages (and far better in hybrid languages that provide OO and generic abstractions) than in purely procedural ones. This isn’t the ’80s anymore. No one is claiming that OO is a magic bullet that solves all of your problems.

                                                                  1. 1

                                                                    Are you trying to claim that C standard library qsort is the pinnacle of API design?

                                                                    Personal attacks are not welcomed in this forum or any forum. If you can’t use technical arguments to debate you are never going to win.

                                                                    It is an example of code reuse that absolutely doesn’t break.

                                                                    Absolutely none of that is intrinsic to OO. If you rename a C struct field or a function, people will need to rewrite their code.

                                                                    It is absolutely intrinsic to OO because interfaces, classes are multiple level deep. It is a fractal of bad design. Change one thing everything breaks.

                                                                    There is a strong culture of not breaking interfaces in C and using versioning but the opposite is true for OO where changing the base class and interface happens for every release. Do you actually have fun rewriting code between every new release of an MVC framework ?

                                                                    Why use any language feature when you can just roll your own in macro assembly?

                                                                    Again, personal attacks are not welcomed in this forum or any forum.

                                                                    Vtables are trivial. They are not a new feature. All your optimzations can equally apply to vtables.

                                                                    This isn’t the ’80s anymore.

                                                                    Lies don’t become truths just because time has passed.

                                                                    If code in your library needs to call into code provided by library consumers, then this is not the case.

                                                                    Use function pointers to provide hooks or I am missing something.

                                                                    OO is fragile. Procedural code is resilient.

                                                                    1. 6

                                                                      Are you trying to claim that C standard library qsort is the pinnacle of API design?

                                                                      Personal attacks are not welcomed in this forum or any forum. If you can’t use technical arguments to debate you are never going to win.

                                                                      That was not an ad hominem, that was an attempt to clarify your claims. It was unclear what you were claiming with references to a sort function. An ad hominem attack looks more like this:

                                                                      Do you think homeopathy is medicine just because people dress up and play the role of doctors doing science ?

                                                                      This is an ad hominem attack and one that I ignored when you made it, because I’m attempting to have a discussion on technical aspects.

                                                                      It is an example of code reuse that absolutely doesn’t break.

                                                                      It’s also an example of an interface with trivial semantics (it’s covered in the first term of most undergraduate computer science course) and whose requirements have been stable for longer than C has been around. The C++ std::sort template is also stable and defaults to using OO interfaces for defining the comparison (overloads of the compare operators). The Objective-C -sort family of methods on the standard collection classes are also unchanged since they were standardised in 1992. The Smalltalk equivalents have remained stable since 1980.

                                                                      You have successfully demonstrated that it’s possible to write stable APIs in situations where the requirements are stable. That’s orthogonal to OO vs procedural. If you want to produce a compelling example, please present something where a C library has changed the semantics of how it interacts with a type provided by the library consumer (for example a plug-in filter to a video processing library, a custom view in a GUI, or similar) and an OO library making the same change has required more code modification.

                                                                      Absolutely none of that is intrinsic to OO. If you rename a C struct field or a function, people will need to rewrite their code.

                                                                      It is absolutely intrinsic to OO because interfaces, classes are multiple level deep. It is a fractal of bad design. Change one thing everything breaks.

                                                                      This is an assertion, but it is not supported by evidence. I have provided examples of the same kinds of breaking changes being required in widely used C libraries that do non-trivial things. You have made a few claims here:

                                                                      • Something about interfaces. I’m not sure what this is, but COM objects are defined in terms of interfaces and Microsoft is still able to support the same interfaces in 2021 that we were shipping for Windows 3.1 (though since we no longer support 16-bit binaries these required a recompile at some point between 1995 and now).
                                                                      • Classes are multiple levels deep. This is something that OO enables, but not something that it requires. The original GoF design patterns book recommended favouring composition over inheritance and some OO languages don’t even support inheritance. Most modern C++ style guides favour composition with templates over inheritance. Inheritance is useful when you want to define a subtype relationship with code reuse.
                                                                      • Something (OO in general? A specific set of OO patterns? Some OO library that you don’t like?) is a fractal of bad design. This is an emotive and subjective claim, not one that you have supported. Compare your posts with the article that I believe coined that phrase: It contains dozens of examples of features in PHP that compose poorly.

                                                                      There is a strong culture of not breaking interfaces in C and using versioning but the opposite is true for OO where changing the base class and interface happens for every release. Do you actually have fun rewriting code between every new release of an MVC framework ?

                                                                      You’re comparing culture, not language features. You can write code today against the OpenStep specification from 1992 that will compile and run fine on modern macOS with Cocoa (I know of some code that has been through this process). That’s an OO MVC API that’s retained source compatibility for almost 30 years. The only breaking changes were the switch from int to NSInteger for better support for 64/32-bit compatibility and these changes also affected the purely procedural APIs. They were not breaking changes for code targeting 32-bit platforms. The changes over the ’90s in the Classic MacOS Toolbox (C APIs) were far more invasive.

                                                                      A lot of JavaScript frameworks and pretty much everything from Google make breaking API changes every few months but that’s an issue of developer culture, not one of the language abstractions.

                                                                      Why use any language feature when you can just roll your own in macro assembly?

                                                                      Again, personal attacks are not welcomed in this forum or any forum.

                                                                      This is not a personal attack. It is your point. You are saying that you should not use a feature of a language because you can implement it in a lower-level language. Why stop at vtables?

                                                                      Vtables are trivial. They are not a new feature. All your optimzations can equally apply to vtables.

                                                                      No they can’t. It is undefined behaviour to write to the vtable pointer in a C++ object for the lifetime of an object. Modern C++ compilers use this optimisation for devirtualisation. If the concrete type of a C++ object is known at compile time (after inlining) then calls to virtual functions can be replaced with direct calls.

                                                                      Here is a reduced example. The C version with custom vtables is called in the function can_not_inline the C++ version using C++ vtables is called in the function can_inline. In both cases, the object is passed to a function that the compiler can’t see before the call. In the C case, the language semantics allow this to modify the vtable pointer, in the C++ case they do not. This means that the C++ version knows that the foo call has a specific target, the C version must be conservative. The C++ version can then inline the call, which doesn’t do anything in this trivial example and so elides it completely.

                                                                      This isn’t the ’80s anymore.

                                                                      Lies don’t become truths just because time has passed.

                                                                      No, but claims that were believed to be true and were debunked are no longer claimed. In the ’80s, OO was claimed to be a panacea that solved all problems. That turned out to be untrue. Like many other things in programming, it is a set of useful tools that can be applied to make things better or worse.

                                                                      If code in your library needs to call into code provided by library consumers, then this is not the case.

                                                                      Use function pointers to provide hooks or I am missing something.

                                                                      You are missing a lot of detail. Yes, you can provide function pointers as hooks. Now what happens when a new version of your library needs to add a new hook? What happens when that hook interacts in subtle ways with the others? These are the kinds of problems that make OO APIs fragile, but they also make procedural APIs fragile.

                                                                      OO is fragile. Procedural code is resilient.

                                                                      Assertions are not evidence. Assertions that contradict the experience of folks who have been working with these APIs for decades need strong evidence.

                                                                      1. 0

                                                                        The only breaking changes were the switch from int to NSInteger for better support for 64/32-bit compatibility and these changes also affected the purely procedural APIs.

                                                                        And that doesn’t count as evidence. Please read what I wrote. OO programmers constantly rename things to break backwards compatibility for no good reason at all. Code rewrite is not code reuse, by definition. Do C programmers do this ?

                                                                        We are discussing how C does things and maintains backwards compatibility not COM. You say COM and I say POSIX / libc which is older. The fact that you cite COM is in-itself proof that objects are insufficient.

                                                                        In Python3 … print was made into a function and almost overnight 100% code was made useless. This is the daily life of OO programmers for the release of every major version of a framework.

                                                                        In database how many times do you change the schema ? Well structs and classes are like schema. Inheritance changes the schema. Interface renames change the schema. Changing method names is like changing the column name. Just like in database design you should not change the schema but use foreign keys to extend the tables with additional data. Perhaps OO needs a new “View” layer like SQL.

                                                                        No, but claims that were believed to be true and were debunked are no longer claimed …. like many other things in programming, it is a set of useful tools that can be applied to make things better or worse.

                                                                        The keyword is “debunked” like snake oil.

                                                                        I propose mandatory version suffix for all classes to avoid this. The compiler creates a new class for every change made to a class, no matter how small. If you are changing the class substantially create a completely new name, don’t ship it by the same name and break all code. For ABI do something like COM if that worked.

                                                                        These are the kinds of problems that make OO APIs fragile, but they also make procedural APIs fragile.

                                                                        You are right. They make procedural APIs using vtables fragile, not to mention slow. So use it sparingly ? 99% of code should be procedural. I only see vtables being useful in creating bags of event handlers.

                                                                        1. 7

                                                                          The only breaking changes were the switch from int to NSInteger for better support for 64/32-bit compatibility and these changes also affected the purely procedural APIs.

                                                                          And that doesn’t count as evidence. Please read what I wrote. OO programmers constantly rename things to break backwards compatibility for no good reason at all. Code rewrite is not code reuse, by definition. Do C programmers do this ?

                                                                          You’ve now changed your argument. You were saying that OO is fragile, now you’re saying that OO programmers (which OO programmers?) rename things and that breaks things. Okay, but if procedural programmers rename things that also breaks things. So now you’re not talking about OO in general, you’re talking about some specific examples of OO (but you’re not naming them). You’ve been given examples of widely used rich OO APIs that have retained huge degrees of backwards compatibility, so your argument seems now to be nothing to do with OO in general but an attack on some unspecified people that you don’t like who write bad code.

                                                                          We are discussing how C does things and maintains backwards compatibility not COM. You say COM and I say POSIX / libc which is older. The fact that you cite COM is in-itself proof that objects are insufficient.

                                                                          Huh? COM is a standard for representing objects that can be shared across different languages. I also cited OpenStep / Cocoa (the latter is an implementation of the former), which uses the Objective-C object model.

                                                                          POSIX provides a much simpler set of abstractions than either of these. If you want to compare something equivalent, how about GTK? It’s a C library that’s a bit newer than POSIX but that lets you do roughly the same set of things as OpenStep. How many GTK applications from even 10 years ago work with a modern version of GTK without modification? GTK 1 to GTK 2 and GTK 2 to GTK 3 both introduced significant backwards compatibility breaks.

                                                                          In Python3 … print was made into a function and almost overnight 100% code was made useless. This is the daily life of OO programmers for the release of every major version of a framework.

                                                                          Wait, so your argument is that a procedural API, in a multi-paradigm language changed, which broke everything, and that’s a reason why OO is bad?

                                                                          In database how many times do you change the schema ? Well structs and classes are like schema. Inheritance changes the schema. Interface renames change the schema. Changing method names is like changing the column name. Just like in database design you should not change the schema but use foreign keys to extend the tables with additional data. Perhaps OO needs a new “View” layer like SQL.

                                                                          I don’t even know where to go with that. OO provides away of expressing the schema. The schema doesn’t change because of OO, the schema changes because the requirements change. OO provides mechanisms for constraining the impact of that change.

                                                                          Again, your argument seems to be:

                                                                          1. There exists a set of things in OO that, if modified, break backwards compatibility.
                                                                          2. People who write OO code will change these things
                                                                          3. OO is bad.

                                                                          But it’s also possible to say the same thing with OO replaced with procedural, functional, generic, or any other style of programming. If you want to make this point convincingly then you need to demonstrate that the set of things that break backwards compatibility in OO are more likely to be changed than in another style. So far, you have made a lot of assertions, but where I have presented examples of OO APIs with a long history of backwards compatibility and procedural APIs performing equivalent things with weaker guarantees, you have failed to present any examples.

                                                                          I propose mandatory version suffix for all classes to avoid this.

                                                                          So, like COM?

                                                                          The compiler creates a new class for every change made to a class, no matter how small. If you are changing the class substantially create a completely new name, don’t ship it by the same name and break all code.

                                                                          So, like COM?

                                                                          For ABI do something like COM if that worked.

                                                                          So, you want COM? But you want COM without OO? In spite of the fact that COM is an OO standard?

                                                                          These are the kinds of problems that make OO APIs fragile, but they also make procedural APIs fragile.

                                                                          You are right. They make procedural APIs using vtables fragile, not to mention slow. So use it sparingly ? 99% of code should be procedural. I only see vtables being useful in creating bags of event handlers.

                                                                          It’s not just about vtables, it’s about any kind of rich abstraction that introduces coupling between the producers and consumers of an interface.

                                                                          Let’s go back to the C sort function that you liked. There’s a C standard qsort. Let’s say you want to sort an array of strings by their locale-aware order. It has a callback, so you can define a comparison function. Now you want to sort an array that has an external indexing structure for quickly finding the first entry with a particular prefix. Ooops, qsort doesn’t have any kind of hook for defining how to do the move or for receiving a notification when things are moved, so you can’t keep the data structure up to date, you need to recalculate it after the sort. After a while, you realise that resizing the array is expensive and so you replace it with a skip list. Oh dear, qsort can’t sort anything other than an array, so you now have to implement your own sorting function.

                                                                          Compare that to C++‘s std::sort. It is given two random-access iterators. These are objects that define how to access the start and end of some collection. If I need to update some other data structure when entries in the list move, then I overload their copy or move constructors to do this. The iterators know how to move through the collection, so when I move to a skip list I don’t even have to modify the call to std::stort, I just modify the begin() and end() methods on my data structure.

                                                                          I am lazy. I regularly work on projects with millions of lines of code. I want to write the smallest amount of code possible to achieve my goal and I want to have to modify the smallest amount of code when the requirements change. Object orientation gives me some great tools for this. So does generic programming. Pure procedural programming would make my life much harder and I don’t like inflicting pain on myself, so I avoid it where possible.

                                                                          1. 5

                                                                            You have the patience of a saint to continue arguing with this person as they continue to disregard your experience. I certainly don’t have the stamina for it, but despite the bizarreness of the slapfight, your replies are really insightful when it comes to API design.

                                                                            1. 3

                                                                              I had a lot of the same misconceptions (and complete conviction that I was right) in my early ‘20s, and I am very grateful to the folks who had the patience to educate me. In hindsight, I’m astonished that they put up with me.

                                                                            2. 1

                                                                              This page lists all the changes in Objective C since the last 10 years. Plenty of renames.

                                                                              I think more languages could benefit from COM’s techniques but I don’t think it is a part of the C++ core. I would use a minimal and flexible version of it but it seems to be doing way too many Win32 specific things.

                                                                              1. 4

                                                                                As @david_chisnall has pointed out many times already, this has nothing to do with OO. GTK has exhibited the exact same thing. GCC has done something similar with its internals. Renaming things such that code that relies on the API having to change has nothing at all to do with any specific programming paradigm.

                                                                                Please stop your screed on this topic. It’s pretty clear from the discussion you are not grasping what is being said. I urge you to spend some time and study the replies above.

                                                                                1. 1

                                                                                  Fine. I would compare GUI development with Tk which is more idiomatic in C.

                                                                                  As I have pointed out if people used versioning for interfaces things won’t break every time an architecture astronaut or an undisciplined programmer change a name, amplifying code rewrites. It is clear that the problem applies to vtables as well and naming in general and not solved within OO which exasperates the effects of simple changes.

                                                                3. 6

                                                                  You can conclude whatever you like, but after taking a look at your blog, I’m going to back away slowly from this discussion and find a better use for my time. Best of luck with your jihad.

                                                                  1. 2

                                                                    Glad you discovered my blog. I’d recommend you start with Simula the Misunderstood. The language is a bit coarse though. The entire discussion has however inspired me to write - Interfaces a fractal of bad design. I see myself more like James Randi exposing homeopathy, superstitions, faith healers and fortune telling.

                                                    1. 7

                                                      One thing that really bugs me is how “dreaded” is defined in the survey: “languages that had a high percentage of developers who are currently using them, but have no interest in continuing to do so”.

                                                      Those two are not the same thing at all. For example, Apple has made clear that Swift is the future and that Objective-C is on the way out, so developers are somewhat obviously interested in learning/using the new thing and anticipate stopping to use the old thing. Does that mean they “dread” Objective-C? They might, but the answer to that question does not indicate either way.

                                                      1. 4

                                                        Apple have never stated anything like that. Apple’s internal use of ObjC remains extremely high, though some of that has to do with legacy factors and the fact that Swift didn’t become ABI stable until two years ago.

                                                        I think Swift’s position as an outlier here somewhat disproves the author’s thesis that all languages must pass through a “honeymoon” phase only to inevitably end up in “dreaded” territory.

                                                      1. 4

                                                        Ranged numeric types, especially combined with linear types (as per my understanding; not a PLT expert) are a pretty awesome feature, it’s encouraging to see the idea being put into practical use.

                                                        Unfortunately, the language seems to target an extremely narrow niche. I get that it’s a Google-internal project, and that niche is probably worth a cool $100M+ for them so there’s no pressure to expand its scope.

                                                        Things that look like they’re dealbreakers for just about anything I’d like to use this for:

                                                        • It looks like you’re only supposed to implement “leaf” libraries with it. It doesn’t look like there’s any kind of FFI, so if what I want to implement would require internally calling down to a C API for another (system) library, I’d effectively have to extend the language or standard library.
                                                        • Memory safety is primarily achieved by not supporting dynamic memory allocations. It’d be a different story if combined with something like Rust’s borrow checker. I mean the support for linear types is already there to some extent…

                                                        On the other hand, the ability to compile down to C is a bonus compared to Rust.

                                                        I guess if you fix those 2 aspects I mentioned you probably end up with ATS anyway… and I have to admit I haven’t quite wrapped my head around that one yet.

                                                        1. 1

                                                          Ranged numeric types…it’s encouraging to see the idea being put into practical use.

                                                          Pascal had subrange types in 1970.

                                                          1. 1

                                                            It’s been a few decades since I worked with Pascal, but I’m fairly sure Pascal’s support wasn’t as comprehensive as this is. Without linear types, they’re fairly cumbersome to use comprehensively. Specifically, I’m talking about:

                                                            if (a < 200)
                                                            {
                                                               // inside here, a's type is automatically restricted with an upper bound of 200 exclusive
                                                            }
                                                            

                                                            I don’t think Pascal supported this?

                                                        1. 9

                                                          When you get tied up in developing software for testing, and in design patterns and best practice, all your enthusiasm is lost and you loose precious energy that could have been spend creatively coming up with better and more efficient ways to deal with the problems you’re facing.

                                                          Couldn’t disagree more, and I wonder if the original poster is doing test-after or test-first. If you’re bing “frantic” about tests, you’re probably doing it wrong:

                                                          The attitude I see is that testing is like eating your vegetables, you know it’s supposed to be good for you and you do it, grudgingly, but it really is rather annoying and the benefits are more something you know intellectually.

                                                          For me with MPWTest, TDD is also still intellectually a Good Thing™, but also viscerally fun, less like vegetables and more like tasty snacks, except that those snacks are not just yummy, but also healthy. It helps me stay in the flow and get things done.

                                                          From: MPWTest: Reducing Test Friction by Going Beyond the xUnit Model

                                                          Back to the original post:

                                                          The method I use is a rigorously usage of inserted breaks, where you halt the application at specific places during runtime and then check that the values and data is correct, all the time.

                                                          Please…don’t. There’s nothing rigorous about this. And it’s manual. What do programmers do with manual tasks? They automate them. And it’s only done during original development. What happens during refactoring? How do I even know what values are correct?

                                                          Really, please write a test instead. And write it first, as specification. Make sure the test is red. Then write just enough code to make the test pass. Feel free to check in at this point. Then refactor, keeping the test green. It’s a fantastic flow.

                                                          1. 8

                                                            Please…don’t. There’s nothing rigorous about this. And it’s manual. What do programmers do with manual tasks? They automate them. And it’s only done during original development. What happens during refactoring? How do I even know what values are correct?

                                                            Really, please write a test instead. And write it first, as specification. Make sure the test is red. Then write just enough code to make the test pass. Feel free to check in at this point. Then refactor, keeping the test green. It’s a fantastic flow.

                                                            Or if you don’t want to do TDD, replace as much as possible your breaks with assertions checking your invariants instead of losing time and energy doing it yourself.

                                                          1. 3

                                                            The “right” thing to do is to spend time un-tangling your changes, staging them in groups and committing with well thought-out commit messages

                                                            Why?

                                                            git commit -a -m "<describe your changes>"

                                                            There is this assumed notion, never much justified, that commit histories must be a pristine sequence of logically consistent additions.

                                                            And yes, it would be slightly nicer if they looked like that. But the actual benefits of such a history are at best marginal, because commit histories just aren’t looked at that much. Or at least shouldn’t: if you have an intense relationship with your or your colleague’s commit histories, you need to look at your development practices. IMHO, they’re actually harmful.

                                                            Because development doesn’t actually work like that, and logs should track what actually happened, with as little filtering as possible, rather than what you (retroactively) wanted to happen. Just imagine tracking a bug in a program using logs and the program edited those logs retroactively to reflect what it thought should have happened. It would be impossible.

                                                            And in addition to being directly harmful, you also need to expend effort creating them, there is a significant chance of inconsistent commits entering the repository, and you are incentivised to adjust your actual development to reflect this unrealistic model (in order to avoid the extra work) rather then letting it flow.

                                                            Just say no.

                                                            1. 3

                                                              Yes, one of the many things I talk about in performance book is that this trend is pretty much inevitable:

                                                              Hard disk capacity is determined by their area. Access speed is linear. So if you improve the (linear) density two-fold, meaning you’ve doubled throughput, well congratulations, you’ve quadrupled capacity!

                                                              So our ability to actually get at the data on our hard disks has inevitably shrunk and shrunk, and will continue to do so. (Access times are controlled by physics, how fast you can start/stop a mass. Physics don’t change much, though the heads get a little lighter with time. The faster you start/stop them, the noisier they get, again, physics, so wanting quieter HDs works against them getting faster).

                                                              And of course SSD are also something with an area, and the access to a particular block is serial. Same overall issue, minus starting/stopping the read head. And RAM as well.

                                                              1. 2

                                                                This is going to seem like a bit if a weird question but I’ve been wondering this for a while after going toe deep into a bunch of different programming languages: Do they not all seem pretty much the same aside from the syntax? Sometimes not even that. J is cool because it’s lists n stuff but it’s still a similar logic behind it.

                                                                I mean, after a certain point everything seems to boil down into modify this, assign this to that, store this data here, send this data there, iterate, traverse, wrap this function/value, find this pattern and text replace then evaluate, etc, etc. I’ve looked into a couple of esoteric languages like brainfuck and INTERCAL but they’re not appreciably different either. I used to get my mind blown by the way some languages do things but now after looking at older programs, libraries and books that have already been written everything seems to be a rehash or just a reconfiguration of what already exists. It feels more like looking at a bunch of hammers than a bunch of toys.

                                                                I don’t really know where I’m going with this…

                                                                1. 4

                                                                  A few things:

                                                                  1. In the static realm, different languages’ type systems make fundamentally different guarantees. If your language’s type system makes more guarantees than another language’s, you can have more confidence in the robustness and correctness of programs written in that language.

                                                                  2. It’s not what the language allow you to do, it’s what they push you towards. Let’s say that, for some problem, there is a robust solution and a hacky solution. If a language makes hacky solutions easier to write, then programs written in that language are unlikely to be robust. This is somewhat related to, but yet distinct from, #1.
                                                                    An unlikely exemplar of this is perl. Despite its reputation for unmaintainability, I find programs written in perl tend to be rather robust and well-made. Partly, this is due its tradition of modularity and stability (in keeping with unix). But partly it’s because, since it’s so expressive, the error handling code you could have just skipped is not actually that hard to write, so you might as well write it.

                                                                  3. Related is what the ecosystem does for you. Perl has an excellent, robust package repository and a culture of writing high quality packages. Java has really high quality autocompletion. Common lisp’s dynamism (a language property) is taken advantage of by REPLs (not a language property), enabling an interactive development style that wouldn’t otherwise be possible.

                                                                  4. You may be selling syntax short.

                                                                  1. 2

                                                                    “What we currently call general purpose languages are actually domain specific languages for the domain of algorithms.” – Objective-S

                                                                    1. 1

                                                                      Kind of, ultimately we need to think about the hardware we’re running on so we end up with languages that are similar since we care (at least a bit) about execution speed.

                                                                      Most languages, C, Python, Assembly, Java, Go, Rust, etc, are based on playing around with register machines.

                                                                      Other paradigms give you different flavours.

                                                                      The ML family of languages give you a more usable lambda calculus which does not feel anything like a register machine.

                                                                      The Lisp family is based around the idea that since every expression is ultimately a tree we might as well get rid of the sugar and see what happens, then liberally salt it with a whole bunch of other ideas depending on the version.

                                                                      Forth et al. are based on the insane idea that since a two stack pushdown automaton is equivalent to a Turing machine we might as well program on one.

                                                                      The APL family is similar to register machines but you assume that most of the work will be done on long stretches of contiguous memory containing the same type, and that you can pass these arrays between as many functions as you could possibly want.

                                                                      Logical languages like (a subset of) SQL and Prolog are yet another way to program by giving constraints and letting the computer sort it out.

                                                                      Distributed languages are more about the communication between functions/machines and assuming that things will go wrong and messages lost. I’ve only used Erlang from this group.

                                                                      Term rewriting is another interesting paradigm that’s not at all popular, the wolfram language is the only one I know of, but I frequently write dsl based around these for maths work.

                                                                      There’s more but it gets difficult to decide what’s what since no language has only one idea.

                                                                      1. 1

                                                                        Term rewriting is another interesting paradigm that’s not at all popular, the wolfram language is the only one I know of, but I frequently write dsl based around these for maths work.

                                                                        Is there a rewrite engine you use regularly? Like Maude?

                                                                        1. 1

                                                                          No, I write most of them in Guile. It’s quite straight forward and I get access to a first class scripting language for GNU utils. The fact that the term rewriting system is also s-expression based means I can evaluate the results of whatever I do as Scheme programs when whatever transformations I want are done. Something that most other systems can’t actually do.

                                                                      2. 1

                                                                        What’s an idea to you that feels different? Not asking for a coherent, fleshed out thought here, just the vague sense in your head that’s driving some of this.

                                                                        Worth noting that BF and IC aren’t particularly esoteric as languages go, I’ll share some more interesting ones when it’s not 1 AM here

                                                                        1. 1

                                                                          I don’t think this is a weird question. I’ll be honest, I find most languages quite similar and quite quotidian. J and APL are probably some of the few languages I find fairly “different” from others (try building a scheduler in APL or J ;). I maintain that algorithms and data structures almost always end up much more important than a programming language. I also find both much more interesting than PL (or at least the treatment it gets here, which feels superficial.) I think the invite tree has just hit a local maximum of folks who are very interested in programming languages, so we get mostly that. I’m trying to change this by posting non-PL posts, but my language posts are much more popular than anything else I post so shrug

                                                                          1. 1

                                                                            try building a scheduler in APL

                                                                            Honestly, that sounds like a pretty fun exercise. People have done ‘real’ apps before, probably most notable of which is aaron hsu’s compiler.

                                                                            What’s hard is actually performing the context switch, but you can’t do that without help from assembly in c, either.

                                                                            1. 1

                                                                              I did it actually, but not as a first-time thing. My first attempt at anything like this was a maze solver in J. Now that was a mind-bender that’s for sure. For me, it hit the same spot as a good math puzzle does. I highly recommend the exercise, it’s tough but fun!

                                                                        1. 1

                                                                          An unsolvable problem! I wonder what happened since then! /s

                                                                          1. 2

                                                                            I see your /s, but to quote from the Go generics proposal:

                                                                            We believe that this design permits different implementation choices. Code may be compiled separately for each set of type arguments, or it may be compiled as though each type argument is handled similarly to an interface type with method calls, or there may be some combination of the two.

                                                                            In other words, this design permits people to stop choosing slow programmers, and permits the implementation to decide between slow compilers (compile each set of type arguments separately) or slow execution times (use method calls for each operation on a value of a type argument).

                                                                            1. 2

                                                                              Rust chose a similar path, where you can have monomorphized generic calls or use trait objects for dynamic dispatch, and it’s clear in the code which one you’re using.

                                                                              1. 0

                                                                                Let’s be realistic: The specialization path will win, because the Go compiler isn’t that fast to begin with (and everyone seems to be fine with that), and for generic methods (which are currently left out) you will need it anyway.

                                                                                1. 3

                                                                                  How is the Go compiler not fast? It is faster than Haskell, OCaml, Rust, C++? What are you comparing it to?

                                                                                  1. 1

                                                                                    [Genuine question, not sarcastic or in point-scoring mode:] Is it fast because it’s faster than those at the same tasks, or because it does so much less? I would assume that a large part of why it’s so much faster is because it’s just doing so many fewer things, but I would also be willing to believe that there are things that would let it be faster if it were doing the same things, if someone made a case for why that would be so.

                                                                                    1. 0

                                                                                      D was a magnitude faster a while back. And that’s with putting all things in favor of Go:

                                                                                      • Before the Go compiler was migrated to Go (and got slower).
                                                                                      • Assuming that 1 line of Go is equivalent to 1 line of D (which is silly).
                                                                                      • Pretending the emitted code is of comparable quality (it isn’t).
                                                                                      1. 0

                                                                                        So was Turbo Pascal, so is tcc, so is Oberon, …

                                                                                        But just because there are things that are faster doesn’t mean that something isn’t fast. A Tesla Roadster is generally considered pretty fast. New Horizons is a lot faster, but does that mean the Tesla Roadster is slow?

                                                                                        In both the cohort of compiled languages that are currently popular (and it is) and the cohort of languages that came out around that time, the go compiler ranks among the fastest.

                                                                                        1. 1

                                                                                          I think the important point is that the balance between “Go people claiming things” and “Go people actually delivering” is out of whack when it comes to “Go compiles fast”.

                                                                                          both the cohort of compiled languages that are currently popular (and it is)

                                                                                          Given that logic, McDonalds must have the best food ever invented!

                                                                                          the cohort of languages that came out around that time

                                                                                          Find me a language from the 70ies that compiles slower.

                                                                              1. 15

                                                                                This matches my experience.

                                                                                The promise was a language without non-ref-counting GC, with straight-forward async support, and that hasn’t been delivered. I mean it’s fine so long as you don’t use traits, which is like saying C is fine if you don’t use pointers.

                                                                                1. 12

                                                                                  Understanding ownership is the price of admission.

                                                                                  For users who are used to GC languages it is admittedly very hard to switch the mental model. It’s a common mistake to assume Rust’s references are a substitute for pointers/passing-by-reference semantics, and makes people completely stuck with what seems like a language that is absolutely impossible to use.

                                                                                  But once you “get” single ownership and moves, it’s really not a big deal.

                                                                                  1. 10

                                                                                    AFAICT people who “fight” the borrow checker want to use refs for everything. If you default to move and clone when you need something in two places this gets you 80%+ of the way there IME. Then treat refs or fancy smart pointer types as optimizations or for advanced cases. Over time one gets comfortable using refs for short-lived cases and such.

                                                                                    1. 1

                                                                                      Maybe those people are right? As in: refs are good enough for 80-90% of the cases and you reserve the move/clone and other fancy mechanisms for advanced cases.

                                                                                      1. 8

                                                                                        No, they really aren’t. Rust refs are temporary* scope-bound borrows. If something can’t be guaranteed to be used only temporarily, they’re not good for it. If something can’t be bound to a single scope, they’re not good for it. If something doesn’t already have an owning binding, they’re not good for it.

                                                                                        * except &'static, but that is effectively a memory leak, so it’s not a solution in general.

                                                                                        In GC languages references are ownership-agnostic and their lifetime can be extended, which makes them suitable for everything. But in the land of Rust, references are very specific tool with narrow use-cases.

                                                                                        If you find yourself fighting with the borrow checker: use references only for function arguments, and nothing else. That’s a 90% accurate guidance.

                                                                                        1. 1

                                                                                          I guess I should have been less terse, apologies.

                                                                                          This is obviously wrong within the constraints of the current Rust design.

                                                                                          My point was that maybe those people are right and this aspect of Rust’s design is wrong.

                                                                                        2. 5

                                                                                          That is exactly the recipe for “fighting the borrow checker”. Move is what you want more thn half the time. If you use move and clone when you can’t the main case you can’t cover are shared mutability (which you usually want to avoid anyway). Can go very far this way.

                                                                                      2. 1

                                                                                        Understanding ownership is fine but the issue is around the way that async generates types that can’t be typed. The async_trait hack is a confusing, imperfect hack but impossible to live without in an async rust world.

                                                                                    1. 12

                                                                                      Some context for Apple tech newcomers:

                                                                                      A pillar of Swift’s initial uptake was smooth interop with Objective-C, in which all of Apple’s application frameworks were written (most still are). “NSArray” and similarly-prefixed symbols are Objective-C classes from the Foundation framework — more or less ObjC’s standard library. In Objective-C, classes are reference types, and dispatch is always dynamic. Swift is performance-focused and has value type structs and enums, plus reference type classes. All of the above can have methods, and Swift will compile static, vtable, or dynamic dispatch depending on runtime possibilities.

                                                                                      Usually, only classes can be compatible with Objective-C for interop. But as distinct from Objective-C’s NSArray class, Swift arrays are structs. The same is true for other basic types like other collections and the string type. So for the standard library types, there has always had to be a plan for how that bridging between statically-called value types and dynamically-called classes can happen without developer effort. That’s goal 2 in this article. The underscore-prefixed symbols used to accomplish that are the standard library implementation, not something an app developer uses.

                                                                                      On Linux and Windows, where Objective-C was never part of the baseline, all the Objective-C interop logic is simply omitted by design. In the far future that may be true on Apple platforms as well. I believe Swift has C interop on any platform regardless of Objective-C.

                                                                                      The ArrayTypes have full value semantics via copy-on-write (COW)

                                                                                      “Value semantics” in the Swift world is as opposed to “reference semantics”. It just means “does it act like a value type or act like a reference type,” under situations like passing it to a function or saving it to a new variable. A class object is a reference, so you expect it to copy the reference and share the object. A struct is a value, so you expect it to make a unique copy, or at least behave like it did. But to have both growable arrays and known allocation sizes to hold them, a collection must back its storage with a reference. When a struct contains a reference, by default that will just get copied, which would share array storage and break value semantics. In order to let programmers think about arrays like they are values, the data should be copied. And to preserve performance under that constraint, it will only Copy On Write: When an Array gets a call that would mutate, it checks whether its backing store has only one owner. If not, it makes its own copy first. Then the mutation proceeds. Making a COW type is something a Swift programmer might have to do once in a blue moon, but usually the standard library has what you need.

                                                                                      This is a pretty good example of the compiler and standard library doing an awful lot for you, while not abstracting it so far away that you never learn it. A design principle of Swift is that you don’t have to keep Objective-C bridging or COW types or ARC memory management top of mind every day, but that you should understand them and have to think about them sometimes.

                                                                                      1. 5

                                                                                        Arrays being COW sounds like awfully dangerous performance footgun. Though considering that the language is largely used for macOS/iOS apps and nothing critical, it’s probably fine.

                                                                                        1. 4

                                                                                          It doesn’t cater to anything system-level yet, that’s right. The current intended use case is application-level stuff, they’re working on server stuff, and system level is an eventual goal. There are unimplemented plans for opt-in Rust-like ownership, for instance.

                                                                                          As for footgun severity, well, it does hide something from you in the sense that copies happen on some but not all mutations. But this copy is shallow, not deep, and the idiomatic norm is for everything to be a constant and not a variable, which prevents mutations and therefore COW operations. At the application level the iOS / Mac dev community has been pleased with the performance, and we’ll see how it goes as the language spreads to more performance-critical use cases.

                                                                                          1. 3

                                                                                            At the application level the iOS / Mac dev community has been pleased with the performance

                                                                                            Only insofar as they haven’t actually measured it.

                                                                                            For example, I have seen several GitHub repos stating “we use Swift Codable, so it is fast”.

                                                                                            Swift Codable is horrendously slow at around 10MB/s for encoding/decoding JSON on a fairly top-of-the-line 13” MBP/Intel. To compare, the actual SSD does 2 GB/s, and so is 200 times faster.

                                                                                            I had an article series showing how to get around 20-30x faster speed using Objective-C.

                                                                                            1. 3

                                                                                              Your work looks promising for an app that heavily consumes JSON! I imagine the reason some of us haven’t focused on that problem is because we’re calling simple APIs, the objects are not huge, and decoding doesn’t show itself to be an outstanding bottleneck compared to the network call.

                                                                                              In the first few years of Swift, everyone wanted to use its richer types in our model layers, and so our JSON-serialized domain models had to come along for the ride. Without Objective-C’s dynamism that was a tall order. There were a ton of libraries about it; none of them amazing. Like you wrote in your series, Codable was faster than those libraries, so it was fast compared to what early Swift devs had grown used to, but not compared to NSJSONSerialization producing plain untyped dictionaries. Really it was more of a standardization win than anything else.

                                                                                              1. 1

                                                                                                So we’ve come from “Swift is performance focused” first to “well, our users are pleased with the performance” (as long as they don’t measure) and finally to, essentially “well, performance isn’t really that important”.

                                                                                                Hmm….

                                                                                                1. 3

                                                                                                  My post was about context for people who are interested in how Swift arrays work. I’m not clear on why you’re picking a fight, but I’m not interested. Good luck with the book.

                                                                                        2. 4

                                                                                          Swift is performance-focused and has value type structs and enums

                                                                                          While Swift may be “performance focused” its performance is generally quite a bit worse than Objective-C. Sometimes amazingly worse, very rarely comparable or slightly better and usually noticeably worse.

                                                                                          And also very significantly less predictable, which is arguably even worse for real-world performance.

                                                                                          And of course Objective-C also has value type structs (and reference type structs) and enums, though admittedly the enums are nothing like Swift enums.

                                                                                          And you can use C arrays for those value types, and even have those allocated on the stack (making sure they don’t blow it) or inline. While there may be a way to create a Swift array where the underlying storage is guaranteed to not be heap allocated, I haven’t figure it out yet.

                                                                                          1. 3

                                                                                            Safety is probably Swift’s first focus before performance. You can’t just drop down to C like ObjC could, and although you can write unsafe code you have to jump through significant hoops. The level of dynamism in Objective-C while still being so performant, even without any C escape hatches, was really something special. One thing I really miss is the speed of the compiler, though the safety and richer types feel worth it to me.

                                                                                            Regarding the two languages’ structs and enums, yeah, they’re not the same beasts at all. Objective-C has what C has, while Swift uses the same keywords for two kinds of value-type objects that can have methods, conform to protocols, and participate in polymorphic dispatch. What I was trying to establish with that quoted line is the reason Swift uses multiple kinds of function dispatch depending on the situation.

                                                                                            1. 3

                                                                                              Yeah I’ve run into a lot surprising performance pitfalls with Swift. For performance sensitive code, I just switch to Objective-C now. It is as you said much more predictable, and IMO much easier to optimize being simpler than swift and having the option to easily fallback to C or C++ when necessary. That being said, for general purpose code I find the Swift’s robust type system a pleasure to use. I’ll take the performance tradeoff. Optionals and enums have been such a huge boost to the reliability and comprehensibility of my apps.

                                                                                              1. 2

                                                                                                For performance sensitive code, I just switch to Objective-C now

                                                                                                Yeah, that’s pretty much what I recommend in my performance book.

                                                                                          1. 3

                                                                                            I’m not entirely convinced a new model is needed. We already have memory mapped files in all the major operating systems. And file pages can already be as small as 4KiB, which is tiny compared to common file sizes, these days. Perhaps it would make sense to have even smaller pages for something like Opteron, but do we really need to rethink everything? What would we gain?

                                                                                            1. 4

                                                                                              What we’d gain is eliminating 50+ years of technical debt.

                                                                                              I recommend the Twizzler presentation mentioned a few comments down. It explains some of the concepts much better than I can. These people have really dug into the technical implications far deeper than me.

                                                                                              The thing is this: persistent memory blows apart the computing model that has prevailed for some 60+ years now. This is not the Von Neumann model or anything like that; it’s much simpler.

                                                                                              There are, in all computers for since about the late 1950s, a minimum of 2 types of storage:

                                                                                              • primary storage, which the processor can access directly – it’s on the CPUs’ memory bus. Small, fast, volatile.
                                                                                              • secondary storage, which is big, slow, and persistent. It is not on the memory bus and not in the memory map. It is held in blocks, and the processor must send a message to the disk controller, ask for a particular block, wait for it to be loaded from 2y store and place into 1y store.

                                                                                              The processor can only work on data in 1y store, but everything must be fetched from it, worked on, and put back.

                                                                                              This is profoundly limiting. It’s slow. It doesn’t matter how fast the storage is, it’s slow.

                                                                                              PMEM changes that. You have RAM only RAM, but some of your RAM keeps its contents when the power is off.

                                                                                              Files are legacy baggage. When all your data is in RAM all the time, you don’t need files. Files are what filesystems hold; filesystems are an abstraction method for indexing blocks of secondary storage. With no secondary storage, you don’t need filesystems any more.

                                                                                              1. 7

                                                                                                I feel like there are a bunch of things conflated here:

                                                                                                Filesystems and file abstractions provide a global per-device namespace. That is not a great abstraction today, where you often want a truly global namespace (i.e. one shared between all of your devices) or something a lot more restrictive. I’d love to see more of the historical capability systems research resurrected here: for typical mobile-device UI abstractions, you really want a capability-based filesystem. Persistent memory doesn’t solve any of the problems of naming and access. It makes some of them more complicated: If you have a file on a server somewhere, it’s quite easy to expose remote read and write operations, it’s very hard to expose a remote mmap - trying to run a cache coherency protocol over the Internet does not lead to good programming models.

                                                                                                Persistence is an attribute of files but in a very complicated way. On *NIX, the canonical way of doing an atomic operation on a file is to copy the file, make your changes, and then move the old file over the top. This isn’t great and it would be really nice if you could have transactional updates over ranges of files (annoyingly, ZFS actually implements all of the machinery for this, it just doesn’t expose it at the ZPL). With persistent memory, atomicity is hard. On current implementations, atomic operations with respect to CPU cache coherency and atomic operations with respect to committing data to persistent storage are completely different things. Getting any kind of decent performance out of something that directly uses persistent memory and is resilient in the presence of failure is an open research problem.

                                                                                                Really using persistent memory in this way also requires memory safety. As one of the The Machine developers told me when we were discussing CHERI: with persistent memory, your memory-safety bugs last forever. You’ve now turned your filesystem abstractions into a concurrent GC problem.

                                                                                                1. 1

                                                                                                  Excellent points; thank you.

                                                                                                  May I ask, are you the same David Chisnall of “C is not a low-level language” paper? That is probably my single most-commonly cited paper. My compliments on it.

                                                                                                  Your points are entirely valid, and that is why I have been emphasizing the “just for fun” angle of it. I do not have answers to some of these hard questions, but I think that at first, what is needed is some kind of proof of concept. Something that demonstrates the core point: that we can have a complex, rich, capable environment that is able to do real, interesting work, which in some ways exceeds the traditional *nix model for a programmer, which runs entirely in a hybrid DRAM/PMEM system, on existing hardware that can be built today.

                                                                                                  Once this point has been made by demonstration, then perhaps it will be possible to tackle much more sophisticated systems, which provide reliability, redundancy, resiliency, and all that nice stuff that enterprises will pay lots of money for.

                                                                                                  There is a common accusation, not entirely unjust, that the FOSS community is very good at imitating and incrementally improving existing implementations, but not so good at creating wholly new things. I am not here to fight that battle. What I was trying to come up with was a proposal to use some existing open technology – things that are already FOSS, already out there, and not new and untested and immature, but solid, time-proven tools that have survived despite decades in obscurity – and assemble them into something that can be used to explore new and largely uncharted territory.

                                                                                                  ISTM, based on really very little evidence at all, that HPE got carried away with the potential of someting that came out of their labs. It takes decades to go from a new type of component to large-scale highly-integrated mass production. Techies know that; marketing people do not. We may not have competitive memristor storage until the 2030s at the earliest, and HPE wanted to start building enterprise solutions out of it. Too much, too young.

                                                                                                  Linux didn’t spring fully-formed from Torvalds’ brow ready to defeat AIX, HP-UX and Solaris in battle. It needed decades to grow up.

                                                                                                  The Machine didn’t get decades.

                                                                                                  Smalltalk has already had decades.

                                                                                                  1. 1

                                                                                                    Reply notifications are working again, so I just saw this!:

                                                                                                    May I ask, are you the same David Chisnall of “C is not a low-level language” paper? That is probably my single most-commonly cited paper. My compliments on it.

                                                                                                    That’s me, thanks! I’m currently working on a language that aims to address a lot of my criticisms of the C abstract machine.

                                                                                                    Something that demonstrates the core point: that we can have a complex, rich, capable environment that is able to do real, interesting work, which in some ways exceeds the traditional *nix model for a programmer, which runs entirely in a hybrid DRAM/PMEM system, on existing hardware that can be built today.

                                                                                                    I do agree with the ‘make it work, make it correct, make it fast’ model, but I suspect that you’ll find with a lot of these things that the step from ‘make it work’ to ‘make it correct’ is really hard. A lot of academic OS work fails to make it from research to production because they focus on making something that works for some common cases and miss the bits that are really important in deployment. For persistent memory systems, how you handle failure is probably the most important thing.

                                                                                                    With a file abstraction, there’s an explicit ‘write state for recovery’ step and a clear distinction in the abstract machine between volatile and non-volatile storage. I can quite easily do two-phase commit to a POSIX filesystem (unless my disk is lying about sync) and end up with something that leaves my program in a recoverable state if the power goes out at any point. I may lose uncommitted data, but I don’t lose committed data. Doing the same thing with a single-level store is much harder because caches are (as their name suggests) hidden. Data that’s written back to persistent memory is safe, data in caches isn’t. I have to ensure that, independent of the order that things are evicted from cache, my persistent storage is in a consistent state. This is made much harder on current systems by the fact that atomic with respect to other cores is done via the cache coherency protocol, whereas atomic with respect to main memory (persistent or otherwise) is done via cache evictions and so guaranteeing that you have a consistent view of your data structures with respect to both other cores and persistent storage is incredibly hard.

                                                                                                    The only systems that I’ve seen do this successfully segregated persistent and volatile memory and provided managed abstractions for interacting with it. I particularly like the FaRM project from some folks downstairs.

                                                                                                    There is a common accusation, not entirely unjust, that the FOSS community is very good at imitating and incrementally improving existing implementations, but not so good at creating wholly new things.

                                                                                                    I think there’s some truth to that accusation, though I’m biased from having tried to do something very different in an open source project. It’s difficult to get traction for anything different because you start from a position of unfamiliarity when trying to explain to people what the benefits are. Unless it’s solving a problem that they’ve hit repeatedly, it’s hard to get the message across. This is true everywhere, but in projects that depend on volunteers it is particularly problematic.

                                                                                                    ISTM, based on really very little evidence at all, that HPE got carried away with the potential of someting that came out of their labs. It takes decades to go from a new type of component to large-scale highly-integrated mass production. Techies know that; marketing people do not. We may not have competitive memristor storage until the 2030s at the earliest, and HPE wanted to start building enterprise solutions out of it. Too much, too young.

                                                                                                    That’s not an entirely unfair characterisation. The Machine didn’t depend on memristers though, it was intended to work with the kind of single-level store that you can build today and be ready to adopt memrister-based memory when it became available. It suffered a bit from the same thing that a lot of novel OS projects do: they wanted to build a Linux compat layer to make migration easy, but once they have a Linux compat layer it was just a slow way of running Linux software. One of my colleagues likes to point out that a POSIX compatibility layer tends to be the last piece of native software written for any interesting OS.

                                                                                                2. 4

                                                                                                  I think files are more than just an abstraction over block storage, they’re an abstraction over any storage. They’re crucial part of the UX as well. Consider directories… Directories are not necessary for file systems to operate (it could just all be flat files) but they exist, purely for usability and organisation. I think even in the era of PMEM users will demand some way to organise information and it’ll probably end up looking like files and directories.

                                                                                                  1. 2

                                                                                                    Most mobile operating systems don’t expose files and directories and they are extremely popular.

                                                                                                    1. 3

                                                                                                      True, but those operating systems still expose filesystems to developers. Users don’t necessarily need to be end users. iOS and Android also do expose files and directories to end users now, although I know iOS didn’t for a long time.

                                                                                                      1. 3

                                                                                                        iOS also provides Core Data, which would be a better interface in the PMEM world anyway.

                                                                                                        1. 2

                                                                                                          True, but those operating systems still expose filesystems to developers.

                                                                                                          Not all of them don’t, no.

                                                                                                          NewtonOS didn’t. PalmOS didn’t. The reason being that they didn’t have filesystems.

                                                                                                          iOS is just UNIX. iOS and Android devices are tiny Unix machines in your pocket. They have all the complexity of a desktop workstation – millions of lines of code in a dozen languages, multiuser support, all that – it’s just hidden.

                                                                                                          I’m proposing not just hiding it. I am proposing throwing the whole lot away and putting something genuinely simple in its place. Not hidden complexity: eliminating the complexity.

                                                                                                        2. 2

                                                                                                          They tried. Really hard. But in the end, even Apple had to give up and provide the Files app.

                                                                                                          Files are an extremely useful abstraction, which is why they were invented in the first place. And why they get reinvented every time someone tries to get rid of them.

                                                                                                          1. 4

                                                                                                            Files (as a UX and data interchange abstraction) are not the same thing as a filesystem. You don’t need a filesystem to provide a document abstraction. Smalltalk-80 had none. (It didn’t have documents itself, but I was on a team that added documents and other applications to it.) And filesystems tend to lack stuff you want for documents, like metadata and smart links and robust support for updating them safely.

                                                                                                            1. 1

                                                                                                              I’m pretty sure the vast majority of iOS users don’t know Files exist.

                                                                                                              I do, but I almost never use it.

                                                                                                            2. 1

                                                                                                              And extremely limiting.

                                                                                                      1. 6

                                                                                                        Huh?

                                                                                                        Performance is horrible on Macs.

                                                                                                        No it’s not. And then there are no numbers to back up that claim, just …

                                                                                                        And yet the performance is tangibly worse for Docker on Macs, because you have to run it inside a VM.

                                                                                                        Well, that could be a reason why performance might be worse, but VMs are actually pretty darn efficient these days for many tasks. It certainly in no way justifies the “horrible” from the headline. It doesn’t even justify “tangibly”, for that we would at least need to see some numbers.

                                                                                                        When making performance claims, always be measuring. Otherwise you will get egg on your face.

                                                                                                        Furthermore:

                                                                                                        Windows cleverly skirts this problem by running the Linux kernel directly, not inside a VM;

                                                                                                        That turns out not to be the case. It used to run directly, but, er, performance was “horrible” particularly for file-intensive operations. So with WSL2, Microsoft improved performance (dramatically) by switching to running the Linux kernel inside a VM.

                                                                                                        So exactly the opposite of what the author claims.

                                                                                                        (This is all easily duckable, for example Initial Impressions of WSL 2 - 13× Faster than WSL 1

                                                                                                        1. 1

                                                                                                          Compiling C++ in Docker is definitely not the same as compiling it on bare metal. If I could spit out Linux binaries from Mac I’d consider it, but the build process for some stuff at $work not Mac ready.