This has an odd focus on preventing piracy of media; my concern is being able to run whatever OS and application software I want, with whatever patches I want. Additionally, as long as I correctly implement the network protocols, I would rather like to be able to interact with the rest of the world.
As part of their “Defective by Design” anti-DRM campaign, the FSF recently made the following claim:
Today, most of the major streaming media platforms utilize the TPM to decrypt media streams, forcefully placing the decryption out of the user’s control
So this post is a response to the FSF’s comment, and it was the FSF’s choice to put the focus there.
I believe that the common advice of “always store time in UTC” is a bit overplayed. There are many cases where you actually need to store local time (AND a corresponding timezone name). Basically when you handle any time in the future. Only rarely you actually need “now + X * 86400 seconds”, most of the time it’s “October 10, 2030 at 20:00 German time”.
I would say that storing this as an integer: 20301010200000 would even make some sense because it will discard the time-related assumptions.
I rarely store timestamps in the future. I cannot recall when was the last time I needed that. And for times in future (like calendar events) I agree, but that is super rare case that you need to store such timestamp.
Leap seconds are another reason not to store future times in UTC, though it seems those might be finally going away. UTC always for past times is probably very reasonable, though.
UTC is for timestamps. When something actually happened.
Rarely for things like “expire in 24 hours”.
I mean we all understand how many times people were burned by timestamps stored in local time, so this overshotted advice seems to be kind of a valid precaution.
You can safely ignore leap seconds in basically all cases, e.g. UNIX time does not include them. A day in UNIX time is always 86400 seconds, and in practice clocks are adjusted around leap seconds with leap smearing, or just setting the clock back 1 second at the arranged time. If you actually care about highly precise timing, you have to use specialized libraries that are aware of leap seconds. Most things ignore them, and mostly 1 extra second of real time here or there doesn’t matter.
That 1 second matters even less when you consider that all clocks you have access to through conventional APIs are some degree of wrong—either non-monotonic, unsteady, or both.
That said, I agree UNIX time shouldn’t be used for future dates/times that have any calendar semantics.
The only way to avoid leap seconds is to use TAI internally, and keep a separate table for the UTC offset (kind of like timezones). Not sure what applications need second-level accuracy in the mid-to-long future, though.
Epoch time is just another representation of UTC, as far as I’m aware. So if you’re storing future dates then you have the downside described in the OP.
I sense some over-optimization going on here. Perhaps slightly OT, but then, not having to write unsafe code at all is certainly one solution to the issues raised here, right?
No allocations under steady-state use. Allocations are a source of contention, failure, and overhead, especially when using slow system allocators.
In a program where the items being queued are files, and the processing involves generating ML embeddings, memory allocation in the channel is going to be lost in the noise.
And the slowness of allocators is pretty OS dependent — from what I’ve heard here, Linux (or glibc) has a particularly slow malloc implementation.
The channel cannot know how many tasks are blocked, so we can’t use a fixed-size array. Using a Vec means we allocate memory every time we queue a waker.
Unless I’m missing something, this is untrue. A Vec only allocates memory when it has to grow its capacity. Once the queue warms up, there should be no more allocations.
Unless I’m missing something, this is untrue. A Vec only allocates memory when it has to grow its capacity. Once the queue warms up, there should be no more allocations.
That’s my understanding as well. It should happen very quickly for most systems, and remain stable for the lifetime of the channel. That fact removes the need for all the other complexity.
We also provide Rust bindings in snmalloc. Rust tends to encourage the kind of thing that we optimise for (producer-consumer things with allocation in one thread and deallocation in another are a pathological case for thread-caching allocators like jemalloc and are very fast in message-passing allocators like snmalloc)l
We got big gains switching to jemalloc at $JOB, I’ll have to give snmalloc a try when I have some spare time. I’m not sure how much cross-thread message passing we do in practice. I think many of our messages go between Tokio tasks on the same thread, but it’s hard to know without measuring.
Just tried to use snmalloc in Rust. Let me describe two problems. I say all this to help you make snmalloc better.
First. My first instinct was to cargo add snmalloc. Unfortunately, crate https://crates.io/crates/snmalloc seems to be empty. So I suggest reaching out to its owner and asking for transferring this name
Second (more important). Building Rust project with snmalloc dependency needs C++ compiler (as opposed to C compiler). This makes building slightly harder. I have no problems with installing C++ compiler, but I don’t want to complicate building instructions for users of my code. Rust has great advantage: nearly all projects can be built with just cargo build. Yet if a project depends on snmalloc you have to apt-get install g++ (or similar) first. Also, snmalloc doesn’t give me any significant benefits over jemalloc or mimalloc, so I will just use jemalloc or mimalloc for my projects. Simpler build is more important than hypothetical speed advantages I don’t care about in my projects. Yes, jemalloc and mimalloc depend on C compiler. So what? Very popular crate reqwest (and many other popular crates) depends on C compiler in its default settings, too. So, most Rust developers already have C compiler installed. But not necessary C++ compiler.
So, my personal conclusion is so: I will use default (system) allocator in all my Rust projects. If it performs badly, I will switch to jemalloc or mimalloc. But never to snmalloc, because it doesn’t give any real advantage over jemalloc or mimalloc, while complicating build, which is important for me (because I care about my users). If, say, snmalloc will make my whole program (which consists not only of allocations) 2 times faster than with jemalloc and mimalloc, then this will be possibly enough reason for switching to snmalloc.
All these is not attempting to insult you. I’m just trying to deliver constructive critique, so you will make snmalloc better
If you actually want to improve it, the right place to post this is on the issue tracker. The Rust integration is contributed by someone who uses it, so they’ll see the friction for other people and maybe fix it.
We obviously can’t do anything about the C++ requirement, big given that every platform supported by Rust has a default C compiler that is also a C++ compiler, I have no idea why that would be a problem.
Yeah, the original version of the crate did not have any unsafe code. But I previously worked on projects where every malloc() hurt, even with jemalloc, and sometimes you have to use the system allocator. (Certainly library crates cannot dictate choice of allocator to downstream consumers.)
You’re slightly misinterpreting the use case: some of the items being queued are inodes, likely served from kernel cache, and straight-line performance matters sometimes. (Of course, the second you have to hash a large file, then yes, that’s going to dominate.)
I do have benchmarks, but I did not think to include them. Someone on Reddit asked the same question, so I’ll see about publishing something.
On amortized allocation and Vec, if you sit down and try, it’s actually hard to amortize the allocation cost in this particular use case. In the least, you have to introduce a second mutex acquisition to store the backbuffer somewhere. More details here: https://news.ycombinator.com/item?id=41947924
That said, yes, I could have ignored the small allocation cost, but it was worth the exercise to learn the rules, and if Tokio and LILOS can do it, so can I!
What is there to amortize? A channel will quickly grow to its steady-state size and usually stay there. And that size is much, much less than the number of items that go through it. So you might have a handful of allocations before the Vec grows to 100 items or whatever, then no more. If that concerns you, you can initialize the Vec with that size.
You just described amortization. It would amortize to 0 allocations per task sleep.
Either way, I described why it can’t easily work that way in this use case without an extra lock or some global lock-free pool. (Which I did implement! https://docs.rs/wakerpool/latest/wakerpool/)
And the slowness of allocators is pretty OS dependent — from what I’ve heard here, Linux (or glibc) has a particularly slow malloc implementation.
And it’s pretty easy to switch to jemalloc, mimalloc, snmalloc, or probably any number of other allocators, if the concern is that the OS-supplied allocator will be too slow.
you’re better off defaulting to sequential consistency memory order until the operation on the atomic variable becomes the performance bottleneck of your code
I think the opposite: you’re better off defaulting to relaxed. Atomics are always atomic with respect to themselves. You don’t need anything stronger than relaxed unless you’re using an atomic to synchronize access to memory other than the atomic itself. And if you are synchronizing access to memory between 2 threads in any way, you’re usually better off using a mutex, channel, condvar, or other primitive that abstracts the appropriate synchronization pattern you need.
You don’t program your computer with LISP or Smalltalk. Your computer runs a separate LISP or Smalltalk computer that you can then program on its own terms and in its own world. (And attempts to bring machines closer to these languages mostly did not succeed).
Emphasis mine. The author addresses this point literally two sentences later.
It’s a strange contention. There were commercially successful Lisp machines from multiple companies for a decade (Symbolics, LMI, Texas Instruments, etc.). What killed them was the Unix workstations: the technology reached the point where you could take an off the shelf microprocessor like an m68k, run a BSD variant on it, and produce a machine with impressive specs for the day at a fraction of the cost of building hardware and OS from the ground up. The same force killed off Burroughs large machines and Connection Machines and a bunch of others. You could make the same contention that Unix workstations mostly did not succeed since Sun, DEC, and SGI died when you could take an off the shelf PC and produce a machine with impressive specs for the day at a fraction of the cost of a dedicated Unix workstation. They had a similar run of roughly a decade, just like the transistor machines with magnetic core memory killed off vacuum tube and delay line machines and machines with integrated circuit memory killed off transistor plus magnetic core machines.
They were commercially viable for a while but “successful” may be a bit of an overstatement. Between them, Symbolics, LMI, PERQ, Xerox and TI had sold less than 10,000 machines (on the order of 7-8000 IIRC?) by the end of the 1980s. The workstation market wasn’t exactly a mass market so it’s not a bad figure, but it’s not exactly a resounding success, either. E.g. SGI alone sold 3000+ units of its first-generation IRIS system, and they were a small player at the time.
It looked more commercially viable than it was because much of the early development cost had been pre-paid in a way. E.g. early on, Symbolics, LMI and TI largely used the same software, licensed from MIT, and they all started pretty much with MIT CADR machines. Symbolics initially planned to start working on their own hardware right away but it took them a couple of years to kickstart the 3600 project.
Even at the peak of Lisp popularity, three (i.e. HP, DEC and Sun) of the five major companies with important Lisp offerings didn’t sell Lisp machines. The other two were Symbolics and TI, but even within Symbolics, lots of business groups weren’t profitable. I think the only one that remained profitable throughout its existence was the Macsyma group.
What killed them was the Unix workstations: the technology reached the point where you could take an off the shelf microprocessor like an m68k, run a BSD variant on it, and produce a machine with impressive specs for the day at a fraction of the cost of building hardware and OS from the ground up.
It also didn’t help that their hardware, despite being very much at the edge of what was possible at the time, was still not enough. Early Symbolics Lisp machines actually had an M68K along with the 3600 CPU. The M68K (the front-end processor, FEP for short) handled startup and some of the peripherals. Merely integrating Lisp machine hardware was expensive.
Unix workstations were one of the things that killed Lisp machines, but not quite by using off-the-shelf hardware. Symbolics hardware was actually contemporary with M68K-era Unix workstations (LM-2 was launched the same year as the DN100, 3600-series was launched the same year as Sun-1 and IRIS).
By the time that Lisp peaked in popularity in the second half of the 1980s, most big workstation manufacturers had actually moved on to their own stuff (SGI started using MIPS RISC in 1987, Sun started using SPARC in 1988, DEC were doing their VAX thing, IBM’s main workstation offering was trying to make ROMP take off since 1986). When Symbolics really started going downwards, the Unix machines that beat their Lisp machine counterparts used their own proprietary CPUs, too. NeXT were, I think, the only ones doing high-tier workstations with M68K CPUs, and even those were slow enough that they were largely excluded from particularly high-performance market segments (e.g. high-end CAD applications).
You don’t program your computer with LISP or Smalltalk.
Well, do you?
The Lisp Machines would like a word. Actually, a whole list of words.
Smalltalk machines were a thing, too.
They existed and were exactly what I alluded to. They were never “a thing”
It seems to come from the use of thing in the sense of a popular phenomenon—cf. “Ecigs are the new thing”. However, its meaning also extends to differentiating set phrases, names, or terms of art from normal productive constructions. For example:
This article’s viewpoint is too narrow: it lacks important historical context.
Hmm…or you could have just just read the next two sentences, where that historical context was provided:
Your computer runs a separate LISP or Smalltalk computer that you can then program on its own terms and in its own world. (And attempts to bring machines closer to these languages mostly did not succeed).
I find this a very strange and hard to parse reply.
From your tone and back-checking a few links, I deduce that this is your blog post, correct? If that is the case then why didn’t you say so? Why do you obscure this simple fact by using the passive voice and so on?
Well, do you?
Me, personally? No. I stopped programming when it stopped being fun, which for me was roughly when Windows started to take off.
They existed and were exactly what I alluded to. They were never “a thing”
I entirely disagree. Indeed this line contradicts itself: the second sentence is invalidated by the first sentences. They existed, therefore, they were a thing.
That means that your supposition in your blog post, that Lisp and Smalltalk only exist inside their own isolated boxes on top of other OSes, is false… and you knew it was false when you wrote it and not only that but you admitted it a line later.
This is simply unfathomable to me.
Hmm…or you could have just just read the next two sentences, where that historical context was provided:
To be perfectly honest, I stopped when I reached that false assertion and wrote my reply. Then, I read the rest before posting, because I’m not 12 years old and I’ve been on the internet for 39 years now. I am not claiming I’m perfect – I’m not – but I commented on the bit that struck me.
I am a bit baffled by your apparently angry response, TBH.
From your tone and back-checking a few links, I deduce that this is your blog post, correct? If that is the case then why didn’t you say so? Why do you obscure this simple fact by using the passive voice and so on?
Marcel said (and you quoted!) “They existed and were exactly what I alluded to.” How’s that obscuring?
I am a bit baffled by your apparently angry response, TBH.
If I’m not mistaken, you write articles for a living. Surely you understand why an author might be annoyed at someone for “commenting on the bit that struck them” while seemingly ignoring that the point was addressed later?
For what it’s worth, I’m under the impression that you’re trying to pick a fight and I hope I’m wrong. :)
Because, TBH, that’s the moment I realised “hang on, is he saying he wrote this?”
If I’m not mistaken, you write articles for a living. Surely you understand why an author might be annoyed at someone for “commenting on the bit that struck them” while seemingly ignoring that the point was addressed later?
Sure, that’s a fair point. :-)
Wider context for reference: I am a lifelong sceptic and disbelieve in all forms of religion, the paranormal, alternate medicine, etc.
If I read an essay that starts “homeopathy shows us that X can do Y” then I am not going to pay much attention to the rest of the essay that talks about – well, anything really – because homeopathy does not show that. Homeopathy does not work, full stop, the end.
So when an article about programming languages says “X is never used for Y, and you can’t do Y with X” when I have stood on stage and told a thousand people about how and why doing Y with X is important, yeah, you are damned right I am going to stop right there and say “hey, that is not right.”
Marcel seems to feel that because something was not commercially successful it didn’t happen. That is not true.
For what it’s worth, I’m under the impression that you’re trying to pick a fight and I hope I’m wrong. :)
Not at all, but I’ve spent over 40 years calling out BS when I see it, and I actively enjoy making people angry by telling them they are wrong.
I am not picking fights. I am trying to point out logical errors and fallacious claims. Not the same thing.
The grown up way to respond when someone points out a hole in your argument is to change your argument. The schoolkids’ way is to get louder defending it.
I think Marcel’s argument rests on a core point that’s wrong. I am only interested in that core point, not what sits on top of it, which seems to be the bit he wants to talk about.
So when an article about programming languages says “X is never used for Y, and you can’t do Y with X” when I have stood on stage and told a thousand people about how and why doing Y with X is important, yeah, you are damned right I am going to stop right there and say “hey, that is not right.”
You could have stopped with your first comment and just said “hey you know lisp machines were actually a bigger thing”, and gone into a bit more detail yourself, but instead just dismissed the rest of the article as lacking historical context.
Context that is utterly irrelevant to the rest of the article. The author doesn’t say you can’t use Smalltalk or LISP to do systems programming. He says that people don’t. Perhaps try being charitable and read this as “people don’t tend to”. Which is not untrue.
Ambiguity isn’t uncommon in writing, and sometimes readers should just be able to infer things themselves from common sense.
I entirely disagree. Indeed this line contradicts itself: the second sentence is invalidated by the first sentences. They existed, therefore, they were a thing.
Once again: “X is a thing” is an idiom for X being something that was popular.
How many LISP and Smalltalk machines were built and sold, in your opinion?
What market share did they achieve in the workstation market?
Once again: “X is a thing” is an idiom for X being something that was popular.
Nope. It means “they existed”, “they were real”.
How many LISP and Smalltalk machines were built and sold, in your opinion?
Doesn’t matter. They were on commercial sale, were real products loved by legions of influential people, and inspired important works that you seem unaware of, such as the Unix Hater’s Handbook, which I submit you need to read.
What market share did they achieve in the workstation market?
Irrelevant. This is not a popularity contest. You are attempting to talk about absolutes – “this only runs in this way” – which are not true. Which is why I responded “this article lacks important context.”
Because with every reply you send, you reinforce my impression that you do not understand the bigger picture here.
The author of the post has linked to a discussion about the origin and evolution of the term, where being popular, especially in a surprising context, is one of the accepted meanings of the word. It’s not just the assertion of the person who asked about it, but also one of the quoted definitions:
An action, fashion style, philosophy, musical genre, or other popularly recognized subsection of popular culture.
Early use of the term seems to have been an ellided version of “is there such a thing as”, but several later instances of its use obviously lean towards the aspect of popularity. Presumably the secondary sense was developed by contamination from the pejorative “a thing”, as in “PCs will be a thing of the past in 20 years”. Regardless, it’s a thing now. See e.g. this entry.
FWIW it’s definitely how I use it, too. Seeing how this is slang, yeah, on behalf everyone else, I would like to take this chance to not apologize for using it in a way that you disapprove of. I’m sure that, if a whole generation of stationery users woke up one day in a world where “to clip” meant both “to cut” and “to fasten” and managed to get over it, you’ll eventually come to terms with this horrifying misuse of language, too.
Is it ambiguous? Yes. Does minimal charity resolve virtually every instance of ambiguity? For heaven’s sake also yes, is it really not obvious that an article that literally says “attempts to bring machines closer to these languages mostly did not succeed” acknowledges that attempts to bring machines closer to these languages existed? What exactly do you think is missing here and isn’t obvious to a community that has a higher incidence of Lisp per capita than the daily appointment list of a speech therapy clinic?
This is how it is in C and C++, which Go is closely related to. I think this is only surprising when compared to languages where literally everything is a reference (e.g. Java, Python, JS, or Ruby).
That is irrelevant. When you do a = b in C++, in all cases where the type of a and b is a struct (or class), you will get either copy of b in a, or you will get a compile failure. Whether that’s a deep or shallow copy depends on the copy-assignment operator for the type of a, but it will always be a copy of the struct.
They are… sort of. As in, the copy constructor of members will get executed. However, the copy constructor of a pointer just copies the pointer.
So they’re “deep” as in “members also get copies”, but not “deep” as in “members which are pointer types will allocate a new object and point to the new object”.
In C++ copied strings and vectors will result in deep copies with new allocations, as they are NOT pointer types. Go strings and slices are copied shallowly as they ARE pointer types.
Also the original problem was Go ranges iterating by value. C++ can iterate by reference.
“Members are copied recursively” does sound like a deep copy, in a way, though. And you could totally make a CopiedPtr class which is sorta like a std::unique_ptr except it has a copy constructor which allocates and copy-constructs a new pointed-to object with new.
It’s not a memcpy though. It’s recursively running copy constructors. Strings and vectors will allocate a new buffer and copy their contents over; that doesn’t sound totally “shallow” to me.
And in fact, you could make a CopiedPtr class which allocates and copy-constructs a new pointed-to object with new. An object tree where all pointers are CopiedPtr rather than the other pointer/reference types would be deeply copied.
C++ is too complicated for simple statements like “copies are shallow” or “copies are deep”. Copies are customizable. And the language’s and standard library’s pointer types are copied shallowly (or refuse to be copied altogether, as is the case with unique_ptr), while the standard library’s container types are copied deeply.
This exploit is currently being used by ticket scalpers. And bad actors who are selling fraudulent already-sold tickets. And, perhaps, a few honest people who just want to sell their ticket due to a change of plans.
I come from a family of professional, touring musicians. Scalpers are a nuisance, but Ticket Master is an abusive monopoly. Regulate them if you can; hurt them if you can’t.
Ticketmaster’s solution is bad, Ticketmaster itself is bad, but scalpers are still the scum of the earth. Scalpers harm artists, venues, and attendees, benefiting only themselves.
Resellers simply take over the sales business from whomever they buy their tickets from. Venues also hire marketers to determine how to maximize profits from the sale of tickets; I don’t know why it’s suddenly worse if the marketer is independent rather than working for the venue.
I would also like to know why TicketMaster is not the scum of the earth, since they also harm artists, venues, and attendees. Plus they have the market dominance to reshape the ticketing business and fuck the world to a much greater degree than resellers.
Resellers simply take over the sales business from whomever they buy their tickets from.
Scalpers engage in arbitrage by using automation to rush ticket sites, buy as many tickets as possible, and then use their control over supply to drive profit. It’s monopolistic behavior, not market behavior. If everyone had access to scalper-level technology for getting tickets quickly, you might be right; though I think that would be worse in other ways (can you imagine trying to keep a server up in those conditions?).
The alternative approach is to try to force scalpers to use only the technology available to consumers – which means things like hardware-bound tickets that can’t be resold.
Scalpers engage in arbitrage by using automation to rush ticket sites, buy as many tickets as possible, and then use their control over supply to drive profit. It’s monopolistic behavior, not market behavior.
How is that any different from what ticket sites do themselves?
How is that any different from what ticket sites do themselves?
Ticket sites don’t compete with attendees for tickets. They compete with other ticketsellers to be the agent of the event owner. Ticketmaster/LiveNation are monopolists who engage in terrible anticompetitive conduct to maintain their advantage, but it’s a different problem than scalping. Tickets are naturally non-fungible whereas ticket-selling services are fungible.
But do these differences imply that one is worse than the other? Is there a value judgement in the observation that scalpers monopolize the ticket supply through automated purchasing whereas ticket sites monopolize the ticket supply through a business relationship with the venue?
No, but I want neither to hold monopolies. If Ticketmaster didn’t have a monopoly, nothing would change in what I said about why scalpers are a problem. So I was talking about why it’s worse for “the marketer to be independent” as you put it.
Scalpers are not marketers. When’s the last time you bought tickets to an event you only heard about because of a scalper? When’s the last time you couldn’t figure out who the ticket seller was for an event until you saw promotion from a scalper?
Scalpers engage in arbitrage by using automation to rush ticket sites, buy as many tickets as possible, and then use their control over supply to drive profit. It’s monopolistic behavior, not market behavior. If they were just marketers, what they were selling would be available even without going through them, just as you can buy from Lowes because you’ve seen their advertising, or even if you haven’t.
Certainly Lowes’s marketing team doesn’t buy as much of the stock as they can until the store has no supply, then stand outside the store reselling the stock at a much higher price. And again, fungability is important here: in my hypothetical, there’s no difference between the screwdriver the marketing team is selling and one inside the store, or at another store. Not so with tickets, where (assuming reserved seating etc) each one is non-fungible.
Scalpers are a very abstract concept for me because I’ve never actually encountered one. But I have to imagine they do something after buying the tickets in order to find people to sell them to, no?
Certainly Lowes’s marketing team doesn’t buy as much of the stock as they can until the store has no supply, then stand outside the store reselling the stock at a much higher price.
But the effect for the consumer is the same: The marketing team sets the highest prices that they think the market will bear in order to extract maximum value from customers. So what exactly makes it worse? That they’re standing outside the store?
And again, fungability is important here: in my hypothetical, there’s no difference between the screwdriver the marketing team is selling and one inside the store, or at another store. Not so with tickets, where (assuming reserved seating etc) each one is non-fungible.
I don’t follow this at all. Why use screwdrivers as the example then? What if it’s the MSG marketing team selling tickets outside of MSG?
because in the real world “oh if you can buy this ticket you can sell it for a $100+ profit” leads to a bunch of people buying tickets purely to resell them, instead of people buying tickets to use them.
Of course sometimes things happen and people want to resell the ticket. But honestly “100% refund on tickets are available up until the day of” for most events would help resolve a lot of that sort of issue.
Reselling a ticket because you can’t go is fine and normal obviously. I have a hard time seeing a justification for needing to make a profit off of that activity.
because in the real world “oh if you can buy this ticket you can sell it for a $100+ profit” leads to a bunch of people buying tickets purely to resell them, instead of people buying tickets to use them.
I’m having trouble interpreting your “because.” Are you saying this is the reason Ticketmaster is not the scum of the earth?
It’s not about sympathy for Ticketmaster, it’s sympathy for the people who want to buy reasonably-priced tickets. Ticketmaster is part of the problem, but scalpers systematically price tickets out of most people’s comfort zone.
How widespread is the scalping problem? Have you ever actually had to buy a ticket from a scalper at a markup because scalpers bought all the tickets from the original seller? I certainly haven’t, going back well before TicketMaster’s insane restrictions.
I suspect that it’s pretty uncommon because a good opportunity for a scalper depends on the venue selling tickets for less than they could be selling them, thus forgoing profit.
On the other hand scalpers will also sell tickets below the original price if people don’t buy them. It can allow people to attend events that they otherwise couldn’t afford.
How widespread is the scalping problem? Have you ever actually had to buy a ticket from a scalper at a markup because scalpers bought all the tickets from the original seller? I certainly haven’t, going back well before TicketMaster’s insane restrictions.
For background: I used to go to concerts a lot (one year, I went to a concert every third day) and still go quite a bit. Scalping used to be a huge problem, for popular bands, about a third of the tickets went to scalpers. And that was in the times you’d usually go to the ticket office and buy manually. As it’s an arbitrage business, if the whole thing fails, you’d see people in front of the venue with about 12 tickets each, selling them off.
I suspect that it’s pretty uncommon because a good opportunity for a scalper depends on the venue selling tickets for less than they could be selling them, thus forgoing profit.
Well, concerts used to be a lot cheaper. Venues have figured that part out. 50 EUR used to be an expensive concert, nowadays, it’s just Tuesday. As with any form of arbitrage, things move towards an equilibrium.
I implied above that i think it became less, but in particular i think it’s because venues themselves have closed the delta between the possible ticket price and what they are asking, making scalping more risky, because margins are lower. Essentially: Monster Magnet touring Germany isn’t worth scalping anymore (MM were relatively cheap, but have a cult following and played relatively small venues that often sold out), while Rage against the Machine may still be worth it (drops by every few decades, sells out immediately).
On the other hand scalpers will also sell tickets below the original price if people don’t buy them. It can allow people to attend events that they otherwise couldn’t afford.
I’ve never seen that TBH, they’d rather eat the loss, maybe you can negotiate at the door a bit. I think that’s painting those in too good of a light.
I see no gain in scalping and the prices for a scalped ticket for a pretty rare tour (e.g. A perfect circle) are easily 5x the original price. None of that goes to artists or the venue.
So, now we have ticket buy restrictions (4 per sale), crappy apps, etc. But I do agree on the level that scalpers are just the convenient argument to introduce them. There’s also a lot of environmental changes: concerts are now the “authentic, non-canned” experience and for many artists a much more important source of income rather then selling music online.
HOWEVER, that doesn’t mean Ticketmaster and all of its shenanigans aren’t bad. None of that is in the interest of the customer and most of it is about controlling the resale market and profiting again. Selling or giving away tickets between friends is just a normal thing.
One more thing: scalpers also went from the local music nerd who made a few hundred on the side by exploiting their knowledge about the market to highly automated operations trying to skim the top line of the what people are willing to pay.
I don’t think any of them care much about the music fan not getting a ticket because it’s scalped and ticketmaster in particular isn’t the fix. Both groups will extract as much money out of the venues, the fans and the artists, and not care, and will happily use each other as an argument.
My fix for this is indeed that I’m back to buying paper tickets, dropping by the venue if possible.
I don’t think any of this has helped making concerts more accessible. If you were okay with just seeing a band take stage and using some instruments, tickets were much more affordable around the 2000s, scalping or not. That also shows in the crowds.
Sounds like you actually understand the situation. :)
One thing though:
On the other hand scalpers will also sell tickets below the original price if people don’t buy them…
I’ve never seen that TBH, they’d rather eat the loss, maybe you can negotiate at the door a bit.
But you also said “if the whole thing fails, you’d see people in front of the venue with about 12 tickets each, selling them off.” Are you saying when that happens, they still refuse to drop the price below the initial price because “they’d rather eat the loss”? I would think a scalper who cuts their losses would have a competitive advantage over one who doesn’t.
The scalper already misjudged the market - the concert was “sold out”, but there wasn’t that much interest anyways, so there’s a gap. That means there’s a high likelihood to not sell at all, which may incentivize to at least sell the few items you have at face value.
It’s more likely then that everyone who wants a ticket already has one and there’s few people heading out to the venue. Those who want though are probably fine with just buying a ticket at market price.
Few of them are good negotiators and may not be used to tactics like “wait until the last second before the show starts”.
Both the scalper and the buyer needs to make their way out to the venue, which is often outside of town. Some may do that, some not.
So you usually end up with 1-2 people even standing in front of the show trying to sell and a few that want to buy.
Not buying for the buyer means they have to take the long way back into town.
I think, and this is me spitballing, this leads to a situation where as a seller don’t want to go to low to soon and rather keep the price up as long as possible.
The scalper already misjudged the market - the concert was “sold out”, but there wasn’t that much interest anyways, so there’s a gap. That means there’s a high likelihood to not sell at all, which may incentivize to at least sell the few items you have at face value.
I don’t follow. Did the scalper buy a only a few tickets? Or is the concert sold out because the scalper bought up all the tickets? Obviously the scalper is incentivized to sell tickets at the highest price he can, but what if all of the people who are willing to pay face value already have tickets?
Both the scalper and the buyer needs to make their way out to the venue, which is often outside of town. Some may do that, some not.
Why assume that the venue is outside of town?
So you usually end up with 1-2 people even standing in front of the show trying to sell and a few that want to buy.
Are you saying this is your personal experience, and you saw that the people buying tickets from scalpers before the show were paying face value?
I think, and this is me spitballing, this leads to a situation where as a seller don’t want to go to low to soon and rather keep the price up as long as possible.
Then are you agreeing that they will drop the price eventually? Especially if the venue gets foot traffic from uncommitted potential customers.
I don’t know how widespread it is, but, yes, I’ve personally purchased a ticket from a scalper at an absurd markup because scalpers bought all the available tickets from the original seller. The year was 1996. We wanted to watch the University of North Carolina Tar Heels defeat the Duke Blue Devils in Chapel Hill. The face value of the tickets was around $40. Relatively few of the tickets in the Smith Center (which at the time held ~19k people, I think?) were available for sale to the public after the two universites’ obligations to their stakeholders were met. We wound up paying about $200 per ticket for a pair of seats, and I personally know someone who paid nearly twice that for modestly better seats with a similar face value.
If $200+ per seat is the rate the market supports, so be it. But I’d have preferred that the university take that profit directly from me and my friend and use it for their mission, as opposed to having the majority of our expenditure line the pockets of a scalper.
UNC won by a single point in an exciting contest, which was one of the legendary coach Dean Smith’s final contests against our largest rival, and we felt it was very much worth our outlay.
Also, I’m pretty certain selling the tickets to us above face value was what made them “scalpers” in the first place; that is a term of art referring to someone who sells tickets at a markup. People who sell tickets below face value are not scalpers, at least in the legal sense where it’s banned in many jurisdictions…
If $200+ per seat is the rate the market supports, so be it. But I’d have preferred that the university take that profit directly from me and my friend and use it for their mission, as opposed to having the majority of our expenditure line the pockets of a scalper.
That makes sense; I guess the real harm comes when you can afford the face value price but can’t afford the marked up price and then you miss out.
Also, I’m pretty certain selling the tickets to us above face value was what made them “scalpers” in the first place; that is a term of art referring to someone who sells tickets at a markup. People who sell tickets below face value are not scalpers, at least in the legal sense where it’s banned in many jurisdictions…
People who sell tickets at a markup and people who sell tickets below face value are not mutually exclusive categories. You’re saying if a scalper tries to cut their losses they cease to be a scalper?
I’m saying that the offense the police used to charge you for (that was colloquially known as “scalping”) was tied to selling the tickets above face value.
If you were only caught reselling tickets below face value, you would not be charged with that specific offense, as I recall. It’s been quite a while since I’ve been very interested in this, though. I suspect things have changed a bit.
We’re splitting a strange hair. I suppose, to be more precise, I was saying that reselling a ticket for less than the sum of face value plus whatever service fee is allowed by law does not constitute “scalping.” Scalpers could also engage in non-scalping resale, to be sure. And in bank robbery. And in unlicensed dog walking.
I took that as you disagreeing with my statement that “scalpers will also sell tickets below the original price if people don’t buy them.” If you’re saying something that’s consistent with that, then OK but it’s strange to make that point if it has no bearing on anything I said.
“Adjective first” is one of my least favorite parts of English. Even as a native speaker, I find it terribly awkward to have to buffer all of the adjectives in my mind before I hear what the noun is that they’re applied to: “the big fluffy white … dog”
Like ISO 8601 dates, this article is correct to say that the noun should go first and all modifiers/adjectives come after.
Even more fun, “big fluffy white dog” is canonical and other adjective orderings like “white fluffy big dog” are incorrect! And adjective ordering is not universal across languages, or even for some closely related languages.
Many adjective second languages aren’t exclusively adjective second either. In Spanish, “zapatos viejos” means “old shoes,” but “viejos zapatos” means “OLD shoes” (i.e. emphasis on old).
For an unspecced language like English, both “canonical” and “incorrect” raise the descriptivist’s ire in me.
Further to your second point, many languages differ on whether certain concepts are expressed using adjectives or other parts of speech, or whether they even separate them cleanly.
Other adjective orderings “feel wrong” to virtually all native English speakers, even though most speakers are completely unaware the rule exists.
If descriptive linguistics didn’t care about “canonical” and “incorrect” then the field would be extremely shallow. “Just like, say any words dawg. Believed by all, eye of, within is held correctness, the beholder’s semantics irrelevant linguists!” You can probably piece together what that’s supposed to mean but it’s most definitely incorrect.
Yet you still said the canonical “big fluffy white dog” instead of any of those other orderings, as the vast majority of native speakers would. Other adjective orders aren’t incorrect because they’re bad, they’re incorrect because descriptively those orders are almost never used in practice.
Put another way, if a non-native English speaker wanted to sound more natural to the widest audience, they would learn the correct ordering, even though the meaning of their words would be perfectly clear out of order.
Yet you still said the canonical “big fluffy white dog” instead of any of those other orderings
? I think you’re mixing up me and who you replied to initially. Your point is not made.
edit: to be clear, the one I’d most likely use (though it’s hard to say now for sure, obviously!) is “big white fluffy dog”. Why? I enjoy dogs a lot, and “fluffy dog” a bit of a unit in my mind. “big fluffy white dog” sounds ever so slightly awkward. “big dog” is also a bit of a semantic unit in my mind, and I could see those going together, though at that stage it’d probably become “fluffy white big dog”.
Ah yes, I did mix you two up. Either way, that is the way most people will say it in normal circumstances. My mix up doesn’t change that adjective ordering is well understood by linguists.
The “wrong” adjective orderings aren’t necessarily “incorrect” in English. Intentional atypical order is used to change meaning. It can split adjectives and what they apply to.
They are incorrect if used unintentionally. Although now we are splitting hairs over exactly what “incorrect” means, and I think we both agree there is a “typical” and “atypical” ordering.
Then in the example, should it become ScreenFull.tsx too? UserCurrent? Using the English order, those names are not ordered, sure, but at least it doesn’t sound like gibberish phrasing. Maybe it’s just a habit though.
If I imagine further, UserCurrent.tsx sitting right next to User.tsx doesn’t seem so bad.
ScreenFull.tsx seems a little silly, but I feel that’s because — unless there’s also other kinds of Screens — FullScreen is being used as an adjective in its wholeness (or adverb perhaps), i.e. “this is the component we use when we go full-screen”, as opposed to “this is the full screen, and this is the empty one”.
I spotted that, and I think it applies best to ButtonGoLive.tsx, since you’re likely to have more buttons than screens.
I don’t honestly have a strong opinion on Noun-Adjective vs Adjective-Noun (vs Noun-Verb, Verb-Noun, Noun-Adjective-Verb, …..). Most have their merits and it seems largely a matter of taste.
That said, I think it’s a vital part of any code style guide. As long as the codebase is consistent, the discoverability boon is substantial. As soon as it’s not standardized, finding anything becomes a chore.
At least English is reasonably consistent about adjective-first. Some “noun-first” languages require you to memorize special exceptions for some adjectives that come first despite the general rule:
Français: « Il y a un petit chien dans ce joli café »
Español: “El otro hombre está aquí”
etc.
(Spanish is actually quite annoying in that you can flip the order of a bunch of otherwise postpositive adjectives for emphasis or “poetic” reasons; French at least just kinda has its list of exceptions and once you know them you know them)
French-native speaker here. What you’re saying about the French language is erroneous. Most adjectives tend to appear after the name, but there’s no general rule. Additionally, the position of the adjective may change the meaning , e.g. “un homme grand is a tall man while”un grand homme” is a “great man”.
Part of my comment was meant to be joking/exaggerating for humor, but I will say my genuine experience with both languages is that Spanish and Spanish speakers are much more flexible with word order than French and French speakers.
The complexity in French, for me as non-native speaker, is the large number of irregular verbs, the inconsistencies of the pronunciation/spelling, and the more complex gender/number agreement. Spanish verbs are much more regular, Spanish pronunciation is much more regular, and Spanish gender/number agreement is simpler. But then Spanish is complex in its flexibility, playing with word order and completely dropping many pronouns and even nouns.
For some definition of “better”, yes. It won’t be as fast but it’s well tested and supports pretty much everything it needs to for it to be the default.
I can’t count the the number of times that I’ve been rude to someone in my family because I turned the buildfarm red and had to spend the afternoon, or the evening, fixing it, or often enough, just reverting my ill-considered changes.
It sounds like developers trigger CI by pushing to a branch shared with all other developers. That just seems like a bad idea. Why can’t they trigger CI on personal development branches? Then if it breaks then it doesn’t affect anyone else much.
I contributed to postgres recently by highlighting a documentation bug, and even fixing that took a little time because I’m not used to the email-based workflow and don’t have a mail client set up to handle git-format-patch.
I ended up just describing the issue and someone else made a patch that a third person committed.
I remember a case where a serious data-corrupting bug that I introduced wasn’t found for something like two years, and that kind of thing isn’t uncommon.
Things here could definitely be a bit better than they are. You can run the full test suite locally, and I do, a lot. You can also trigger a CI run on a bunch of common platforms fairly easily, and I do that sometimes, but for various reasons it’s a little more of a hassle than would be ideal. Even if you do that, the buildfarm contains an even wider variety of machines and configurations, so it is possible to break the buildfarm even if you did a CI run and it came back clean. A good number of my mistakes, though, come from thinking that a local run of the tests is good enough and that I don’t need to hassle with CI, and then finding out that it wasn’t and I did.
I realize a lot of people here like email-based workflows, but I really think we need systems that can do both. Even if it meant that I generate a patch and then upload it through a web interface. I’ve absolutely abandoned (admittedly small) contributions to projects before because I didn’t want to set up a special email client. But I do understand why people who are used to it like it.
I’ll plug my friend and colleague’s answer to this problem, GitGitGadget: https://gitgitgadget.github.io/. It translates between GitHub PRs and the patch-based email workflow, bidirectionally.
It’s presently only suitable for the Git project, but I bet could be adapted for another project.
You don’t need a special email client. In the CHICKEN project, we typically use something like git format-patch and then e-mail the resulting files as attachments to the mailing list. To apply, save the attachments and use git am to apply them. Any halfway decent mail client should be able to handle that. The only “special” requirement is that it doesn’t try to get clever with text attachments and mangle them somehow.
Interesting, I had been under the impression that most projects wanted them inline for code review purposes? If all it took was to attach the patch, though, I’d do that.
Working on PostgreSQL, we almost always attach them, not include them inline. I actually find inline very inconvenient; I can’t easily extract the patch if it’s done that way.
Why can’t they trigger CI on personal development branches?
Databases typically have massive test suites that can’t reasonably run for every dev branch. Back when I worked on a DB engine we had a standard set you ran for every commit, along with any new tests or other relevant tests. Then every night our full test suite would run, executing several CPU-years of tests. We then re-ran failed tests the next day to isolate issues if necessary.
I can’t speak for the postgres test environment, but I wouldn’t be surprised if the situation is similar.
An approach that can work well here is a merge queue with integrated testing. So you batch up a lot of changes and test the merged result. If the tests fail you don’t commit it but try again with half of the commits. It can be slow with slow CI but it is typically manageable as long as your number of bad commits aren’t too great.
The downside is that you still need to pass full review before merging. But at least you don’t break the main branch, your commit just bounces back and you can fix the issues before merging.
Ideally you would still have some pre-review tests but that can be a sample of common issues without needing to run the entire suite.
Given how critical the PostgreSQL project - and assuming the test suite can run in parallel - in my opinion it would justify having a much larger set of CI workers than most projects. Firing up a thousand CPU cores to run tests for a trusted developer could be a worthwhile investment of project sponsorship money.
In total, 114.7 MB of data was requested from my site in just under five minutes
If a web server has a problem serving 114MB of data in five minutes… maybe improve the web server performance? 114MB is peanuts by modern standards. Expecting a centralized platform to work as a free-of-charge CDN doesn’t seem sustainable at all — how will the server cope if people start clicking on those links en masse?
I find it hilarious the author is complaining about 114.7 MB when they have zero respect for the user’s bandwidth. Open up this page without an ad blocker, scroll through it, and take a look at the network tab:
I find it hilarious how few people actually read the article. The quote about 114.7 MB is from a different person who independently observed the effect on a different website.
I read it. I’m not talking about Chris Partridge, who may legitimately complain about 114.7 MB. I’m talking about the author of the It’s FOSS article Ankush Das, who is sharing the quote as an example of excessive bandwidth, and complaining (hypocritically) that Mastodon is DDoSing It’s FOSS as well.
Right, I replied to you in a “things I also find hilarious” sense, not because I think you’re wrong, though I see how that’s unclear. The comments about server performance to me seem to be confusing which site is which.
To be charitable to the author, that was a third party engineer’s test and not necessarily representative of the load itsfoss is seeing. The full quote:
Quoting Chris Partridge’s (a security engineer’s) older findings, he mentioned:
However, I got a bit of a nasty surprise when I looked into how much traffic this had consumed - a single roughly ~3KB POST to Mastodon caused servers to pull a bit of HTML and… fuck, an image. In total, 114.7 MB of data was requested from my site in just under five minutes - making for a traffic amplification of 36704:1.
The crucial point is that someone needs to serve that traffic anyway, if we assume that link previews are a good thing. Facebook, Ex-Twitter, etc. will query the server of itsfoss.com just like Fediverse instances do. Then they will serve that data to everyone who views the timeline, so the “traffic amplification” is still there — it’s just someone else who picks up the tab. It’s only less noticeable because there are five of them. That is like saying “I want fewer social media platforms on the Internet” without saying that explicitly.
From this perspective, why have a website at all? If you want to offload everything on centralized megacorp-owned platforms, just post to them directly.
This is not to say that link previews cannot be more efficient: I suppose one problem is that a social media server fetches a complete page just to make a low-resolution version of the header image and extract some headings and first paragraphs. Web servers could generate their own previews and provide endpoints for querying just the previews.
But still, if you can’t serve a page to thousands Fediverse servers, you are already in trouble if thousands users come to your page.
The site itself can provide og: meta tags with content and the social site should use those to create the preview. It only needs to fetch the raw HTML for whatever link - and to optimise more only the is really needed. You can stop parsing as soon as you hit
Worth noting your HTML tags here have been treated as HTML tags and sanitised out. You can escape them like \<this>, or put them in backticks to get inline code spans (i.e. `<this>`).
I don’t think you can assume that link previews are a good thing. Also a service like Facebook or Ex-Twitter will generate the preview once and then use that preview for every other occurrence, which limits the amount of amplification that occurs. The Fediverse doesn’t have that mitigation.
This isn’t a traffic amplification attack. That is when the traffic goes to someone other than who made the request. In general TCP protocols (such as HTTP) are immune to this issue.
For example it looks like this:
10.0.0.1 sends small DNS query to 10.1.0.0 with source address 10.0.0.2
10.1.0.0 sends large DNS response to 10.0.0.2.
In this way 10.0.0.1 has amplified its attack traffic to attack 10.0.0.2.
You could get a similar “amplification” to what is mentioned in this article by downloading an image with a small HTTP request. It is slightly worse because you get various hosts to help out. But it really isn’t that effective because it only works if you can post to accounts where people are following you or where people re-share your post.
If it were a practical attack that number would matter. But then we’d also see it happening as an attack, and I’m not aware of “high-reach accounts on Mastodon post links to take sites down” being a thing?
I think the concept of an amplification factor is not very relevant here. One could imagine “make a network of fake accounts following each other and spamming links” as a potential vector, but it’d also likely be caught and cleaned up fairly quickly, and such low traffic levels is just trivial to do in many other ways. Amplification factor is mostly interesting if you can scale it to large-scale attacks, and you can’t do that here.
I literally Ctrl + f’d “hash”. I knew that would be part of this.
By switching to a HashMap based on aHash, we achieved nearly a 40% performance improvement in table lookup.
This is the first thing I do when I audit a crate for performance. By far this is the single most significant performance footgun you will run into. I know that there was way more than this, but I have had this exact situation happen “that Go code is ~X times faster” and I closed almost the entire gap just by tuning hashmap usage.
If you are interested, you can also try to refactor the deserialization code using slices, and we’d be happy if you shared your experience with us.
If I’d read this post a year or two ago I can pretty much guarantee I’d stay up all night doing exactly that. But I am tired.
Yeah, the hash algorithm is a major difference. Rust uses siphash 1-3, Go uses some sorta aes based algorithm. I’m a bit reluctant to put numbers down but I think we’re talking ~10x performance penalty with siphash.
Not really, or at least not to the extent that it would materially affect this sort of comparisons. The hash was chosen for its reliability and resistance to adversarial inputs (especially as the hashmap was designed to have a very high load factor).
But the hash is pluggable so it’s reasonably easy to use cheaper ones when the situation allows.
I don’t know. It’s already changed from an even slower siphash, so I suppose there’s always the option. Or there could be a faster hash in the stdlib for people to easily switch to, whereas you need a 3rd party crate currently.
I would love to hear from an expert on this topic, if anyone is aware. Is what Go does insufficient?
I look at ahash and I’m just not sure that it’s meaningfully less safe that the std library’s approach.
You can always incorporate a random seed, such as a per-process random seed. I suspect that this is sufficient if an attacker can’t easily derive the seed or characteristics of it? Or that the space of keys for that seed is sufficiently small such that they can not brute force through that mitigation.
How strong do the properties of the hash really have to be? Algorithms like what go uses seem to have pretty good distributions. Is “pretty good” not enough? How do we think of the threshold here? Perhaps random seed + fnv is no good, because fnv is so easy to trigger collisions in that even with a seed an attacker could easily explore a space that leads to issues. But aeshash really isn’t good enough? ahash isn’t? What are the properties and thresholds that siphash meets but ahash does not?
I have read it before but I’ll refresh myself on this, sorry if the questions are answered in that document, I posted this and then remembered that the doc exists.
They can’t be compared directly, because Go uses separate chaining while Rust uses open addressing (with a very high load factor) so distribution / conflict requirements don’t match.
Using AES doesn’t automatically guarantee robustness. Ahash applies it in its own way, which in the past had some suboptimal cases and a side channel leak.
I think there are a few reasons stopping libstd from switching:
SipHash claims to be cryptographically secure-ish, but AFAIK there has been no proper cryptoanalysis of ahash. So even though ahash could be as good or even better, it’s unknown at the moment.
ahash takes advantage of AES hardware, which is not available on all platforms. It would require fallback for some other platforms, and it’s not great to have unexpectedly large differences in performance or security between platforms.
Rust prefers safe defaults over performance. The users who don’t know if they need DoS resistance, get it by default.
I didn’t mean to imply that aeshash would be good by virtue of AES, only that the hash has good distribution. I don’t think cryptographic requirements matter here, I would assume the issue is statistical distribution in the face of malicious keys.
I’m not sure if Rust should switch but I definitely think there should be an alternative hasher in the stdlib.
This sounds like it would be very sensitive to the order of the numbers in the input. For example say I have a steam of coordinates for vertices. If I write them out as x0,y0,z0,x1,y1,z1… then I would expect to get much better compression by switching to x0,x1,…,y0,y1,…,z0,z1,…. order instead, since the x coordinates are much more likely to be correlated with each other than the y coordinates or the z coordinates.
Separately I’m thinking that if you have noisy data near 0, you probably won’t get much compression since the sign bit keeps changing and you don’t get to take advantage of zeroes between the sign bit and the top of the mantissa.
This sounds like it would be very sensitive to the order of the numbers in the input.
Yes, and there are whole classes of compression algorithms for multi-dimensional data that start out with simple algorithms like this XOR one, but wrap them inside of shuffle-and-slice preprocessing steps to make “similar” values adjacent in the stream.
Separately I’m thinking that if you have noisy data near 0, you probably won’t get much compression since the sign bit keeps changing
If you expect that, you could probably get a savings by changing the coding to more efficiently deal with sign bit flips instead of having to extend the window back to bit 0.
It still might not be great, because the kind of data that crosses zero a lot probably has a lot of variability in the exponent, and a more-or-less random distribution of mantissas — but it might still benefit: say the values are all between -1 and 1 and only have a few decimal places of precision. That’s another case of “well, why not just code them as 16-bit fixed-point then?”, but supposing that’s not an option, then, well, the exponent is mostly only going to vary in the last few bits and the mantissa only in the first few, so it works out tolerably.
This is really clever. It’s hardly obvious to another reader, though.
In C and C++, I have written
#if 0
code variant 1
#else
code variant 2
#endif
The intent is clear. I can switch between variants by modifying a single character, changing 0 to 1 or vice versa. The code highlighting in my editor (vim) figures this out and greys out the disabled code.
If C/C++ can do this, maybe you should demand that your favourite language should support this feature as well (statically switching between code variants I mean, not necessarily by adding the C preprocessor).
This kind of technique is done strictly for your eyes only, in the heat of debugging. It’s only for the convenience of the author/debugger. If it needs to be committed longer-term, yeah: please do anything else. However such techniques are still extremely valuable to have in your personal toolbox; do not disregard them just because they shouldn’t be committed for others to read.
It’s the same difference as writing for thinking vs. writing for an audience.
I use crate features for this in Rust, which is somewhat tedious if the condition is only in one place, and immediately wonderful as soon as I need the same condition in a second place. Temporarily of course. IntelliJ highlights enabled/disables features appropriately.
Inserting with multiple VALUES at a time is definitely a good idea. You may also want to switch to preparing the INSERT and reusing it multiple times. Both for inserting a single row many times and for batching multiple VALUES many times in one transaction. You may also want to take a look at COPY FROM or LOAD DATA LOCAL (if on MySQL).
In my project i am using sqlite due to the offline nature of the application. I am not really versed in prepared statements but i will have a look at the documentation.
Oh, it’s the year of Linux on the desktop again? I am a heavy user of Linux, but it. Just. Never. Works. Flawlessly. I use a ThinkPad with Fedora for my daily work/home needs because they seem to be well-supported, but still the power management just doesn’t work for me. It’s still better than Windows though (Mac is too expensive for me).
I wanted to try Arch, but didn’t want to fuss too much like my old Gentoo days, so I installed EndeavourOS on my old laptop. Worked basically flawlessly. I’m very pleased with it.
I know the m1 macs are expensive, but its just day and night from the intel ones, not even mentioning fixing the shitty keyboard, but just the overall performance is amazing, I hope they will get cheaper if apple keep releasing new versions, but as I say in the article, the issue is Apple’s far reaching control over the code my computer executes.
It’s interesting how people’s experiences can differ. I have mostly used macOS since 2007 and I cannot remember when I last had an issues with suspend/resume.
I had a Linux-based ThinkPad in 2020/early 2021. One of the models that a lot of Linux users recommended and certified for Linux. Sleep set in the firmware to S3 Sleep (Linux). The battery would drain overnight in S3 sleep. Resume would often come back with various hardware (often the trackpad or track point, sometimes the display) non-operational. Battery life was often four hours, after all the tuning (powertop, CPU governers, etc.). Windows would work better, except it would sometimes wake the machine from sleep while it was in the sleeve, almost cooking the laptop.
I know quite some people (mostly former colleagues) who use Linux on laptops and many complain that suspend/resume is still a tire fire. Quite a few just shut down their laptop and boot it again to avoid the issues. Many friends and family members have MacBooks and they open/close their lids many times a day without any issues.
I have mostly used macOS since 2007 and I cannot remember when I last had an issues with suspend/resume.
Oh goodness, I have them all the time; Mac laptops tend to get cranky when you let the power fall below 10% and usually refuse to turn on if the power is below 1%. Inevitably I find this out when I have to switch from my work MacBookPro to my Personal for a client call or the like haha
The intel MacBooks from roughly 2016-2017 onwards had a lot of issues. There was a definite quality drop in the years before M1. But my 2013 MacBook Pro had instantaneous and flawless suspend/resume as the current MacBooks do.
Former Red Hat staffer here. They issued me a shiny new Thinkpad X240… and 3 different and conflicting sets of docs on how to configure it with Fedora. All, incidentally, were wrong in some details.
At the end, my trackpad didn’t work, if I used both of the supplied 2 external screens then my internal screen didn’t work, and other issues. I complained to my colleagues. I was told: “oh, yes, that’s normal.”
In other words, for clarity: Fedora did not fully support some of the hardware that Fedora is developed, tested and documented upon. >_<
I’ve tried most Linux distros out there. It’s my job. Fedora is consistently problematic and has among the worst installers in the whole industry, along with some of the poorest driver support. It’s better now than it was a decade ago, but it’s gone from “unusably dire” to “poor” in my professional assessment.
FWIW, a few years after I left RH, I went to SUSE. OpenSUSE is substantially better in almost all respects and if you dislike and disdain the Debian family for some reason and want to stay in RPM-land, try openSUSE.
Echoing nomemory, I first tried Fedora on my ThinkPad, and it was painful. I guess part of that was the kernel didn’t yet support the hardware? Anyway stuff didn’t work, and stuff broke on upgrade. After distro surfing, I am quite happy with Linux Mint on the same Thinkpad.
JXL is such a cool image format, I hope we get widespread support (meaning e.g Chromium eventually adds back support). These results are impressive. And I really like this concept of “the Pareto front”.
However, this:
Version 0.10 of libjxl, the reference implementation for JPEG XL, has just been released.
Would currently make me think twice about adding JXL support in my own software. I hope we eventually get a version 1.0 of at least a reference decoder, so that those of us who do t want to beta test a 0.x library can support JXL.
And I really like this concept of “the Pareto front”.
This concept has come up a few times in relation to JXL the last few years. A few years ago, Luca Versari (another JXL dev) created fjxl and fpnge to demonstrate extremely fast encoding. At the time, people compared them to QOI and other formats and encoders, with the Pareto frontier being a big point of discussion. fpnge got speeds over 1 GPix/s in screenshots, and over 500MPix/s in photos. fjxl was eventually integrated into libjxl, as the modular mode e1 speed. Luca made this pdf which has some technical details on how they work, if you’re interested.
Would currently make me think twice about adding JXL support in my own software. I hope we eventually get a version 1.0 of at least a reference decoder, so that those of us who do t want to beta test a 0.x library can support JXL.
libjxl seems to work pretty well, with a decent number of vendors using it (Samsung, Apple, Adobe, for example), it seems like it probably works decently well. It’s not 1.0 yet because it lacks some features they want to implement first, it has some issues, and the API isn’t final. However, they are trying to not break anything in the API (0.9 -> 0.10 is a drop in upgrade, IIRC). So I would suggest that rather than looking at the number, you look at their 1.0 issues on GitHub. Some of them look gnarly but most of them are niche-specific, and may not apply to your use cases.
There is also jxl-oxide which the JPEG committee has tested and found to be conformant. It isn’t as fast as libjxl, but it seems like a good decoder.
I’m not saying that libjxl doesn’t work well, but the JPEG-XL people are telling me not to use it yet, and I respect that. (Also, https://github.com/libjxl/libjxl/issues/1450 this issue in particular is pretty big and wide reaching)
jxl-oxide is 0.7, so the jxl-oxide developers are also telling me not to use the library yet, and I respect that too.
the JPEG-XL people are telling me not to use it yet
jxl-oxide is 0.7, so the jxl-oxide developers are also telling me not to use the library yet
I saw this in the last thread on JXL as well, but since when does 0.x mean “don’t use this”? I can assure you that that is not what the libjxl developers mean, in any case.
What would the JXL people mean by 0.x if not “don’t expect this to be ‘production-ready’ yet”? Have they communicated what they mean, or are you just conjecturing?
0.x pretty universally communicates that software isn’t “ready” in some way, to the point that semver defined 0.x versions to mean “initial development”; versions before the software is properly “released”.
FWIW, issues like “will terminate the whole process on error” is exactly the sort of thing I would expect on a 0.x “development” version that’s not meant to be used in production yet.
What would the JXL people mean by 0.x if not “don’t expect this to be ‘production-ready’ yet”? Have they communicated what they mean, or are you just conjecturing?
The question of versioning came up in the JXL Discord. Jon compared the 1.0 milestone with something like libjpeg version 6.
0.x pretty universally communicates that software isn’t “ready” in some way, to the point that semver defined 0.x versions to mean “initial development”; versions before the software is properly “released”.
Semver is why they’re using 0.x. Even though they’re keeping the API as stable as possible, it isn’t final.
Really, going by semvar rules, lots of annoying things like abort() all over the place would be perfectly fine in 1.x versions.
So my point is to not treat “production ready” as a binary, nor to tie it to semver. Concrete issues and how they relate to your use case are what really matter. For example, if you already sandbox all media codecs anyway, maybe you don’t care about abort(). Hell, maybe you’re compiling your code with bounds checking so that all OOB accesses turn into aborts, so a few extra explicit aborts don’t matter much. I’m not telling you to go use libjxl right now, you’ve identified an issue that prevents you from using it right now, so that’s that. If they release 0.11 with all aborts removed, suddenly libjxl would be “production ready” for more people, regardless of that 0 in front, so I don’t think treating it as an indicator is very useful. Just about the only thing it indicates is that you shouldn’t use it yet if you don’t want breaking API changes.
All that is to say, yes, 0.x here means that it isn’t ready in some ways (API not done, missing features, etc), but that’s not the same thing as saying it isn’t ready to be used in production at all.
Semver is why they’re using 0.x. Even though they’re keeping the API as stable as possible, it isn’t final.
Semver doesn’t dictate that you have to use 0.x until you have figured out the final iteration of the API, but rather:
Major version zero (0.y.z) is for initial development. Anything MAY change at any time. The public API SHOULD NOT be considered stable.
Semver just says that once you’re out of 0.x, breaking changes must be in major versions.
Anyway, the project is communicating that the library isn’t to be considered production ready (no, I don’t count random Discord conversations. I haven’t said anywhere that “0.x means you shouldn’t use it in production”, just that you’re essentially using beta software. Of course there are situations where you may decide to use “not generally production ready” software in production, you just need to be prepared to handle it as such. And I’m not.
I don’t really understand what you’re trying to get out of this conversation.
Lots of 0.x software is used in production with the expectation of production-readiness, and a lot of people treat 0.x as jxl does, as @lonjil describes. You may choose to ignore the realities of modern software engineering culture because “dictionary says this lol,” if that’s how you wish to see the world. But you can’t say things like “the JPEG-XL people are telling me not to use it” on the same basis, and pretend the dictionary gives you absolute authority to ignore everything else that has ever been said about JPEG-XL, even by its creators.
This release is for evaluation purposes and may contain bugs, including security bugs, that may not be individually documented when fixed. See the SECURITY.md file for details. Always prefer to use the latest release.
For evaluation purposes is the opposite of for production purposes. It’s not just the “0.x” version number, they directly state it there.
Because all libraries should only be evaluated on test data and never used in any user facing software for any purpose. The 1.0 issues lonjil linked represent caveats that may or may not apply to your particular use case, but if the developers deleted that list and released 1.0 with the exact same code then it’s now production ready and will never have bugs.
My point is, lots of software can be production ready by some metrics to plenty of people, while still being evaluated and refined. The libjxl authors are up front about which issues keep them from releasing as 1.0. If those issues don’t apply to you, there are no problems.
So the warning that the software is for evaluation purposes is meaningless, because all software can be evaluated on production data. It’s a meaningless warning and should be removed.
Not at all. But there are different levels of production-readiness, that a single 0 / >0 bit doesn’t express exhaustively. While two releases at 0.1 and 0.10 may both be considered “for evaluation,” the context in which they should be evaluated is probably different.
I’m just saying I’ll respect the clearly communicated request of the authors to treat libjxl as pre-release software in its current state. I don’t understand why you’re telling me to ignore their wishes. You’d have a point if you could link to any official communication which says that libjxl is supposed to be treated as production ready, but you can’t (the best thing you have is seriously alleged Discord messages?), which leaves the pre-release version number as the only piece of official communication on the topic.
The lack of a 1.x decoder library for JXL is an adoption issue, no matter how much you screech at me to ignore it. It’s not like convincing me will change anything; the goal is wide-spread JXL adoption, not to successfully convince me to incorporate it in its pre-release state into whatever hypothetical software with 2 users I may eventually write in the future, right?
It’s not like convincing me will change anything; the goal is wide-spread JXL adoption, not to successfully convince me to incorporate it in its pre-release state into whatever hypothetical software with 2 users I may eventually write in the future, right?
idk about Peter, but I said that you shouldn’t use it if it has issues that prevents you from using it. My point was purely that the number in front isn’t relevant when you have real concrete issues to base your decision on.
The fact that the project clearly communicates “please don’t use this yet if you don’t want to deal with the issue you’d expect from pre-release software” is the issue that prevents me from using it, for the most part.
Punishing users to show the middle finger to a regulator is a weird approach. A few days back, Amazon was a no-show at European Parliament’s hearing on working conditions in their Polish and German mega-warehouses. Is the Chinese market the single one where these companies are not so cocky?
you can either keep the current contract, or, accept the new one which enables you to distribute outside the app store and also use any kind of in-app processing you want, but in that case you also agree with the Core Technology Fee. This fee makes really popular free (and without monetisation) apps unsustainable.
No, you can opt to keep the current one apparently, but that also prevents you from shipping outside de app store or using other browser engines or using your own in-app payment processor. The new DMA benefits are only for the new contract. So is a choose your evil kind of scenario.
As far as I can make out the new terms only apply if you distribute outside of the main App Store.
There is no requirement I can see that prevents you shipping browsers in the main App Store, if you can point to where you’re seeing the rule prohibiting it it would help.
If I’m understanding the terms correctly, it also applies if you want to distribute your app on alternate marketplaces, which means that free devs have to choose between paying the 0,50€ per install surcharge over 1m installs and still being locked into the App Store.
Also, there are a few exceptions that apply to browsers, so Firefox (which is not just a skin) could come both to the AppStore or an alternative store just fine, with minimal fees (developer account).
if you think that a free app with two million users in the EU in the new contract is sustainable, then you have a different standard than I do because I just used Apple’s own fee calculator and that amounts to 45,290 USD per month.
I think this is more an issue for free apps that are not possible now. E.g. it is conceivable that an emulator could get > 1 million users, but emulators are not allowed in the Apple App Store.
I have several emulators and full blown qemu (UTM) on my ipad/iphone right now. Technically free though I do pay 3$ per month for patreon beta access (for altstore/dolphin).
This has an odd focus on preventing piracy of media; my concern is being able to run whatever OS and application software I want, with whatever patches I want. Additionally, as long as I correctly implement the network protocols, I would rather like to be able to interact with the rest of the world.
It’s not an odd focus, it’s just something you don’t care about. The motivating quote of the article is specifically about media streaming DRM.
The very first lines of the post are:
So this post is a response to the FSF’s comment, and it was the FSF’s choice to put the focus there.
Luckily, the author has written about that topic at length as well, just not in this specific blog post addressing one other specific claim.
I believe that the common advice of “always store time in UTC” is a bit overplayed. There are many cases where you actually need to store local time (AND a corresponding timezone name). Basically when you handle any time in the future. Only rarely you actually need “now + X * 86400 seconds”, most of the time it’s “October 10, 2030 at 20:00 German time”.
I would say that storing this as an integer: 20301010200000 would even make some sense because it will discard the time-related assumptions.
I rarely store timestamps in the future. I cannot recall when was the last time I needed that. And for times in future (like calendar events) I agree, but that is super rare case that you need to store such timestamp.
Leap seconds are another reason not to store future times in UTC, though it seems those might be finally going away. UTC always for past times is probably very reasonable, though.
UTC is for timestamps. When something actually happened.
Rarely for things like “expire in 24 hours”.
I mean we all understand how many times people were burned by timestamps stored in local time, so this overshotted advice seems to be kind of a valid precaution.
You can safely ignore leap seconds in basically all cases, e.g. UNIX time does not include them. A day in UNIX time is always 86400 seconds, and in practice clocks are adjusted around leap seconds with leap smearing, or just setting the clock back 1 second at the arranged time. If you actually care about highly precise timing, you have to use specialized libraries that are aware of leap seconds. Most things ignore them, and mostly 1 extra second of real time here or there doesn’t matter.
That 1 second matters even less when you consider that all clocks you have access to through conventional APIs are some degree of wrong—either non-monotonic, unsteady, or both.
That said, I agree UNIX time shouldn’t be used for future dates/times that have any calendar semantics.
The only way to avoid leap seconds is to use TAI internally, and keep a separate table for the UTC offset (kind of like timezones). Not sure what applications need second-level accuracy in the mid-to-long future, though.
I have always just stored times as epoch timestamps and found it infinitely easier. What is the downside?
Epoch time is just another representation of UTC, as far as I’m aware. So if you’re storing future dates then you have the downside described in the OP.
Oh, of course. Thanks.
An epoch timestamp still wouldn’t be able to recompute to an intended date if the underlying timezone/DST/similar assumption changes.
I sense some over-optimization going on here. Perhaps slightly OT, but then, not having to write unsafe code at all is certainly one solution to the issues raised here, right?
In a program where the items being queued are files, and the processing involves generating ML embeddings, memory allocation in the channel is going to be lost in the noise.
And the slowness of allocators is pretty OS dependent — from what I’ve heard here, Linux (or glibc) has a particularly slow malloc implementation.
Unless I’m missing something, this is untrue. A Vec only allocates memory when it has to grow its capacity. Once the queue warms up, there should be no more allocations.
That’s my understanding as well. It should happen very quickly for most systems, and remain stable for the lifetime of the channel. That fact removes the need for all the other complexity.
It’s also pretty easy to use jemalloc in Rust, just a few lines of boilerplate and you’re all set.
We also provide Rust bindings in snmalloc. Rust tends to encourage the kind of thing that we optimise for (producer-consumer things with allocation in one thread and deallocation in another are a pathological case for thread-caching allocators like jemalloc and are very fast in message-passing allocators like snmalloc)l
We got big gains switching to jemalloc at $JOB, I’ll have to give snmalloc a try when I have some spare time. I’m not sure how much cross-thread message passing we do in practice. I think many of our messages go between Tokio tasks on the same thread, but it’s hard to know without measuring.
For single-threaded apps, we typically see a 1-3% speedup relative to jemalloc. For things with high inter-thread communication, we’ve seen 20-100%.
Just tried to use snmalloc in Rust. Let me describe two problems. I say all this to help you make snmalloc better.
cargo add snmalloc. Unfortunately, crate https://crates.io/crates/snmalloc seems to be empty. So I suggest reaching out to its owner and asking for transferring this namesnmallocdependency needs C++ compiler (as opposed to C compiler). This makes building slightly harder. I have no problems with installing C++ compiler, but I don’t want to complicate building instructions for users of my code. Rust has great advantage: nearly all projects can be built with justcargo build. Yet if a project depends on snmalloc you have toapt-get install g++(or similar) first. Also,snmallocdoesn’t give me any significant benefits over jemalloc or mimalloc, so I will just use jemalloc or mimalloc for my projects. Simpler build is more important than hypothetical speed advantages I don’t care about in my projects. Yes, jemalloc and mimalloc depend on C compiler. So what? Very popular cratereqwest(and many other popular crates) depends on C compiler in its default settings, too. So, most Rust developers already have C compiler installed. But not necessary C++ compiler.So, my personal conclusion is so: I will use default (system) allocator in all my Rust projects. If it performs badly, I will switch to jemalloc or mimalloc. But never to snmalloc, because it doesn’t give any real advantage over jemalloc or mimalloc, while complicating build, which is important for me (because I care about my users). If, say, snmalloc will make my whole program (which consists not only of allocations) 2 times faster than with jemalloc and mimalloc, then this will be possibly enough reason for switching to snmalloc.
All these is not attempting to insult you. I’m just trying to deliver constructive critique, so you will make snmalloc better
If you actually want to improve it, the right place to post this is on the issue tracker. The Rust integration is contributed by someone who uses it, so they’ll see the friction for other people and maybe fix it.
We obviously can’t do anything about the C++ requirement, big given that every platform supported by Rust has a default C compiler that is also a C++ compiler, I have no idea why that would be a problem.
Yeah, the original version of the crate did not have any unsafe code. But I previously worked on projects where every malloc() hurt, even with jemalloc, and sometimes you have to use the system allocator. (Certainly library crates cannot dictate choice of allocator to downstream consumers.)
You’re slightly misinterpreting the use case: some of the items being queued are inodes, likely served from kernel cache, and straight-line performance matters sometimes. (Of course, the second you have to hash a large file, then yes, that’s going to dominate.)
I do have benchmarks, but I did not think to include them. Someone on Reddit asked the same question, so I’ll see about publishing something.
On amortized allocation and Vec, if you sit down and try, it’s actually hard to amortize the allocation cost in this particular use case. In the least, you have to introduce a second mutex acquisition to store the backbuffer somewhere. More details here: https://news.ycombinator.com/item?id=41947924
That said, yes, I could have ignored the small allocation cost, but it was worth the exercise to learn the rules, and if Tokio and LILOS can do it, so can I!
What is there to amortize? A channel will quickly grow to its steady-state size and usually stay there. And that size is much, much less than the number of items that go through it. So you might have a handful of allocations before the Vec grows to 100 items or whatever, then no more. If that concerns you, you can initialize the Vec with that size.
You just described amortization. It would amortize to 0 allocations per task sleep.
Either way, I described why it can’t easily work that way in this use case without an extra lock or some global lock-free pool. (Which I did implement! https://docs.rs/wakerpool/latest/wakerpool/)
And it’s pretty easy to switch to jemalloc, mimalloc, snmalloc, or probably any number of other allocators, if the concern is that the OS-supplied allocator will be too slow.
I think the opposite: you’re better off defaulting to relaxed. Atomics are always atomic with respect to themselves. You don’t need anything stronger than relaxed unless you’re using an atomic to synchronize access to memory other than the atomic itself. And if you are synchronizing access to memory between 2 threads in any way, you’re usually better off using a mutex, channel, condvar, or other primitive that abstracts the appropriate synchronization pattern you need.
From the article:
The Lisp Machines would like a word. Actually, a whole list of words.
Smalltalk machines were a thing, too.
This article’s viewpoint is too narrow: it lacks important historical context.
Emphasis mine. The author addresses this point literally two sentences later.
It’s a strange contention. There were commercially successful Lisp machines from multiple companies for a decade (Symbolics, LMI, Texas Instruments, etc.). What killed them was the Unix workstations: the technology reached the point where you could take an off the shelf microprocessor like an m68k, run a BSD variant on it, and produce a machine with impressive specs for the day at a fraction of the cost of building hardware and OS from the ground up. The same force killed off Burroughs large machines and Connection Machines and a bunch of others. You could make the same contention that Unix workstations mostly did not succeed since Sun, DEC, and SGI died when you could take an off the shelf PC and produce a machine with impressive specs for the day at a fraction of the cost of a dedicated Unix workstation. They had a similar run of roughly a decade, just like the transistor machines with magnetic core memory killed off vacuum tube and delay line machines and machines with integrated circuit memory killed off transistor plus magnetic core machines.
They were commercially viable for a while but “successful” may be a bit of an overstatement. Between them, Symbolics, LMI, PERQ, Xerox and TI had sold less than 10,000 machines (on the order of 7-8000 IIRC?) by the end of the 1980s. The workstation market wasn’t exactly a mass market so it’s not a bad figure, but it’s not exactly a resounding success, either. E.g. SGI alone sold 3000+ units of its first-generation IRIS system, and they were a small player at the time.
It looked more commercially viable than it was because much of the early development cost had been pre-paid in a way. E.g. early on, Symbolics, LMI and TI largely used the same software, licensed from MIT, and they all started pretty much with MIT CADR machines. Symbolics initially planned to start working on their own hardware right away but it took them a couple of years to kickstart the 3600 project.
Even at the peak of Lisp popularity, three (i.e. HP, DEC and Sun) of the five major companies with important Lisp offerings didn’t sell Lisp machines. The other two were Symbolics and TI, but even within Symbolics, lots of business groups weren’t profitable. I think the only one that remained profitable throughout its existence was the Macsyma group.
It also didn’t help that their hardware, despite being very much at the edge of what was possible at the time, was still not enough. Early Symbolics Lisp machines actually had an M68K along with the 3600 CPU. The M68K (the front-end processor, FEP for short) handled startup and some of the peripherals. Merely integrating Lisp machine hardware was expensive.
Unix workstations were one of the things that killed Lisp machines, but not quite by using off-the-shelf hardware. Symbolics hardware was actually contemporary with M68K-era Unix workstations (LM-2 was launched the same year as the DN100, 3600-series was launched the same year as Sun-1 and IRIS).
By the time that Lisp peaked in popularity in the second half of the 1980s, most big workstation manufacturers had actually moved on to their own stuff (SGI started using MIPS RISC in 1987, Sun started using SPARC in 1988, DEC were doing their VAX thing, IBM’s main workstation offering was trying to make ROMP take off since 1986). When Symbolics really started going downwards, the Unix machines that beat their Lisp machine counterparts used their own proprietary CPUs, too. NeXT were, I think, the only ones doing high-tier workstations with M68K CPUs, and even those were slow enough that they were largely excluded from particularly high-performance market segments (e.g. high-end CAD applications).
Well, do you?
They existed and were exactly what I alluded to. They were never “a thing”
The idiom “be a thing”
Hmm…or you could have just just read the next two sentences, where that historical context was provided:
I find this a very strange and hard to parse reply.
From your tone and back-checking a few links, I deduce that this is your blog post, correct? If that is the case then why didn’t you say so? Why do you obscure this simple fact by using the passive voice and so on?
Me, personally? No. I stopped programming when it stopped being fun, which for me was roughly when Windows started to take off.
I entirely disagree. Indeed this line contradicts itself: the second sentence is invalidated by the first sentences. They existed, therefore, they were a thing.
That means that your supposition in your blog post, that Lisp and Smalltalk only exist inside their own isolated boxes on top of other OSes, is false… and you knew it was false when you wrote it and not only that but you admitted it a line later.
This is simply unfathomable to me.
To be perfectly honest, I stopped when I reached that false assertion and wrote my reply. Then, I read the rest before posting, because I’m not 12 years old and I’ve been on the internet for 39 years now. I am not claiming I’m perfect – I’m not – but I commented on the bit that struck me.
I am a bit baffled by your apparently angry response, TBH.
Marcel said (and you quoted!) “They existed and were exactly what I alluded to.” How’s that obscuring?
If I’m not mistaken, you write articles for a living. Surely you understand why an author might be annoyed at someone for “commenting on the bit that struck them” while seemingly ignoring that the point was addressed later?
For what it’s worth, I’m under the impression that you’re trying to pick a fight and I hope I’m wrong. :)
Because, TBH, that’s the moment I realised “hang on, is he saying he wrote this?”
Sure, that’s a fair point. :-)
Wider context for reference: I am a lifelong sceptic and disbelieve in all forms of religion, the paranormal, alternate medicine, etc.
If I read an essay that starts “homeopathy shows us that X can do Y” then I am not going to pay much attention to the rest of the essay that talks about – well, anything really – because homeopathy does not show that. Homeopathy does not work, full stop, the end.
https://www.howdoeshomeopathywork.com/
So when an article about programming languages says “X is never used for Y, and you can’t do Y with X” when I have stood on stage and told a thousand people about how and why doing Y with X is important, yeah, you are damned right I am going to stop right there and say “hey, that is not right.”
Marcel seems to feel that because something was not commercially successful it didn’t happen. That is not true.
Not at all, but I’ve spent over 40 years calling out BS when I see it, and I actively enjoy making people angry by telling them they are wrong.
I am not picking fights. I am trying to point out logical errors and fallacious claims. Not the same thing.
The grown up way to respond when someone points out a hole in your argument is to change your argument. The schoolkids’ way is to get louder defending it.
I think Marcel’s argument rests on a core point that’s wrong. I am only interested in that core point, not what sits on top of it, which seems to be the bit he wants to talk about.
Do that somewhere else.
You could have stopped with your first comment and just said “hey you know lisp machines were actually a bigger thing”, and gone into a bit more detail yourself, but instead just dismissed the rest of the article as lacking historical context.
Context that is utterly irrelevant to the rest of the article. The author doesn’t say you can’t use Smalltalk or LISP to do systems programming. He says that people don’t. Perhaps try being charitable and read this as “people don’t tend to”. Which is not untrue.
Ambiguity isn’t uncommon in writing, and sometimes readers should just be able to infer things themselves from common sense.
Pick one, mate.
And your belief is incorrect. Language moved on and the idiom under contention is listed in Wiktionary.
How’s it feel to be on the other side of that?
Once again: “X is a thing” is an idiom for X being something that was popular.
How many LISP and Smalltalk machines were built and sold, in your opinion?
What market share did they achieve in the workstation market?
Nope. It means “they existed”, “they were real”.
Doesn’t matter. They were on commercial sale, were real products loved by legions of influential people, and inspired important works that you seem unaware of, such as the Unix Hater’s Handbook, which I submit you need to read.
https://web.mit.edu/~simsong/www/ugh.pdf
Irrelevant. This is not a popularity contest. You are attempting to talk about absolutes – “this only runs in this way” – which are not true. Which is why I responded “this article lacks important context.”
Because with every reply you send, you reinforce my impression that you do not understand the bigger picture here.
The author of the post has linked to a discussion about the origin and evolution of the term, where being popular, especially in a surprising context, is one of the accepted meanings of the word. It’s not just the assertion of the person who asked about it, but also one of the quoted definitions:
Early use of the term seems to have been an ellided version of “is there such a thing as”, but several later instances of its use obviously lean towards the aspect of popularity. Presumably the secondary sense was developed by contamination from the pejorative “a thing”, as in “PCs will be a thing of the past in 20 years”. Regardless, it’s a thing now. See e.g. this entry.
FWIW it’s definitely how I use it, too. Seeing how this is slang, yeah, on behalf everyone else, I would like to take this chance to not apologize for using it in a way that you disapprove of. I’m sure that, if a whole generation of stationery users woke up one day in a world where “to clip” meant both “to cut” and “to fasten” and managed to get over it, you’ll eventually come to terms with this horrifying misuse of language, too.
Is it ambiguous? Yes. Does minimal charity resolve virtually every instance of ambiguity? For heaven’s sake also yes, is it really not obvious that an article that literally says “attempts to bring machines closer to these languages mostly did not succeed” acknowledges that attempts to bring machines closer to these languages existed? What exactly do you think is missing here and isn’t obvious to a community that has a higher incidence of Lisp per capita than the daily appointment list of a speech therapy clinic?
Definitely not in 2024, as you can check easily by googling for /what does “it’s a thing” means/
This is how it is in C and C++, which Go is closely related to. I think this is only surprising when compared to languages where literally everything is a reference (e.g. Java, Python, JS, or Ruby).
Debatable, as copy constructors are usually deep.
That is irrelevant. When you do
a = bin C++, in all cases where the type ofaandbis astruct(orclass), you will get either copy ofbina, or you will get a compile failure. Whether that’s a deep or shallow copy depends on the copy-assignment operator for the type ofa, but it will always be a copy of the struct.I don’t think default ones are.
They are… sort of. As in, the copy constructor of members will get executed. However, the copy constructor of a pointer just copies the pointer.
So they’re “deep” as in “members also get copies”, but not “deep” as in “members which are pointer types will allocate a new object and point to the new object”.
This is exactly what’s described in the article, and isn’t considered a deep copy.
In C++ copied strings and vectors will result in deep copies with new allocations, as they are NOT pointer types. Go strings and slices are copied shallowly as they ARE pointer types.
Also the original problem was Go ranges iterating by value. C++ can iterate by reference.
“Members are copied recursively” does sound like a deep copy, in a way, though. And you could totally make a
CopiedPtrclass which is sorta like astd::unique_ptrexcept it has a copy constructor which allocates and copy-constructs a new pointed-to object withnew.This is generally what’s referred to as a deep copy. A shallow copy is
memcpy(dst, src, sizeof(*src))It’s not a memcpy though. It’s recursively running copy constructors. Strings and vectors will allocate a new buffer and copy their contents over; that doesn’t sound totally “shallow” to me.
And in fact, you could make a
CopiedPtrclass which allocates and copy-constructs a new pointed-to object withnew. An object tree where all pointers areCopiedPtrrather than the other pointer/reference types would be deeply copied.C++ is too complicated for simple statements like “copies are shallow” or “copies are deep”. Copies are customizable. And the language’s and standard library’s pointer types are copied shallowly (or refuse to be copied altogether, as is the case with unique_ptr), while the standard library’s container types are copied deeply.
That’s true. It is in between a bit.
But given the original scenario of pointers, where the default copy constructor is basically memcpy, I feel like it worked for that example.
I still feel like the phrase “copies are usually deep” is overlooking a lot.
And really, things like strings don’t use the default constructor (the thing I specified), they use a custom one
This exploit is currently being used by ticket scalpers. And bad actors who are selling fraudulent already-sold tickets. And, perhaps, a few honest people who just want to sell their ticket due to a change of plans.
I come from a family of professional, touring musicians. Scalpers are a nuisance, but Ticket Master is an abusive monopoly. Regulate them if you can; hurt them if you can’t.
You say “exploit”, I say “documentation”. You say “scalpers”, I say “resellers”.
If you want me to have sympathy for Ticketmaster (i.e., “evil monopoly”) you’ll need to work harder than that.
Ticketmaster’s solution is bad, Ticketmaster itself is bad, but scalpers are still the scum of the earth. Scalpers harm artists, venues, and attendees, benefiting only themselves.
Resellers simply take over the sales business from whomever they buy their tickets from. Venues also hire marketers to determine how to maximize profits from the sale of tickets; I don’t know why it’s suddenly worse if the marketer is independent rather than working for the venue.
I would also like to know why TicketMaster is not the scum of the earth, since they also harm artists, venues, and attendees. Plus they have the market dominance to reshape the ticketing business and fuck the world to a much greater degree than resellers.
Scalpers engage in arbitrage by using automation to rush ticket sites, buy as many tickets as possible, and then use their control over supply to drive profit. It’s monopolistic behavior, not market behavior. If everyone had access to scalper-level technology for getting tickets quickly, you might be right; though I think that would be worse in other ways (can you imagine trying to keep a server up in those conditions?).
The alternative approach is to try to force scalpers to use only the technology available to consumers – which means things like hardware-bound tickets that can’t be resold.
How is that any different from what ticket sites do themselves?
Ticket sites don’t compete with attendees for tickets. They compete with other ticketsellers to be the agent of the event owner. Ticketmaster/LiveNation are monopolists who engage in terrible anticompetitive conduct to maintain their advantage, but it’s a different problem than scalping. Tickets are naturally non-fungible whereas ticket-selling services are fungible.
But do these differences imply that one is worse than the other? Is there a value judgement in the observation that scalpers monopolize the ticket supply through automated purchasing whereas ticket sites monopolize the ticket supply through a business relationship with the venue?
No, but I want neither to hold monopolies. If Ticketmaster didn’t have a monopoly, nothing would change in what I said about why scalpers are a problem. So I was talking about why it’s worse for “the marketer to be independent” as you put it.
I don’t follow the logic. Can you explain more directly why it’s worse for the marketer to be independent?
Scalpers are not marketers. When’s the last time you bought tickets to an event you only heard about because of a scalper? When’s the last time you couldn’t figure out who the ticket seller was for an event until you saw promotion from a scalper?
Scalpers engage in arbitrage by using automation to rush ticket sites, buy as many tickets as possible, and then use their control over supply to drive profit. It’s monopolistic behavior, not market behavior. If they were just marketers, what they were selling would be available even without going through them, just as you can buy from Lowes because you’ve seen their advertising, or even if you haven’t.
Certainly Lowes’s marketing team doesn’t buy as much of the stock as they can until the store has no supply, then stand outside the store reselling the stock at a much higher price. And again, fungability is important here: in my hypothetical, there’s no difference between the screwdriver the marketing team is selling and one inside the store, or at another store. Not so with tickets, where (assuming reserved seating etc) each one is non-fungible.
Scalpers are a very abstract concept for me because I’ve never actually encountered one. But I have to imagine they do something after buying the tickets in order to find people to sell them to, no?
But the effect for the consumer is the same: The marketing team sets the highest prices that they think the market will bear in order to extract maximum value from customers. So what exactly makes it worse? That they’re standing outside the store?
I don’t follow this at all. Why use screwdrivers as the example then? What if it’s the MSG marketing team selling tickets outside of MSG?
because in the real world “oh if you can buy this ticket you can sell it for a $100+ profit” leads to a bunch of people buying tickets purely to resell them, instead of people buying tickets to use them.
Of course sometimes things happen and people want to resell the ticket. But honestly “100% refund on tickets are available up until the day of” for most events would help resolve a lot of that sort of issue.
Reselling a ticket because you can’t go is fine and normal obviously. I have a hard time seeing a justification for needing to make a profit off of that activity.
How is it different from buying books just to resell them? We call that a book store
I’m having trouble interpreting your “because.” Are you saying this is the reason Ticketmaster is not the scum of the earth?
Sorry, my usage of “because” is incorrect. But I do believe that not all resellers are created equal.
certainly
It’s not about sympathy for Ticketmaster, it’s sympathy for the people who want to buy reasonably-priced tickets. Ticketmaster is part of the problem, but scalpers systematically price tickets out of most people’s comfort zone.
How widespread is the scalping problem? Have you ever actually had to buy a ticket from a scalper at a markup because scalpers bought all the tickets from the original seller? I certainly haven’t, going back well before TicketMaster’s insane restrictions.
I suspect that it’s pretty uncommon because a good opportunity for a scalper depends on the venue selling tickets for less than they could be selling them, thus forgoing profit.
On the other hand scalpers will also sell tickets below the original price if people don’t buy them. It can allow people to attend events that they otherwise couldn’t afford.
For background: I used to go to concerts a lot (one year, I went to a concert every third day) and still go quite a bit. Scalping used to be a huge problem, for popular bands, about a third of the tickets went to scalpers. And that was in the times you’d usually go to the ticket office and buy manually. As it’s an arbitrage business, if the whole thing fails, you’d see people in front of the venue with about 12 tickets each, selling them off.
Well, concerts used to be a lot cheaper. Venues have figured that part out. 50 EUR used to be an expensive concert, nowadays, it’s just Tuesday. As with any form of arbitrage, things move towards an equilibrium.
I implied above that i think it became less, but in particular i think it’s because venues themselves have closed the delta between the possible ticket price and what they are asking, making scalping more risky, because margins are lower. Essentially: Monster Magnet touring Germany isn’t worth scalping anymore (MM were relatively cheap, but have a cult following and played relatively small venues that often sold out), while Rage against the Machine may still be worth it (drops by every few decades, sells out immediately).
I’ve never seen that TBH, they’d rather eat the loss, maybe you can negotiate at the door a bit. I think that’s painting those in too good of a light.
I see no gain in scalping and the prices for a scalped ticket for a pretty rare tour (e.g. A perfect circle) are easily 5x the original price. None of that goes to artists or the venue.
So, now we have ticket buy restrictions (4 per sale), crappy apps, etc. But I do agree on the level that scalpers are just the convenient argument to introduce them. There’s also a lot of environmental changes: concerts are now the “authentic, non-canned” experience and for many artists a much more important source of income rather then selling music online.
HOWEVER, that doesn’t mean Ticketmaster and all of its shenanigans aren’t bad. None of that is in the interest of the customer and most of it is about controlling the resale market and profiting again. Selling or giving away tickets between friends is just a normal thing.
One more thing: scalpers also went from the local music nerd who made a few hundred on the side by exploiting their knowledge about the market to highly automated operations trying to skim the top line of the what people are willing to pay.
I don’t think any of them care much about the music fan not getting a ticket because it’s scalped and ticketmaster in particular isn’t the fix. Both groups will extract as much money out of the venues, the fans and the artists, and not care, and will happily use each other as an argument.
My fix for this is indeed that I’m back to buying paper tickets, dropping by the venue if possible.
I don’t think any of this has helped making concerts more accessible. If you were okay with just seeing a band take stage and using some instruments, tickets were much more affordable around the 2000s, scalping or not. That also shows in the crowds.
Sounds like you actually understand the situation. :)
One thing though:
But you also said “if the whole thing fails, you’d see people in front of the venue with about 12 tickets each, selling them off.” Are you saying when that happens, they still refuse to drop the price below the initial price because “they’d rather eat the loss”? I would think a scalper who cuts their losses would have a competitive advantage over one who doesn’t.
I think there’s a few things at play here:
I think, and this is me spitballing, this leads to a situation where as a seller don’t want to go to low to soon and rather keep the price up as long as possible.
However, very much anecdata.
It’s an interesting problem to think through.
I don’t follow. Did the scalper buy a only a few tickets? Or is the concert sold out because the scalper bought up all the tickets? Obviously the scalper is incentivized to sell tickets at the highest price he can, but what if all of the people who are willing to pay face value already have tickets?
Why assume that the venue is outside of town?
Are you saying this is your personal experience, and you saw that the people buying tickets from scalpers before the show were paying face value?
Then are you agreeing that they will drop the price eventually? Especially if the venue gets foot traffic from uncommitted potential customers.
I don’t know how widespread it is, but, yes, I’ve personally purchased a ticket from a scalper at an absurd markup because scalpers bought all the available tickets from the original seller. The year was 1996. We wanted to watch the University of North Carolina Tar Heels defeat the Duke Blue Devils in Chapel Hill. The face value of the tickets was around $40. Relatively few of the tickets in the Smith Center (which at the time held ~19k people, I think?) were available for sale to the public after the two universites’ obligations to their stakeholders were met. We wound up paying about $200 per ticket for a pair of seats, and I personally know someone who paid nearly twice that for modestly better seats with a similar face value.
If $200+ per seat is the rate the market supports, so be it. But I’d have preferred that the university take that profit directly from me and my friend and use it for their mission, as opposed to having the majority of our expenditure line the pockets of a scalper.
UNC won by a single point in an exciting contest, which was one of the legendary coach Dean Smith’s final contests against our largest rival, and we felt it was very much worth our outlay.
Also, I’m pretty certain selling the tickets to us above face value was what made them “scalpers” in the first place; that is a term of art referring to someone who sells tickets at a markup. People who sell tickets below face value are not scalpers, at least in the legal sense where it’s banned in many jurisdictions…
That makes sense; I guess the real harm comes when you can afford the face value price but can’t afford the marked up price and then you miss out.
People who sell tickets at a markup and people who sell tickets below face value are not mutually exclusive categories. You’re saying if a scalper tries to cut their losses they cease to be a scalper?
I’m saying that the offense the police used to charge you for (that was colloquially known as “scalping”) was tied to selling the tickets above face value.
If you were only caught reselling tickets below face value, you would not be charged with that specific offense, as I recall. It’s been quite a while since I’ve been very interested in this, though. I suspect things have changed a bit.
Right but someone who robs banks is a bank robber even if they are between robberies.
We’re splitting a strange hair. I suppose, to be more precise, I was saying that reselling a ticket for less than the sum of face value plus whatever service fee is allowed by law does not constitute “scalping.” Scalpers could also engage in non-scalping resale, to be sure. And in bank robbery. And in unlicensed dog walking.
I took that as you disagreeing with my statement that “scalpers will also sell tickets below the original price if people don’t buy them.” If you’re saying something that’s consistent with that, then OK but it’s strange to make that point if it has no bearing on anything I said.
I reject the implication that ticket scalpers are not honest and that TIcketMaster is not a bad actor.
I’m sure the author already knows this, but you could accomplish the stated goal for the SIMD swizzle use case with a
const fn:Full playground with FetchEven and SwapPairs examples.
“Adjective first” is one of my least favorite parts of English. Even as a native speaker, I find it terribly awkward to have to buffer all of the adjectives in my mind before I hear what the noun is that they’re applied to: “the big fluffy white … dog”
Like ISO 8601 dates, this article is correct to say that the noun should go first and all modifiers/adjectives come after.
Even more fun, “big fluffy white dog” is canonical and other adjective orderings like “white fluffy big dog” are incorrect! And adjective ordering is not universal across languages, or even for some closely related languages.
Many adjective second languages aren’t exclusively adjective second either. In Spanish, “zapatos viejos” means “old shoes,” but “viejos zapatos” means “OLD shoes” (i.e. emphasis on old).
For an unspecced language like English, both “canonical” and “incorrect” raise the descriptivist’s ire in me.
Further to your second point, many languages differ on whether certain concepts are expressed using adjectives or other parts of speech, or whether they even separate them cleanly.
Other adjective orderings “feel wrong” to virtually all native English speakers, even though most speakers are completely unaware the rule exists.
If descriptive linguistics didn’t care about “canonical” and “incorrect” then the field would be extremely shallow. “Just like, say any words dawg. Believed by all, eye of, within is held correctness, the beholder’s semantics irrelevant linguists!” You can probably piece together what that’s supposed to mean but it’s most definitely incorrect.
Is it “canonical incorrect version” or “incorrect canonical version”? :P
I only made the point because several other orderings of those particular adjectives don’t feel wrong to me, a native (Australian) English speaker.
(And fair point about “canonical” and “incorrect”—I just felt they were a bridge too far as applied here.)
Yet you still said the canonical “big fluffy white dog” instead of any of those other orderings, as the vast majority of native speakers would. Other adjective orders aren’t incorrect because they’re bad, they’re incorrect because descriptively those orders are almost never used in practice.
Put another way, if a non-native English speaker wanted to sound more natural to the widest audience, they would learn the correct ordering, even though the meaning of their words would be perfectly clear out of order.
? I think you’re mixing up me and who you replied to initially. Your point is not made.
edit: to be clear, the one I’d most likely use (though it’s hard to say now for sure, obviously!) is “big white fluffy dog”. Why? I enjoy dogs a lot, and “fluffy dog” a bit of a unit in my mind. “big fluffy white dog” sounds ever so slightly awkward. “big dog” is also a bit of a semantic unit in my mind, and I could see those going together, though at that stage it’d probably become “fluffy white big dog”.
Ah yes, I did mix you two up. Either way, that is the way most people will say it in normal circumstances. My mix up doesn’t change that adjective ordering is well understood by linguists.
Why the green great dragon can’t exist.
Maybe “opinion-size-age-shape-colour-origin-material-purpose” is different in Australia?
Is “fluffy” a shape, or a material? :3
The “wrong” adjective orderings aren’t necessarily “incorrect” in English. Intentional atypical order is used to change meaning. It can split adjectives and what they apply to.
Big Blue House parses as (Big and Blue) (House)
Blue Big House parses as (Blue) (Big House)
They are incorrect if used unintentionally. Although now we are splitting hairs over exactly what “incorrect” means, and I think we both agree there is a “typical” and “atypical” ordering.
My brain wants to say “big white fluffy dog”, interestingly.
Then in the example, should it become
ScreenFull.tsxtoo?UserCurrent? Using the English order, those names are not ordered, sure, but at least it doesn’t sound like gibberish phrasing. Maybe it’s just a habit though.If I imagine further,
UserCurrent.tsxsitting right next toUser.tsxdoesn’t seem so bad.ScreenFull.tsxseems a little silly, but I feel that’s because — unless there’s also other kinds ofScreens —FullScreenis being used as an adjective in its wholeness (or adverb perhaps), i.e. “this is the component we use when we go full-screen”, as opposed to “this is the full screen, and this is the empty one”.I spotted that, and I think it applies best to
ButtonGoLive.tsx, since you’re likely to have more buttons than screens.I don’t honestly have a strong opinion on
Noun-AdjectivevsAdjective-Noun(vsNoun-Verb,Verb-Noun,Noun-Adjective-Verb, …..). Most have their merits and it seems largely a matter of taste.That said, I think it’s a vital part of any code style guide. As long as the codebase is consistent, the discoverability boon is substantial. As soon as it’s not standardized, finding anything becomes a chore.
At least English is reasonably consistent about adjective-first. Some “noun-first” languages require you to memorize special exceptions for some adjectives that come first despite the general rule:
Français: « Il y a un petit chien dans ce joli café »
Español: “El otro hombre está aquí”
etc.
(Spanish is actually quite annoying in that you can flip the order of a bunch of otherwise postpositive adjectives for emphasis or “poetic” reasons; French at least just kinda has its list of exceptions and once you know them you know them)
French-native speaker here. What you’re saying about the French language is erroneous. Most adjectives tend to appear after the name, but there’s no general rule. Additionally, the position of the adjective may change the meaning , e.g. “un homme grand is a tall man while”un grand homme” is a “great man”.
Part of my comment was meant to be joking/exaggerating for humor, but I will say my genuine experience with both languages is that Spanish and Spanish speakers are much more flexible with word order than French and French speakers.
The complexity in French, for me as non-native speaker, is the large number of irregular verbs, the inconsistencies of the pronunciation/spelling, and the more complex gender/number agreement. Spanish verbs are much more regular, Spanish pronunciation is much more regular, and Spanish gender/number agreement is simpler. But then Spanish is complex in its flexibility, playing with word order and completely dropping many pronouns and even nouns.
the same is true for english
Is
rust-lldbetter thanmold[1]?[1] https://github.com/rui314/mold
So it’s just LLVM’s
lld, if that helps.For some definition of “better”, yes. It won’t be as fast but it’s well tested and supports pretty much everything it needs to for it to be the default.
It sounds like developers trigger CI by pushing to a branch shared with all other developers. That just seems like a bad idea. Why can’t they trigger CI on personal development branches? Then if it breaks then it doesn’t affect anyone else much.
I contributed to postgres recently by highlighting a documentation bug, and even fixing that took a little time because I’m not used to the email-based workflow and don’t have a mail client set up to handle git-format-patch.
I ended up just describing the issue and someone else made a patch that a third person committed.
This does not inspire confidence :/
Things here could definitely be a bit better than they are. You can run the full test suite locally, and I do, a lot. You can also trigger a CI run on a bunch of common platforms fairly easily, and I do that sometimes, but for various reasons it’s a little more of a hassle than would be ideal. Even if you do that, the buildfarm contains an even wider variety of machines and configurations, so it is possible to break the buildfarm even if you did a CI run and it came back clean. A good number of my mistakes, though, come from thinking that a local run of the tests is good enough and that I don’t need to hassle with CI, and then finding out that it wasn’t and I did.
I realize a lot of people here like email-based workflows, but I really think we need systems that can do both. Even if it meant that I generate a patch and then upload it through a web interface. I’ve absolutely abandoned (admittedly small) contributions to projects before because I didn’t want to set up a special email client. But I do understand why people who are used to it like it.
I’ll plug my friend and colleague’s answer to this problem, GitGitGadget: https://gitgitgadget.github.io/. It translates between GitHub PRs and the patch-based email workflow, bidirectionally.
It’s presently only suitable for the Git project, but I bet could be adapted for another project.
Really neat idea!
You don’t need a special email client. In the CHICKEN project, we typically use something like
git format-patchand then e-mail the resulting files as attachments to the mailing list. To apply, save the attachments and usegit amto apply them. Any halfway decent mail client should be able to handle that. The only “special” requirement is that it doesn’t try to get clever with text attachments and mangle them somehow.Interesting, I had been under the impression that most projects wanted them inline for code review purposes? If all it took was to attach the patch, though, I’d do that.
Working on PostgreSQL, we almost always attach them, not include them inline. I actually find inline very inconvenient; I can’t easily extract the patch if it’s done that way.
I use
mutt, which automatically displays attached plaintext files after the end of the e-mail (or I configured it to do so, I don’t remember).Databases typically have massive test suites that can’t reasonably run for every dev branch. Back when I worked on a DB engine we had a standard set you ran for every commit, along with any new tests or other relevant tests. Then every night our full test suite would run, executing several CPU-years of tests. We then re-ran failed tests the next day to isolate issues if necessary.
I can’t speak for the postgres test environment, but I wouldn’t be surprised if the situation is similar.
An approach that can work well here is a merge queue with integrated testing. So you batch up a lot of changes and test the merged result. If the tests fail you don’t commit it but try again with half of the commits. It can be slow with slow CI but it is typically manageable as long as your number of bad commits aren’t too great.
The downside is that you still need to pass full review before merging. But at least you don’t break the main branch, your commit just bounces back and you can fix the issues before merging.
Ideally you would still have some pre-review tests but that can be a sample of common issues without needing to run the entire suite.
That’s essentially what we did. Batch of changes overnight with the full suite, pre-review tests for common issues and core functionality.
Given how critical the PostgreSQL project - and assuming the test suite can run in parallel - in my opinion it would justify having a much larger set of CI workers than most projects. Firing up a thousand CPU cores to run tests for a trusted developer could be a worthwhile investment of project sponsorship money.
I agree, that’s what our standard set was like. The nightly full set ran on many more cores, all night long.
The PostgreSQL project must be drowning in resources contributed by all of its beneficiaries /s
If a web server has a problem serving 114MB of data in five minutes… maybe improve the web server performance? 114MB is peanuts by modern standards. Expecting a centralized platform to work as a free-of-charge CDN doesn’t seem sustainable at all — how will the server cope if people start clicking on those links en masse?
I find it hilarious the author is complaining about 114.7 MB when they have zero respect for the user’s bandwidth. Open up this page without an ad blocker, scroll through it, and take a look at the network tab:
That was after a couple minutes. It keeps going up with no sign of stopping.
I find it hilarious how few people actually read the article. The quote about 114.7 MB is from a different person who independently observed the effect on a different website.
I read it. I’m not talking about Chris Partridge, who may legitimately complain about 114.7 MB. I’m talking about the author of the It’s FOSS article Ankush Das, who is sharing the quote as an example of excessive bandwidth, and complaining (hypocritically) that Mastodon is DDoSing It’s FOSS as well.
Right, I replied to you in a “things I also find hilarious” sense, not because I think you’re wrong, though I see how that’s unclear. The comments about server performance to me seem to be confusing which site is which.
Ah I understand now, that’s fair.
To be charitable to the author, that was a third party engineer’s test and not necessarily representative of the load itsfoss is seeing. The full quote:
The crucial point is that someone needs to serve that traffic anyway, if we assume that link previews are a good thing. Facebook, Ex-Twitter, etc. will query the server of itsfoss.com just like Fediverse instances do. Then they will serve that data to everyone who views the timeline, so the “traffic amplification” is still there — it’s just someone else who picks up the tab. It’s only less noticeable because there are five of them. That is like saying “I want fewer social media platforms on the Internet” without saying that explicitly.
From this perspective, why have a website at all? If you want to offload everything on centralized megacorp-owned platforms, just post to them directly.
This is not to say that link previews cannot be more efficient: I suppose one problem is that a social media server fetches a complete page just to make a low-resolution version of the header image and extract some headings and first paragraphs. Web servers could generate their own previews and provide endpoints for querying just the previews.
But still, if you can’t serve a page to thousands Fediverse servers, you are already in trouble if thousands users come to your page.
That’s why we have the opengraph standard: https://www.opengraph.xyz
The site itself can provide og: meta tags with content and the social site should use those to create the preview. It only needs to fetch the raw HTML for whatever link - and to optimise more only the is really needed. You can stop parsing as soon as you hit
Worth noting your HTML tags here have been treated as HTML tags and sanitised out. You can escape them like \<this>, or put them in backticks to get inline code spans (i.e.
`<this>`).Oh, thanks, I didn’t know about that standard.
note though that that is what Mastodon is already using - the complaint is exactly about it fetching the page to look for those meta tags.
I don’t think you can assume that link previews are a good thing. Also a service like Facebook or Ex-Twitter will generate the preview once and then use that preview for every other occurrence, which limits the amount of amplification that occurs. The Fediverse doesn’t have that mitigation.
You can from the perspective of a site operator though? Embeds with images, titles and intros drive more engagement.
They’re nicer for me as a user, too. Seeing the real page title gives me more information, and in a much nicer form, than looking at a raw URL.
I think that probably depends on the site operator. Probably it’s good for certain customers and sites. I don’t think it’s a universal rule though.
Wouldn’t it be possible to use a lower-quality image when serving to Mastodon?
That’s not the important part of that test. This is the important numbers from the test:
traffic amplification of 36704:1.This isn’t a traffic amplification attack. That is when the traffic goes to someone other than who made the request. In general TCP protocols (such as HTTP) are immune to this issue.
For example it looks like this:
In this way
10.0.0.1has amplified its attack traffic to attack 10.0.0.2.You could get a similar “amplification” to what is mentioned in this article by downloading an image with a small HTTP request. It is slightly worse because you get various hosts to help out. But it really isn’t that effective because it only works if you can post to accounts where people are following you or where people re-share your post.
If it were a practical attack that number would matter. But then we’d also see it happening as an attack, and I’m not aware of “high-reach accounts on Mastodon post links to take sites down” being a thing?
I think the concept of an amplification factor is not very relevant here. One could imagine “make a network of fake accounts following each other and spamming links” as a potential vector, but it’d also likely be caught and cleaned up fairly quickly, and such low traffic levels is just trivial to do in many other ways. Amplification factor is mostly interesting if you can scale it to large-scale attacks, and you can’t do that here.
I often use my own wrapper type for JoinHandle that does cancel it on Drop. “Fire and forget” pattern seems to me like a less common case.
Do you abort the task on drop, or have a more elaborate wrapper that passes cancellation tokens into tasks?
I hate cancellation tokens, so I use them only if necessary to abort blocking code. For async, just Drop works for me in pretty much all cases.
I literally Ctrl + f’d “hash”. I knew that would be part of this.
This is the first thing I do when I audit a crate for performance. By far this is the single most significant performance footgun you will run into. I know that there was way more than this, but I have had this exact situation happen “that Go code is ~X times faster” and I closed almost the entire gap just by tuning hashmap usage.
If I’d read this post a year or two ago I can pretty much guarantee I’d stay up all night doing exactly that. But I am tired.
Is there a good reason the Rust std hash map is slower than the Go one?
Yeah, the hash algorithm is a major difference. Rust uses siphash 1-3, Go uses some sorta aes based algorithm. I’m a bit reluctant to put numbers down but I think we’re talking ~10x performance penalty with siphash.
Is changing the hash algorithm on the table for Rust?
Not really, or at least not to the extent that it would materially affect this sort of comparisons. The hash was chosen for its reliability and resistance to adversarial inputs (especially as the hashmap was designed to have a very high load factor).
But the hash is pluggable so it’s reasonably easy to use cheaper ones when the situation allows.
I don’t know. It’s already changed from an even slower siphash, so I suppose there’s always the option. Or there could be a faster hash in the stdlib for people to easily switch to, whereas you need a 3rd party crate currently.
I would love to hear from an expert on this topic, if anyone is aware. Is what Go does insufficient?
I look at ahash and I’m just not sure that it’s meaningfully less safe that the std library’s approach.
You can always incorporate a random seed, such as a per-process random seed. I suspect that this is sufficient if an attacker can’t easily derive the seed or characteristics of it? Or that the space of keys for that seed is sufficiently small such that they can not brute force through that mitigation.
How strong do the properties of the hash really have to be? Algorithms like what go uses seem to have pretty good distributions. Is “pretty good” not enough? How do we think of the threshold here? Perhaps random seed + fnv is no good, because fnv is so easy to trigger collisions in that even with a seed an attacker could easily explore a space that leads to issues. But aeshash really isn’t good enough? ahash isn’t? What are the properties and thresholds that siphash meets but ahash does not?
Here is a doc from ahash: https://github.com/tkaitchuck/aHash/wiki/How-aHash-is-resists-DOS-attacks
I have read it before but I’ll refresh myself on this, sorry if the questions are answered in that document, I posted this and then remembered that the doc exists.
They can’t be compared directly, because Go uses separate chaining while Rust uses open addressing (with a very high load factor) so distribution / conflict requirements don’t match.
You can read the basics of Rust’s hashmap at https://faultlore.com/blah/hashbrown-tldr/ (and more by looking up swisstable / abseil from which it’s derived).
https://www.dolthub.com/blog/2023-03-28-swiss-map/ could also provide useful information as it specifically compared a Go version of SwissTable to the builtin map.
Using AES doesn’t automatically guarantee robustness. Ahash applies it in its own way, which in the past had some suboptimal cases and a side channel leak.
I think there are a few reasons stopping libstd from switching:
SipHash claims to be cryptographically secure-ish, but AFAIK there has been no proper cryptoanalysis of ahash. So even though ahash could be as good or even better, it’s unknown at the moment.
ahash takes advantage of AES hardware, which is not available on all platforms. It would require fallback for some other platforms, and it’s not great to have unexpectedly large differences in performance or security between platforms.
Rust prefers safe defaults over performance. The users who don’t know if they need DoS resistance, get it by default.
I didn’t mean to imply that aeshash would be good by virtue of AES, only that the hash has good distribution. I don’t think cryptographic requirements matter here, I would assume the issue is statistical distribution in the face of malicious keys.
I’m not sure if Rust should switch but I definitely think there should be an alternative hasher in the stdlib.
Very cute! I like it.
This sounds like it would be very sensitive to the order of the numbers in the input. For example say I have a steam of coordinates for vertices. If I write them out as x0,y0,z0,x1,y1,z1… then I would expect to get much better compression by switching to x0,x1,…,y0,y1,…,z0,z1,…. order instead, since the x coordinates are much more likely to be correlated with each other than the y coordinates or the z coordinates.
Separately I’m thinking that if you have noisy data near 0, you probably won’t get much compression since the sign bit keeps changing and you don’t get to take advantage of zeroes between the sign bit and the top of the mantissa.
It’s very sensitive to the order. The algorithm comes from a time-series database, and is mostly used in column oriented storage as far as I know.
Yes, and there are whole classes of compression algorithms for multi-dimensional data that start out with simple algorithms like this XOR one, but wrap them inside of shuffle-and-slice preprocessing steps to make “similar” values adjacent in the stream.
If you expect that, you could probably get a savings by changing the coding to more efficiently deal with sign bit flips instead of having to extend the window back to bit 0.
It still might not be great, because the kind of data that crosses zero a lot probably has a lot of variability in the exponent, and a more-or-less random distribution of mantissas — but it might still benefit: say the values are all between -1 and 1 and only have a few decimal places of precision. That’s another case of “well, why not just code them as 16-bit fixed-point then?”, but supposing that’s not an option, then, well, the exponent is mostly only going to vary in the last few bits and the mantissa only in the first few, so it works out tolerably.
This is really clever. It’s hardly obvious to another reader, though. In C and C++, I have written
The intent is clear. I can switch between variants by modifying a single character, changing 0 to 1 or vice versa. The code highlighting in my editor (vim) figures this out and greys out the disabled code.
If C/C++ can do this, maybe you should demand that your favourite language should support this feature as well (statically switching between code variants I mean, not necessarily by adding the C preprocessor).
This kind of technique is done strictly for your eyes only, in the heat of debugging. It’s only for the convenience of the author/debugger. If it needs to be committed longer-term, yeah: please do anything else. However such techniques are still extremely valuable to have in your personal toolbox; do not disregard them just because they shouldn’t be committed for others to read.
It’s the same difference as writing for thinking vs. writing for an audience.
I use crate features for this in Rust, which is somewhat tedious if the condition is only in one place, and immediately wonderful as soon as I need the same condition in a second place. Temporarily of course. IntelliJ highlights enabled/disables features appropriately.
For a single place use, this should solve your problem? https://stackoverflow.com/a/72154186
Oh, that’s clever. Why didn’t I think of that? Thank you!
Inserting with multiple VALUES at a time is definitely a good idea. You may also want to switch to preparing the INSERT and reusing it multiple times. Both for inserting a single row many times and for batching multiple VALUES many times in one transaction. You may also want to take a look at COPY FROM or LOAD DATA LOCAL (if on MySQL).
I’ve done some basic ballparking for MySQL, Postgres and SQLite here: https://github.com/eatonphil/databases-intuition?tab=readme-ov-file#mariadb-load-data-local.
In my project i am using sqlite due to the offline nature of the application. I am not really versed in prepared statements but i will have a look at the documentation.
If you’re running offline, take a look at the
.importcommand for sqlite.https://sqlite.org/cli.html#importing_files_as_csv_or_other_formats
.importjust uses a single insert prepared statement in a transaction (source in latest release 3.45.2).Oh, it’s the year of Linux on the desktop again? I am a heavy user of Linux, but it. Just. Never. Works. Flawlessly. I use a ThinkPad with Fedora for my daily work/home needs because they seem to be well-supported, but still the power management just doesn’t work for me. It’s still better than Windows though (Mac is too expensive for me).
For me the distro that works is Linux Mint. I recently switched from a Mac to Linux Mint and I am supper happy with my choice.
Seconded, Mint is everything great about Ubuntu plus Cinnamon, which is my favorite DE by far.
I wanted to try Arch, but didn’t want to fuss too much like my old Gentoo days, so I installed EndeavourOS on my old laptop. Worked basically flawlessly. I’m very pleased with it.
Arch these days is astonishingly non-fussy. The installer makes it kind of a breeze to setup.
I’ve read that, but EndeavourOS still made it slightly easier. It’s seems like a pretty thin layer on top of Arch which I’m comfortable with.
I know the m1 macs are expensive, but its just day and night from the intel ones, not even mentioning fixing the shitty keyboard, but just the overall performance is amazing, I hope they will get cheaper if apple keep releasing new versions, but as I say in the article, the issue is Apple’s far reaching control over the code my computer executes.
Mac power management isn’t all that. When I had a work Mac it had all kinds of suspend/resume issues. Definitely no better than Linux & Windows.
It’s interesting how people’s experiences can differ. I have mostly used macOS since 2007 and I cannot remember when I last had an issues with suspend/resume.
I had a Linux-based ThinkPad in 2020/early 2021. One of the models that a lot of Linux users recommended and certified for Linux. Sleep set in the firmware to S3 Sleep (Linux). The battery would drain overnight in S3 sleep. Resume would often come back with various hardware (often the trackpad or track point, sometimes the display) non-operational. Battery life was often four hours, after all the tuning (powertop, CPU governers, etc.). Windows would work better, except it would sometimes wake the machine from sleep while it was in the sleeve, almost cooking the laptop.
I know quite some people (mostly former colleagues) who use Linux on laptops and many complain that suspend/resume is still a tire fire. Quite a few just shut down their laptop and boot it again to avoid the issues. Many friends and family members have MacBooks and they open/close their lids many times a day without any issues.
Oh goodness, I have them all the time; Mac laptops tend to get cranky when you let the power fall below 10% and usually refuse to turn on if the power is below 1%. Inevitably I find this out when I have to switch from my work MacBookPro to my Personal for a client call or the like haha
The intel MacBooks from roughly 2016-2017 onwards had a lot of issues. There was a definite quality drop in the years before M1. But my 2013 MacBook Pro had instantaneous and flawless suspend/resume as the current MacBooks do.
Yeah, there’s your problem.
Former Red Hat staffer here. They issued me a shiny new Thinkpad X240… and 3 different and conflicting sets of docs on how to configure it with Fedora. All, incidentally, were wrong in some details.
At the end, my trackpad didn’t work, if I used both of the supplied 2 external screens then my internal screen didn’t work, and other issues. I complained to my colleagues. I was told: “oh, yes, that’s normal.”
In other words, for clarity: Fedora did not fully support some of the hardware that Fedora is developed, tested and documented upon. >_<
I’ve tried most Linux distros out there. It’s my job. Fedora is consistently problematic and has among the worst installers in the whole industry, along with some of the poorest driver support. It’s better now than it was a decade ago, but it’s gone from “unusably dire” to “poor” in my professional assessment.
FWIW, a few years after I left RH, I went to SUSE. OpenSUSE is substantially better in almost all respects and if you dislike and disdain the Debian family for some reason and want to stay in RPM-land, try openSUSE.
Echoing nomemory, I first tried Fedora on my ThinkPad, and it was painful. I guess part of that was the kernel didn’t yet support the hardware? Anyway stuff didn’t work, and stuff broke on upgrade. After distro surfing, I am quite happy with Linux Mint on the same Thinkpad.
JXL is such a cool image format, I hope we get widespread support (meaning e.g Chromium eventually adds back support). These results are impressive. And I really like this concept of “the Pareto front”.
However, this:
Would currently make me think twice about adding JXL support in my own software. I hope we eventually get a version 1.0 of at least a reference decoder, so that those of us who do t want to beta test a 0.x library can support JXL.
This concept has come up a few times in relation to JXL the last few years. A few years ago, Luca Versari (another JXL dev) created fjxl and fpnge to demonstrate extremely fast encoding. At the time, people compared them to QOI and other formats and encoders, with the Pareto frontier being a big point of discussion. fpnge got speeds over 1 GPix/s in screenshots, and over 500MPix/s in photos. fjxl was eventually integrated into libjxl, as the modular mode e1 speed. Luca made this pdf which has some technical details on how they work, if you’re interested.
libjxl seems to work pretty well, with a decent number of vendors using it (Samsung, Apple, Adobe, for example), it seems like it probably works decently well. It’s not 1.0 yet because it lacks some features they want to implement first, it has some issues, and the API isn’t final. However, they are trying to not break anything in the API (0.9 -> 0.10 is a drop in upgrade, IIRC). So I would suggest that rather than looking at the number, you look at their 1.0 issues on GitHub. Some of them look gnarly but most of them are niche-specific, and may not apply to your use cases.
There is also jxl-oxide which the JPEG committee has tested and found to be conformant. It isn’t as fast as libjxl, but it seems like a good decoder.
I’m not saying that libjxl doesn’t work well, but the JPEG-XL people are telling me not to use it yet, and I respect that. (Also, https://github.com/libjxl/libjxl/issues/1450 this issue in particular is pretty big and wide reaching)
jxl-oxide is 0.7, so the jxl-oxide developers are also telling me not to use the library yet, and I respect that too.
Yeah, that issue is one of the 1.0 blockers.
I saw this in the last thread on JXL as well, but since when does 0.x mean “don’t use this”? I can assure you that that is not what the libjxl developers mean, in any case.
What would the JXL people mean by 0.x if not “don’t expect this to be ‘production-ready’ yet”? Have they communicated what they mean, or are you just conjecturing?
0.x pretty universally communicates that software isn’t “ready” in some way, to the point that semver defined 0.x versions to mean “initial development”; versions before the software is properly “released”.
FWIW, issues like “will terminate the whole process on error” is exactly the sort of thing I would expect on a 0.x “development” version that’s not meant to be used in production yet.
The question of versioning came up in the JXL Discord. Jon compared the 1.0 milestone with something like libjpeg version 6.
Semver is why they’re using 0.x. Even though they’re keeping the API as stable as possible, it isn’t final.
Really, going by semvar rules, lots of annoying things like abort() all over the place would be perfectly fine in 1.x versions.
So my point is to not treat “production ready” as a binary, nor to tie it to semver. Concrete issues and how they relate to your use case are what really matter. For example, if you already sandbox all media codecs anyway, maybe you don’t care about abort(). Hell, maybe you’re compiling your code with bounds checking so that all OOB accesses turn into aborts, so a few extra explicit aborts don’t matter much. I’m not telling you to go use libjxl right now, you’ve identified an issue that prevents you from using it right now, so that’s that. If they release 0.11 with all aborts removed, suddenly libjxl would be “production ready” for more people, regardless of that 0 in front, so I don’t think treating it as an indicator is very useful. Just about the only thing it indicates is that you shouldn’t use it yet if you don’t want breaking API changes.
All that is to say, yes, 0.x here means that it isn’t ready in some ways (API not done, missing features, etc), but that’s not the same thing as saying it isn’t ready to be used in production at all.
Semver doesn’t dictate that you have to use 0.x until you have figured out the final iteration of the API, but rather:
Semver just says that once you’re out of 0.x, breaking changes must be in major versions.
Anyway, the project is communicating that the library isn’t to be considered production ready (no, I don’t count random Discord conversations. I haven’t said anywhere that “0.x means you shouldn’t use it in production”, just that you’re essentially using beta software. Of course there are situations where you may decide to use “not generally production ready” software in production, you just need to be prepared to handle it as such. And I’m not.
I don’t really understand what you’re trying to get out of this conversation.
Lots of 0.x software is used in production with the expectation of production-readiness, and a lot of people treat 0.x as jxl does, as @lonjil describes. You may choose to ignore the realities of modern software engineering culture because “dictionary says this lol,” if that’s how you wish to see the world. But you can’t say things like “the JPEG-XL people are telling me not to use it” on the same basis, and pretend the dictionary gives you absolute authority to ignore everything else that has ever been said about JPEG-XL, even by its creators.
All libjxl releases have the note:
For evaluation purposes is the opposite of for production purposes. It’s not just the “0.x” version number, they directly state it there.
Because all libraries should only be evaluated on test data and never used in any user facing software for any purpose. The 1.0 issues lonjil linked represent caveats that may or may not apply to your particular use case, but if the developers deleted that list and released 1.0 with the exact same code then it’s now production ready and will never have bugs.
My point is, lots of software can be production ready by some metrics to plenty of people, while still being evaluated and refined. The libjxl authors are up front about which issues keep them from releasing as 1.0. If those issues don’t apply to you, there are no problems.
So the warning that the software is for evaluation purposes is meaningless, because all software can be evaluated on production data. It’s a meaningless warning and should be removed.
Not at all. But there are different levels of production-readiness, that a single 0 / >0 bit doesn’t express exhaustively. While two releases at 0.1 and 0.10 may both be considered “for evaluation,” the context in which they should be evaluated is probably different.
I’m just saying I’ll respect the clearly communicated request of the authors to treat libjxl as pre-release software in its current state. I don’t understand why you’re telling me to ignore their wishes. You’d have a point if you could link to any official communication which says that libjxl is supposed to be treated as production ready, but you can’t (the best thing you have is seriously alleged Discord messages?), which leaves the pre-release version number as the only piece of official communication on the topic.
The lack of a 1.x decoder library for JXL is an adoption issue, no matter how much you screech at me to ignore it. It’s not like convincing me will change anything; the goal is wide-spread JXL adoption, not to successfully convince me to incorporate it in its pre-release state into whatever hypothetical software with 2 users I may eventually write in the future, right?
idk about Peter, but I said that you shouldn’t use it if it has issues that prevents you from using it. My point was purely that the number in front isn’t relevant when you have real concrete issues to base your decision on.
The fact that the project clearly communicates “please don’t use this yet if you don’t want to deal with the issue you’d expect from pre-release software” is the issue that prevents me from using it, for the most part.
Not the only one, there’s the caveat in the release announcements quoted by @cosarara
So, Apple just killed popular free apps on iOS in the EU…
Punishing users to show the middle finger to a regulator is a weird approach. A few days back, Amazon was a no-show at European Parliament’s hearing on working conditions in their Polish and German mega-warehouses. Is the Chinese market the single one where these companies are not so cocky?
“GDPR” cookie banners are the exact same malicious-compliance approach, and have been around for years.
We might find out if China ever had an open market.
By offering the option of choosing a payment model that’s not as suitable for very popular free apps?
you can either keep the current contract, or, accept the new one which enables you to distribute outside the app store and also use any kind of in-app processing you want, but in that case you also agree with the Core Technology Fee. This fee makes really popular free (and without monetisation) apps unsustainable.
… and thus popular free apps will keep the current contract. How does having to do that “kill free apps”?
If you intended to say “the new model is terrible for free apps”, sure, I agree.
Are EU developers required to switch over to this new contract?
No, you can opt to keep the current one apparently, but that also prevents you from shipping outside de app store or using other browser engines or using your own in-app payment processor. The new DMA benefits are only for the new contract. So is a choose your evil kind of scenario.
So it doesn’t actually make free apps unsustainable in any way, because nothing has to change.
Imagine how expensive it would be for Mozilla if Firefox for iOS actually had more than 1 million users.
As I understand it Mozilla could just distribute for free via the existing App Store?
they wouldn’t be able to put their own engine on Firefox without accepting the new contract.
Yes they would, the new contract only applies to apps using the new capabilities for alternative distribution or alternative payment processing.
As far as I can make out the new terms only apply if you distribute outside of the main App Store.
There is no requirement I can see that prevents you shipping browsers in the main App Store, if you can point to where you’re seeing the rule prohibiting it it would help.
Not relevant to Firefox but note: it also applies if you want to use the App Store but want the alternative payment model.
If I’m understanding the terms correctly, it also applies if you want to distribute your app on alternate marketplaces, which means that free devs have to choose between paying the 0,50€ per install surcharge over 1m installs and still being locked into the App Store.
💀💀💀
More seriously, it would actually be $0 because the new business terms aren’t tied to the browser engine stuff:
They mention an exception for NGO’s. In theory, Mozilla could get a pass. They are also allowing alternative browser engines in the announcement.
Nonprofits are exempted from the fees.
Also, there are a few exceptions that apply to browsers, so Firefox (which is not just a skin) could come both to the AppStore or an alternative store just fine, with minimal fees (developer account).
Does Firefox come from Mozilla Foundation or Mozilla Corporation?
if you think that a free app with two million users in the EU in the new contract is sustainable, then you have a different standard than I do because I just used Apple’s own fee calculator and that amounts to 45,290 USD per month.
peter isn’t saying that the new contract is sustainable, they’re saying that free apps can just continue using the old one.
I think this is more an issue for free apps that are not possible now. E.g. it is conceivable that an emulator could get > 1 million users, but emulators are not allowed in the Apple App Store.
I have several emulators and full blown qemu (UTM) on my ipad/iphone right now. Technically free though I do pay 3$ per month for patreon beta access (for altstore/dolphin).
No? Using other browser engines doesn’t seem to be tied to the new contract?
I think that may be a violation of the DMA. Not sure, though I guess we’ll find out soonish.