I like Gruber but I’m very much in the opposite camp. I hate Apple shipping software because they said they would in a keynote. I don’t turn to Apple because they hit all their OKRs on a spreadsheet.
If they’re not happy with it, don’t ship it. That’s why I pay the Apple Tax. I don’t want half done mostly working sorta broken crap like almost all the AI products I’ve tried. Do it right or don’t ship it. I actually don’t care at all if these features ever launch, but I’m certainly not going to get upset with Apple for doing what I’ve begged them to do for years.
I don’t think Gruber’s argument is that Apple should have shipped the feature even if it doesn’t work - he’s arguing that Apple shouldn’t have announced it (and used it in marketing materials to sell more iPhones) if they didn’t yet have a working project that they could reliably ship.
That’s not the point. Assuming the software is shipped when ready anyway, there’s still the problem of advertising features and selling hardware based on something that is far away from shipping, and may have even been just a fake concept art.
Apple is generally very strict about not talking about future features, and they typically announce features when they exist, and only need polish and bugfixing before shipping. The AI Siri was apparently different — they’ve announced it as a real feature before they’ve implemented it.
It’s the discrepancy of showing a feature “working” in TV ads for general public, but not having the feature working enough to prove to anyone outside Apple that it even exists.
The c# author chose go for this project rather than c#. It’s not because go is “better” but it’s better for tsc. He said that c# is fundamentally a bytecode first language and he doesn’t consider it’s aot capabilities to be that mature. So essentially he thought binaries built by go would be a better deployment target due to better (in his opinion not mine) cross compilation support. He also mentioned that he considers low level memory/struct layout to be easier to do in go and presumably he wanted that in this project.
In a way go won because it was a syntactically similar natively compiled garbage collected language with the best cross compilation story.
Rust seems to have been ruled out because it either would have required reworking the object graph to comply with rust ownership rules (believed to be harder than line by line transliteration) or they would have had to GC<Box<T>> all the things. I presume they ruled that out because it was ugly and painful to work with but I don’t know if that’s true.
Despite how good javascript jits can be, they don’t shine at the polymorphic graph analysis compilers do. If everything was being manipulated as a big byte array, v8 would probably have been more competitive.
So the tl;Dr is that ultimately golang was closest 1:1 transpile target that met their needs.
Is this the first big Go project at Microsoft? I had no idea they could choose Go, I would assume that it just wouldn’t be acceptable, because they wouldn’t want to unnecessarily legitimize a Google project.
They had already lost the web platform by the time they switched to Chromium.
When I think of MS and ‘developers developers developers develop’, I was lead to believe that they’d have more pride in their own development platforms.
I wonder if the switch to Chromium, the embrace of Linux in Azure, the move from VS to VSCode, any of these, would have happened under Ballmer or Gates. I suspect Gates’ new book is like Merkel’s: retrospective but avoiding the most controversial questions: Russian gas and giving up dogfooding. Am I foolish to expect a more critical introspection?
I think corporate open source is now “proven”, and companies don’t worry so much about this kind of thing.
Google has used TypeScript for quite awhile now (even though they had their own Closure compiler, which was not staffed consistently, internally)
All of these companies are best thought of as “frenemies” :) They cooperate in some ways, and compete in others.
They have many common interests, and collaborate on say lobbying the US govt
When Google was small, and Microsoft was big, the story was different. They even had different employee bases.
But now they are basically peers, and employees go freely back and forth
I can see a blog post saying they made changes for FIPS compliance, which is usually understood to be for satisfying government requirements and not a security improvement. I can’t see a summary of any other changes.
MS is a pretty big company, so I wouldn’t be surprised if they have a ton of internal services in Go. Plus with Azure being a big part of their business, I doubt they care what language people use as long as you deploy it on their platforms.
Can confirm that some internal services were written in go as of a couple years ago. (I worked for Microsoft at the time.) I didn’t have a wide enough view to know if it was “a ton,” though.
I’m too lazy to look up a source but some of the error messages I’ve seen in azure make it clear Microsoft uses go to build their cloud.
Message like this one:
dial tcp 192.168.1.100:3000: connect: timeout
Keep in mind Microsoft also maintains their own go fork for fips compliance. I don’t know exactly their use case but I can tell you at Google we did the same thing for the same reason. I think the main fips compliant go binaries being shipped were probably gke/k8s related.
(Edit removed off topic ramble, sorry I have ADHD)
One of the benefits of microservices is that if you have a malicious dependency that requires the ability to talk to arbitrary public up addresses you can just isolate it. No need to put it in the same process/ address space as the code that mints API keys, etc.
To me, that’s the solution to these problems. Arbitrary egress isn’t something you typically need and it’s sketchy enough to move it to a sidecar if you do. Of course, very few people do that, but it seems way cheaper than auditing every dependency, though maybe AI would help.
TBH I was a bit surprised to see this run as part of the deployed server. Seems like running in a build script is going to yield better results/ wins but that’s just how I’d do it personally.
TBH I was a bit surprised to see this run as part of the deployed server. Seems like running in a build script is going to yield better results/ wins but that’s just how I’d do it personally.
The Go SDK has no way to execute arbitrary code when building.
Go has some stuff like go generate but again that doesn’t actually cause go build to execute code.
But yeah most of the weird stuff that pops up when looking at go build and dependency history can be explained by “the people who wrote go thought of building and dependency management as being the responsibility of a monorepo build system which has explicit support for generated files as something distinct from source code and binaries”.
One time someone found a way to make cgo linker and compiler flags execute arbitrary code, and that was considered a vulnerability. https://github.com/golang/go/issues/42556
For comparison, it’s normal for cargo build to execute arbitrary code (build.rs).
You’re right that the size of types are statically known, but Go arrays are actually not small fixed sized structs.
In particular if x is an array of at least size one then &x==&x[0] (except for of course go not allowing this comparison based on different types). You can confirm with the below playground link which prints out the address of the array as well as the address of each item.
As in different types (can) have different sizes, as opposed to reference based languages where all types take a pointer and memory layout is only a concern for a type’s internal metadata.
Hence my surprise that they used a row layout, if all types have the same size then row v column is not (as much of) a concern, every value takes a pointer and there’s no padding or storage loss. But if the key and value can have different alignments then you generally want to segregate keys and values to avoid waste.
Go has no variable size types. Arrays, strings, slices, etc. are small fixed-size structs that point to variable-length blocks
I assume that cache coherency / locality of reference is the rationale for putting keys and values together. Performance trumps compactness. Also, the typical types you use in maps contain pointers or are at least pointer-sized, so they’ll have compatible alignment.
Swiss Tables do a lot less probing due to that cute little byte-array table.
As for the other points, I dunno, it’s clear Google should have had you in the room suggesting these things, since of course they wouldn’t bother to actually profile and compare different layouts.
I’ve actually had success with log all the things before, but usually in the form of what are essentially traces or request logs. Even this has it’s problems. It’s expensive, although often worth the cost. It’s almost certain you will need to do ongoing compliance work.
The thing that makes this work is that it’s a structured log with a pretty uniform schema. There’s also a pretty well trodden pathway to sampling (including sampling complete transactions not just randomly dropping log line).
What is usually a disaster though is log shipping unstructured printf style logs. As the article mentions, if you don’t know who’s going consuming them you don’t know which debug statements are load bearing. I’ve seen big companies trying to reduce logging costs but even though they knew 99% of the data was useless, they didn’t know which 99% it was.
I think that a lot of companies are religiously saving random library output on stderr/stdout to long term storage. Instead it’s often better to have a local ring buffer or some sort.
[edit]
I should add the domain I work in because I think that matters. I’ve spent time in e-commerce and ad-tech. In e-commerce, if you have an average order value of $50 the percentage of revenue going to logging http requests is relatively small. In ad-tech, you just can’t budget as much storage for every impression.
Except you don’t know that errors are being handled, and the logic isn’t local to the caller.
Nothing I’m aware of in Go requires you to actually check if err != nil, meaning your program will still compile if you don’t do it, and will still run. Just when the error condition finally does happen, you’ll get a mysterious problem at some other location.
Compare this to exceptions or sum types, both of which force you to address the problem before moving on. An uncaught exception doesn’t allow execution to proceed, and in languages with sum types a failure to resolve them before use is generally a compile-time error.
Similarly, the logic is non-local; the near-universal pattern in Go is if err != nil {return nil, err} or variant forms of it that wrap err in some sort of nice reporting structure. That’s not “logic”, that’s just yeeting the error up the call stack in hope it will finally encounter an actual error handler (and if we’re being really cynical, is just try/catch but implemented manually). Figuring out which handler that might be and what it will do can easily be a non-trivial bit of analysis, and is basically never obvious from just reading the surrounding code.
Nothing I’m aware of in Go requires you to actually check if err != nil
I’ve only contributed to one open-source project written in Go, and they had a linter that made sure that err returns were either checked or forwarded to the caller. I think it was a standard go thing, I don’t know why it isn’t a compiler warning.
My biggest problem with Go is that the type system does nothing to help you avoid footguns. In Go, it is undefined behaviour to concurrently modify an object from two goroutines (and, yes, this can break memory safety if you do). Yet Go has primitives that make it trivial to send references to objects to other goroutines. In Erlang, you have the same kind of send (messages to actors rather than messages via channels), but Erlang always passes objects by value (or by reference as an optimisation, but they’re immutable so this isn’t visible in the abstract machine). Concurrent mutation is impossible by construction. In Rust, the Send trait ensures that objects are passed as a move operation and the borrow checker ensures that you don’t hold any pointers to objects dominated by the sent object after you send it. In Go, sending a pointer via a channel is just as easy as sending a by-value copy. Easier, in fact, because copies are shallow so sending an object by value will send any referenced objects by reference and end up with accidental aliasing.
All of this means that there’s a huge cognitive load writing concurrent Go, yet concurrency was meant to be one of the main selling points of Go. The people I’ve spoken to who are happy with Go treat it as a replacement for Python that produces a statically linked binary.
Yeah that limitation of channel sends was annoying, and I had to adopt conventions to reduce the “cognitive load” and avoid errors.
I’d have liked the ability to have some form of “const/immutable pointer” such that a read only reference to an object could be passed around.
As for the lack of move operation, I simply adopted a convention of immediately nil’ing the pointer on the sending side after sending it. Possibly a linter could be written to verify that convention?
Obviously someone subsequently updating the code could fail to be as rigorous, and introduce an issue. I was pondering catching that with my UTs, by building upon the finalisation facility in the runtime, but never got around to doing it.
All languages have their issues, and sharp corners…
Nothing I’m aware of in Go requires you to actually check if err != nil, meaning your program will still compile if you don’t do it, and will still run.
True. What we rely on is a weaker property: unused variables are a compiler error.
So at first this seems almost as good. You do in fact have to do something with err. It gets a little trickier when you realize that err can be reused so you could do 2 a, err = operations and only check the second operation and that would make the compiler happy.
But in a worse is better sort of way Go code frequently won’t compile if you don’t check the err.
You can choose at what level to handle it, and i think it’s pretty strong consensus that exceptions are a bad model over something like returning errors/result types
You can choose at what level to handle it, and i think it’s pretty strong consensus that exceptions are a bad model over something like returning errors/result types
Reiterating for clarity: if you follow all the recommended best practices for error handling in Go, what you will wind up with is a sort of Greenspun’s-Rule-ish ad-hoc implementation, not of Common Lisp, but of a try/catch exception system. Which is to say: you’ll halt execution at the point where err != nil and begin passing some type of error object up the stack, wrapping it with new frame-local information as it bubbles up through each frame of the stack, until it encounters code which actually handles the error and stops it from propagating further.
But you’ll get none of the ergonomics of an actual try/catch exception language because Go will demand that you implement it yourself every single time, and will demand that you remember to do it at every point where err might not be nil, because Go cannot force you to check that.
you can also lint rules like forcing error checks
Compared to error-handling systems which actually make you handle the error, “you can lint for it” is not great. You see how that’s not great, right?
FYI, this feels like a farce from both the UK legislature and small site owners blocking the UK over this. Prosecutors and courts are not computers with infinite capacity that blindly apply a law by the letter to the entire world. The chance that a case relating to something on lobste.rs is opened is minute. The chance that the prosecution deems it likely they can prevent real harm and secure a conviction from a court despite the risk of creating new case law that is disadvantageous to them is even smaller. The chance that any such conviction would lead to extradition of the site operator to the UK rather than a site block mandated by OFCOM is next to zero.
Without even going through the legalese, I do not see blocking 67 million people from the site to be a measured response. I don’t think anyone went through all the already existing laws in all the western countries to make sure lobste.rs wasn’t breaking one of those. If a bobby ever has a problem with lobste.rs, they will talk to you first to remedy it in a friendly manner because that’s much less work for them.
If you want to make your statement about overregulation and block the UK over this, feel free to do so, but I don’t think you can claim with a straight face that it was a necessary act you were forced into.
The chance is low but the punishments are ruinous. I look both ways when I cross the street for the same reason.
You also haven’t addressed the problem of jurisdiction. If I submit to the friendly bobby who shows up, should I also submit to law enforcement of every jurisdiction? Without getting into the tempting distraction of naming names, there are many polities with flatly incompatible worldviews who’d also like to claim authority over the entire web.
I don’t think anyone went through all the already existing laws in all the western countries to make sure lobste.rs wasn’t breaking one of those.
Probably not. The difference? Well this particular historically important Western country has explicitly drafted this legislation to go after people in other countries.
I am one of the people who began to wonder “Is my phone listening to me?” because of Instagram ads. I never really believed that they were, but it felt like they could be given how targeted the ads were. Here’s a dilemma.
Instagram/Meta/Whoever was listening and sending me microphone-based targeted ads. That’s definitely bad.
Instagram/Meta/Whoever was not listening but they still had a method to send me (and people I talked with) ads (well) targeted enough that they felt as if they could have been based on microphone data. (E.g., I talk to my wife about needing sweaters. Within minutes, both wife and me have buckets of ads for men’s sweaters on Instagram.) That’s also definitely bad.
Either way, I’m glad I quit Instagram (and all social media) as a 2022 New Year’s resolution.
There are plenty ways to track users nowadays: cookies, pixels, tcp hello handshake,… Each comes with a different “resolution” allowing advertisers to send you more relevant ads. Chance is that Meta, Google, Adobe, Alibaba, Bytedance are just really good at building these data pipelines and segment them with different clustering algorithms, ML powered recommendations. It’s next to impossible to disable these completely given that many of these services also design and sell the underlying compute platform that you are using: android, chrome, search, email, isp, etc…
I think all these fear mongers created a really good selling pitch for Apple’s private compute pitch. However, i doubt that it gona last long bc Apple could start selling ads themselves
“Ads that are delivered by Apple’s advertising platform may appear on the App Store, Apple News, Stocks, and Apple TV app. Apple’s advertising platform does not track you, meaning that it does not link user or device data collected from our apps with user or device data collected from third parties for targeted advertising or advertising measurement purposes, and does not share user or device data with data brokers.”
Yes. Why does nobody believe anything a company says any more?
When companies are caught lying even a tiny bit it’s headline news. And yet a lot of people seem convinced you can’t believe anything any company says about anything.
I guess the big problem here is probably around telling the difference between marketing and material statements. If a company says “our product is the best solution on the market” that’s a different category of statement from “our products do not listen to your microphone to target ads” or “we don’t train our LLMs on your inputs”.
Why does nobody believe anything a company says any more?
Because they’re incentivized to lie by the only factor that they care about, money. If they can make more money by lying, they will, then pay a fine or a PR agency or fire some token employee if it comes out. Doing otherwise would be failing the great god of “maximizing shareholder value”. I mean, look who the richest man in the world is right now; what’s his history with materially false statements?
I also believe that “they are listening”. I’m a software engineer who’s worked in ad-tech and has developed mobile apps in the past.
I’m aware that, for example, the Facebook app may not be able to literally use my microphone to listen at that exact second. But I am also aware at a high level that lots of data is collected and exchanged between companies for advertising.
So whether or not Google’s app is listening to me, or my “smart TV” is listening and sending that info with identifying IP or other identity resolution methods, or someone else’s phone is listening and sharing text plus ip and geo, the result is the same. I have many times said incredibly specific things and immediately gone from seeing zero ads about that product to seeing an ad for that product.
It’s kind of like solving a crime or debugging a production issue. The advertisers possess the motive. I believe that they do possess the means (other devices or maybe other more unscrupulous apps on your phone).
More often than not the xkcd observation is true “Correlation doesn’t imply causation, but it does waggle its eyebrows suggestively and gesture furtively while mouthing ‘look over there’”.
You make a good point. While I am comfortable with Apple, and their promise of protecting users, my home network has several IoT devices of questionable origin (like the 4K projector that allows me to login to Netflix and plays sound), and I cannot be sure that they aren’t listening in.
As an example, Chinese random projector brands offer their $300+ projectors for peanuts (like under $50) with coupon codes. I won’t be surprised if these are actually CCP survellience devices. I cannot prove it either way, but I am inclined to believe the no-name cheap Chinese projector is doing something nasty.
On one hand “secure by default” can end in the same place as “won’t someone please think of the children!”. Which is to say it never ends and is used as a justification for increasingly questionable restraints. In some cases trying to protect users from “mistakes” just makes things worse. See mysql_escape_string, mysql_real_escape_string, and PHP in general for an example of bad historically bad defaults and bad fixes for bad defaults.
OTOH, I consider some of the issues raised pretty relevant. Here’s my take, if you don’t agree you can have your money back.
Exposing a database to the public internet is a level of dumbness that I honestly don’t think is worth trying to recover from in the django framework. “What if attackers can arbitrarily edit our data?” I think it was Raymond Chen who characterizes these issues as “rather requires being on the wrong side of the airlock”.
settings.DEBUG should be a bool and only a bool. I’m an unrepentant fan of strong types (ok python people, strong static types) just about anywhere I can in order to avoid stuff like settings.DEBUG = "False" turning DEBUG on.
Pickle: Around the time django was built, using native serialization for session data was common. For java, perl, etc. But in 2024 I think running eval() on stuff in your database (or redis cache) by default is a crap default. Arbitrary object deserialization is just a gift to hackers that keeps on giving. In case anyone isn’t aware: Pickle essentially allows you to run arbitrary code. But also in 2024 signing data by default is not a terrible idea, in the same way that mTLS makes sense even within the datacenter.
At least with languages like C# and python (and anything that allows operator overloading), you can potentially make the above collection.Count() > 0 work in a performant way.
I’ve come to prefer being more explicit in order to get more predictable performance, but there’s a place for overly clever “let’s act like an optimizing compiler” and SQL query planners/ORMs/LINQ probably is that place.
The Google Domains thing is so confusing to me. The negative impact on how much people trust Google Cloud (even though Google Domains was, presumably, in another part of the company) surely must have dwarfed the amount of money they could save not continuing to run that service.
Some day I’d love to hear the inside story of who was arguing for what and how the internal company politics played out. I bet there were some very loud voices arguing against what happened.
In my opinion as someone who no longer works there but was close to at least one big shutdown:
Google’s entire DNA and incentive structure is really about “10x new multi-billion user innovations”.
My best guess is that a new exec came innand saw an opportunity to get some quick cash for their next promo project.
I was nowhere near close enough to get the real inside story. Every coworker was stunned, I think it has recently bec profitable or something. Which is hilarious at this point. “Nobody could have seen this product cancellation coming, says employees of company famous for senseless product cancellatios”.
This is what’s so shocking to me. The ROI / reputation hit avoidance seems so obvious. Like, how many engineers does it take to keep Google Domains up?
Let’s assume it cost 3 million dollars per year to keep up. At companies I’ve worked for, that’s less than our yearly AWS spend. If GCP comes up as an alternative to lower that spend, I will absolutely speak out against it and explain why I think it’s a terrible idea. And I’m obviously not the only one.
i dont know their internal reasons, but domain registrars are one of those very human problems. the staffing overhead for dealing with support and fraud/takedown/compliance/etc was probably not insubstantial
Support people tend to cost way less. Regardless, I budgeted 3 million dollars - spend it how you like. Do you think you’d need more? Because IMO this is easily going to cost them > 3 million.
I didn’t say it wasn’t an engineering problem - it is that as well. FWIW i dont disagree with the core conceit that they are making money on this product, as well. They were probably very revenue positive.
For a registrar operating at Google’s scale? Dealing with international laws and legal challenges? I honestly think it’d be substantially more than $3m.
You’d need product and infra engineering, a few on-call engineers (24 hours), a communications team (to deal with downtime without impacting the google brand), someone who’s familiar with icann and the various registrar related and dns related compliance/ecosystem, a few support engineers, first tier support (24 hours), not to mention a pretty decently staffed and competent legal team - especially one used to dealing with that niche. You’re going to be dealing with a LOT of subpoenas and takedown requests from many different jurisdictions.
And that doesn’t include the attention and time required from senior/executive management overseeing the org that Google Domains is under.
FWIW My back of the hand math comes to about $7million a year of staff costs. Not including redundant, HA infra. I think I’m being conservative with that, as well. If you told me the total staffing expenditure was north of $20m I wouldn’t even blink.
My conservative estimate is that about 20-30 employees run the whole thing. A competing company (such as Namecheap) with about twice the amount of domains (and other services, to be honest), has about 1500 employees. A much smaller company (about half the domains of Google Domains) had about 50 employees.
It’s clearly a revenue positive product, but may not be worth it to them. Which is the usual annoying Google attitude. But also, the reputation hit of such a complex and important product might not have been worth it either - I wouldn’t be surprised if the amount of risk involved was too high for them.
I realize that larger companies have more to do but I feel like that’s a very bloated team you’re describing - like, why would this have to be its own team, for starters? It could be rolled up into a larger team, sharing the support staff and much of the product/infra engineers. Why would you need a dedicated legal team for this? No company operates that way afaik - “normal” lawyers at Google are going to do just fine. Hell, they could have even just stopped selling it entirely - radically cuts down on the overhead around sales, execs, etc.
And then on top of that they earn money, and were even profiting. So idk I just can’t see the cost/benefit here.
It’s not bloated at all. It’s lean. I think you are strongly underestimating the amount of people required to run something this size and with these requirements.
It could be rolled up into a larger team, sharing the support staff and much of the product/infra engineers
Strongly doubt it, or if it was, they still had at least 20 dedicated staff. I don’t know their internal structure. Again, this is a team that serves hundreds of thousands if not millions of customers in dozens of countries.
It takes a LOT of DEDICATED people to run a 24/7 High Availability service that operates in many countries with millions of customers. This is a service, that if it goes down, will make international news. They could literally break the internet for a day.
I considered all of that, that’s why I suggested 20 instead of 50.
As for support staff, you may not need many first tier (you will need a fair amount of second tier), but you’re going to need dedicated technical support.
Why would you need a dedicated legal team for this? “normal” lawyers at Google are going to do just fine
It’s a bit about skillset, and a bit about workload and volume.
For the legal team, for the volume of requests you get as a registrar, you’re going to need/want dedicated people - especially those with understanding of intl law. At Google’s scale, they have someone who’s probably a liason for icann and other registrars - to handle issues and complexities with transfers and regulatory concerns. Some domains can’t be transferred to some companies, for example.
They probably dealt with dozens of law enforcement requests per day.
On top of that, considering how domains intersect with international politics, if you want to operate in all those countries that google did, you’re going to need at least two or three people dedicated just to compliance and regulatory concerns.
You’re going to need multiple on-call engineers JUST for domains - they had about 100million under their watch. I don’t know if you’ve ever staffed a truly 24/7 on-call rotation, but if you’re going to be responsible for any major customers’ domains, you’re going more people than you think (to account for time zones, illness, burnout, etc).
I said before, and I’ll say it again: this was clearly revenue positive for them. If I had to guess, it’s a headache and/or risk thing, or just the normal google bs.
Interesting! I’ve used Otto extensively, which I think is the original pure-Go JSVM. But Otto is rather slow and doesn’t support modern JS features. I also experimented with using V8, but the build process is quite a pain. Goja might be a good replacement.
You might also look into quickjs. It’s pretty solid in my experience so far. It is a c based runtime, but for me dealing with cgo is the lesser evil compared to dealing with a less mature js implementation.
Especially with esbuild you can use all the modern stuff and transpile to what quickjs supports.
I did look at quickjs and otto. Goja had ESM 5.1 support, which I could transpile with esbuild to support, with some extra padding code. quickjs seems great, but I didn’t need it with goja.
V8 was speedy. The bridge to Go and Javascript, however, was tedious. It turns out I didn’t need V8’s speed. It turns out it was fast enough if I had an excellent bridge to Go code.
If you want to make sure the method doesn’t mutate the struct t, use a value receiver.
I think the biggest footguns with value receivers is that, if you forget that you have one (and it’s just a one character difference in the signature), and modifications you make to the receiver are silently discarded. If I remember correctly there are no warnings for this, even if there’s no code that ever reads back your writes.
I wish that the Go team hadn’t copied C’s mistake of using *T for the type “pointer to T”. It should be &T, so & always means “take a pointer” and * always means “dereference a pointer”. func (s &MyStruct) Frobnicate() would at least have a big & instead of a tiny * in it. Or they could have used @ or something. Maybe a use a keyword for deref because you don’t actually do it that often.
The symbol * in types is inconsistent with the symbol & in values.
This is stated as fact but no proof is provided. I don’t see how it could even be a valid proposition. Values use both * and &, but in Go there is only one annotation used for types. Any choice would be arbitrary.
Mind you, this choice is not only consistent with C, but consistent with Rust as well. Rust has both *T and &T, and Go’s pointers are clearly analogous to Rust’s raw pointers and not to Rust references since Go pointers are nullable.
though both have the same general meaning of “take a pointer”.
The meaning of *T is a pointer to T, no more and no less. No “taking of” action is implied. And in Rust *T and &T are both some sort of reference with the notational choice simply indicating which type of reference it is and not indicating some sort of asymmetry, unlike * and & when applied to values.
The symbol * is easy to overlook in function signatures, as compared to &, @, or a keyword.
Citation needed.
Notating types by &T is a perfectly sensible choice, but I object to claims that *T is inconsistent or less readable than &T.
I think the reasoning is quite simple: on values, &thing gives you a pointer to thing; it would be nice if on types, &Thing would give you the type of pointers to Things. Or at least that’s how i understood it.
Aside: i think C has a good rationale for using * as the symbol for pointers types, even if it is the dereference operator. The declaration int foo says “foo is an int”, and the declaration of a function int foo(char) says “calling foo with a char is a an int”. Similarly, int *foo says “dereferencing foo is an int”, which i find pretty elegant and consistent TBH (and also a nice argument for putting the * besides the variable name rather than the type).
Regarding why * would be easier to overlook than &, @, or a keyword, I thought it was visually apparent, so no citation was needed. The asterisk * (or, in another font, *) is a visually smaller symbol – it has fewer black pixels than the other symbols. Also, it cannot be seen when the upper half of the line is scrolled offscreen. Just as the symbol * is easier to see than ' (straight quote), the symbol @ or keyword pointer would be easier to see than *.
Of course, the difference in difficulty between noticing * and noticing other symbols depends on the viewer’s font, font size, color scheme, screen DPI, and so on. Also, I haven’t personally had a problem noticing * symbols in the programs I write (though those programs usually aren’t in languages that use * as part of types). But I can easily believe that those programmers who do overlook * symbols would, on average, be less likely to overlook a larger symbol.
this is one thing I like about languages like Rust that use keywords instead of characters for this, it makes it much more obvious when you see a “mut” (highlighted too, since it’s a keyword) compared to “*” (which is not highlighted and easily missed sometimes) it still catches me even after a decade of Go!
So judging by the comments I’m not sure if folks took a look at the implementation. Say whatever you want about braces, but the implementation of bython (python with braces) is pretty gnarly.
It’s all parsing python code with a bunch of ad-hoc regular expressions. Judging by the code comments there’s been a lot of whack a mole being played to get various test cases to pass.
I actually use job control all day. I use emacs, but don’t always like using emacsclient or the gui.
Ctrl-z to leave emacs, fg to bring it back. That’s about 90% of what I do, occasionally I bg a job. Also don’t forget Ctrl-s,Ctrl-q which stop/resume. If for no other reason than to understand why sometimes my terminal accidentally freezes!
Scheduling heuristics are tough. A while back when goroutines weren’t fully preemptive they made a release which said that function calls could potentially yield to the scheduler. So I no longer had to worry about a runaway for loop blocking everything else.
At the time I was doing some high performance logging so I had a for loop that called write and a simple time.After call to cancel the loop after one second.
To my surprise it didn’t seem as if the loop was yielding. The process would regularly take 5 seconds to respond.
I reported what I thought was a bug and learned a few technical and non-technical lessons. The responses from the Go team mostly followed the narcissists prayer: https://www.thelifedoctor.org/the-narcissist-s-prayer
“Well time.After is guaranteed to sleep at least n seconds.”
Technically true. The best kind of true. But while we are all aware desktop Linux is not a rtos, there’s a fairly large difference between one second and five seconds.
“Why are you calling a function in a loop? Artificial micro benchmark!”
Well. I heard functions would now yield and I assured them our real life code (which was basically some big data processing using stdio like Hadoop streaming used to work) did in fact write a lot of data in a loop.
Then I learned things like “write is on a list of syscalls that is supposed to be ‘fast’ so it might not yield”.
Finally one of the more interesting responses was “the loop is yielding to the scheduler and the scheduler heuristic is essentially saying that this goroutine clearly has a lot of work to do so it keeps passing control right back”. That one really opened my eyes to the complexity of “fair scheduling “.
While my bug was dismissed, I was fortunate enough to be on Dave Cheney’s good side at the time due to running a go arm5 builder. So a nice word from him made someone, Dmitry iirc, decide I wasn’t being unreasonable and throw in some extra fairness or randomness.
Remember kids: it’s not what you know, it’s who you know.
P.s. I’m writing from memory from events a decade ago. Apologies in advance for the likely many details I’ve misremembered.
P.p.s. after writing a bunch of async Rust code that does something that is in go’s sweet spot: make a bunch of parallel http requests that do CPU intensive stuff, I miss the go scheduler.
Looks like the ban works for its intended purpose: preventing radio interference. If someone is willing to put in this much effort to evade the ban, I’d let them get away with it. :)
I like Neighborhood Wifi as a social engineering trick. I wouldn’t given it a second thought.
This is the strategy I went for in a similar situation!
My dorms in a Welsh university had rules banning switches/wifi routers/etc from being connected to their network … but at the same time, they had DPI which blocked connections to the Debian archives on the grounds that it was a “freeware” site, so I just set up a PC (£50 SFF Dell Optiplex … power was free) with a 4 port Intel NIC.
Installed VyOS, and configured it as a gateway which, from the uni’s perspective, was just a PC establishing a WireGuard VPN over a certain port for a popular online MMORPG that I was well informed the DPI wouldn’t mess with. (No policies against VPNs from what I saw, but it didn’t work unless I used an acceptable port…)
Bonus: I got BGP feeds from a friend who was giving me the WireGuard endpoint, so announced some of my own v6 space :-)
On the other side, this was connected to a managed switch and a cheap Unifi AP broadcasting “Joris Bohnson’s iPhone” on low TX power — this went all year without a peep from anyone.
After a month or so, I did swap the Dell for an EdgeRouter X and just spoofed its MAC address to the Dell I’d registered, for less fan noise, and so I could repurpose the Dell as a server.
Of course, I maintain that I was compliant, as the AP, etc. wasn’t connected to their network, rather through (using an encrypted tunnel is a Jisc-approved way for unis to connect guest wifi through Janet, at that) :-)
I love to hear stories of undergrads testing the limits of the rules to set up interesting network services from their student rooms :-)
I had the enormous privilege of 10 Mbit/s internet to my rooms, 1995-1997. I had a public IP address, minimally filtered - there were some basic port blocks on 25 and 53 which was mildly annoying since I wanted to use those protocols, but I could run a web server no problem. Years later when I became a member of staff I was happy that my colleagues had not followed the crowd in relaxing the rules for email because it made it much easier for me to control outgoing spam. I am still not sure if there is a lesson to be learned or if it depended too much on the weird circumstances at the time…
My big thing at USU was realizing that our email was served by a VMS cluster called the 7 dwarves. Your student email also gave ftp access to publish docs to something like usu.edu/~username. You also had ssh access with port forwarding so you could socks proxy.
Ssh access put you on a VMS machine which was a weird environment but it had perl so you could play around a bit.
Oh that and realizing the student info kiosks were just running firefox full screen. It wasn’t that hard to figure out how to bring up a print dialog and change the print command from lpr to xterm. It was just a knoppix livecd so any changes didn’t persist.
Sounds fun! :-) I didn’t get a public IP (all NAT’ed through Sonicwall(?)), but was definitely spoiled by symmetric low-latency gigabit ethernet, coming from shite Virgin Media in Plymouth…
It’s nice your uni left most ports open – I suppose P2P file sharing probably killed a lot of that fun later on. Out of interest, why block just 53 - was the issue of open resolvers being used for an amplification attack a particularly known/prevalent risk back then?
P2P was a big issue, especially before the post-dot-com bandwidth glut kicked in. I can’t remember whether it led to port blocks, tho. There were other bigger issues:
There weren’t enough public IP addresses for everyone, so student rooms got stuck behind NAT, preventing them from running servers.
Windows security was dogshit; it was the main reason for putting more port blocks in the network.
The impression I get, which I hope is not too far off the mark, is that port 53 was blocked because there was a lingering attitude that registering hosts was a serious matter that should be done properly, with things like matching forward/reverse DNS. The computing service staff couldn’t ensure that DNS servers were operated correctly if they didn’t know about them. At that time, domain names still cost $100 per year and registration was largely manual. And open resolvers were the norm rather than the exception! My memory is vague but I don’t think that changed until after 2000.
Thought about that actually! But as stated, the ban seems to intended to apply to all wireless networks. Tethering a phone might technically violate it or trip detection. And it would definitely suggest it’s in your building. Neighborhood WiFi would appear to be from somewhere else
I like Gruber but I’m very much in the opposite camp. I hate Apple shipping software because they said they would in a keynote. I don’t turn to Apple because they hit all their OKRs on a spreadsheet.
If they’re not happy with it, don’t ship it. That’s why I pay the Apple Tax. I don’t want half done mostly working sorta broken crap like almost all the AI products I’ve tried. Do it right or don’t ship it. I actually don’t care at all if these features ever launch, but I’m certainly not going to get upset with Apple for doing what I’ve begged them to do for years.
I don’t think Gruber’s argument is that Apple should have shipped the feature even if it doesn’t work - he’s arguing that Apple shouldn’t have announced it (and used it in marketing materials to sell more iPhones) if they didn’t yet have a working project that they could reliably ship.
That’s not the point. Assuming the software is shipped when ready anyway, there’s still the problem of advertising features and selling hardware based on something that is far away from shipping, and may have even been just a fake concept art.
Apple is generally very strict about not talking about future features, and they typically announce features when they exist, and only need polish and bugfixing before shipping. The AI Siri was apparently different — they’ve announced it as a real feature before they’ve implemented it.
It’s the discrepancy of showing a feature “working” in TV ads for general public, but not having the feature working enough to prove to anyone outside Apple that it even exists.
His entire point was that, unlike other releases, they never even got to half done or sorta broken.
They just advertised pure vaporware. Straight up fraud.
Echoing and consolidating what others have said.
The c# author chose go for this project rather than c#. It’s not because go is “better” but it’s better for
tsc. He said that c# is fundamentally a bytecode first language and he doesn’t consider it’s aot capabilities to be that mature. So essentially he thought binaries built by go would be a better deployment target due to better (in his opinion not mine) cross compilation support. He also mentioned that he considers low level memory/struct layout to be easier to do in go and presumably he wanted that in this project.In a way go won because it was a syntactically similar natively compiled garbage collected language with the best cross compilation story.
Rust seems to have been ruled out because it either would have required reworking the object graph to comply with rust ownership rules (believed to be harder than line by line transliteration) or they would have had to
GC<Box<T>>all the things. I presume they ruled that out because it was ugly and painful to work with but I don’t know if that’s true.Despite how good javascript jits can be, they don’t shine at the polymorphic graph analysis compilers do. If everything was being manipulated as a big byte array, v8 would probably have been more competitive.
So the tl;Dr is that ultimately golang was closest 1:1 transpile target that met their needs.
Is this the first big Go project at Microsoft? I had no idea they could choose Go, I would assume that it just wouldn’t be acceptable, because they wouldn’t want to unnecessarily legitimize a Google project.
Their web browser runs on Chromium, I think it’s a little late to avoid legitimizing Google projects
They had already lost the web platform by the time they switched to Chromium.
When I think of MS and ‘developers developers developers develop’, I was lead to believe that they’d have more pride in their own development platforms.
I wonder if the switch to Chromium, the embrace of Linux in Azure, the move from VS to VSCode, any of these, would have happened under Ballmer or Gates. I suspect Gates’ new book is like Merkel’s: retrospective but avoiding the most controversial questions: Russian gas and giving up dogfooding. Am I foolish to expect a more critical introspection?
I think corporate open source is now “proven”, and companies don’t worry so much about this kind of thing.
Google has used TypeScript for quite awhile now (even though they had their own Closure compiler, which was not staffed consistently, internally)
All of these companies are best thought of as “frenemies” :) They cooperate in some ways, and compete in others.
They have many common interests, and collaborate on say lobbying the US govt
When Google was small, and Microsoft was big, the story was different. They even had different employee bases. But now they are basically peers, and employees go freely back and forth
Microsoft has actually maintained a security-hardened build of Go for a while now: https://devblogs.microsoft.com/go/
I can see a blog post saying they made changes for FIPS compliance, which is usually understood to be for satisfying government requirements and not a security improvement. I can’t see a summary of any other changes.
MS is a pretty big company, so I wouldn’t be surprised if they have a ton of internal services in Go. Plus with Azure being a big part of their business, I doubt they care what language people use as long as you deploy it on their platforms.
Can confirm that some internal services were written in go as of a couple years ago. (I worked for Microsoft at the time.) I didn’t have a wide enough view to know if it was “a ton,” though.
We embrace and we extend!
I’m too lazy to look up a source but some of the error messages I’ve seen in azure make it clear Microsoft uses go to build their cloud.
Message like this one:
Keep in mind Microsoft also maintains their own go fork for fips compliance. I don’t know exactly their use case but I can tell you at Google we did the same thing for the same reason. I think the main fips compliant go binaries being shipped were probably gke/k8s related.
(Edit removed off topic ramble, sorry I have ADHD)
Could this message just be coming from Kubernetes?
One of the benefits of microservices is that if you have a malicious dependency that requires the ability to talk to arbitrary public up addresses you can just isolate it. No need to put it in the same process/ address space as the code that mints API keys, etc.
To me, that’s the solution to these problems. Arbitrary egress isn’t something you typically need and it’s sketchy enough to move it to a sidecar if you do. Of course, very few people do that, but it seems way cheaper than auditing every dependency, though maybe AI would help.
TBH I was a bit surprised to see this run as part of the deployed server. Seems like running in a build script is going to yield better results/ wins but that’s just how I’d do it personally.
The Go SDK has no way to execute arbitrary code when building.
Oh interesting. I guess that makes sense, the expectations is that you’d use bazel or something.
Go has some stuff like
go generatebut again that doesn’t actually causego buildto execute code.But yeah most of the weird stuff that pops up when looking at go build and dependency history can be explained by “the people who wrote go thought of building and dependency management as being the responsibility of a monorepo build system which has explicit support for generated files as something distinct from source code and binaries”.
One time someone found a way to make cgo linker and compiler flags execute arbitrary code, and that was considered a vulnerability. https://github.com/golang/go/issues/42556
For comparison, it’s normal for cargo build to execute arbitrary code (build.rs).
I think the idea was avoiding a series of vulnerabilities related to this
I assume because Go was built at Google so bazel is already the build tool. And bazel already solves a ton of these problems, which is great.
Most of the Go SDK that isn’t the compiler is not used when compiling Go code with Bazel.
Also https://github.com/golang/go/issues/71368
Using KV rows seems like a terrible idea for a langage with variable type size, is abseil’s also laid out like that?
In both Go and C++ the key and value types are known at compile time so for any instance the key and value size are fixed.
(Go has no variable size types. Arrays, strings, slices, etc. are small fixed-size structs that point to variable-length blocks.)
You’re right that the size of types are statically known, but Go arrays are actually not small fixed sized structs.
In particular if x is an array of at least size one then &x==&x[0] (except for of course go not allowing this comparison based on different types). You can confirm with the below playground link which prints out the address of the array as well as the address of each item.
https://go.dev/play/p/TcbxqnjIuFo
Not variable size types, variable type size.
As in different types (can) have different sizes, as opposed to reference based languages where all types take a pointer and memory layout is only a concern for a type’s internal metadata.
Hence my surprise that they used a row layout, if all types have the same size then row v column is not (as much of) a concern, every value takes a pointer and there’s no padding or storage loss. But if the key and value can have different alignments then you generally want to segregate keys and values to avoid waste.
Not true of arrays, but not the point.
I assume that cache coherency / locality of reference is the rationale for putting keys and values together. Performance trumps compactness. Also, the typical types you use in maps contain pointers or are at least pointer-sized, so they’ll have compatible alignment.
Except now you’re losing that during probing, or when just iterating keys, especially if the value is not behind a pointer.
Unless compactness is what yields performances.
Unless they don’t because e.g. go not having sets means byte values are common.
Swiss Tables do a lot less probing due to that cute little byte-array table.
As for the other points, I dunno, it’s clear Google should have had you in the room suggesting these things, since of course they wouldn’t bother to actually profile and compare different layouts.
I’ve actually had success with log all the things before, but usually in the form of what are essentially traces or request logs. Even this has it’s problems. It’s expensive, although often worth the cost. It’s almost certain you will need to do ongoing compliance work.
The thing that makes this work is that it’s a structured log with a pretty uniform schema. There’s also a pretty well trodden pathway to sampling (including sampling complete transactions not just randomly dropping log line).
What is usually a disaster though is log shipping unstructured printf style logs. As the article mentions, if you don’t know who’s going consuming them you don’t know which debug statements are load bearing. I’ve seen big companies trying to reduce logging costs but even though they knew 99% of the data was useless, they didn’t know which 99% it was.
I think that a lot of companies are religiously saving random library output on stderr/stdout to long term storage. Instead it’s often better to have a local ring buffer or some sort.
[edit]
I should add the domain I work in because I think that matters. I’ve spent time in e-commerce and ad-tech. In e-commerce, if you have an average order value of $50 the percentage of revenue going to logging http requests is relatively small. In ad-tech, you just can’t budget as much storage for every impression.
This feels emotionally charged
I like the err != nil because I know that errors are clearly being handled somewhere, and the logic for that is local to the caller
Except you don’t know that errors are being handled, and the logic isn’t local to the caller.
Nothing I’m aware of in Go requires you to actually check
if err != nil, meaning your program will still compile if you don’t do it, and will still run. Just when the error condition finally does happen, you’ll get a mysterious problem at some other location.Compare this to exceptions or sum types, both of which force you to address the problem before moving on. An uncaught exception doesn’t allow execution to proceed, and in languages with sum types a failure to resolve them before use is generally a compile-time error.
Similarly, the logic is non-local; the near-universal pattern in Go is
if err != nil {return nil, err}or variant forms of it that wraperrin some sort of nice reporting structure. That’s not “logic”, that’s just yeeting the error up the call stack in hope it will finally encounter an actual error handler (and if we’re being really cynical, is justtry/catchbut implemented manually). Figuring out which handler that might be and what it will do can easily be a non-trivial bit of analysis, and is basically never obvious from just reading the surrounding code.I’ve only contributed to one open-source project written in Go, and they had a linter that made sure that err returns were either checked or forwarded to the caller. I think it was a standard go thing, I don’t know why it isn’t a compiler warning.
My biggest problem with Go is that the type system does nothing to help you avoid footguns. In Go, it is undefined behaviour to concurrently modify an object from two goroutines (and, yes, this can break memory safety if you do). Yet Go has primitives that make it trivial to send references to objects to other goroutines. In Erlang, you have the same kind of send (messages to actors rather than messages via channels), but Erlang always passes objects by value (or by reference as an optimisation, but they’re immutable so this isn’t visible in the abstract machine). Concurrent mutation is impossible by construction. In Rust, the Send trait ensures that objects are passed as a move operation and the borrow checker ensures that you don’t hold any pointers to objects dominated by the sent object after you send it. In Go, sending a pointer via a channel is just as easy as sending a by-value copy. Easier, in fact, because copies are shallow so sending an object by value will send any referenced objects by reference and end up with accidental aliasing.
All of this means that there’s a huge cognitive load writing concurrent Go, yet concurrency was meant to be one of the main selling points of Go. The people I’ve spoken to who are happy with Go treat it as a replacement for Python that produces a statically linked binary.
Yeah that limitation of channel sends was annoying, and I had to adopt conventions to reduce the “cognitive load” and avoid errors.
I’d have liked the ability to have some form of “const/immutable pointer” such that a read only reference to an object could be passed around.
As for the lack of move operation, I simply adopted a convention of immediately nil’ing the pointer on the sending side after sending it. Possibly a linter could be written to verify that convention?
Obviously someone subsequently updating the code could fail to be as rigorous, and introduce an issue. I was pondering catching that with my UTs, by building upon the finalisation facility in the runtime, but never got around to doing it.
All languages have their issues, and sharp corners…
I seem to recall that go has no compiler warnings. Only compiler errors. Warnings are for linters.
True. What we rely on is a weaker property: unused variables are a compiler error.
So at first this seems almost as good. You do in fact have to do something with err. It gets a little trickier when you realize that err can be reused so you could do 2 a, err = operations and only check the second operation and that would make the compiler happy.
But in a worse is better sort of way Go code frequently won’t compile if you don’t check the err.
You can choose at what level to handle it, and i think it’s pretty strong consensus that exceptions are a bad model over something like returning errors/result types
you can also lint rules like forcing error checks
Reiterating for clarity: if you follow all the recommended best practices for error handling in Go, what you will wind up with is a sort of Greenspun’s-Rule-ish ad-hoc implementation, not of Common Lisp, but of a try/catch exception system. Which is to say: you’ll halt execution at the point where
err != niland begin passing some type of error object up the stack, wrapping it with new frame-local information as it bubbles up through each frame of the stack, until it encounters code which actually handles the error and stops it from propagating further.But you’ll get none of the ergonomics of an actual try/catch exception language because Go will demand that you implement it yourself every single time, and will demand that you remember to do it at every point where
errmight not benil, because Go cannot force you to check that.Compared to error-handling systems which actually make you handle the error, “you can lint for it” is not great. You see how that’s not great, right?
Yeah but it’s habit now for me, but I can see why someone that’s not used to it might feel uncomfortable with it
FYI, this feels like a farce from both the UK legislature and small site owners blocking the UK over this. Prosecutors and courts are not computers with infinite capacity that blindly apply a law by the letter to the entire world. The chance that a case relating to something on lobste.rs is opened is minute. The chance that the prosecution deems it likely they can prevent real harm and secure a conviction from a court despite the risk of creating new case law that is disadvantageous to them is even smaller. The chance that any such conviction would lead to extradition of the site operator to the UK rather than a site block mandated by OFCOM is next to zero.
Without even going through the legalese, I do not see blocking 67 million people from the site to be a measured response. I don’t think anyone went through all the already existing laws in all the western countries to make sure lobste.rs wasn’t breaking one of those. If a bobby ever has a problem with lobste.rs, they will talk to you first to remedy it in a friendly manner because that’s much less work for them.
If you want to make your statement about overregulation and block the UK over this, feel free to do so, but I don’t think you can claim with a straight face that it was a necessary act you were forced into.
The chance is low but the punishments are ruinous. I look both ways when I cross the street for the same reason.
You also haven’t addressed the problem of jurisdiction. If I submit to the friendly bobby who shows up, should I also submit to law enforcement of every jurisdiction? Without getting into the tempting distraction of naming names, there are many polities with flatly incompatible worldviews who’d also like to claim authority over the entire web.
To answer one small part of what you said:
Probably not. The difference? Well this particular historically important Western country has explicitly drafted this legislation to go after people in other countries.
I am one of the people who began to wonder “Is my phone listening to me?” because of Instagram ads. I never really believed that they were, but it felt like they could be given how targeted the ads were. Here’s a dilemma.
Either way, I’m glad I quit Instagram (and all social media) as a 2022 New Year’s resolution.
There are plenty ways to track users nowadays: cookies, pixels, tcp hello handshake,… Each comes with a different “resolution” allowing advertisers to send you more relevant ads. Chance is that Meta, Google, Adobe, Alibaba, Bytedance are just really good at building these data pipelines and segment them with different clustering algorithms, ML powered recommendations. It’s next to impossible to disable these completely given that many of these services also design and sell the underlying compute platform that you are using: android, chrome, search, email, isp, etc…
I think all these fear mongers created a really good selling pitch for Apple’s private compute pitch. However, i doubt that it gona last long bc Apple could start selling ads themselves
https://www.apple.com/legal/privacy/data/en/apple-advertising/
Apple already sells ads.
“Ads that are delivered by Apple’s advertising platform may appear on the App Store, Apple News, Stocks, and Apple TV app. Apple’s advertising platform does not track you, meaning that it does not link user or device data collected from our apps with user or device data collected from third parties for targeted advertising or advertising measurement purposes, and does not share user or device data with data brokers.”
And you’re just going to take their word for it?
Yes. Why does nobody believe anything a company says any more?
When companies are caught lying even a tiny bit it’s headline news. And yet a lot of people seem convinced you can’t believe anything any company says about anything.
I guess the big problem here is probably around telling the difference between marketing and material statements. If a company says “our product is the best solution on the market” that’s a different category of statement from “our products do not listen to your microphone to target ads” or “we don’t train our LLMs on your inputs”.
Because they’re incentivized to lie by the only factor that they care about, money. If they can make more money by lying, they will, then pay a fine or a PR agency or fire some token employee if it comes out. Doing otherwise would be failing the great god of “maximizing shareholder value”. I mean, look who the richest man in the world is right now; what’s his history with materially false statements?
None of the companies I have ever worked for have seemed like that as an insider.
So what? That still allows selling targeted ads.
If there’s no user or device data shared with data brokers, how are those brokers targeting ads?
People buying ads from apple can target them at “segments” based on personal info, so long as each segment contains at least 5000 people. It’s in the link above. https://www.apple.com/legal/privacy/data/en/apple-advertising/
I also believe that “they are listening”. I’m a software engineer who’s worked in ad-tech and has developed mobile apps in the past.
I’m aware that, for example, the Facebook app may not be able to literally use my microphone to listen at that exact second. But I am also aware at a high level that lots of data is collected and exchanged between companies for advertising.
So whether or not Google’s app is listening to me, or my “smart TV” is listening and sending that info with identifying IP or other identity resolution methods, or someone else’s phone is listening and sharing text plus ip and geo, the result is the same. I have many times said incredibly specific things and immediately gone from seeing zero ads about that product to seeing an ad for that product.
It’s kind of like solving a crime or debugging a production issue. The advertisers possess the motive. I believe that they do possess the means (other devices or maybe other more unscrupulous apps on your phone).
More often than not the xkcd observation is true “Correlation doesn’t imply causation, but it does waggle its eyebrows suggestively and gesture furtively while mouthing ‘look over there’”.
You make a good point. While I am comfortable with Apple, and their promise of protecting users, my home network has several IoT devices of questionable origin (like the 4K projector that allows me to login to Netflix and plays sound), and I cannot be sure that they aren’t listening in.
As an example, Chinese random projector brands offer their $300+ projectors for peanuts (like under $50) with coupon codes. I won’t be surprised if these are actually CCP survellience devices. I cannot prove it either way, but I am inclined to believe the no-name cheap Chinese projector is doing something nasty.
On one hand “secure by default” can end in the same place as “won’t someone please think of the children!”. Which is to say it never ends and is used as a justification for increasingly questionable restraints. In some cases trying to protect users from “mistakes” just makes things worse. See mysql_escape_string, mysql_real_escape_string, and PHP in general for an example of bad historically bad defaults and bad fixes for bad defaults.
OTOH, I consider some of the issues raised pretty relevant. Here’s my take, if you don’t agree you can have your money back.
settings.DEBUG = "False"turning DEBUG on.eval()on stuff in your database (or redis cache) by default is a crap default. Arbitrary object deserialization is just a gift to hackers that keeps on giving. In case anyone isn’t aware: Pickle essentially allows you to run arbitrary code. But also in 2024 signing data by default is not a terrible idea, in the same way that mTLS makes sense even within the datacenter.I’ve seen the same anti-pattern with LINQ.
better:
At least with languages like C# and python (and anything that allows operator overloading), you can potentially make the above
collection.Count() > 0work in a performant way.I’ve come to prefer being more explicit in order to get more predictable performance, but there’s a place for overly clever “let’s act like an optimizing compiler” and SQL query planners/ORMs/LINQ probably is that place.
The Google Domains thing is so confusing to me. The negative impact on how much people trust Google Cloud (even though Google Domains was, presumably, in another part of the company) surely must have dwarfed the amount of money they could save not continuing to run that service.
Some day I’d love to hear the inside story of who was arguing for what and how the internal company politics played out. I bet there were some very loud voices arguing against what happened.
In my opinion as someone who no longer works there but was close to at least one big shutdown:
Google’s entire DNA and incentive structure is really about “10x new multi-billion user innovations”.
My best guess is that a new exec came innand saw an opportunity to get some quick cash for their next promo project.
I was nowhere near close enough to get the real inside story. Every coworker was stunned, I think it has recently bec profitable or something. Which is hilarious at this point. “Nobody could have seen this product cancellation coming, says employees of company famous for senseless product cancellatios”.
This is what’s so shocking to me. The ROI / reputation hit avoidance seems so obvious. Like, how many engineers does it take to keep Google Domains up?
Let’s assume it cost 3 million dollars per year to keep up. At companies I’ve worked for, that’s less than our yearly AWS spend. If GCP comes up as an alternative to lower that spend, I will absolutely speak out against it and explain why I think it’s a terrible idea. And I’m obviously not the only one.
probably not an engineering issue tbh
i dont know their internal reasons, but domain registrars are one of those very human problems. the staffing overhead for dealing with support and fraud/takedown/compliance/etc was probably not insubstantial
Support people tend to cost way less. Regardless, I budgeted 3 million dollars - spend it how you like. Do you think you’d need more? Because IMO this is easily going to cost them > 3 million.
I didn’t say it wasn’t an engineering problem - it is that as well. FWIW i dont disagree with the core conceit that they are making money on this product, as well. They were probably very revenue positive.
For a registrar operating at Google’s scale? Dealing with international laws and legal challenges? I honestly think it’d be substantially more than $3m.
You’d need product and infra engineering, a few on-call engineers (24 hours), a communications team (to deal with downtime without impacting the google brand), someone who’s familiar with icann and the various registrar related and dns related compliance/ecosystem, a few support engineers, first tier support (24 hours), not to mention a pretty decently staffed and competent legal team - especially one used to dealing with that niche. You’re going to be dealing with a LOT of subpoenas and takedown requests from many different jurisdictions.
And that doesn’t include the attention and time required from senior/executive management overseeing the org that Google Domains is under.
FWIW My back of the hand math comes to about $7million a year of staff costs. Not including redundant, HA infra. I think I’m being conservative with that, as well. If you told me the total staffing expenditure was north of $20m I wouldn’t even blink.
My conservative estimate is that about 20-30 employees run the whole thing. A competing company (such as Namecheap) with about twice the amount of domains (and other services, to be honest), has about 1500 employees. A much smaller company (about half the domains of Google Domains) had about 50 employees.
It’s clearly a revenue positive product, but may not be worth it to them. Which is the usual annoying Google attitude. But also, the reputation hit of such a complex and important product might not have been worth it either - I wouldn’t be surprised if the amount of risk involved was too high for them.
I realize that larger companies have more to do but I feel like that’s a very bloated team you’re describing - like, why would this have to be its own team, for starters? It could be rolled up into a larger team, sharing the support staff and much of the product/infra engineers. Why would you need a dedicated legal team for this? No company operates that way afaik - “normal” lawyers at Google are going to do just fine. Hell, they could have even just stopped selling it entirely - radically cuts down on the overhead around sales, execs, etc.
And then on top of that they earn money, and were even profiting. So idk I just can’t see the cost/benefit here.
It’s not bloated at all. It’s lean. I think you are strongly underestimating the amount of people required to run something this size and with these requirements.
Strongly doubt it, or if it was, they still had at least 20 dedicated staff. I don’t know their internal structure. Again, this is a team that serves hundreds of thousands if not millions of customers in dozens of countries.
It takes a LOT of DEDICATED people to run a 24/7 High Availability service that operates in many countries with millions of customers. This is a service, that if it goes down, will make international news. They could literally break the internet for a day.
I considered all of that, that’s why I suggested 20 instead of 50.
As for support staff, you may not need many first tier (you will need a fair amount of second tier), but you’re going to need dedicated technical support.
It’s a bit about skillset, and a bit about workload and volume.
For the legal team, for the volume of requests you get as a registrar, you’re going to need/want dedicated people - especially those with understanding of intl law. At Google’s scale, they have someone who’s probably a liason for icann and other registrars - to handle issues and complexities with transfers and regulatory concerns. Some domains can’t be transferred to some companies, for example.
They probably dealt with dozens of law enforcement requests per day.
On top of that, considering how domains intersect with international politics, if you want to operate in all those countries that google did, you’re going to need at least two or three people dedicated just to compliance and regulatory concerns.
You’re going to need multiple on-call engineers JUST for domains - they had about 100million under their watch. I don’t know if you’ve ever staffed a truly 24/7 on-call rotation, but if you’re going to be responsible for any major customers’ domains, you’re going more people than you think (to account for time zones, illness, burnout, etc).
I said before, and I’ll say it again: this was clearly revenue positive for them. If I had to guess, it’s a headache and/or risk thing, or just the normal google bs.
I guess I just disagree that it would take that much but that’s okay. It doesn’t really matter, it was still stupid of them to shut it down.
Interesting! I’ve used Otto extensively, which I think is the original pure-Go JSVM. But Otto is rather slow and doesn’t support modern JS features. I also experimented with using V8, but the build process is quite a pain. Goja might be a good replacement.
You might also look into quickjs. It’s pretty solid in my experience so far. It is a c based runtime, but for me dealing with cgo is the lesser evil compared to dealing with a less mature js implementation.
Especially with esbuild you can use all the modern stuff and transpile to what quickjs supports.
I did look at quickjs and otto. Goja had ESM 5.1 support, which I could transpile with esbuild to support, with some extra padding code. quickjs seems great, but I didn’t need it with goja.
Goja has been around for a very long time, longer than QuickJS, and is very mature and stable. Also the Go API is incredible.
V8 was speedy. The bridge to Go and Javascript, however, was tedious. It turns out I didn’t need V8’s speed. It turns out it was fast enough if I had an excellent bridge to Go code.
I think the biggest footguns with value receivers is that, if you forget that you have one (and it’s just a one character difference in the signature), and modifications you make to the receiver are silently discarded. If I remember correctly there are no warnings for this, even if there’s no code that ever reads back your writes.
I wish that the Go team hadn’t copied C’s mistake of using
*Tfor the type “pointer to T”. It should be&T, so&always means “take a pointer” and*always means “dereference a pointer”.func (s &MyStruct) Frobnicate()would at least have a big&instead of a tiny*in it. Or they could have used@or something. Maybe a use a keyword for deref because you don’t actually do it that often.Why?
Doesn’t Carlana’s comment already contain her reasoning? Here’s a rephrasing.
*in types is inconsistent with the symbol&in values, though both have the same general meaning of “take a pointer”.*is easy to overlook in function signatures, as compared to&,@, or a keyword.This is stated as fact but no proof is provided. I don’t see how it could even be a valid proposition. Values use both
*and&, but in Go there is only one annotation used for types. Any choice would be arbitrary.Mind you, this choice is not only consistent with C, but consistent with Rust as well. Rust has both
*Tand&T, and Go’s pointers are clearly analogous to Rust’s raw pointers and not to Rust references since Go pointers are nullable.The meaning of
*Tis a pointer toT, no more and no less. No “taking of” action is implied. And in Rust*Tand&Tare both some sort of reference with the notational choice simply indicating which type of reference it is and not indicating some sort of asymmetry, unlike*and&when applied to values.Citation needed.
Notating types by
&Tis a perfectly sensible choice, but I object to claims that*Tis inconsistent or less readable than&T.I think the reasoning is quite simple: on values,
&thinggives you a pointer tothing; it would be nice if on types,&Thingwould give you the type of pointers toThings. Or at least that’s how i understood it.Aside: i think C has a good rationale for using
*as the symbol for pointers types, even if it is the dereference operator. The declarationint foosays “foo is an int”, and the declaration of a functionint foo(char)says “calling foo with a char is a an int”. Similarly,int *foosays “dereferencing foo is an int”, which i find pretty elegant and consistent TBH (and also a nice argument for putting the*besides the variable name rather than the type).Regarding why
*would be easier to overlook than&,@, or a keyword, I thought it was visually apparent, so no citation was needed. The asterisk*(or, in another font, *) is a visually smaller symbol – it has fewer black pixels than the other symbols. Also, it cannot be seen when the upper half of the line is scrolled offscreen. Just as the symbol*is easier to see than'(straight quote), the symbol@or keywordpointerwould be easier to see than*.Of course, the difference in difficulty between noticing
*and noticing other symbols depends on the viewer’s font, font size, color scheme, screen DPI, and so on. Also, I haven’t personally had a problem noticing*symbols in the programs I write (though those programs usually aren’t in languages that use*as part of types). But I can easily believe that those programmers who do overlook*symbols would, on average, be less likely to overlook a larger symbol.As discussed on lobste.rs a few weeks ago IIRC, real men use non-breaking space as their sigil symbol.
Transition saved my life, ROFL.
this is one thing I like about languages like Rust that use keywords instead of characters for this, it makes it much more obvious when you see a “mut” (highlighted too, since it’s a keyword) compared to “*” (which is not highlighted and easily missed sometimes) it still catches me even after a decade of Go!
I almost never want to use a value receiver unless it’s something like a Point type that is two int64s.
A compiler inserted unsynchronized shallow copy is, to a first approximation, never the behavior I want when calling a method.
So judging by the comments I’m not sure if folks took a look at the implementation. Say whatever you want about braces, but the implementation of bython (python with braces) is pretty gnarly.
It’s all parsing python code with a bunch of ad-hoc regular expressions. Judging by the code comments there’s been a lot of whack a mole being played to get various test cases to pass.
A nested doc mentions, almost as an aside, that they also broke map syntax, so any inline maps must use
dict(...)instead of braces. Oops!Set syntax probably, too
I actually use job control all day. I use emacs, but don’t always like using emacsclient or the gui.
Ctrl-z to leave emacs, fg to bring it back. That’s about 90% of what I do, occasionally I bg a job. Also don’t forget Ctrl-s,Ctrl-q which stop/resume. If for no other reason than to understand why sometimes my terminal accidentally freezes!
I am now running my own file system on my laptop. I’ll be fixing bugs and working on improving performance.
As the rootfs? For your homedir? Very curious.
Yes. It’s been my /tmp for the last 3 months.
Excellent technical content. I appreciated both the deep dive into current Linux as well as the historical code spelunking.
Go Dave.
Scheduling heuristics are tough. A while back when goroutines weren’t fully preemptive they made a release which said that function calls could potentially yield to the scheduler. So I no longer had to worry about a runaway for loop blocking everything else.
At the time I was doing some high performance logging so I had a for loop that called write and a simple time.After call to cancel the loop after one second.
To my surprise it didn’t seem as if the loop was yielding. The process would regularly take 5 seconds to respond.
I reported what I thought was a bug and learned a few technical and non-technical lessons. The responses from the Go team mostly followed the narcissists prayer: https://www.thelifedoctor.org/the-narcissist-s-prayer
“Well time.After is guaranteed to sleep at least n seconds.”
Technically true. The best kind of true. But while we are all aware desktop Linux is not a rtos, there’s a fairly large difference between one second and five seconds.
“Why are you calling a function in a loop? Artificial micro benchmark!”
Well. I heard functions would now yield and I assured them our real life code (which was basically some big data processing using stdio like Hadoop streaming used to work) did in fact write a lot of data in a loop.
Then I learned things like “write is on a list of syscalls that is supposed to be ‘fast’ so it might not yield”.
Finally one of the more interesting responses was “the loop is yielding to the scheduler and the scheduler heuristic is essentially saying that this goroutine clearly has a lot of work to do so it keeps passing control right back”. That one really opened my eyes to the complexity of “fair scheduling “.
While my bug was dismissed, I was fortunate enough to be on Dave Cheney’s good side at the time due to running a go arm5 builder. So a nice word from him made someone, Dmitry iirc, decide I wasn’t being unreasonable and throw in some extra fairness or randomness.
Remember kids: it’s not what you know, it’s who you know.
P.s. I’m writing from memory from events a decade ago. Apologies in advance for the likely many details I’ve misremembered.
P.p.s. after writing a bunch of async Rust code that does something that is in go’s sweet spot: make a bunch of parallel http requests that do CPU intensive stuff, I miss the go scheduler.
Looks like the ban works for its intended purpose: preventing radio interference. If someone is willing to put in this much effort to evade the ban, I’d let them get away with it. :)
I like Neighborhood Wifi as a social engineering trick. I wouldn’t given it a second thought.
Even better: “John’s iPhone 13” or something to that effect ;)
This is the strategy I went for in a similar situation!
My dorms in a Welsh university had rules banning switches/wifi routers/etc from being connected to their network … but at the same time, they had DPI which blocked connections to the Debian archives on the grounds that it was a “freeware” site, so I just set up a PC (£50 SFF Dell Optiplex … power was free) with a 4 port Intel NIC.
Installed VyOS, and configured it as a gateway which, from the uni’s perspective, was just a PC establishing a WireGuard VPN over a certain port for a popular online MMORPG that I was well informed the DPI wouldn’t mess with. (No policies against VPNs from what I saw, but it didn’t work unless I used an acceptable port…)
Bonus: I got BGP feeds from a friend who was giving me the WireGuard endpoint, so announced some of my own v6 space :-)
On the other side, this was connected to a managed switch and a cheap Unifi AP broadcasting “Joris Bohnson’s iPhone” on low TX power — this went all year without a peep from anyone.
After a month or so, I did swap the Dell for an EdgeRouter X and just spoofed its MAC address to the Dell I’d registered, for less fan noise, and so I could repurpose the Dell as a server.
Of course, I maintain that I was compliant, as the AP, etc. wasn’t connected to their network, rather through (using an encrypted tunnel is a Jisc-approved way for unis to connect guest wifi through Janet, at that) :-)
I love to hear stories of undergrads testing the limits of the rules to set up interesting network services from their student rooms :-)
I had the enormous privilege of 10 Mbit/s internet to my rooms, 1995-1997. I had a public IP address, minimally filtered - there were some basic port blocks on 25 and 53 which was mildly annoying since I wanted to use those protocols, but I could run a web server no problem. Years later when I became a member of staff I was happy that my colleagues had not followed the crowd in relaxing the rules for email because it made it much easier for me to control outgoing spam. I am still not sure if there is a lesson to be learned or if it depended too much on the weird circumstances at the time…
i guess the largest-scale unsanctioned network* was probably at moscow uni:
https://medium.com/@pv.safronov/moscow-state-university-network-built-by-students-211539855cf9 (alternate link)
* actually, two of them, in competition
My big thing at USU was realizing that our email was served by a VMS cluster called the 7 dwarves. Your student email also gave ftp access to publish docs to something like usu.edu/~username. You also had ssh access with port forwarding so you could socks proxy.
Ssh access put you on a VMS machine which was a weird environment but it had perl so you could play around a bit.
Oh that and realizing the student info kiosks were just running firefox full screen. It wasn’t that hard to figure out how to bring up a print dialog and change the print command from lpr to xterm. It was just a knoppix livecd so any changes didn’t persist.
Sounds fun! :-) I didn’t get a public IP (all NAT’ed through Sonicwall(?)), but was definitely spoiled by symmetric low-latency gigabit ethernet, coming from shite Virgin Media in Plymouth…
It’s nice your uni left most ports open – I suppose P2P file sharing probably killed a lot of that fun later on. Out of interest, why block just 53 - was the issue of open resolvers being used for an amplification attack a particularly known/prevalent risk back then?
P2P was a big issue, especially before the post-dot-com bandwidth glut kicked in. I can’t remember whether it led to port blocks, tho. There were other bigger issues:
The impression I get, which I hope is not too far off the mark, is that port 53 was blocked because there was a lingering attitude that registering hosts was a serious matter that should be done properly, with things like matching forward/reverse DNS. The computing service staff couldn’t ensure that DNS servers were operated correctly if they didn’t know about them. At that time, domain names still cost $100 per year and registration was largely manual. And open resolvers were the norm rather than the exception! My memory is vague but I don’t think that changed until after 2000.
That was me :-) (Well, my colleagues.)
Thought about that actually! But as stated, the ban seems to intended to apply to all wireless networks. Tethering a phone might technically violate it or trip detection. And it would definitely suggest it’s in your building. Neighborhood WiFi would appear to be from somewhere else
Per the article, there probably wasn’t any interference to prevent in the first place. 🙃