But if the relay phone number can be used to reach my real phone number, does it matter? Maybe gaps in my understanding of this privacy enhancing feature.
Part of the reasoning behind email masks is that when you don’t give out your real email address, then it’s impossible to cross-reference your identity across multiple online services. No-one knows who’s at the other end of bunchofrandomnumbers@firefoxrelay.com or whatever. I guess the thinking is similar here.
I’m not sure how I feel about all my calls/texts going through Twilio, though. In general I think Mozilla could improve here – they also use AWS’ Simple Email Service to forward all Relay emails. I get that it’s easier than running their own mailserver, but it really highlights that privacy has become a marketing buzzword, IMO.
(disclaimer: I work on a competing service to Relay as a side project)
If you find yourself receiving too many unwanted spam calls or texts, you can easily turn it off for all phone numbers or select the specific ones you want to block.
Sounds like you can disable/“turn off” a relay number, so I guess future calls to it would be dropped(?) without being forwarded to your real number?
Has anyone used this? I’m very curious which carrier(s) they use, and whether it works for signing up with services that are notoriously picky about which phone numbers they like, such as Discord, Instagram, Telegram, Google, and WhatsApp.
100% this. I signed up for a GOOG voice number and then also a Trello number to try and avoid giving my phone number to every damn mobile app that demands one. Somehow, about half of the time, they know it is a virtual number and lock me out. Including, ironically enough, apps powered by Trello. A month to month burner turned out to be much easier.
There’s a lot of “should” in this article and very little argumentation. This distinction also seems arbitrary:
Of course you cannot do that with everything, you cannot read all the source code for the kernel of the operating system you’re running, you cannot read all the code that makes up the compiler or interpreter you’re using, but that is not the point at all, of course some level of trust is always required. The point is that you need to do it when you’re dealing with code you are writing and importing!
I recently interviewed for a new role and I really liked how they structured their technical interview: 2 hours for a task, with a short introduction over a video call, then independent work with another check-in halfway through. Finally, half an hour’s call to talk through your solution.
It was a good balance of having space to think about the problem without someone breathing down your neck, while still having an interviewer available for questions and discussion.
Trying to tie up loose ends in the last couple of weeks at my current job. I think things are getting to a pretty good state for a smooth handover!
Otherwise, I’m working a lot on my side project, an email alias service. I’m migrating from a hosted email sender to a self-hosted MTA, which is necessary but comes with a whole lot of (ongoing) work!
Woof. Self hosting an MTA is quite a lot of work indeed.
I admire anyone who stays on top of this. I feel like GOOG, MSFT and a couple of other key players these days control the horizontal and the vertical, and don’t much care how many smaller players they tread on in the process.
Woof. Self hosting an MTA is quite a lot of work indeed.
That hasn’t actually been my experience, surprisingly. I started self-hosting my own email in… 2019 I think, and since early 2021 or so have used it for essentially everything. It took a few days to set up, using postfix + postgres for virtual accounts and dovecot for IMAP. But since then it has probably taken an average of one hour per year to actually support, maybe less. It’s not a perfect setup, but works pretty well and I have zero fears of it suddenly breaking. The only actual outage I’ve had was when I upgraded postgres and had to re-import the user database from backups.
Now that I actually say this, I’m a little surprised by it myself. But I can heartily recommend it to anyone likes being self-reliant, and is running their own VPS anyway.
Yeah I have SPF and reverse DNS, but no DKIM set up at all IIRC. And as far as I can tell I’ve never had problems sending to gmail or any other mail server. Maybe I just got lucky and my VPS’s IP address has never been owned by a spammer? ¯\_(ツ)_/¯
Edit: Just remembered that I still have a gmail account as a backup and it’s set up to forward email to my own server. Maybe that makes a difference to Google?
I’m still sad about the state of messengers. One one hand you have Telegram and Signal which are two walled gardens with a closed-source server. (Signal being slightly worse as the author does not want it to be packaged for fdroid, and released a few versions without publishing the source code prior to integrating his cryptocurrency-scam in it.)
On the other hand, the decentralised alternatives are miles behind WhatsApp/Telegram/Signal. Jami just does not work for me and my friends (messages are not going through) and is a battery killer. (Messages are not being received) Tox has one of the worst UI/UX I’ve ever experienced (let alone push notifications being broken) And Matrix’s support for E2EE is not widespread.
I appreciate the community effort, but unfortunately I still use Telegram :( .
I’m not sure it’s fair to place Telegram and Signal in the same category, at least not if E2EE is important to you.
The vast majority of Telegram’s chats are stored in plaintext on their servers. Group chats cannot be end-to-end encrypted, and you have to manually start a secret chat between two people if you want E2EE. In spite of this, Telegram continues to be seen as a secure messenger – which is why a 2017 study found that many Telegram users mistakenly think their chats are end-to-end encrypted when they’re not.
I have no expectation that Telegram is actually secure. I just find it moderately interesting it’s one of the few modern IM services with open client APIs.
I’m with you on the Telegram and Signal, though I still use both for some contacts. What has been working better for me, though, was setting up a hosted Snikket instance and then I got my family using that. It’s XMPP/Jabber (with OMEMO for the e2ee, which is based on the Signal protocol), and actually has good mobile clients. It’s one of the few ways the iOS family members get consistent notifications and experience. Might be worth a try if you’re ever up for trying another one.
People keep saying this, but I don’t know when it’ll actually be the case. Every time I try it, I get disappointed by mediocre clients, resource hogging server, and inadequate moderation tools for groups.
I tried matrix about 3 years ago I think, it worked, but it wasn’t great.
In fact I self-hosted a matrix server, and while playing it filled my server with garbage from IRC-bridges, etc…
So after one year or so, I uninstalled matrix, and just used a self-hosted IRC server, and for my group of geeky friends it was ok.
Then, after a while, IRC show some limits, so we’ve tried to use matrix again, about 1 year ago, and there were a lot of progress. The client is a lot better, the server too, I just forgot about it, and it “just works”.
The e2ee user interface is a lot better than a few years ago.
So if you haven’t tried matrix in a while you should really try it again. Really, now I think matrix is ready.
conduit.rs has been running flawlessly for me, for a couple months now - way less resource-hungry than the Synapse stack (albeit, with some missing features).
I have and it’s still not great. The interface is a bit confusing (i.e. the multiple levels of hierarchy for chat, the weird spaces thing that’s trying to paper over federation, etc.) and lacks this je ne sais quoi other clients have that makes it feel wrong for lack of a better term.
just wanted to say: matrix/element has come a long way. Due to it being used across orgs without federation (work vs family) I’d say the biggest downside is that there is no recommended way for multiple accounts on one device (especially mobile). Otherwise it works great.
It REALLY has. But sadly I think the fact that @acatton did a whole ‘state of messengers’ and didn’t even mention Matrix is a great example of how it’s gaining mindshare, but VERY slowly and not with non technical users.
The clients are IMO finally ready for that to happen, now we just need more people to actually try the updated versions and figure out how good it can be.e
Meanwhile sadly 99% of the communities I actually want to chat on are Discord or Slack :(
I use prosody xmpp server with Conversations from f-droid on mobile. It works great for me and my friends and family. It is also very fast and easy to setup. Server to server works flawlessly so I can connect with others easily and without creating an account.
Can you provide more context or links around Signal’s cryptocurrency-scam that you mention? I could only locate a Verge article, but nothing more technical. Thanks.
I know it wasn’t your intention, but just to clarify, this makes it seem as if these three are of similar quality. However, Telegram is miles ahead of WhatsApp.
In late January 2022, Okta detected an attempt to compromise the account of a third party customer support engineer working for one of our subprocessors. The matter was investigated and contained by the subprocessor. We believe the screenshots shared online are connected to this January event. Based on our investigation to date, there is no evidence of ongoing malicious activity beyond the activity detected in January.
I actually just moved away from them because of pricing – didn’t like that you had to pay a monthly/yearly fee in addition to the cost of the domains themselves. Otherwise they were solid though.
I do think that “clean code” can be a useful shorthand in casual conversation, but the author is right – we can do better.
I think a similar thing could be said for the maxim “use the best tool for the job” – I keep seeing people say this, but it’s too general to really mean anything at all!
This kind of thing comes up in discussions of implicit bias. The problem isn’t describing something as ‘clean’, it’s using that to shut down discussion, which can happen accidentally. No one wants to criticise the clean design (who wants to advocate for dirty code?) and by being imprecise it’s hard for someone who isn’t part of the in group to know whether their objections make sense in the context of the evaluation criteria that everyone else is using.
If you’re praising someone’s work, telling them that it’s clean, beautiful, or other non-specific adjectives is completely fine. The purpose in the communication is to convey approval and agreement that they’ve made good decisions, not to critique the design. If you’re evaluating approaches, using this kind of terminology is not so good.
I’m attempting to fix a mysterious character encoding bug in one of my side projects. It’s one of those debugging journeys with lots of twists and turns, but I’m trying to enjoy the challenge of it!
I’m afraid it’s perfectly possible to ship one version of your code to GitHub and a different version to npm.
Is anyone aware of any cool cryptography that might help assert code on Github matches the code on NPM? Or asserting code on Github matches code running on a server?
If the builds are reproducible, you can, by simply comparing your own build results (assuming you trust your own compiler chain, and with webpack having 780 dependencies, I’m not so sure about that) and the ones you get from NPM. I’m not sure how viable setting up reproducible builds is in JS ecosystem though.
This is what Code Transparency is intended to do. You run your builds in a known environment in a confidential computing environment that gives you an attestation over the initial state of the environment (VM image contents plus git hash of the repo). The VM is configured so that the only thing it will do is your build (no accepting incoming connections, for example) and produce the output. The output is signed with a key derived from the attestation. You then append this hash into a public ledger. Anyone else can then audit your build system and VM image and can verify that running the same build gives the same output.
Whether anyone will actually do that auditing is an open question. If your VM image has a custom container with a load of stuff that you’ve built then that’s all in the TCB and who knows what’s going on there.
This is all predicated on reproduceable builds, of course. If running the same build twice gives different output then there’s no automated way of verifying the process.
Cryptography can make it easier to simply trust an authority’s signature on a compiled blob, but besides that, I don’t think it will help much. You can certainly download all the source code for each module and run the build for each module yourself. To be honest I don’t know if npm or yarn support that type of operation out of the box. Something tells me even if they did, it would probably fail on many packages for unforeseen reasons; you would have to fix a bunch of “broken builds”.
I do feel like this is kind of silly though. This isn’t a problem unique to JavaScript and npm. Will say regarding javascript and analytics however; IMO malicious or “criminal” intent is not even required for these kinds of “bad things” to happen. It can also happen straight through the front door without having to hide anything, without doing all this silly malware style stuff like changing the behavior based on the time of day.
Plugging my own project, but: the fact that we don’t have a good answer to this is why I made xray.computer which lets you view the code that’s actually published to npm. It’s only a plaster on a major wound, though – npm packages often include pre-minified/compiled code, which is what your project will actually use, and it’s almost impossible to review since it’s essentially obfuscated.
Reproducible builds and/or a cryptographically signed web of trust are the best proposals I know of, though it’s worth noting that npm does appear to do some relatively advanced work on detecting malicious packages behind the scenes.
The code for [nose] in its source repository was fine, only PyPI was bad.
[…]
[The maintainers] disclaimed any knowledge of what went on.
So, an unknown entity was able to insert a certain code into a package at PyPI, and pip(1) was downloading it for years. This only came to light because the inserted code failed on my Fedora test box.
The author provides no details of what difference existed between what they saw in the repo and what they saw in the package, what effect it actually had, or anything else that would allow readers to verify or investigate for themselves, and simply declares the only possible conclusion to be that PyPI itself has been silently and completely compromised. Not that a maintainer’s account for an unmaintained (coming up on 7 years since last release) package could have been. Not that it’s possible the last release legitimately wasn’t generated from what the author saw in the GitHub repo. Not any of the other many possible explanations. And I haven’t seen this allegedly huge “PyPI compromise” mentioned anywhere else despite that post being over a week old.
Smells an awful lot like FUD, and until the author chooses to provide some more information that will be my stance.
The fact that the NFT spec only stores an image URL on the blockchain, and no hash to verify the image, is just absolutely astounding. Moxie is more generous than I am; IMO it highlights the grift that NFTs are – like whoever threw it together gave no thought to security or longevity.
Yeah to me this is the “tell” that the main thing driving it is other people’s FOMO money.
Basically software devs in this ecosystem realized they didn’t have to do their jobs to make money. The whole system keeps working even if you don’t do your job!!!
You just have to repeat the memes long enough, and it doesn’t matter if the tech actually does what it says it does, because nobody’s checking! Nobody really wants to stand up their own copies of these things and check if it works.
There’s little benefit in that. The benefit is talking about it and repeating it to your friends.
I was interested in IPFS before it grew the blockchain component. I went back the original paper, and it is genuinely a good idea, and something we still need: a cross between git and BitTorrent (it mentions this in the abstract). I have long wanted Debian repos, PyPI, CPAN, etc. to be stored in such a system, i.e. consistently versioned with metadata.
But it’s 6 years later, and when I go to look at it, apparently it just doesn’t work very well. And it probably won’t because it grew so many components. (Gall’s law: to get a big working system, you have to start from a small working system.)
So what should we expect of IPFS? At five years old, is this a project that’s usable ‘here and now’, as the homepage promised in 2017? Are all the parts in place, just waiting for web and application developers to see the light? Have the stumbling blocks I noticed in 2017 been smoothed over?
So I just did it via CoinBase and I suppose it worked. So I would say there’s non-zero number of real use cases for cryptocurrency and blockchain. I think IPFS started out as genuine too, but it basically got ruined as working tech by an influx of money and employees.
The situation with IPFS also affected gittorrent, another combination of git and bittorrent. Blockchains are like fig trees; as they grow, they choke whatever software projects originally gave them structure and purpose.
Hm what happened to it? It doesn’t look like very much code.
Thinking more about the original comment … To “steel man” the blockchain, I do think there is a place for regular old web apps to serve data from a blockchain. I don’t think that is inherently a bad architecture. You just have to be honest about it!
I think there will always be a place for money in the blockchain, however niche, as the Scihub example shows.
It’s more logical for an end user like me to use Coinbase, which is centralized, but that’s OK. (Also big irony: it relies on sending your driver’s license in for verification, so it’s building on top of US state regulations which they want to be free of.)
One viewpoint I heard is “Bitcoin is a settlement network, not a payment network”. That is, the logical end user is banks and companies like Coinbase, not consumers. It could make sense to have a two-tiered system.
(Although then you have the same problem that those banks are subject to regulation of the countries they operate in, so you’ve lost a lot of the purported benefit of blockchain. I think people will gradually come to understand this and a much smaller set of use cases will be ironed out.)
But I do think the logical evolution of blockchain is to evolve to be less “consumer” and more “enterprise”.
I think the problems with the centralization of the cloud are real, and while web3 is mostly a BS term, and it’s not really a solution as is, I can see this type of distributed consensus as a useful primitive for small but important parts of real systems.
I believe that the Dat protocol and its successor Hypercore are both basically Bittorrent plus version control, but with no connection to blockchains or cryptocurrency that I know of. Please correct me if I’m incorrect :)
I think Beaker Browser is a really cool project that suggests what could be done with Hypercore, but unfortunately a lot of websites and apps that were designed for its first iteration using Dat didn’t succeed in making the transition to the Hypercore version. I think the tech change cost it some momentum / users. Hopefully it will build up steam again, but I’m afraid IPFS has stolen its thunder by providing similar tech plus a chance to get rich quick :(
Dat is so good aside from one critical flaw (IMHO, anyway): the “spec” is more or less “whatever the version you pull from NPM right now does.” Yes, there is some additional documentation, and a running reference implementation is great, but going a decade+ without a compatible version in some environment other than Node is a pretty big handicap and IMHO a major oversight by the project owners.
Secure Scuttlebutt suffers from the same problem, and the “rewrite it in Rust” efforts for both are at best implicitly— if not explicitly, as in the most recent SSB -> Rust work — aimed at performance, not broad interop or adoption.
So neither one can effectively run without hundreds of MBs of JS packages, there’s no support for languages that offer better native platform integration or type safety…heck, they don’t even ship TS definitions, and the internal APIs are so idiosyncratic it’s extremely difficult to build new apps on the platform.
In a world where I had infinite time and attention to spend on an open software + network stack I would love to built a libdat or libssb that was straightforward to link in a Python project, or an iOS app, or really anything that can do normal C FFI. Alas, I don’t, so I haven’t. Maybe someday.
Hm I heard of Dat a few years ago, but I didn’t know about Hypercore. Thanks for the pointer!
I think eventually we will get something like this … just because there is a need. In the meantime I might write my own with shell scripts invoking git, and then curl or BitTorrent :)
I forget which post I read this in, but there are really 2 kinds of “decentralized systems”, i.e. ones that don’t rely on “the cloud”:
Something like git, where it’s trivial to stand up your own. BitTorrent is also in this category. As well as a web server with Apache or Nginx.
Something like BitCoin where there’s no central node.
Notably, it’s not only hard to run your own Ethereum node, but you also don’t want to run your own Ethereum network!
So the word is overloaded, and they have very different architectures. I care more about the first kind. So really I don’t want a global file system like IPFS – I’d rather have distributed storage that runs on a few nodes, is easy to administer, etc.
I’m not surprised but I am really sad about this. I thought it had a lot of potential for building a resilient web that could e.g. handle network outages by making it trivial to continue sharing websites over a LAN. It seemed to be great at sharing static websites which are where I’m putting my development efforts these days, but it appears the developer was frustrated by this and wanted people to share web apps instead.
Agregore looks like it may be continuing on with support for Hypercore, but it also supports IPFS and I saw not interacting with Protocol Labs and their reliance on / promotion of cryptocurrency as a significant feature.
IPFS is a very pure “better BitTorrent” and the fact that some web3 stuff hosts content on IPFS should not taint the protocol, even if you happen to hate web3. Lots of web1 content on IPFS too, the protocol predates web3 by a lot, etc
Part of the problem here is that “hosts content on IPFS” is a misnomer, for the same reason that things aren’t “hosted on BitTorrent”. IPFS is a distribution mechanism, not a storage mechanism (despite what their marketing implies), and so something can only be distributed through IPFS - it needs to be hosted somewhere else, and that “somewhere else” is left undefined by IPFS.
That might sound like pedantry, but it has some crucial implications: it means that by default, the network cannot provide meaningfully better availability than BitTorrent can, which is to say the availability is really bad and unsuitable for hosting websites on. You could address this by seeding your content from a server, but then what have you really accomplished, other than a potential ‘download accelerator’ (which is again distribution, not storage)?
which is to say the availability is really bad and unsuitable for hosting websites on. You could address this by seeding your content from a server
This seems like a contradiction. Of course you cannot host an IPFS powered website without seeding it from somewhere! That doesn’t make availability bad or unsuitable any more than HTTP is bad or unsuitable. What I love about IPFS is that I can pin my websites on any computer connected to the internet, and usually much more than one! No special setup is needed, and if I want to change what computer pins the content I can easily do so at any time without changing any settings anywhere else. It just seamlessly keeps working.
Websites do not have user-mutable state. I guess if you want your blog to have a user theme toggle or some other appy feature a website might want you can use a cookie or localStorage just as you would on the regular web.
I don’t think bad faith is necessary to provide valid criticism. The fact that NFTs are sold to the public as ‘stored forever in the blockchain’ and the reality is that there’s no actual mechanism storing anything other than a URL pointing to the content is almost by definition a scam. If I was going around selling ownership contracts to a house I don’t own, I’d be charged with fraud.
I don’t think bad faith is necessary to provide valid criticism
That’s not what I meant. When @tao_oat said “Moxie is more generous than I am”, the difference between a “generous” take and a “not generous” take can often be the assumption of good faith. Are you trying to take a keep a neutral eye while evaluating this argument, or are you just looking for more evidence confirm your existing bias, that’s the difference between having good faith or approaching a topic cynically.
I think you’re prejudiced against NFTs and are just looking to criticize them. That’s fine, I think NFTs being used to attest art and their associated speculation is pretty stupid, so you aren’t gonna find me defending any of this. I also agree with each of Moxie’s criticisms. That said, I just don’t think the sort of cynical, charged rhetoric in this thread is indicative of good faith. Your comment even assumes a position from me that I don’t have. Good faith keeps discussions intellectually interesting IMO. I don’t have that much more to say here, I’ll let everyone else continue ranting in anger.
FWIW I agree with you – on doing more research, a good-faith take would be that whoever wrote the ERC721 spec might have seen the promise of an image that changes over time. They might not have foreseen that art collectibles would be the primary driver behind NFTs. In the spec they write:
A mechanism is provided to associate NFTs with URIs. We expect that many implementations will take advantage of this to provide metadata for each NFT. The image size recommendation is taken from Instagram, they probably know much about image usability. The URI MAY be mutable (i.e. it changes from time to time). We considered an NFT representing ownership of a house, in this case metadata about the house (image, occupants, etc.) can naturally change.
(Though having a sentence like “The image size recommendation is taken from Instagram, they probably know much about image usability” in an official spec doesn’t exactly scream professionalism or care to me).
I should direct my critique against those who push art NFTs without mentioning the serious technical issues, and not the spec authors themselves.
I think the broader question is: when you keep seeing red flags in an ecosystem/community how long can or should you make an effort to retain good faith?
I think the broader question is: when you keep seeing red flags in an ecosystem/community how long can or should you make an effort to retain good faith?
I think that’s the wrong question to ask. Technology usually has two components. One is the design and engineering that goes into it. Think with XMPP the protocol and standards used to send/receive messages. The other is adoption: this can be success in number of users, its usage, or its revenue. There’s plenty of technologies, ones much less complicated and more “clearly” incremental than a blockchain, that have failed purely because their adoption was lacking, despite years of attempts to make themselves relevant. Examples here are Laserdisc, Betamax, HD-DVD, and more. Adoption of a technology has much more to do with culture, user-experience, or business considerations than the engineering behind the technology. These questions you’re asking, about whether the space is filled with hucksters and such, are questions affecting the adoption of the technology. Discussions concerning the adoption of a technology are much more complicated (well IMO at least, probably because I’m an engineer first and foremost!) than discussions dealing with the design and engineering behind a technology, and also off-topic for Lobsters (probably due to the challenge of dealing with the nature of those discussions properly.) I will say though, a space filled with fraud doesn’t exactly instill confidence, especially when it’s a person’s money or other resources on the line…
But I also don’t think it’s necessary to get inordinately angry at the blockchain space. Businesses are made and die every day, some full of hucksters, others just blatant copies of existing businesses. Community projects are made and die every day. It’s the churn of human creativity. If humans knew exactly which projects would succeed and which would fail, then we’d already have solved our problems, wouldn’t we?
I don’t really care for collectibles, real or virtual, but to be fair the NFT is stored in the blockchain. The media is not the NFT, just an associated decoration.
This sounds like a solution looking for a problem, and then someone inventing a problem for the solution to fix. One thing I’ll give ‘crypto bros’ - specially those pumping the NFT racket - is that they’ve managed to befuddle the world into thinking they created something actually revolutionary. Every single problem the article describes is stuff anyone who understands this technology works could’ve guessed from day 1. Ultimately ‘web3’ is just a nebulous term that means nothing and everything, depending on who you ask and what time you ask them.
But nobody says that an NFT is a URL, i.e. a string starting with “https://…” Because nobody would find such a string, in itself, interesting enough to pay money for.
The scam is that people describe and sell NFTs as being a file, or even a unique fingerprint of such a file, when they’re no such thing.
It’s like selling you a painting, only it turns out you only bought the frame, and the gallery reserves the right to swap out the canvas for another one, or just take it away, at their pleasure.
Yeah but the funny thing is: who cares if it was actually pinned to the right file?
What’s to stop me from minting another NFT for the same artwork and selling it? I just have to convince enough people it’s valuable.
As far as I can tell, the thing that makes NFTs “work” in any sense is specific pockets of social media, e.g. Twitter. Reality is socially constructed.
Like the artist Beeple has to go on Twitter and say he’s selling some number that represents art that he created.
And other people witness that and they believe it is him, i.e. the person who keeps creating all the enjoyable pictures. Twitter and other centralized social networks provide some continuity of identity. lobste.rs does this too.
If somebody else tries to sell an NFT of his artwork, maybe he can use Twitter to shame them or whatever. That’s about it.
As far as I can see, social media is really the thing that matters, and not anything in the blockchain. It seems clear that no end users every really look at the blockchain and verify things. (Hell I didn’t when I sent the donation to the Scihub founder. Did it really get sent? I just trusted Coinbase.)
So I think there’s no notion of identity, exclusivity, or authenticity on the blockchain. You always have to make some jump between the number/hash and the “thing”, and other people can make that jump too.
I’d be interested in any arguments otherwise … I have never actually used NFTs or Ethereum, but it seems like there is an obvious hole either way.
i.e. the blockchain is really built on trust built by social media; it can’t stand alone. Social media is very powerful – the former US president got elected in a large part because of it, and then he got blocked from it with huge consequences, and now he wishes he had his own Twitter and there are multiple efforts in that direction, etc. It has a lot of real consequences in the world, and is intimately connected to it. Blockchain doesn’t have that property at all!
Right — I can just touch one pixel in the image, or add a no-op EXIF tag, and suddenly it’s a different file with a different digest that I can sell as a different NFT.
That was my day-one objection to NFTs. In terms of my analogy, it’s like buying a limited edition print where I only have the artist’s promise she won’t issue more. But the realization that it’s just a URL makes them orders of magnitude sillier and more scam-like.
Yeah this has been beat to death, but I was reading conversations earlier this year, and one analogy is the company that used to sell naming rights to the stars:
Like they would sell you a certificate that a star was named after you.
Never mind that anybody can rename the same star for a different person, and sell that. Nobody ever used those names – not even the one scientist who might have a picture of that star or care about it. (Again, reality is socially constructed.)
Another problem is that you’re literally just getting the NFT number itself. You’re not getting the copyright to that work !!!
Like you could buy the NFT, and then the artist can sell the actual artwork to somebody else, which IS
enforceable by copyright law !!! But your NFT isn’t.
Also with music, there are separate rights to perform songs in public places, to play recordings at bars, and to publish the sheet music. You do NOT get that if you buy an NFT of a song.
In fact I remember seeing a VC blog or podcast where this was brought up ….
And so that basically proves that it doesn’t matter if the NFT has an immutable hash or not. You’re buying a useless pointer to a thing, not anything related to the thing itself … so it doesn’t matter what’s in it!
I don’t understand this sticking point. An NFT is just a provably-unique (i.e. non-fungible) thing maintained on a chain. What it maps to off-chain, or how it does so, is basically incidental. It’s a contract, meaningful only in a specific domain: for legal contracts, that domain is typically a government jurisdiction; for NFTs, it’s the chain on which they exist.
I think it’s because the loudest use is for art collectibles so people get hung up on that. For me the best analogy is baseball cards. No one would say “but the card doesn’t contain an actual baseball player! You don’t get the right to boss the real human around!” But somehow NFTs the detractors think should “be” the art or “be” the copyright in a way that they were never intended to be.
The problem with that analogy is that the issuer of the baseball card could, if it were an NFT, blank the contents of the card or change it to something else at any point. If baseball cards had that property, would people trade them? If a baseball card just had a QR code on it that let you go to a web page of stats about the player, would they be as valuable? Would they keep being valuable once some of the servers hosting the stats went offline (or would ones pointing to dead URLs become more valuable?)? What about when someone buys the domain for a popular card and points it at a porn site?
The problem with that analogy is that the issuer of the baseball card could, if it were an NFT, blank the contents of the card or change it to something else at any point.
In this analogy the NFT, the contract which exists on-chain, is itself the baseball card. The fact that one of the metadata fields of the NFT is a URL that may or may not resolve is more or less incidental.
Would they keep being valuable once some of the servers hosting the stats went offline (or would ones pointing to dead URLs become more valuable?)?
The important properties of NFTs are that, in the context of the chain on which they exist, they’re provably unique, non-fungible, and owned. A broken URL in the metadata doesn’t impact those properties. Of course, value is determined by the market, and the crypto markets are wildly irrational, so you may have a point.
In this analogy the NFT, the contract which exists on-chain, is itself the baseball card. The fact that one of the metadata fields of the NFT is a URL that may or may not resolve is more or less incidental.
I’ll buy this, but practically speaking, what then is the use of the NFT? If you’re saying the URL isn’t important, what exactly are you buying?
You’re buying something which is guaranteed to be unique and non-fungible, in the context of a specific chain. There is no intrinsic value, any more than a specific painting or whatever is valuable. The value relies on the market belief that the NFT’s chain-specific scarcity is valuable.
It’s kind of like a deed. The property isn’t legally yours until the relevant legal regime accepts that you own the deed. The deed isn’t the property but it uniquely represents the property in a specific domain.
But like value is not the only interesting thing about NFTs. The non-fungibility itself is novel and opens the door to lots of interesting things.
A lot of NFT projects use IPFS, Filecoin or similar decentralized storage.
These solutions have a similar problem as with Bittorrent. The stuff that’s not popular won’t get seeded/hosted. It just adds one layer of indirection to the storage issue, but once the business that’s pushing NFTs goes out of business these files are probably not going to get shared.
My understanding of how this works is: the NFT is an object on the blockchain, whose rules enforce its uniqueness. We’re gonna assume the chain is going to be continued to be mined and therefore “exist” indefinitely.
The NFT contains a hash denoting a location on IPFS. Accessing it using an IPFS gateway will show the user the JPG portraying whatever they paid $2.4M in funny money for. But that JPG has to reside on disk somewhere. And when the company or user goes out of business or the VPS is decommissioned or they get kicked out of AWS for scamming, where is the JPG?
Of course, concerned parties can… right click on the image, save it to disk, and then use that as a source of data for the IPFS hash, but that does kind of put a lie to the popular imagination on how all this nonsense works.
Just like the baseball player the “hash” (picture) on a baseball card may die, so may the hosting for an NFTs associated JPG go away. Of course if you want to preserve it you can simply pin it yourself and thus your own computer becomes a host for it. So it’s actually more resiliant than the baseball player :). The NFT is not the image, the image is an associated decoration
Location: Remote, UK timezone +- 2 hours. We have an office in London if you’re based there, but our engineering team is remote-first.
Description: We’re making cybersecurity easy. Intruder is a vulnerability scanner that continually scans your attack surface for known CVEs, configuration weaknesses, and common vulnerabilities. We focus on making security accessible to all through an easy-to-use platform and simple, understandable descriptions.
Tech stack: Ruby/Rails and Python/Django. Our frontend is written in Vue.js with Tailwind, and we’re starting to introduce TypeScript. Mostly running on Kubernetes.
I’ve been working with a similar paradigm through Elixir + Phoenix + LiveView, and it works extremely well for my one-person side projects. I’m excited to see Hotwire become the default in Rails.
I wonder, though, does anyone have experience working with technologies like these on very large apps? I can imagine it’s challenging to scale it to larger teams – even for my small projects, I’ve found myself missing the concept of full-featured UI components.
I don’t really agree with a lot of the claims in the article (and I say this as someone who was very actively involved with XMPP when it was going through the IETF process and who wrote two clients and continued to use it actively until 2014 or so):
Truly Decentralized and Federated (meaning people from different servers can talk to each other while no central authority can have influence on another server unlike Matrix)
This is true. It also means that you need to do server reputation things if your server is public and you don’t want spam (well, it did for a while - now no one uses XMPP so no one bothers spamming the network). XMPP, unlike email, validates that a message really comes from the originating domain, but that doesn’t stop spammers from registering millions of domains and sending spam from any of them. Google turned off federation because of spam and the core problems remain unsolved.
End-To-End Encryption (unlike Telegram, unless you’re using secret chats)
This is completely untrue for the core protocol. End-to-end encryption is (as is typical in the XMPP world) multiple, incompatible, extensions to the core protocol and most clients don’t support any of them. Looking at the list of clients almost none of them support the end-to-end encryption XEP that the article recommends. I’d not looked at XEP-0384 before, but a few things spring to mind:
It’s not encrypting any metadata (i.e. the stuff that the NSA thinks is the most valuable bit to intercept), this is visible to the operators of both party’s servers.
You can’t encrypt presence stanzas (so anything in your status message is plaintext) without breaking the core protocol.
Most info-query stanzas will need to be plain-text as well, so this only affects direct messages, but some client-to-client communication is via pub-sub. This is not necessarily encrypted and clients may or may not expose which things are and aren’t encrypted to the user.
The bootstrapping thing involves asking people to trust new fingerprints that exist. This is a security-usability disaster: users will click ‘yes’. Signal does a good job of ensuring that fingerprints don’t change across devices and manages key exchange between clients so that all clients can decrypt a message encrypted with a key assigned to a stable identity. OMEMO requires a wrapped key for every client.
The only protection against MITM attacks is the user noticing that a fingerprint has changed. If you don’t validate fingerprints out-of-band (again, Signal gives you a nice mechanism for doing this with a QR code that you can scan on the other person’s phone if you see them in person) then a malicious server can just advertise a new fingerprint once and now you will encrypt all messages with a key that it can decrypt.
There’s no revocation story in the case of the above. If a malicious fingerprint is added, you can remove it from the advertised set, but there’s no guarantee that clients will stop sending things encrypted with it.
The XEP says that forward secrecy is a requirement and then doesn’t mention it again at all.
There’s no sequence counter or equivalent so a server can drop messages without your being aware (or can reorder them, or can send the same message twice - no protection against replay attacks, so if you can make someone send a ‘yes it’s fine’ message once then you can send it in response to a request to a different question).
There’s no padding, so message length (which provides a lot of information) is available.
This is without digging into the protocol. I’d love to read @soatok’s take on it. From a quick skim, my view is that it’s probably fine if your threat model is bored teenagers.
They recommend looking for servers that support HTTP upload, but this means any file you transfer is stored in plain text on the server.
Cross-Platform Applications (Desktop, Web, and Mobile)
True, with the caveat that they have different feature sets. For example, I tried using XMPP again a couple of years ago and needed to have two clients installed on Android because one could send images to someone using a particular iOS client and the other supported persistent messaging. This may be better now.
Multi-Device Synchronization (available on some servers)
This, at least, is fairly mature. There are some interesting interactions between it and the security gurantees claimed by OMEMO.
Voice and Video Calling (available on most servers)
Servers are the easy part (mostly they do STUN or fall back to relaying if they need to). There are multiple incompatible standards for voice and video calling on top of XMPP. The most widely supported is Jingle which is, in truly fractal fashion, a family of incompatible standards for establishing streams between clients and negotiating a CODEC that both support. It sounds as if clients can now do encrypted Jingle sessions from their article. This didn’t work at all last time I tried, but maybe clients have improved since then.
Strongly agree – claiming that XMPP is secure and/or private without mentioning all the caveats is surprising! There’s also this article from infosec-handbook.eu outlining some of the downsides: XMPP: Admin-in-the-middle
The state of XMPP security is a strong argument against decentralization in messengers, in my opinion.
Spam in XMPP is largely a solved problem today. Operators of open relays, servers where anyone can create an account, police themselves and each other. Anyone running a server that originates spam without dealing with it gets booted off the open federation eventually.
Another part of the solution is ensuring smaller server operators don’t act as open relays, but instead use invites (like Lobste.rs itself). Snikket is a great example of that.
but that doesn’t stop spammers from registering millions of domains and sending spam from any of them.
Bold claim. Citation needed. Where do you register millions of domains cheaply enough for the economics of spam to work out?
Domains tend to be relatively expensive and are easy to block, just like the IP addresses running any such servers. All I hear from server operators is that spammers slowly register lots of normal accounts on public servers with open registration, which are then used once for spam campaigns. They tend to be deleted by proactive operators, if not before, at least after they are used for spam.
Google turned off federation because of spam and the core problems remain unsolved.
That’s what they claim. Does it really seem plausible that Google could not manage spam? It’s not like they have any experience from another federated communications network… Easier for me to believe that there wasn’t much in the way of promotion to be gained from doing anything more with GTalk, so they shut it down and blamed whatever they couldn’t be bothered dealing with at the time.
Your reasonning about most clients not supporting OMEMO is invalid because noone cares about most clients: it’s all about the marketshare. Most XMPP clients probably don’t support images but that doesn’t matter.
For replays, this may be dealt with the double ratchet algorithm since the keys change fairly often. Your unknown replay would also have to make sense in an unknown conversation.
Forward secrecy could be done with the double ratchet algorithm too.
Overall OMEMO should be very similar to Signal’s protocol, which means that it’s quite likely the features and flaws of one are in the other.
Conversations on Android also offers showing and scanning QR codes for validation.
As for HTTP upload, that’s maybe another XEP but there’s encrypted upload with an AES key and a link using the aesgcm:// scheme (as you can guess: where to retrieve the file plus the key).
I concur that bootstrapping is often painful. I’m not sure it’s possible to do much better without a centralized system however.
Finally, self-hosting leads to leaking quite a lot of metadata because your network activity is not hidden in large amounts of network activity coming from others. I’m not sure that there’s really much more that is available by reading the XMPP metadata. Battery saving on mobile means the device needs to tell the server that it doesn’t care about status messages and presence from others but who cares if it’s unencrypted to the server (on the wire, there’s TLS) since a) it’s meant for the server, b) even if for clients instead, you could easily spot the change in network traffic frequency.
I mean, I’m not sure there’s a lot more that is accessible that way (not even mentionning that if you’re privacy-minded, you avoid stuff like typing notifications and if you don’t, traffic patterns probably leak that anyway). And I’m fairly sure that’s the same with Signal for many of these.
now no one uses XMPP so no one bothers spamming the network
I guess you’ve been away for awhile :) there is definitely spam, and we have several community groups working hard to combat it (and trying to avoid the mistakes of email, not doing server/ip rep and blocking and all that)
Cross-Platform Applications (Desktop, Web, and Mobile)
True, with the caveat that they have different feature sets. For example, I tried using XMPP again a couple of years ago and needed to have two clients installed on Android because one could send images to someone using a particular iOS client and the other supported persistent messaging. This may be better now.
Or they’ve also calcified (see: Pidgin). Last time I tried XMPP a few years ago, Conversations on Android was the only tolerable one, and Gajim was janky as hell normally, let alone on Windows.
True, with the caveat that they have different feature sets. For example, I tried using XMPP again a couple of years ago and needed to have two clients installed on Android because one could send images to someone using a particular iOS client and the other supported persistent messaging. This may be better now.
This was the reason I couldn’t get on with XMPP. When I tried it a few years ago, you really needed quite a lot of extensions to make a good replacement for something like WhatsApp, but all of the different servers and clients supported different subsets of the features.
I don’t know enough about all the details of XMPP to pass technical judgement, but the main problems never were the technical decisions like XML or not.
XMPP had a chance, 10-15 years ago, but either because of poor messaging (pun not intended) or not enough guided activism the XEP thing completely backfired and no two parties really had a proper interaction with all parts working. XMPP wanted to do too much and be too flexible. Even people who wanted it to succeed and run their own server and championed for use in the companies they worked for… it was simply a big mess. And then the mobile disaster with undelivered messages to several clients (originally a feature) and apps using up to much battery, etc.pp.
Jitsi also came a few years too late, sadly, and wasn’t exactly user friendly either at the start. (Good people though, they really tried).
I don’t know enough about all the details of XMPP to pass technical judgement, but the main problems never were the technical decisions like XML or not.
XML was a problem early on because it made the protocol very verbose. Back when I started working on XMPP, I had a £10/month plan for my phone that came with 40 MB of data per month. A few extra bytes per message added up a lot. A plain text ‘hi’ in XMPP was well over a hundred bytes, with proprietary messengers it was closer to 10-20 bytes. That much protocol overhead is completely irrelevant now that phone plans measure their data allowances in GB and that folks send images in messages (though the requirement to base64-encode images if you’re using in-band bytestreams and not Jingle still matters) but back then it was incredibly important.
XMPP was also difficult to integrate with push notifications. It was built on the assumption that you’d keep the connection open, whereas modern push notifications expect a single entity in the phone to poll a global notification source periodically and then prod other apps to make shorter-lived connections. XMPP requires a full roster sync on each connection, so will send a couple of megs of data if you’ve got a moderately large contact list (first download and sync the roster, then get a presence stanza back from everyone once you’re connected). The vcard-based avatar mechanism meant that every presence stanza contained the base64-encoded hash of the current avatar, even if the client didn’t care, which made this worse.
A lot of these problems could have been solved by moving to a PubSub-based mechanism, but PubSub and Personal Eventing over PubSub (PEP) weren’t standardised for years and were incredibly complex (much more complex than the core spec) and so took even longer to get consistent implementations.
The main lessons I learned from XMPP were:
Federation is not a goal. Avoiding having an untrusted admin being able to intercept / modify my messages is a goal, federation is potentially a technique to limit that.
The client and server must have a single reference implementation that supports anything that is even close to standards track, ideally two. If you want to propose a new extension then you must implement it at least once.
Most users don’t know the difference between a client, a protocol, and a service. They will conflate them, they don’t care about XMPP, they care about Psi or Pidgin - if the experience isn’t good with whatever client you recommend that’s the end.
XMPP requires a full roster sync on each connection, so will send a couple of megs of data if you’ve got a moderately large contact list (first download and sync the roster, then get a presence stanza back from everyone once you’re connected).
This is not accurate. Roster versioning, which means that only roster deltas, which are seldom, are transferred, is used widely and also specified in RFC 6121 (even though, not mandatory to implement, but given that it’s easy to implement, I am not aware of any mobile client that doesn’t use it)
I want to add that, it may be a good idea to avoid using XMPP jargon to make the test more accessible to a wider audience. Here ‘smacks’ stands for XEP-198: Stream Management.
XMPP had a chance, 10-15 years ago, but either because of poor messaging (pun not intended) or not enough guided activism the XEP thing completely backfired and no two parties really had a proper interaction with all parts working. XMPP wanted to do too much and be too flexible.
I’d argue there is at least one other reason. XMPP on smartohones was really bad for a very long time, also due to limitations on those platforms. This only got better later. For this reason having proper messaging used to require spending money.
Nowadays so you “only” need is too pay a fee to put stuff into the app store and in case of iOS development buy an overpriced piece of hardware to develop on. Oh and of course deal with a horrible experience there and be at the risk of your app being banned from the store, when they feel like. But I’m drifting off. In short: Doing what the Conversation does used to be harder/impossible on both Android and iOS until certain APIs were added.
I think that gave it a pretty big downturn when it started to do okay on the desktop.
I saw a lot of those same issues in the article. Most people don’t realize (myself included until a few weeks ago) that when you stand up Matrix, it still uses matrix.org’s keyserver. I know a few admins who are considering standing up their own keyservers and what that would entail.
And the encryption thing too. I remember OTR back in the day (which was terrible) and now we have OMEMO (which is ….. still terrible).
This is a great reply. You really detailed a lot of problems with the article and also provided a lot of information about XMPP. Thanks for this.
It’s not encrypting any metadata (i.e. the stuff that the NSA thinks is the most valuable bit to intercept), this is visible to the operators of both party’s servers.
You can’t encrypt presence stanzas (so anything in your status message is plaintext) without breaking the core protocol.
Do you know if this situation is any better on Matrix? Completely honest question (I use both and run servers for both). Naively it seems to me that at least some important metadata needs to be unencrypted in order to route messages, but maybe they’re doing something clever?
I haven’t looked at Matrix but it’s typically a problem with any Federated system: you need at least an envelope that tells you the server that a message needs to be routed to to be public. Signal avoids this by not having federation and by using their sealed-sender mechanism to avoid the single centralised component from knowing who the sender of a message is.
I am sure you know that some of your points, like Metadata encryption, are a deliberate design tradeoff. Systems that provide full metadata encryption have other drawbacks. Other “issues” you mention to be generic and apply to most (all?) cryptographic systems. I am not sure why XEP-0384 needs to mention forward secrecy again, given that forward secrecy is provided by the building blocks the XEP uses and discussed there, i.e., https://www.signal.org/docs/specifications/x3dh/. Some points of yous are also outdated and no longer correct. For example, since the newest version of XEP-0384 uses XEP-0420, there is now padding to disguise the actual message length (XEP-0420 borrows this again from XEP-0373: OpenPGP for XMPP).
From a quick skim, my view is that it’s probably fine if your threat model is bored teenagers.
That makes it sound like your threat model shouldn’t be bored teenagers. But I believe that we should also raise the floor for encryption so that everyone is able to use a sufficiently secured connection. Of course, this does not mean that raising the ceiling shouldn’t be researched and tried also. But we, that is, the XMPP community of volunteers and unpaid spare time developers, don’t have the resources to accomplish everything in one strike. And, as I said before, if you need full metadata encryption, e.g., because you are a journalist in a suppressive regime, then the currently deployed encryption solutions in XMPP are probably not what you want to use. But for my friends, my family, and me, it’s perfectly fine.
They recommend looking for servers that support HTTP upload, but this means any file you transfer is stored in plain text on the server.
That depends on the server configuration, doesn’t it? I imagine at least some servers use disk or filesystem-level encryption for user-data storage.
For example, I tried using XMPP again a couple of years ago and needed to have two clients installed on Android because one could send images to someone using a particular iOS client and the other supported persistent messaging. This may be better now.
It got better. But yes, this is the price we pay for the modularity of XMPP due to its extensibility. I also believe it isn’t possible to have it any other way. Unlike other competitors, most XMPP developers are not “controlled” by a central entity, so they are free to implement what they believe is best for their project. But there is also a strong incentive to implement extensions that the leading Implementations support for compatibility. So there are some checks and balances in the system.
It’s a good question. I’m not sure that npm is all that different from most other dependency managers. My feeling is that it’s more cultural than anything – why do JS developers like to create such small packages, and why do they use so many of them? The install script problem is exacerbated because of this, but really the same issue applies to RubyGems, PyPI, etc.
I’m not sure that npm is all that different from most other dependency managers. My feeling is that it’s more cultural than anything – why do JS developers like to create such small packages, and why do they use so many of them?
I thought this was fairly-well understood, certainly it’s been discussed plenty: JS has no standard library, and so it has been filled-in over many years by various people. Some of these libraries are really quite tiny, because someone was scratching their own itch and published the thing to npm to help others. Sometimes there are multiple packages doing essentially the same thing, because people had different opinions about how to do it, and no canonical std lib to refer to. Sometimes it’s just the original maintainers gave up, or evolved their package in a way that people didn’t like, and other packages moved in to fill the void.
I’m also pretty sure most people developing applications rather than libraries aren’t directly using massive numbers of dependencies, and the ones they pulling in aren’t “small”. Looking around at some projects I’m involved with, the common themes are libraries like react, lodash, typescript, tailwind, material-ui, ORMs, testing libraries like Cypress, or enzyme, client libraries eg for Elasticsearch or AWS, etc… The same stuff you find in any language.
It’s more than just library maintainers wanting to “scratch their own itch.” Users must download the js code over the wire everytime they navigate to a website. Small bundle sizes is a unique problem that only JS and embedded systems need to worry about. Large utility libraries like lodash are not preferred without treeshaking — which is easy to mess up and non-trivial.
People writing python code don’t have to worry about numpy being 30MB, they just install it an move on with their lives. Can you imagine if a website required 30MB for a single library? There would be riots.
Sure, but that’s just the way it is? There is no standard library available in the browser, so you have to download all the stuff. It’s not the fault of JS devs, and it’s not a cultural thing. At first people tried to solve it with common CDNs and caching. Now people use tree-shaking, minification, compression etc, and many try pretty hard to reduce their bundle size.
I was thinking about Deno as well. The permission model is great. I’m less sure about URL-based dependencies. They’ve been intentionally avoiding package management altogether.
It’s at least interesting to consider that with deno, a package might opt to require limited access - and the installer/user might opt to invoke (a hypothetical js/deno powered dependency resolver/build system) with limited permissions. It won’t fix everything, but might at least make it easier for a package to avoid permissions it does not need?
I’ve had the same problem, so I’ve been working on a side project to help with this: xray.computer. It lets you view the published code of any npm package, and supports auto-formatting with Prettier for minified code. It’s not a panacea (sometimes, the published code is still unreadable, even when formatted) but it helps.
I also like to look up packages on Snyk Advisor to ensure that they get timely updates when vulnerabilities are found.
(Hopefully it’s okay to promote your own side projects here; please let me know if not.)
Thanks for sharing your side-project! It is helpful to see the differences between versions. I can use it to see if a security issue was indeed solved between versions of a package. I’m also using Snyk to get an idea of certain packages.
However, I think my problem is a bit more complex: a WordPress plugin may contain hundreds of interdependent npm packages all neatly bundled and minified. Without access to a package.json or package-lock.json it is quite hard to find out which individual packages have been used. Quite often there is also no public repo available of the development files.
To give an example of my process thus far:
Someone in my team wants to see if we can use plugin X. I’m downloading the plugin to have a look at the code. Luckily this plugin has included a non-minified version of the js file. I can derive the use of npm packages from this file. Using Snyk I have a look at the first package mentioned. It’s axios. The included version is vulnerable (high & medium severity) and has been for almost a year (Note: the last version of the plugin is 3 months old and does not exclude this vulnerable version in it’s package.json which I found in a Github repo later on).
Since I have no package.json nor package-lock.json (all I have is the distributed build) I can’t easily update the npm package. I have no clue as to how this package relates to the other packages and how their version might depend on each other. Even if I would update the package, all other users of this plugin are still vulnerable. I contacted the plugin author. He tells me he will update the plugin as soon as possible. The plugin is (as of today) still not updated & has not released a new version. In the meantime there have been two new versions of the axios package released.
Every user of plugin X is still vulnerable to the issues mentioned on Snyk, but is this a real problem in this specific WordPress plugin context? I’m not sure how to interpret the high & medium severity in the context of this plugin. How exploitable are these issues & what is the impact of the exploits in the context of this plugin? Do I need to be a logged in user? Is this something which can be triggered by any visitor? What am I able to do when I can exploit these vulnerabilities? I can only try to find answers to these questions if I’m willing to invest a lot more time into this, which more or less beats the purpose of using a ‘ready-made’ WordPress plugin. And this is just one package of multiple npm packages used in this plugin. Packages which also have their own dependencies as well….
At this moment I’m wondering if any WordPress plugin using npm packages can be trusted at all.
ps: The way the npm ecosystem is structured is, in my view at least, problematic. Often packages are not like libraries as I’d expect, but look more like a function call or method call. I’d prefer to write these short pieces of code myself instead of depending on external code which also includes extra risks. The very rapid release schedules makes it even harder to trust external software (like a WordPress plugin) using npm packages as it seems they cannot keep up with it.
I’m sorry if this seems like a npm rant, but I’m seriously looking for methods on how to deal with these issues so we can use external software (like WordPress plugins) built with npm packages.
I think it’s perfectly reasonable to say no to this plugin.
A WordPress plugin may contain hundreds of interdependent npm packages all neatly bundled and minified. Without access to a package.json or package-lock.json it is quite hard to find out which individual packages have been used. Quite often there is also no public repo available of the development files… At this moment I’m wondering if any WordPress plugin using npm packages can be trusted at all.
I’d be pretty skeptical of any proprietary software that has npm dependencies but doesn’t include its package files. Just taking woocommerce (one of the more popular plugins) for example, they do include their package files in their source code. That ought to be the norm. When you have access to the package files, you can run the following to automatically install patches to vulnerable dependencies and subdependencies.
npm audit fix
Without the package files, the plugin developer left you up a creek.
The included version [of axios] is vulnerable (high & medium severity) and has been for almost a year… I’m not sure how to interpret the high & medium severity in the context of this plugin. How exploitable are these issues & what is the impact of the exploits in the context of this plugin?
It really depends on the context. Some subdependencies are only used in the development or build of the dependency. If the vulnerability assumes the subdependency is running in a public node server, it’s probably not relevant to you. The hard part is knowing the difference. Take this vulnerability in glob-parent for example. I see it a lot in single page apps that have a webpack dev server. It’s hard to imagine how a DoS vulnerability is relevant when the only place the code is going to be run is your local development environment. In your case, it’s worth asking, is the axios vulnerability in the HTTP client or the server? If the vulnerability is just in the server and you’re not running a node server, you might be able to ignore it.
The way the npm ecosystem is structured is, in my view at least, problematic. Often packages are not like libraries as I’d expect, but look more like a function call or method call. I’d prefer to write these short pieces of code myself instead of depending on external code which also includes extra risks. The very rapid release schedules makes it even harder to trust external software (like a WordPress plugin) using npm packages as it seems they cannot keep up with it.
I agree that npm is problematic, but I have a slightly different take on it. There was a time when it was popular to mock projects like Bower for keeping front-end dependencies separate from node.js dependencies. The thinking was, why use two package managers when there’s one perfectly decent one? As it turns out, security vulnerabilities are impossible to assess without knowing in what environment the code is going to be run. I’m not suggesting we all go back to using Bower, but it would be nice if npm and its auditing tools would better distinguish between back-end, front-end, and build dependencies. As it is now, most npm users are being blasted with vulnerability warnings that probably aren’t relevant to what they’re doing.
I’d be pretty skeptical of any proprietary software that has npm dependencies but doesn’t include its package files.
I agree. I think this will be one of the main ‘rules’ for deciding if we want to use a particular WordPress plugin. Without the package.json or package-lock.json and npm audit it is near impossible to audit or update (fork) a plugin
It really depends on the context. Some subdependencies are only used in the development or build of the dependency.
You’ve hit the nail right on the head! And that’s what makes it hard to determine the impact: context.
As far as I can tell, determining context can only be done with (extensive) knowledge on how the npm package has been implemented in the WordPress plugin. It requires knowledge of the WordPress plugin, WordPress, the specific npm package, its dependencies and how all these relate to each other. Just like you said with the axios package.
Creating this context thus requires quite some time & skills, which would be fine if we would deal with just a few npm packages. However due to npm’s nature we usually have to deal with way more interdependent packages, which makes manually creating context near impossible. So even though there’s npm audit we’re still left with how to determine the context of the vulnerabilities found by npm audit. And for this I have yet to find a solution.
PS: npm audit fix is in my view not the solution to solve this. It just hides the problems a bit better ;)
Why not use a robots.txt? I am not a big fan of using meta-tags for this, and in the author’s sense, wasting my users’ bandwidth for such redundant tags.
It also saves you from accidentally forgetting to add it to a document.
Combined with the fact that Google might list your site based only on third-party links, robots.txt isn’t an effective way to remove your site from Google’s results.
This is a useful distinction that helps clarify why we have faulty security expectations from module managers, IMO. We’re used to packages being vetted (to some extent) before they’re included in a program manager — you don’t run any great risks by running apt-get install libxyz. In contrast, running npm install xyz is about as (in)secure as running curl randomdomain.com | bash, but most of us aren’t as cautious as we maybe should be when installing new/unknown modules.
But if the relay phone number can be used to reach my real phone number, does it matter? Maybe gaps in my understanding of this privacy enhancing feature.
Part of the reasoning behind email masks is that when you don’t give out your real email address, then it’s impossible to cross-reference your identity across multiple online services. No-one knows who’s at the other end of
bunchofrandomnumbers@firefoxrelay.comor whatever. I guess the thinking is similar here.I’m not sure how I feel about all my calls/texts going through Twilio, though. In general I think Mozilla could improve here – they also use AWS’ Simple Email Service to forward all Relay emails. I get that it’s easier than running their own mailserver, but it really highlights that privacy has become a marketing buzzword, IMO.
(disclaimer: I work on a competing service to Relay as a side project)
Sounds like you can disable/“turn off” a relay number, so I guess future calls to it would be dropped(?) without being forwarded to your real number?
I thought that at least Android supported both blacklist and whitelist approach?
Has anyone used this? I’m very curious which carrier(s) they use, and whether it works for signing up with services that are notoriously picky about which phone numbers they like, such as Discord, Instagram, Telegram, Google, and WhatsApp.
Firefox Relay is open-source, so I went to check – looks like it uses Twilio: https://github.com/mozilla/fx-private-relay/search?q=twilio
100% this. I signed up for a GOOG voice number and then also a Trello number to try and avoid giving my phone number to every damn mobile app that demands one. Somehow, about half of the time, they know it is a virtual number and lock me out. Including, ironically enough, apps powered by Trello. A month to month burner turned out to be much easier.
What is “a month to month burner”? Are you talking about a prepaid SIM?
yep! $15 a month or so for the privilege of using online services and apps that have nothing to do with the telephony.
(also just realized I said Trello when I meant Twilio earlier. Friggin startup naming, heh.)
There’s a lot of “should” in this article and very little argumentation. This distinction also seems arbitrary:
That seems like the point of having a standard.
I recently interviewed for a new role and I really liked how they structured their technical interview: 2 hours for a task, with a short introduction over a video call, then independent work with another check-in halfway through. Finally, half an hour’s call to talk through your solution.
It was a good balance of having space to think about the problem without someone breathing down your neck, while still having an interviewer available for questions and discussion.
Trying to tie up loose ends in the last couple of weeks at my current job. I think things are getting to a pretty good state for a smooth handover!
Otherwise, I’m working a lot on my side project, an email alias service. I’m migrating from a hosted email sender to a self-hosted MTA, which is necessary but comes with a whole lot of (ongoing) work!
Woof. Self hosting an MTA is quite a lot of work indeed.
I admire anyone who stays on top of this. I feel like GOOG, MSFT and a couple of other key players these days control the horizontal and the vertical, and don’t much care how many smaller players they tread on in the process.
Good luck!
That hasn’t actually been my experience, surprisingly. I started self-hosting my own email in… 2019 I think, and since early 2021 or so have used it for essentially everything. It took a few days to set up, using postfix + postgres for virtual accounts and dovecot for IMAP. But since then it has probably taken an average of one hour per year to actually support, maybe less. It’s not a perfect setup, but works pretty well and I have zero fears of it suddenly breaking. The only actual outage I’ve had was when I upgraded postgres and had to re-import the user database from backups.
Now that I actually say this, I’m a little surprised by it myself. But I can heartily recommend it to anyone likes being self-reliant, and is running their own VPS anyway.
I’m really, REALLY glad to hear!
I know a bunch of other folks and myself got impacted recently when GOOG really clamped down on their rules around DKIM signing and domains.
Found it kind of frustrating when all my outgoing mail all of a sudden to Gmail addresses was getting bit-bucketed, but c’est la guerre I guess :)
Yeah I have SPF and reverse DNS, but no DKIM set up at all IIRC. And as far as I can tell I’ve never had problems sending to gmail or any other mail server. Maybe I just got lucky and my VPS’s IP address has never been owned by a spammer? ¯\_(ツ)_/¯
Edit: Just remembered that I still have a gmail account as a backup and it’s set up to forward email to my own server. Maybe that makes a difference to Google?
I’m still sad about the state of messengers. One one hand you have Telegram and Signal which are two walled gardens with a closed-source server. (Signal being slightly worse as the author does not want it to be packaged for fdroid, and released a few versions without publishing the source code prior to integrating his cryptocurrency-scam in it.)
On the other hand, the decentralised alternatives are miles behind WhatsApp/Telegram/Signal. Jami just does not work for me and my friends (messages are not going through) and is a battery killer. (Messages are not being received) Tox has one of the worst UI/UX I’ve ever experienced (let alone push notifications being broken) And Matrix’s support for E2EE is not widespread.
I appreciate the community effort, but unfortunately I still use Telegram :( .
I’m not sure it’s fair to place Telegram and Signal in the same category, at least not if E2EE is important to you.
The vast majority of Telegram’s chats are stored in plaintext on their servers. Group chats cannot be end-to-end encrypted, and you have to manually start a secret chat between two people if you want E2EE. In spite of this, Telegram continues to be seen as a secure messenger – which is why a 2017 study found that many Telegram users mistakenly think their chats are end-to-end encrypted when they’re not.
I did not realise this and now it makes sense to me that they can delete group chats.
Yes. Telegram has to be used with over mitts, I agree. I don’t do any group chat, and only secure chats.
I have no expectation that Telegram is actually secure. I just find it moderately interesting it’s one of the few modern IM services with open client APIs.
I’m with you on the Telegram and Signal, though I still use both for some contacts. What has been working better for me, though, was setting up a hosted Snikket instance and then I got my family using that. It’s XMPP/Jabber (with OMEMO for the e2ee, which is based on the Signal protocol), and actually has good mobile clients. It’s one of the few ways the iOS family members get consistent notifications and experience. Might be worth a try if you’re ever up for trying another one.
matrix works great for me.
People keep saying this, but I don’t know when it’ll actually be the case. Every time I try it, I get disappointed by mediocre clients, resource hogging server, and inadequate moderation tools for groups.
I tried matrix about 3 years ago I think, it worked, but it wasn’t great. In fact I self-hosted a matrix server, and while playing it filled my server with garbage from IRC-bridges, etc… So after one year or so, I uninstalled matrix, and just used a self-hosted IRC server, and for my group of geeky friends it was ok.
Then, after a while, IRC show some limits, so we’ve tried to use matrix again, about 1 year ago, and there were a lot of progress. The client is a lot better, the server too, I just forgot about it, and it “just works”. The e2ee user interface is a lot better than a few years ago.
So if you haven’t tried matrix in a while you should really try it again. Really, now I think matrix is ready.
conduit.rs has been running flawlessly for me, for a couple months now - way less resource-hungry than the Synapse stack (albeit, with some missing features).
If you haven’t tried element in, say the last 3-4 months, then you aren’t up to date.
I have and it’s still not great. The interface is a bit confusing (i.e. the multiple levels of hierarchy for chat, the weird spaces thing that’s trying to paper over federation, etc.) and lacks this je ne sais quoi other clients have that makes it feel wrong for lack of a better term.
just wanted to say: matrix/element has come a long way. Due to it being used across orgs without federation (work vs family) I’d say the biggest downside is that there is no recommended way for multiple accounts on one device (especially mobile). Otherwise it works great.
It REALLY has. But sadly I think the fact that @acatton did a whole ‘state of messengers’ and didn’t even mention Matrix is a great example of how it’s gaining mindshare, but VERY slowly and not with non technical users.
The clients are IMO finally ready for that to happen, now we just need more people to actually try the updated versions and figure out how good it can be.e
Meanwhile sadly 99% of the communities I actually want to chat on are Discord or Slack :(
If you’re ever in the mood to try again, I suggest exploring Snikket.
I use prosody xmpp server with Conversations from f-droid on mobile. It works great for me and my friends and family. It is also very fast and easy to setup. Server to server works flawlessly so I can connect with others easily and without creating an account.
Can you provide more context or links around Signal’s cryptocurrency-scam that you mention? I could only locate a Verge article, but nothing more technical. Thanks.
https://support.signal.org/hc/en-us/articles/360057625692-In-app-Payments
This is using Mobile Coin which is basically a pump and dump by Moxie Marlinspike:
I hope this answer your question.
Thank you. Yes, it does.
I know it wasn’t your intention, but just to clarify, this makes it seem as if these three are of similar quality. However, Telegram is miles ahead of WhatsApp.
This is pretty valid though. F-droid’s whole build process is really strange. It also isn’t built with security in mind.
Statement from the CEO: https://twitter.com/toddmckinnon/status/1506184721922859010
“There was a security breach in January that we didn’t tell you about until we were forced to,” is what I’m reading there.
https://dnsimple.com
Though their pricing has been changing around, so I’m not sure it’s a good deal anymore. I’m grandfathered.
I actually just moved away from them because of pricing – didn’t like that you had to pay a monthly/yearly fee in addition to the cost of the domains themselves. Otherwise they were solid though.
Who did you move to?
I do think that “clean code” can be a useful shorthand in casual conversation, but the author is right – we can do better.
I think a similar thing could be said for the maxim “use the best tool for the job” – I keep seeing people say this, but it’s too general to really mean anything at all!
This kind of thing comes up in discussions of implicit bias. The problem isn’t describing something as ‘clean’, it’s using that to shut down discussion, which can happen accidentally. No one wants to criticise the clean design (who wants to advocate for dirty code?) and by being imprecise it’s hard for someone who isn’t part of the in group to know whether their objections make sense in the context of the evaluation criteria that everyone else is using.
If you’re praising someone’s work, telling them that it’s clean, beautiful, or other non-specific adjectives is completely fine. The purpose in the communication is to convey approval and agreement that they’ve made good decisions, not to critique the design. If you’re evaluating approaches, using this kind of terminology is not so good.
I’m attempting to fix a mysterious character encoding bug in one of my side projects. It’s one of those debugging journeys with lots of twists and turns, but I’m trying to enjoy the challenge of it!
Is anyone aware of any cool cryptography that might help assert code on Github matches the code on NPM? Or asserting code on Github matches code running on a server?
If the builds are reproducible, you can, by simply comparing your own build results (assuming you trust your own compiler chain, and with webpack having 780 dependencies, I’m not so sure about that) and the ones you get from NPM. I’m not sure how viable setting up reproducible builds is in JS ecosystem though.
This is what Code Transparency is intended to do. You run your builds in a known environment in a confidential computing environment that gives you an attestation over the initial state of the environment (VM image contents plus git hash of the repo). The VM is configured so that the only thing it will do is your build (no accepting incoming connections, for example) and produce the output. The output is signed with a key derived from the attestation. You then append this hash into a public ledger. Anyone else can then audit your build system and VM image and can verify that running the same build gives the same output.
Whether anyone will actually do that auditing is an open question. If your VM image has a custom container with a load of stuff that you’ve built then that’s all in the TCB and who knows what’s going on there.
This is all predicated on reproduceable builds, of course. If running the same build twice gives different output then there’s no automated way of verifying the process.
Cryptography can make it easier to simply trust an authority’s signature on a compiled blob, but besides that, I don’t think it will help much. You can certainly download all the source code for each module and run the build for each module yourself. To be honest I don’t know if npm or yarn support that type of operation out of the box. Something tells me even if they did, it would probably fail on many packages for unforeseen reasons; you would have to fix a bunch of “broken builds”.
I do feel like this is kind of silly though. This isn’t a problem unique to JavaScript and npm. Will say regarding javascript and analytics however; IMO malicious or “criminal” intent is not even required for these kinds of “bad things” to happen. It can also happen straight through the front door without having to hide anything, without doing all this silly malware style stuff like changing the behavior based on the time of day.
Plugging my own project, but: the fact that we don’t have a good answer to this is why I made xray.computer which lets you view the code that’s actually published to npm. It’s only a plaster on a major wound, though – npm packages often include pre-minified/compiled code, which is what your project will actually use, and it’s almost impossible to review since it’s essentially obfuscated.
Reproducible builds and/or a cryptographically signed web of trust are the best proposals I know of, though it’s worth noting that npm does appear to do some relatively advanced work on detecting malicious packages behind the scenes.
I’ve heard of https://github.com/in-toto/demo and was able to find it after I made this comment. Haven’t used it, but seems interesting. Apparently datadog uses it: https://www.datadoghq.com/blog/engineering/secure-publication-of-datadog-agent-integrations-with-tuf-and-in-toto/
Related: it seems that something fishy is going on with the
nosepackage on PyPi: https://zaitcev.livejournal.com/263602.htmlThe author provides no details of what difference existed between what they saw in the repo and what they saw in the package, what effect it actually had, or anything else that would allow readers to verify or investigate for themselves, and simply declares the only possible conclusion to be that PyPI itself has been silently and completely compromised. Not that a maintainer’s account for an unmaintained (coming up on 7 years since last release) package could have been. Not that it’s possible the last release legitimately wasn’t generated from what the author saw in the GitHub repo. Not any of the other many possible explanations. And I haven’t seen this allegedly huge “PyPI compromise” mentioned anywhere else despite that post being over a week old.
Smells an awful lot like FUD, and until the author chooses to provide some more information that will be my stance.
The fact that the NFT spec only stores an image URL on the blockchain, and no hash to verify the image, is just absolutely astounding. Moxie is more generous than I am; IMO it highlights the grift that NFTs are – like whoever threw it together gave no thought to security or longevity.
Yeah to me this is the “tell” that the main thing driving it is other people’s FOMO money.
Basically software devs in this ecosystem realized they didn’t have to do their jobs to make money. The whole system keeps working even if you don’t do your job!!!
You just have to repeat the memes long enough, and it doesn’t matter if the tech actually does what it says it does, because nobody’s checking! Nobody really wants to stand up their own copies of these things and check if it works.
There’s little benefit in that. The benefit is talking about it and repeating it to your friends.
I was interested in IPFS before it grew the blockchain component. I went back the original paper, and it is genuinely a good idea, and something we still need: a cross between git and BitTorrent (it mentions this in the abstract). I have long wanted Debian repos, PyPI, CPAN, etc. to be stored in such a system, i.e. consistently versioned with metadata.
But it’s 6 years later, and when I go to look at it, apparently it just doesn’t work very well. And it probably won’t because it grew so many components. (Gall’s law: to get a big working system, you have to start from a small working system.)
https://news.ycombinator.com/item?id=20137918
https://esteroids.medium.com/how-can-ipfs-reach-wide-adoption-42b9a5011bdf
And to give some light on the other side … Despite having some interest in BitCoin since 2013 or so, I first bought it last year.
Because the founder of SciHub asked for donations in crypto on her site. https://en.wikipedia.org/wiki/Alexandra_Elbakyan
So I just did it via CoinBase and I suppose it worked. So I would say there’s non-zero number of real use cases for cryptocurrency and blockchain. I think IPFS started out as genuine too, but it basically got ruined as working tech by an influx of money and employees.
The situation with IPFS also affected gittorrent, another combination of git and bittorrent. Blockchains are like fig trees; as they grow, they choke whatever software projects originally gave them structure and purpose.
Hm what happened to it? It doesn’t look like very much code.
Thinking more about the original comment … To “steel man” the blockchain, I do think there is a place for regular old web apps to serve data from a blockchain. I don’t think that is inherently a bad architecture. You just have to be honest about it!
I think there will always be a place for money in the blockchain, however niche, as the Scihub example shows.
It’s more logical for an end user like me to use Coinbase, which is centralized, but that’s OK. (Also big irony: it relies on sending your driver’s license in for verification, so it’s building on top of US state regulations which they want to be free of.)
One viewpoint I heard is “Bitcoin is a settlement network, not a payment network”. That is, the logical end user is banks and companies like Coinbase, not consumers. It could make sense to have a two-tiered system.
(Although then you have the same problem that those banks are subject to regulation of the countries they operate in, so you’ve lost a lot of the purported benefit of blockchain. I think people will gradually come to understand this and a much smaller set of use cases will be ironed out.)
But I do think the logical evolution of blockchain is to evolve to be less “consumer” and more “enterprise”.
I think the problems with the centralization of the cloud are real, and while web3 is mostly a BS term, and it’s not really a solution as is, I can see this type of distributed consensus as a useful primitive for small but important parts of real systems.
I believe that the Dat protocol and its successor Hypercore are both basically Bittorrent plus version control, but with no connection to blockchains or cryptocurrency that I know of. Please correct me if I’m incorrect :)
I think Beaker Browser is a really cool project that suggests what could be done with Hypercore, but unfortunately a lot of websites and apps that were designed for its first iteration using Dat didn’t succeed in making the transition to the Hypercore version. I think the tech change cost it some momentum / users. Hopefully it will build up steam again, but I’m afraid IPFS has stolen its thunder by providing similar tech plus a chance to get rich quick :(
Dat is so good aside from one critical flaw (IMHO, anyway): the “spec” is more or less “whatever the version you pull from NPM right now does.” Yes, there is some additional documentation, and a running reference implementation is great, but going a decade+ without a compatible version in some environment other than Node is a pretty big handicap and IMHO a major oversight by the project owners.
Secure Scuttlebutt suffers from the same problem, and the “rewrite it in Rust” efforts for both are at best implicitly— if not explicitly, as in the most recent SSB -> Rust work — aimed at performance, not broad interop or adoption.
So neither one can effectively run without hundreds of MBs of JS packages, there’s no support for languages that offer better native platform integration or type safety…heck, they don’t even ship TS definitions, and the internal APIs are so idiosyncratic it’s extremely difficult to build new apps on the platform.
In a world where I had infinite time and attention to spend on an open software + network stack I would love to built a libdat or libssb that was straightforward to link in a Python project, or an iOS app, or really anything that can do normal C FFI. Alas, I don’t, so I haven’t. Maybe someday.
Hm I heard of Dat a few years ago, but I didn’t know about Hypercore. Thanks for the pointer!
I think eventually we will get something like this … just because there is a need. In the meantime I might write my own with shell scripts invoking git, and then curl or BitTorrent :)
I forget which post I read this in, but there are really 2 kinds of “decentralized systems”, i.e. ones that don’t rely on “the cloud”:
Notably, it’s not only hard to run your own Ethereum node, but you also don’t want to run your own Ethereum network!
So the word is overloaded, and they have very different architectures. I care more about the first kind. So really I don’t want a global file system like IPFS – I’d rather have distributed storage that runs on a few nodes, is easy to administer, etc.
Whoops, it seems Beaker Browser is being discontinued: https://github.com/beakerbrowser/beaker/discussions/1944
I’m not surprised but I am really sad about this. I thought it had a lot of potential for building a resilient web that could e.g. handle network outages by making it trivial to continue sharing websites over a LAN. It seemed to be great at sharing static websites which are where I’m putting my development efforts these days, but it appears the developer was frustrated by this and wanted people to share web apps instead.
Agregore looks like it may be continuing on with support for Hypercore, but it also supports IPFS and I saw not interacting with Protocol Labs and their reliance on / promotion of cryptocurrency as a significant feature.
IPFS is a very pure “better BitTorrent” and the fact that some web3 stuff hosts content on IPFS should not taint the protocol, even if you happen to hate web3. Lots of web1 content on IPFS too, the protocol predates web3 by a lot, etc
Part of the problem here is that “hosts content on IPFS” is a misnomer, for the same reason that things aren’t “hosted on BitTorrent”. IPFS is a distribution mechanism, not a storage mechanism (despite what their marketing implies), and so something can only be distributed through IPFS - it needs to be hosted somewhere else, and that “somewhere else” is left undefined by IPFS.
That might sound like pedantry, but it has some crucial implications: it means that by default, the network cannot provide meaningfully better availability than BitTorrent can, which is to say the availability is really bad and unsuitable for hosting websites on. You could address this by seeding your content from a server, but then what have you really accomplished, other than a potential ‘download accelerator’ (which is again distribution, not storage)?
This seems like a contradiction. Of course you cannot host an IPFS powered website without seeding it from somewhere! That doesn’t make availability bad or unsuitable any more than HTTP is bad or unsuitable. What I love about IPFS is that I can pin my websites on any computer connected to the internet, and usually much more than one! No special setup is needed, and if I want to change what computer pins the content I can easily do so at any time without changing any settings anywhere else. It just seamlessly keeps working.
I have been using IPFS for websites for years. It works great.
Where do you persist user-mutable state and how?
Websites do not have user-mutable state. I guess if you want your blog to have a user theme toggle or some other appy feature a website might want you can use a cookie or localStorage just as you would on the regular web.
Indeed. That’s what makes this a balanced critique. Good-faith.
I don’t think bad faith is necessary to provide valid criticism. The fact that NFTs are sold to the public as ‘stored forever in the blockchain’ and the reality is that there’s no actual mechanism storing anything other than a URL pointing to the content is almost by definition a scam. If I was going around selling ownership contracts to a house I don’t own, I’d be charged with fraud.
That’s not what I meant. When @tao_oat said “Moxie is more generous than I am”, the difference between a “generous” take and a “not generous” take can often be the assumption of good faith. Are you trying to take a keep a neutral eye while evaluating this argument, or are you just looking for more evidence confirm your existing bias, that’s the difference between having good faith or approaching a topic cynically.
I think you’re prejudiced against NFTs and are just looking to criticize them. That’s fine, I think NFTs being used to attest art and their associated speculation is pretty stupid, so you aren’t gonna find me defending any of this. I also agree with each of Moxie’s criticisms. That said, I just don’t think the sort of cynical, charged rhetoric in this thread is indicative of good faith. Your comment even assumes a position from me that I don’t have. Good faith keeps discussions intellectually interesting IMO. I don’t have that much more to say here, I’ll let everyone else continue ranting in anger.
FWIW I agree with you – on doing more research, a good-faith take would be that whoever wrote the ERC721 spec might have seen the promise of an image that changes over time. They might not have foreseen that art collectibles would be the primary driver behind NFTs. In the spec they write:
(Though having a sentence like “The image size recommendation is taken from Instagram, they probably know much about image usability” in an official spec doesn’t exactly scream professionalism or care to me).
I should direct my critique against those who push art NFTs without mentioning the serious technical issues, and not the spec authors themselves.
I think the broader question is: when you keep seeing red flags in an ecosystem/community how long can or should you make an effort to retain good faith?
I think that’s the wrong question to ask. Technology usually has two components. One is the design and engineering that goes into it. Think with XMPP the protocol and standards used to send/receive messages. The other is adoption: this can be success in number of users, its usage, or its revenue. There’s plenty of technologies, ones much less complicated and more “clearly” incremental than a blockchain, that have failed purely because their adoption was lacking, despite years of attempts to make themselves relevant. Examples here are Laserdisc, Betamax, HD-DVD, and more. Adoption of a technology has much more to do with culture, user-experience, or business considerations than the engineering behind the technology. These questions you’re asking, about whether the space is filled with hucksters and such, are questions affecting the adoption of the technology. Discussions concerning the adoption of a technology are much more complicated (well IMO at least, probably because I’m an engineer first and foremost!) than discussions dealing with the design and engineering behind a technology, and also off-topic for Lobsters (probably due to the challenge of dealing with the nature of those discussions properly.) I will say though, a space filled with fraud doesn’t exactly instill confidence, especially when it’s a person’s money or other resources on the line…
But I also don’t think it’s necessary to get inordinately angry at the blockchain space. Businesses are made and die every day, some full of hucksters, others just blatant copies of existing businesses. Community projects are made and die every day. It’s the churn of human creativity. If humans knew exactly which projects would succeed and which would fail, then we’d already have solved our problems, wouldn’t we?
I don’t really care for collectibles, real or virtual, but to be fair the NFT is stored in the blockchain. The media is not the NFT, just an associated decoration.
This sounds like a solution looking for a problem, and then someone inventing a problem for the solution to fix. One thing I’ll give ‘crypto bros’ - specially those pumping the NFT racket - is that they’ve managed to befuddle the world into thinking they created something actually revolutionary. Every single problem the article describes is stuff anyone who understands this technology works could’ve guessed from day 1. Ultimately ‘web3’ is just a nebulous term that means nothing and everything, depending on who you ask and what time you ask them.
Sure, I don’t collect baseball or magic cards either, so the while thing doesn’t connect with me personally.
But nobody says that an NFT is a URL, i.e. a string starting with “https://…” Because nobody would find such a string, in itself, interesting enough to pay money for.
The scam is that people describe and sell NFTs as being a file, or even a unique fingerprint of such a file, when they’re no such thing.
It’s like selling you a painting, only it turns out you only bought the frame, and the gallery reserves the right to swap out the canvas for another one, or just take it away, at their pleasure.
Yeah but the funny thing is: who cares if it was actually pinned to the right file?
What’s to stop me from minting another NFT for the same artwork and selling it? I just have to convince enough people it’s valuable.
As far as I can tell, the thing that makes NFTs “work” in any sense is specific pockets of social media, e.g. Twitter. Reality is socially constructed.
Like the artist Beeple has to go on Twitter and say he’s selling some number that represents art that he created.
https://twitter.com/beeple
And other people witness that and they believe it is him, i.e. the person who keeps creating all the enjoyable pictures. Twitter and other centralized social networks provide some continuity of identity. lobste.rs does this too.
If somebody else tries to sell an NFT of his artwork, maybe he can use Twitter to shame them or whatever. That’s about it.
As far as I can see, social media is really the thing that matters, and not anything in the blockchain. It seems clear that no end users every really look at the blockchain and verify things. (Hell I didn’t when I sent the donation to the Scihub founder. Did it really get sent? I just trusted Coinbase.)
So I think there’s no notion of identity, exclusivity, or authenticity on the blockchain. You always have to make some jump between the number/hash and the “thing”, and other people can make that jump too.
I’d be interested in any arguments otherwise … I have never actually used NFTs or Ethereum, but it seems like there is an obvious hole either way.
i.e. the blockchain is really built on trust built by social media; it can’t stand alone. Social media is very powerful – the former US president got elected in a large part because of it, and then he got blocked from it with huge consequences, and now he wishes he had his own Twitter and there are multiple efforts in that direction, etc. It has a lot of real consequences in the world, and is intimately connected to it. Blockchain doesn’t have that property at all!
Right — I can just touch one pixel in the image, or add a no-op EXIF tag, and suddenly it’s a different file with a different digest that I can sell as a different NFT.
That was my day-one objection to NFTs. In terms of my analogy, it’s like buying a limited edition print where I only have the artist’s promise she won’t issue more. But the realization that it’s just a URL makes them orders of magnitude sillier and more scam-like.
Yeah this has been beat to death, but I was reading conversations earlier this year, and one analogy is the company that used to sell naming rights to the stars:
https://news.ycombinator.com/item?id=26488430
Like they would sell you a certificate that a star was named after you.
Never mind that anybody can rename the same star for a different person, and sell that. Nobody ever used those names – not even the one scientist who might have a picture of that star or care about it. (Again, reality is socially constructed.)
Another problem is that you’re literally just getting the NFT number itself. You’re not getting the copyright to that work !!!
Like you could buy the NFT, and then the artist can sell the actual artwork to somebody else, which IS enforceable by copyright law !!! But your NFT isn’t.
Also with music, there are separate rights to perform songs in public places, to play recordings at bars, and to publish the sheet music. You do NOT get that if you buy an NFT of a song.
In fact I remember seeing a VC blog or podcast where this was brought up ….
And so that basically proves that it doesn’t matter if the NFT has an immutable hash or not. You’re buying a useless pointer to a thing, not anything related to the thing itself … so it doesn’t matter what’s in it!
I don’t understand this sticking point. An NFT is just a provably-unique (i.e. non-fungible) thing maintained on a chain. What it maps to off-chain, or how it does so, is basically incidental. It’s a contract, meaningful only in a specific domain: for legal contracts, that domain is typically a government jurisdiction; for NFTs, it’s the chain on which they exist.
I think it’s because the loudest use is for art collectibles so people get hung up on that. For me the best analogy is baseball cards. No one would say “but the card doesn’t contain an actual baseball player! You don’t get the right to boss the real human around!” But somehow NFTs the detractors think should “be” the art or “be” the copyright in a way that they were never intended to be.
The problem with that analogy is that the issuer of the baseball card could, if it were an NFT, blank the contents of the card or change it to something else at any point. If baseball cards had that property, would people trade them? If a baseball card just had a QR code on it that let you go to a web page of stats about the player, would they be as valuable? Would they keep being valuable once some of the servers hosting the stats went offline (or would ones pointing to dead URLs become more valuable?)? What about when someone buys the domain for a popular card and points it at a porn site?
In this analogy the NFT, the contract which exists on-chain, is itself the baseball card. The fact that one of the metadata fields of the NFT is a URL that may or may not resolve is more or less incidental.
The important properties of NFTs are that, in the context of the chain on which they exist, they’re provably unique, non-fungible, and owned. A broken URL in the metadata doesn’t impact those properties. Of course, value is determined by the market, and the crypto markets are wildly irrational, so you may have a point.
I’ll buy this, but practically speaking, what then is the use of the NFT? If you’re saying the URL isn’t important, what exactly are you buying?
You’re buying something which is guaranteed to be unique and non-fungible, in the context of a specific chain. There is no intrinsic value, any more than a specific painting or whatever is valuable. The value relies on the market belief that the NFT’s chain-specific scarcity is valuable.
It’s kind of like a deed. The property isn’t legally yours until the relevant legal regime accepts that you own the deed. The deed isn’t the property but it uniquely represents the property in a specific domain.
But like value is not the only interesting thing about NFTs. The non-fungibility itself is novel and opens the door to lots of interesting things.
I prefer this analogy to signatures: https://jackrusher.com/journal/what-does-it-mean-to-buy-a-gif.html
Sure, but that is absolutely not how NFTs are understood by 99.9999% of people. That mismatch is the scam.
This is not universally true. A lot of NFT projects use IPFS, Filecoin or similar decentralized storage.
But for the ones using Amazon S3 it’s kinda hilarious, yes
These solutions have a similar problem as with Bittorrent. The stuff that’s not popular won’t get seeded/hosted. It just adds one layer of indirection to the storage issue, but once the business that’s pushing NFTs goes out of business these files are probably not going to get shared.
I don’t see how that’s a problem in this context? If the URL is IPFS you have a hash, which was the objection being replied to here.
My understanding of how this works is: the NFT is an object on the blockchain, whose rules enforce its uniqueness. We’re gonna assume the chain is going to be continued to be mined and therefore “exist” indefinitely.
The NFT contains a hash denoting a location on IPFS. Accessing it using an IPFS gateway will show the user the JPG portraying whatever they paid $2.4M in funny money for. But that JPG has to reside on disk somewhere. And when the company or user goes out of business or the VPS is decommissioned or they get kicked out of AWS for scamming, where is the JPG?
Of course, concerned parties can… right click on the image, save it to disk, and then use that as a source of data for the IPFS hash, but that does kind of put a lie to the popular imagination on how all this nonsense works.
You can become a node in the IPFS network yourself, and host just the image.
Just like the baseball player the “hash” (picture) on a baseball card may die, so may the hosting for an NFTs associated JPG go away. Of course if you want to preserve it you can simply pin it yourself and thus your own computer becomes a host for it. So it’s actually more resiliant than the baseball player :). The NFT is not the image, the image is an associated decoration
Company: Intruder
Company site: https://www.intruder.io/
Position(s): Backend Engineer, Full-Stack Software Engineer, Senior Software Engineer
Location: Remote, UK timezone +- 2 hours. We have an office in London if you’re based there, but our engineering team is remote-first.
Description: We’re making cybersecurity easy. Intruder is a vulnerability scanner that continually scans your attack surface for known CVEs, configuration weaknesses, and common vulnerabilities. We focus on making security accessible to all through an easy-to-use platform and simple, understandable descriptions.
Tech stack: Ruby/Rails and Python/Django. Our frontend is written in Vue.js with Tailwind, and we’re starting to introduce TypeScript. Mostly running on Kubernetes.
Compensation: £55,000 - £70,000 (mid-level roles) or £70,000 - £90,000 (senior role). Share options included.
Contact: Apply through the links above, or contact me on tao.bojlen@intruder.io with any questions!
I’ve been working with a similar paradigm through Elixir + Phoenix + LiveView, and it works extremely well for my one-person side projects. I’m excited to see Hotwire become the default in Rails.
I wonder, though, does anyone have experience working with technologies like these on very large apps? I can imagine it’s challenging to scale it to larger teams – even for my small projects, I’ve found myself missing the concept of full-featured UI components.
I don’t think it’s quite there yet for UI components, but it is a very active area of development- see https://surface-ui.org/
I don’t really agree with a lot of the claims in the article (and I say this as someone who was very actively involved with XMPP when it was going through the IETF process and who wrote two clients and continued to use it actively until 2014 or so):
This is true. It also means that you need to do server reputation things if your server is public and you don’t want spam (well, it did for a while - now no one uses XMPP so no one bothers spamming the network). XMPP, unlike email, validates that a message really comes from the originating domain, but that doesn’t stop spammers from registering millions of domains and sending spam from any of them. Google turned off federation because of spam and the core problems remain unsolved.
This is completely untrue for the core protocol. End-to-end encryption is (as is typical in the XMPP world) multiple, incompatible, extensions to the core protocol and most clients don’t support any of them. Looking at the list of clients almost none of them support the end-to-end encryption XEP that the article recommends. I’d not looked at XEP-0384 before, but a few things spring to mind:
This is without digging into the protocol. I’d love to read @soatok’s take on it. From a quick skim, my view is that it’s probably fine if your threat model is bored teenagers.
They recommend looking for servers that support HTTP upload, but this means any file you transfer is stored in plain text on the server.
True, with the caveat that they have different feature sets. For example, I tried using XMPP again a couple of years ago and needed to have two clients installed on Android because one could send images to someone using a particular iOS client and the other supported persistent messaging. This may be better now.
This, at least, is fairly mature. There are some interesting interactions between it and the security gurantees claimed by OMEMO.
Servers are the easy part (mostly they do STUN or fall back to relaying if they need to). There are multiple incompatible standards for voice and video calling on top of XMPP. The most widely supported is Jingle which is, in truly fractal fashion, a family of incompatible standards for establishing streams between clients and negotiating a CODEC that both support. It sounds as if clients can now do encrypted Jingle sessions from their article. This didn’t work at all last time I tried, but maybe clients have improved since then.
Strongly agree – claiming that XMPP is secure and/or private without mentioning all the caveats is surprising! There’s also this article from infosec-handbook.eu outlining some of the downsides: XMPP: Admin-in-the-middle
The state of XMPP security is a strong argument against decentralization in messengers, in my opinion.
Spam in XMPP is largely a solved problem today. Operators of open relays, servers where anyone can create an account, police themselves and each other. Anyone running a server that originates spam without dealing with it gets booted off the open federation eventually.
Another part of the solution is ensuring smaller server operators don’t act as open relays, but instead use invites (like Lobste.rs itself). Snikket is a great example of that.
Bold claim. Citation needed. Where do you register millions of domains cheaply enough for the economics of spam to work out?
Domains tend to be relatively expensive and are easy to block, just like the IP addresses running any such servers. All I hear from server operators is that spammers slowly register lots of normal accounts on public servers with open registration, which are then used once for spam campaigns. They tend to be deleted by proactive operators, if not before, at least after they are used for spam.
That’s what they claim. Does it really seem plausible that Google could not manage spam? It’s not like they have any experience from another federated communications network… Easier for me to believe that there wasn’t much in the way of promotion to be gained from doing anything more with GTalk, so they shut it down and blamed whatever they couldn’t be bothered dealing with at the time.
Your reasonning about most clients not supporting OMEMO is invalid because noone cares about most clients: it’s all about the marketshare. Most XMPP clients probably don’t support images but that doesn’t matter.
For replays, this may be dealt with the double ratchet algorithm since the keys change fairly often. Your unknown replay would also have to make sense in an unknown conversation.
Forward secrecy could be done with the double ratchet algorithm too.
Overall OMEMO should be very similar to Signal’s protocol, which means that it’s quite likely the features and flaws of one are in the other.
Conversations on Android also offers showing and scanning QR codes for validation.
As for HTTP upload, that’s maybe another XEP but there’s encrypted upload with an AES key and a link using the aesgcm:// scheme (as you can guess: where to retrieve the file plus the key).
I concur that bootstrapping is often painful. I’m not sure it’s possible to do much better without a centralized system however.
Finally, self-hosting leads to leaking quite a lot of metadata because your network activity is not hidden in large amounts of network activity coming from others. I’m not sure that there’s really much more that is available by reading the XMPP metadata. Battery saving on mobile means the device needs to tell the server that it doesn’t care about status messages and presence from others but who cares if it’s unencrypted to the server (on the wire, there’s TLS) since a) it’s meant for the server, b) even if for clients instead, you could easily spot the change in network traffic frequency. I mean, I’m not sure there’s a lot more that is accessible that way (not even mentionning that if you’re privacy-minded, you avoid stuff like typing notifications and if you don’t, traffic patterns probably leak that anyway). And I’m fairly sure that’s the same with Signal for many of these.
I guess you’ve been away for awhile :) there is definitely spam, and we have several community groups working hard to combat it (and trying to avoid the mistakes of email, not doing server/ip rep and blocking and all that)
Or they’ve also calcified (see: Pidgin). Last time I tried XMPP a few years ago, Conversations on Android was the only tolerable one, and Gajim was janky as hell normally, let alone on Windows.
This was the reason I couldn’t get on with XMPP. When I tried it a few years ago, you really needed quite a lot of extensions to make a good replacement for something like WhatsApp, but all of the different servers and clients supported different subsets of the features.
I don’t know enough about all the details of XMPP to pass technical judgement, but the main problems never were the technical decisions like XML or not.
XMPP had a chance, 10-15 years ago, but either because of poor messaging (pun not intended) or not enough guided activism the XEP thing completely backfired and no two parties really had a proper interaction with all parts working. XMPP wanted to do too much and be too flexible. Even people who wanted it to succeed and run their own server and championed for use in the companies they worked for… it was simply a big mess. And then the mobile disaster with undelivered messages to several clients (originally a feature) and apps using up to much battery, etc.pp.
Jitsi also came a few years too late, sadly, and wasn’t exactly user friendly either at the start. (Good people though, they really tried).
XML was a problem early on because it made the protocol very verbose. Back when I started working on XMPP, I had a £10/month plan for my phone that came with 40 MB of data per month. A few extra bytes per message added up a lot. A plain text ‘hi’ in XMPP was well over a hundred bytes, with proprietary messengers it was closer to 10-20 bytes. That much protocol overhead is completely irrelevant now that phone plans measure their data allowances in GB and that folks send images in messages (though the requirement to base64-encode images if you’re using in-band bytestreams and not Jingle still matters) but back then it was incredibly important.
XMPP was also difficult to integrate with push notifications. It was built on the assumption that you’d keep the connection open, whereas modern push notifications expect a single entity in the phone to poll a global notification source periodically and then prod other apps to make shorter-lived connections. XMPP requires a full roster sync on each connection, so will send a couple of megs of data if you’ve got a moderately large contact list (first download and sync the roster, then get a presence stanza back from everyone once you’re connected). The vcard-based avatar mechanism meant that every presence stanza contained the base64-encoded hash of the current avatar, even if the client didn’t care, which made this worse.
A lot of these problems could have been solved by moving to a PubSub-based mechanism, but PubSub and Personal Eventing over PubSub (PEP) weren’t standardised for years and were incredibly complex (much more complex than the core spec) and so took even longer to get consistent implementations.
The main lessons I learned from XMPP were:
This is not accurate. Roster versioning, which means that only roster deltas, which are seldom, are transferred, is used widely and also specified in RFC 6121 (even though, not mandatory to implement, but given that it’s easy to implement, I am not aware of any mobile client that doesn’t use it)
Also important to remember that with smacks people are rarely fully disconnected and doing a resync.
Also, the roster itself is fully optional. I consider it one of the selling points and would not use it for IM without, but nothing prevents you.
Correct.
I want to add that, it may be a good idea to avoid using XMPP jargon to make the test more accessible to a wider audience. Here ‘smacks’ stands for XEP-198: Stream Management.
I’d argue there is at least one other reason. XMPP on smartohones was really bad for a very long time, also due to limitations on those platforms. This only got better later. For this reason having proper messaging used to require spending money.
Nowadays so you “only” need is too pay a fee to put stuff into the app store and in case of iOS development buy an overpriced piece of hardware to develop on. Oh and of course deal with a horrible experience there and be at the risk of your app being banned from the store, when they feel like. But I’m drifting off. In short: Doing what the Conversation does used to be harder/impossible on both Android and iOS until certain APIs were added.
I think that gave it a pretty big downturn when it started to do okay on the desktop.
I agree with the rest though.
I saw a lot of those same issues in the article. Most people don’t realize (myself included until a few weeks ago) that when you stand up Matrix, it still uses matrix.org’s keyserver. I know a few admins who are considering standing up their own keyservers and what that would entail.
And the encryption thing too. I remember OTR back in the day (which was terrible) and now we have OMEMO (which is ….. still terrible).
This is a great reply. You really detailed a lot of problems with the article and also provided a lot of information about XMPP. Thanks for this.
Do you know if this situation is any better on Matrix? Completely honest question (I use both and run servers for both). Naively it seems to me that at least some important metadata needs to be unencrypted in order to route messages, but maybe they’re doing something clever?
I haven’t looked at Matrix but it’s typically a problem with any Federated system: you need at least an envelope that tells you the server that a message needs to be routed to to be public. Signal avoids this by not having federation and by using their sealed-sender mechanism to avoid the single centralised component from knowing who the sender of a message is.
Thanks.
There is a bit of metadata leaking in matrix, because of federation. But it’s something the team is working to improve.
Fellow active XMPP developer here.
I am sure you know that some of your points, like Metadata encryption, are a deliberate design tradeoff. Systems that provide full metadata encryption have other drawbacks. Other “issues” you mention to be generic and apply to most (all?) cryptographic systems. I am not sure why XEP-0384 needs to mention forward secrecy again, given that forward secrecy is provided by the building blocks the XEP uses and discussed there, i.e., https://www.signal.org/docs/specifications/x3dh/. Some points of yous are also outdated and no longer correct. For example, since the newest version of XEP-0384 uses XEP-0420, there is now padding to disguise the actual message length (XEP-0420 borrows this again from XEP-0373: OpenPGP for XMPP).
That makes it sound like your threat model shouldn’t be bored teenagers. But I believe that we should also raise the floor for encryption so that everyone is able to use a sufficiently secured connection. Of course, this does not mean that raising the ceiling shouldn’t be researched and tried also. But we, that is, the XMPP community of volunteers and unpaid spare time developers, don’t have the resources to accomplish everything in one strike. And, as I said before, if you need full metadata encryption, e.g., because you are a journalist in a suppressive regime, then the currently deployed encryption solutions in XMPP are probably not what you want to use. But for my friends, my family, and me, it’s perfectly fine.
That depends on the server configuration, doesn’t it? I imagine at least some servers use disk or filesystem-level encryption for user-data storage.
It got better. But yes, this is the price we pay for the modularity of XMPP due to its extensibility. I also believe it isn’t possible to have it any other way. Unlike other competitors, most XMPP developers are not “controlled” by a central entity, so they are free to implement what they believe is best for their project. But there is also a strong incentive to implement extensions that the leading Implementations support for compatibility. So there are some checks and balances in the system.
What would an ideal JavaScript dependency management system look like?
It’s a good question. I’m not sure that npm is all that different from most other dependency managers. My feeling is that it’s more cultural than anything – why do JS developers like to create such small packages, and why do they use so many of them? The install script problem is exacerbated because of this, but really the same issue applies to RubyGems, PyPI, etc.
There are some interesting statistics in Veracode’s State of Software Security - Open Source Edition report (PDF link). Especially the chart on page 15!
Deno’s use of permissions looks very interesting too, but I haven’t tried it myself.
I thought this was fairly-well understood, certainly it’s been discussed plenty: JS has no standard library, and so it has been filled-in over many years by various people. Some of these libraries are really quite tiny, because someone was scratching their own itch and published the thing to npm to help others. Sometimes there are multiple packages doing essentially the same thing, because people had different opinions about how to do it, and no canonical std lib to refer to. Sometimes it’s just the original maintainers gave up, or evolved their package in a way that people didn’t like, and other packages moved in to fill the void.
I’m also pretty sure most people developing applications rather than libraries aren’t directly using massive numbers of dependencies, and the ones they pulling in aren’t “small”. Looking around at some projects I’m involved with, the common themes are libraries like
react,lodash,typescript,tailwind,material-ui, ORMs, testing libraries like Cypress, orenzyme, client libraries eg for Elasticsearch or AWS, etc… The same stuff you find in any language.It’s more than just library maintainers wanting to “scratch their own itch.” Users must download the js code over the wire everytime they navigate to a website. Small bundle sizes is a unique problem that only JS and embedded systems need to worry about. Large utility libraries like lodash are not preferred without treeshaking — which is easy to mess up and non-trivial.
People writing python code don’t have to worry about
numpybeing 30MB, they just install it an move on with their lives. Can you imagine if a website required 30MB for a single library? There would be riots.I wrote more about it in blog article:
https://erock.io/2021/03/27/my-love-letter-to-front-end-web-development.html
Sure, but that’s just the way it is? There is no standard library available in the browser, so you have to download all the stuff. It’s not the fault of JS devs, and it’s not a cultural thing. At first people tried to solve it with common CDNs and caching. Now people use tree-shaking, minification, compression etc, and many try pretty hard to reduce their bundle size.
I was thinking about Deno as well. The permission model is great. I’m less sure about URL-based dependencies. They’ve been intentionally avoiding package management altogether.
It’s at least interesting to consider that with deno, a package might opt to require limited access - and the installer/user might opt to invoke (a hypothetical js/deno powered dependency resolver/build system) with limited permissions. It won’t fix everything, but might at least make it easier for a package to avoid permissions it does not need?
hideous, I assume
apt
apt also has install scripts
with restrictions to ensure they are from a trusted source
You mean policy restrictions? Because that only applies if you don’t add any repos or install random downloaded Debs, both of which many routinely do
yeah
Yes, but when you use Debian you know packages go through some sort of review process.
I’ve had the same problem, so I’ve been working on a side project to help with this: xray.computer. It lets you view the published code of any npm package, and supports auto-formatting with Prettier for minified code. It’s not a panacea (sometimes, the published code is still unreadable, even when formatted) but it helps.
I also like to look up packages on Snyk Advisor to ensure that they get timely updates when vulnerabilities are found.
(Hopefully it’s okay to promote your own side projects here; please let me know if not.)
It is absolutely acceptable to mention your own side projects here, so long as it’s not spammy. We have the “show” tag for a reason. :)
Thanks for sharing your side-project! It is helpful to see the differences between versions. I can use it to see if a security issue was indeed solved between versions of a package. I’m also using Snyk to get an idea of certain packages.
However, I think my problem is a bit more complex: a WordPress plugin may contain hundreds of interdependent npm packages all neatly bundled and minified. Without access to a package.json or package-lock.json it is quite hard to find out which individual packages have been used. Quite often there is also no public repo available of the development files.
To give an example of my process thus far:
Someone in my team wants to see if we can use plugin X. I’m downloading the plugin to have a look at the code. Luckily this plugin has included a non-minified version of the js file. I can derive the use of npm packages from this file. Using Snyk I have a look at the first package mentioned. It’s axios. The included version is vulnerable (high & medium severity) and has been for almost a year (Note: the last version of the plugin is 3 months old and does not exclude this vulnerable version in it’s package.json which I found in a Github repo later on).
Since I have no package.json nor package-lock.json (all I have is the distributed build) I can’t easily update the npm package. I have no clue as to how this package relates to the other packages and how their version might depend on each other. Even if I would update the package, all other users of this plugin are still vulnerable. I contacted the plugin author. He tells me he will update the plugin as soon as possible. The plugin is (as of today) still not updated & has not released a new version. In the meantime there have been two new versions of the axios package released.
Every user of plugin X is still vulnerable to the issues mentioned on Snyk, but is this a real problem in this specific WordPress plugin context? I’m not sure how to interpret the high & medium severity in the context of this plugin. How exploitable are these issues & what is the impact of the exploits in the context of this plugin? Do I need to be a logged in user? Is this something which can be triggered by any visitor? What am I able to do when I can exploit these vulnerabilities? I can only try to find answers to these questions if I’m willing to invest a lot more time into this, which more or less beats the purpose of using a ‘ready-made’ WordPress plugin. And this is just one package of multiple npm packages used in this plugin. Packages which also have their own dependencies as well….
At this moment I’m wondering if any WordPress plugin using npm packages can be trusted at all.
ps: The way the npm ecosystem is structured is, in my view at least, problematic. Often packages are not like libraries as I’d expect, but look more like a function call or method call. I’d prefer to write these short pieces of code myself instead of depending on external code which also includes extra risks. The very rapid release schedules makes it even harder to trust external software (like a WordPress plugin) using npm packages as it seems they cannot keep up with it.
I’m sorry if this seems like a npm rant, but I’m seriously looking for methods on how to deal with these issues so we can use external software (like WordPress plugins) built with npm packages.
I think it’s perfectly reasonable to say no to this plugin.
I’d be pretty skeptical of any proprietary software that has npm dependencies but doesn’t include its package files. Just taking woocommerce (one of the more popular plugins) for example, they do include their package files in their source code. That ought to be the norm. When you have access to the package files, you can run the following to automatically install patches to vulnerable dependencies and subdependencies.
Without the package files, the plugin developer left you up a creek.
It really depends on the context. Some subdependencies are only used in the development or build of the dependency. If the vulnerability assumes the subdependency is running in a public node server, it’s probably not relevant to you. The hard part is knowing the difference. Take this vulnerability in glob-parent for example. I see it a lot in single page apps that have a webpack dev server. It’s hard to imagine how a DoS vulnerability is relevant when the only place the code is going to be run is your local development environment. In your case, it’s worth asking, is the axios vulnerability in the HTTP client or the server? If the vulnerability is just in the server and you’re not running a node server, you might be able to ignore it.
I agree that npm is problematic, but I have a slightly different take on it. There was a time when it was popular to mock projects like Bower for keeping front-end dependencies separate from node.js dependencies. The thinking was, why use two package managers when there’s one perfectly decent one? As it turns out, security vulnerabilities are impossible to assess without knowing in what environment the code is going to be run. I’m not suggesting we all go back to using Bower, but it would be nice if npm and its auditing tools would better distinguish between back-end, front-end, and build dependencies. As it is now, most npm users are being blasted with vulnerability warnings that probably aren’t relevant to what they’re doing.
I agree. I think this will be one of the main ‘rules’ for deciding if we want to use a particular WordPress plugin. Without the package.json or package-lock.json and npm audit it is near impossible to audit or update (fork) a plugin
You’ve hit the nail right on the head! And that’s what makes it hard to determine the impact: context.
As far as I can tell, determining context can only be done with (extensive) knowledge on how the npm package has been implemented in the WordPress plugin. It requires knowledge of the WordPress plugin, WordPress, the specific npm package, its dependencies and how all these relate to each other. Just like you said with the axios package.
Creating this context thus requires quite some time & skills, which would be fine if we would deal with just a few npm packages. However due to npm’s nature we usually have to deal with way more interdependent packages, which makes manually creating context near impossible. So even though there’s
npm auditwe’re still left with how to determine the context of the vulnerabilities found bynpm audit. And for this I have yet to find a solution.PS:
npm audit fixis in my view not the solution to solve this. It just hides the problems a bit better ;)Why not use a robots.txt? I am not a big fan of using meta-tags for this, and in the author’s sense, wasting my users’ bandwidth for such redundant tags. It also saves you from accidentally forgetting to add it to a document.
Hi, author here – Google stopped respecting robots.txt noindex a few years ago: https://developers.google.com/search/blog/2019/07/a-note-on-unsupported-rules-in-robotstxt
Combined with the fact that Google might list your site based only on third-party links, robots.txt isn’t an effective way to remove your site from Google’s results.
https://developers.google.com/search/docs/advanced/crawling/block-indexing#http-response-header indicates that
X-Robots-Tag: noindexas a HTTP header should have the same effect as the meta tag. It’s relatively easy to manage that per-server in the webserver config.That doesn’t save much on bandwidth but at least the “accidentally forgetting” issue is averted.
Today I learned. Thank you very much, Sir!
This is a useful distinction that helps clarify why we have faulty security expectations from module managers, IMO. We’re used to packages being vetted (to some extent) before they’re included in a program manager — you don’t run any great risks by running
apt-get install libxyz. In contrast, runningnpm install xyzis about as (in)secure as runningcurl randomdomain.com | bash, but most of us aren’t as cautious as we maybe should be when installing new/unknown modules.