I find it a stretch to call these packages malicious. Copyright violation? Sure. Terms and Conditions violation? Sure. But finding some dev’s stash of movies being stored in NPM hardly seems like “this package is inherently malicious.”
It is very much an abuse of a service that is provided for the good of everyone. npm is there to store JS code, not as your personal backup archive. It’s malicious towards npm and the people who use it, even if not actually out to harm others directly.
It’s malicious towards npm and the people who use it, even if not actually out to harm others directly.
I have no problem with your calling it very much an abuse of the service, but this definition of “malicious” seems confusingly divorced from the base word “malice”, which is the intent to cause harm.
The definition of maliciousness includes deliberate harm. I see nothing about this which was aimed at deliberate harm towards NPM. If anything, this article reads like security vendors playing up risk (the same as inflating CVEs) to sell their product. For example, the article uses the term “flooded”, when 748 packages is maybe 6 hundredths of one percent of the total packages [1]. That wasn’t flooded.
I could see an argument that the size of the packages was an issue, since all told, this is roughly 40GB of data. But that is a drop in the bucket for a service that moves about 2GB per second over the wire and I highly doubt if this was bumping Microsoft up against their storage capacity it would have taken two months to discover. Nor would they be taking so long to delete all 748 of them.
So that’s why I say this is not malicious. There’s no apparent intention to damage other NPM users (i.e. publishing malicious code, cryptominers, etc), or the service (making one package to rule them all, so to speak). It looks like someone wanted to be able to watch videos at work.
What I would say is malicious would be if someone read this article, then published a package which pulled those down. 40GB isn’t much to NPM, but having that accidentally included in your build pipeline might be blow up some of your CI/CD workers and jack up bandwidth costs pretty quickly.
[1] I don’t have current numbers for the total number of NPM packages and sizes, just went back to find the most recent announcement I could. At that time, 1.3 million packages were 6 PT of bandwidth monthly.
Edit: And to be clear, I agree this is an abuse of the service. I just disagree with labeling it malicious.
In case anyone is curious about the content, these are episodes from one of the most successful mainland Chinese TV shows of all time, “My Own Swordsman” (武林外传, Wulin Waizhuan, hence the “wlwz” initials). Interestingly, English Wikipedia doesn’t have an article for the show, only for the later film adaptation. Another commenter is correct in identifying the JSON as containing strings from the bilibili commentary overlay, where users can submit comments that scroll across the screen at certain timestamps for other users to read. This style of commentary overlay is popular on most large Chinese video content platforms.
This is the cost of offering ‘free’ services. Microsoft’s npm has a long history of various kinds of abuse, but it’s probably willing to tolerate waves of abuse to keep it’s centralized, for-profit position as the JavaScript package manager. If npm Inc. put up bigger hurdles towards joining/pushing that could prevent some of this abuse, folks would realize they could just host their own packages instead of using their service. Package registries ran by communities & foundations do a better job regulating what goes in & what doesn’t.
Package registries ran by communities & foundations do a better job regulating what goes in & what doesn’t.
[citation needed]? I’m not saying it’s definitely wrong, but I don’t see substantially more barriers to this kind of thing on e.g. PyPI. The main difference I see is that npm seems to have far more traffic (not that PyPI is a ghost town or anything, far from it, just that JavaScript is a really popular language and npm is really big).
Within each package are video clips ending in the ‘.ts’ extension, which indicates these were ripped from DVDs and Blu-ray discs.
Seems to be an incorrect assumption. .ts has nice properties for segmented video, like in this case.
edit: And as it came to mind, .ts is also used in HLS streaming if I remember correctly?
Later there is evidence of commentary (Incorrectly assumed to be subtitles) pulled from Bilibili, additionally searching the file name on google points to a similar conclusion.
I would suspect that the videos were perhaps ripped from premium videos on Bilibili. (Though I do not know if Bilibili does such a thing, but Youtube does so I am assuming Bilibili does also.)
Bit disappointing to seemingly only research .ts but not any other string that showed up.
This is why centralised package repositories are a bad idea. I can’t see much excuse for them for any new language ecosystem, but we still see people making new languages designing them with centralised package repositories and repeating the mistake.
I find it a stretch to call these packages malicious. Copyright violation? Sure. Terms and Conditions violation? Sure. But finding some dev’s stash of movies being stored in NPM hardly seems like “this package is inherently malicious.”
It is very much an abuse of a service that is provided for the good of everyone. npm is there to store JS code, not as your personal backup archive. It’s malicious towards npm and the people who use it, even if not actually out to harm others directly.
I have no problem with your calling it very much an abuse of the service, but this definition of “malicious” seems confusingly divorced from the base word “malice”, which is the intent to cause harm.
The definition of maliciousness includes deliberate harm. I see nothing about this which was aimed at deliberate harm towards NPM. If anything, this article reads like security vendors playing up risk (the same as inflating CVEs) to sell their product. For example, the article uses the term “flooded”, when 748 packages is maybe 6 hundredths of one percent of the total packages [1]. That wasn’t flooded.
I could see an argument that the size of the packages was an issue, since all told, this is roughly 40GB of data. But that is a drop in the bucket for a service that moves about 2GB per second over the wire and I highly doubt if this was bumping Microsoft up against their storage capacity it would have taken two months to discover. Nor would they be taking so long to delete all 748 of them.
So that’s why I say this is not malicious. There’s no apparent intention to damage other NPM users (i.e. publishing malicious code, cryptominers, etc), or the service (making one package to rule them all, so to speak). It looks like someone wanted to be able to watch videos at work.
What I would say is malicious would be if someone read this article, then published a package which pulled those down. 40GB isn’t much to NPM, but having that accidentally included in your build pipeline might be blow up some of your CI/CD workers and jack up bandwidth costs pretty quickly.
[1] I don’t have current numbers for the total number of NPM packages and sizes, just went back to find the most recent announcement I could. At that time, 1.3 million packages were 6 PT of bandwidth monthly.
Edit: And to be clear, I agree this is an abuse of the service. I just disagree with labeling it malicious.
[Comment removed by author]
In case anyone is curious about the content, these are episodes from one of the most successful mainland Chinese TV shows of all time, “My Own Swordsman” (武林外传, Wulin Waizhuan, hence the “wlwz” initials). Interestingly, English Wikipedia doesn’t have an article for the show, only for the later film adaptation. Another commenter is correct in identifying the JSON as containing strings from the bilibili commentary overlay, where users can submit comments that scroll across the screen at certain timestamps for other users to read. This style of commentary overlay is popular on most large Chinese video content platforms.
Danmu is a interesting concept, I don’t think I have seen it on any western site yet: https://en.wikipedia.org/wiki/Danmaku_subtitling
That’s really interesting. Pretty similar to a Twitch/Live stream chat overlay in concept
SoundCloud has had this for a long time and I really like it
This is the cost of offering ‘free’ services. Microsoft’s npm has a long history of various kinds of abuse, but it’s probably willing to tolerate waves of abuse to keep it’s centralized, for-profit position as the JavaScript package manager. If npm Inc. put up bigger hurdles towards joining/pushing that could prevent some of this abuse, folks would realize they could just host their own packages instead of using their service. Package registries ran by communities & foundations do a better job regulating what goes in & what doesn’t.
[citation needed]? I’m not saying it’s definitely wrong, but I don’t see substantially more barriers to this kind of thing on e.g. PyPI. The main difference I see is that npm seems to have far more traffic (not that PyPI is a ghost town or anything, far from it, just that JavaScript is a really popular language and npm is really big).
npm isn’t “Microsoft’s”.
yeah to be precise it’s Microsoft’s GitHub’s NPM
Seems to be an incorrect assumption.
.tshas nice properties for segmented video, like in this case.edit: And as it came to mind,
.tsis also used in HLS streaming if I remember correctly?Later there is evidence of commentary (Incorrectly assumed to be subtitles) pulled from Bilibili, additionally searching the file name on google points to a similar conclusion.
I would suspect that the videos were perhaps ripped from premium videos on Bilibili. (Though I do not know if Bilibili does such a thing, but Youtube does so I am assuming Bilibili does also.)
Bit disappointing to seemingly only research
.tsbut not any other string that showed up.People’s ingenuity in seeking out and exploiting unmetered network storage should not be underestimated.
I store my movies here. I just lose a little bit of entropy by having to make each file segment into something that looks like a bona fide comment.
This is why centralised package repositories are a bad idea. I can’t see much excuse for them for any new language ecosystem, but we still see people making new languages designing them with centralised package repositories and repeating the mistake.
Go got this right.