I love Syncthing. It’s totally obviated the need for connecting my phone or media players to my main desktop system - I just go through my normal workflow of downloading, transcoding, and labelling my music with MusicBrainz Picard and it automatically gets transferred to my mobile device once it’s dropped into my Music directory.
I used to have a music collection that was nearly all ripped from CDs, stored in FLAC and transcoded to MP3 for playing on my MP3 player. It was fantastic. Unfortunately, I didn’t have any backups of it, and reformatted the drive. By the time I had realised, it was basically unrecoverable. I couldn’t stomach doing it all over again, so I switched to Spotify.
But recently I’ve just become completely fed up with Spotify: they have a terrible Electron-like desktop programme and Spotify Connect constantly drops its connection. It’s just bad software. I’m ripping all my CDs and I didn’t realise how many of them aren’t on Spotify. I’d say a good half of the CDs aren’t on Spotify, and basically none of my DVDs are on NZ Netflix either.
One day: “it’s here! Buy shares of FirefoxCoin while they’re cheap, and use them as micropayments to the certain genre of sites that Pocket overwhelmingly features”
There has never been a similar behavior around smothering a tool in branding or upsells/spurious features in libre software. Systemd never happened, pulseaudio never happened, gnome3 never happened, firefox/pocket never happened, most client javascript frameworks never happened, etc.
I don’t disagree with you, but I think you’re missing a lot of the picture.
And honestly, if it weren’t for the continuous churn of developers looking to make a name for themselves or being bought into the zealotry, eventually the economics of libre might break down.
Yeah, I’m not saying that free software and capitalism are mutually exclusive; just that this problem arises from capitalism, and when it occurs in free software it’s specifically because free software has fallen under capitalism’s influence.
My machine uses systemd, but it’s completely invisible to me. When I performed the upgrade that brought it in, there were no bright banners heralding its arrival. The only example of a program I’ve actually used that’s done an upsell is Firefox/Pocket, which is a case where the SV startup mentality has been applied to a free software project and caused that ickiness to seep in.
I moved to syncthing as well after suffering from this. Too bad, iCloud could’ve been the simple solution when all you have is Apple devices.
When I went to open source >20 years ago, I did it mostly because the quality was higher already back then. Seems like even though the proprietary world has caught up some, this is still true.
I’ve come to realize that open source is the only type of software worth investing into. No matter how great a commercial piece of software might be, sooner or later it’s going to either disappear or change in a way that doesn’t suit you. Commercial software has to constantly chase profit for the company to stick around. This necessarily means that the product has to continue evolving to chase what’s currently in vogue. And if a company fails to do that, then it will die and the software will stopped being developed.
This is a bad situation to be in as a user since you have little control over the evolution of a product that you rely on. Instead of the product being adapted to your needs, it’s you who has to adapt to the way the product evolves, or spend the time investing in a different product.
On the other hand, open source has a very different dynamic. Projects can survive with little or no commercial incentive because they’re often developed by the users who themselves benefit from these projects. Projects can also be easily forked and taken in different directions by different groups of users. Even when projects become abandoned, they can be picked up again by new teams.
Evolution of GNOME is a great example of this. There are now many flavors of GNOME all catering to different workflows, and users don’t have to compromise their preferred way of doing things to chase how GNOME is evolving. Meanwhile, users of Windows or MacOS have very little choice but to continue adjusting to the ways Apple and Microsoft choose to evolve the desktop. Microsoft even uses DMCA to prevent users from doing customization.
Speaking as someone who abandoned the Microsoft stack for a much better development experience with free software, it’s worth remembering that free software has its own related failure mode:
I switched to Linux as my main computer at the almost same time as you did (if I recall it correctly) and never looked back.
I am using musl-linux + wayland on some of my computers which is little unstable combination with proprietary SW (firefox, chromium) and still, it is ten times better than anything I have ever used. Mostly for the reasons Nikita says in this article.
I use SyncThing and I love it, for pretty much all the reasons listed in the article. However, I could write almost the same article about Jitsi Meet versus Zoom for videoconferencing.
Zoom charges money for meetings, but if everybody had to pay money to use Zoom, nobody would. So Zoom has an artificial distinction between the meeting “host” and “attendees”. Meanwhile, Jitsi Meet is free, so every participant is equal.
I think Zoom charges more for longer meetings? So Zoom meetings have a rigidly-scheduled start time that you can’t join before, and when the host leaves all the attendees get kicked off. Meanwhile, Jitsi Meet lets people show up early and get their technical issues sorted out before the meeting starts, or hang around and socialise after the meeting is over
Maybe for technical reasons, maybe for branding reasons, Zoom has a special binary you have to download and run in order to participate. They do have a purely web-based version, but it’s fairly bare-bones. Meanwhile, Jitsi Meet’s web-based client is the primary client (there are native apps for Android and iOS), and it has all the conveniences I want.
I kind of roll my eyes whenever somebody sends me a Zoom meeting invite. I get that network effects are a thing and people have invested time and money into Zoom and it’s Good Enough, but Zoom’s deliberate anti-features frustrate me.
There’s always this sense of grandiose importance that a lot of end-user software has for itself. It’s like it can’t just do the damn thing it was supposed to do and stay out of the way. Instead, we get:
“MS Office has its own special updater that will bug you, is that okay?”
“Spotify has run automatically at startup, because it’s that important. Do you want to turn this off?”
“Dropbox needs to install a kernel extension despite working fine before”
“Please sign up for an account for which you will inevitably forget the password because you rarely need it”
Thus, when software appears to show even a lick of restraint, it ends up being this huge breath of fresh air from noxious cloud of “engagement.”
My use cases for Dropbox, in descending order of importance:
To ensure photos gets onto the machine which does backups.
To ensure that photos I take are visible to my partner, and likewise, with zero effort. They like to have a look at what I’ve taken recently because it’s fun. I only send photos directly when they are especially I think they will find them especially interesting / amusing.
To make files available on one device that I have on another.
To share large files with people, such as family and friends. Videos, etc.
Syncthing looks great for 3. but I don’t know of a better alternative for 1, 2 or 4, unfortunately.
There are plenty of photo sharing alternatives, but I want my photos to be backed up to several places (you know, actual backups, where you keep every states of the archive over time so that even if you accidentally move / delete / corrupt then you can go back).
There are large file sharing services but they don’t seem as under my control or permanent. I don’t do this enough to care much though.
My use cases for Dropbox are similar to yours, with the addition of Dropbox’s “smart sync”. My laptops have tiny hard drives, whereas my Dropbox is almost two terabytes. Tbh could be that Syncthing has a similar feature, I know at least Resilio Sync does.
I like syncthing, I have used it before, but I’m on an iPhone now and there’s no client for it, so, too bad.
But what I really wanna say is that I’m glad this is not yet another masturbatory nostalgic rant about how things where great when computers where just shitty terminals running BASIC, or shitty CPUs running DOS, and how the web ruined everything and javascript and npm are the literal antichrist.
I also looked for an iOS client for Syncthing, and didn’t find one. I had a look at the protocol specification, and it looked like it should be quite doable to implement it as a Files-compatible thing. If only I had unlimited time to work on side projects…
I love Syncthing, and using it across multiple devices, including Android phones, to sync most of my files.
For Android, I recommend (weirdly named) Syncthing-Fork, it’s some nice power saving features and sync conditions (e.g. so you can only sync select folders when you’re roaming).
The only thing that annoys me is that Android OS doesn’t support symlinks, so I have to maintain separate Syncthing directories for Photos/Pictures/Downloads, etc.
For sync problems: I have a script that’s doing ‘heartbeats’ every hour, merely by writing timestamp into a special file (e.g. /syncthing/.heartbeat/$DEVICE). This is running on every device (so each device has its own hearbeat file). In addition each device is checking for other devices’ heartbeats – if they fall behind (say) for a day, it shows error.
So far that’s been mostly overly paranoid, and on all occasions the problems weren’t with Syncthing itself, but with misconfiguration (e.g. firewall issues, wrong sync conditions on the phone, etc). But I feel like this kind of functionality could be helpful in Syncthing by default, maybe I should contribute it.
I love syncthing but it’s not fun to use in my use case (I haven’t found a solution yet, I suppose it can be solved).
I want to run it on FreeNAS (so I need to rely on the plugin being updated, but at least you can easily replace the binary with the latest FreeBSD release..) but also expose this via SMB to my windows machine where I don’t want to run the sync daemon when I can have it on a network share and I’m only really changing my password manager file anyway… But this seems to be hard. Well maybe I will try the Windows sync app again, I heard it has been improved a lot in the last years, that’s when I originally set up this weird construct :P
Why is serving over SMB hard? Because of some FreeNAS things?
On my Fedora NAS I run Syncthing, serve it over SMB, access with zero effort with my Live ID authenticated windows logon as a breeze. (Mostly, sometimes explorer doesn’t find the shares, must ping the server for explorer to attempt a conncetion. No idea what causes this)
While I too use syncthing, it still has to improve a bit. I have had notes in org-mode files killed without a trace or mbox file corrupted, and sometimes my computers don’t synchronise for days. Part of the reason is different versions running on various machines, but that’s unavoidable unless I have the same installation everywhere.
You talk about simplicity alot in this article, but to load it I had to download 12.9 MB worth of stuff. Surely a blog with a few screenshots does not need to consume that much bandwidth.
It’s images. It’s all images. There’s a 3MiB gif and a bunch of smaller PNGs that add up to that much data. A 1200x848 gif of that many frames is apparently 3MiB.
By the website you mean Syncthing web interface (it runs locally)?
But it’s actually not displaying the conflicts anywhere (unfortunately), so I’ve got an external script that checks for conflicted files every hour and shows a notification if there are any. Perhaps I should contribute and make it the default behaviour.
I love Syncthing. It’s totally obviated the need for connecting my phone or media players to my main desktop system - I just go through my normal workflow of downloading, transcoding, and labelling my music with MusicBrainz Picard and it automatically gets transferred to my mobile device once it’s dropped into my Music directory.
Warms my heart to see someone that still maintains a local music collection, all the way down to transcoding and tagging.
Yeah, the stuff I listen to isn’t really on the big streaming services, lol
Yes, we exist! I use beets.io to autotag my new music (either from bandcamp or deemix) and it gets automatically synced with my phone with syncthing.
I used to have a music collection that was nearly all ripped from CDs, stored in FLAC and transcoded to MP3 for playing on my MP3 player. It was fantastic. Unfortunately, I didn’t have any backups of it, and reformatted the drive. By the time I had realised, it was basically unrecoverable. I couldn’t stomach doing it all over again, so I switched to Spotify.
But recently I’ve just become completely fed up with Spotify: they have a terrible Electron-like desktop programme and Spotify Connect constantly drops its connection. It’s just bad software. I’m ripping all my CDs and I didn’t realise how many of them aren’t on Spotify. I’d say a good half of the CDs aren’t on Spotify, and basically none of my DVDs are on NZ Netflix either.
tl;dr: Dropbox & iCloud are proprietary bloatware, but syncthing isn’t.
tl;dr: capitalism takes things that could be simple/straightforward and smothers them in branding and upsells
looking at you, Firefox
One day: “it’s here! Buy shares of FirefoxCoin while they’re cheap, and use them as micropayments to the certain genre of sites that Pocket overwhelmingly features”
you just gave some precocious PM at Firefox an idea, goddammit
Brave already exists.
There has never been a similar behavior around smothering a tool in branding or upsells/spurious features in libre software. Systemd never happened, pulseaudio never happened, gnome3 never happened, firefox/pocket never happened, most client javascript frameworks never happened, etc.
I don’t disagree with you, but I think you’re missing a lot of the picture.
And honestly, if it weren’t for the continuous churn of developers looking to make a name for themselves or being bought into the zealotry, eventually the economics of libre might break down.
Yeah, I’m not saying that free software and capitalism are mutually exclusive; just that this problem arises from capitalism, and when it occurs in free software it’s specifically because free software has fallen under capitalism’s influence.
My machine uses systemd, but it’s completely invisible to me. When I performed the upgrade that brought it in, there were no bright banners heralding its arrival. The only example of a program I’ve actually used that’s done an upsell is Firefox/Pocket, which is a case where the SV startup mentality has been applied to a free software project and caused that ickiness to seep in.
Not only that, iCloud frequently fails to do the basic thing of exchanging files it’s supposed to :-|
https://discussions.apple.com/thread/8096570
I moved to syncthing as well after suffering from this. Too bad, iCloud could’ve been the simple solution when all you have is Apple devices.
When I went to open source >20 years ago, I did it mostly because the quality was higher already back then. Seems like even though the proprietary world has caught up some, this is still true.
Step three is pairing your devices. Step four is to setup autostart.
SyncThing is great. No need to oversell it.
Step 5: build and maintain a server for syncthing so that the risk of offline devices not catching all the changes is lowered. (Optional I guess).
And then there’s punching holes through networks where you don’t have a peer but want to sync from an Internet server
It’s not a problem in Syncthing, there’s always a pool of public relays
A-ha.
So, how does that effect possible unintended 3rd party storage of your backups?
It should still be impossible, the relays don’t actually store any data, just act as introducers for your data nodes
I may be misunderstanding, but don’t they still need to be active to relay the actual information?
the information they relay is a TLS ciphertext stream, authenticated by the device IDs which are actually sha-256 fingerprints of the TLS certificates
Ah, that’s much more reassuring. Thank you! :)
That must have changed since I last used it. Thanks!
which is only two commands away if you have Arch lying around:
I’ve come to realize that open source is the only type of software worth investing into. No matter how great a commercial piece of software might be, sooner or later it’s going to either disappear or change in a way that doesn’t suit you. Commercial software has to constantly chase profit for the company to stick around. This necessarily means that the product has to continue evolving to chase what’s currently in vogue. And if a company fails to do that, then it will die and the software will stopped being developed.
This is a bad situation to be in as a user since you have little control over the evolution of a product that you rely on. Instead of the product being adapted to your needs, it’s you who has to adapt to the way the product evolves, or spend the time investing in a different product.
On the other hand, open source has a very different dynamic. Projects can survive with little or no commercial incentive because they’re often developed by the users who themselves benefit from these projects. Projects can also be easily forked and taken in different directions by different groups of users. Even when projects become abandoned, they can be picked up again by new teams.
Evolution of GNOME is a great example of this. There are now many flavors of GNOME all catering to different workflows, and users don’t have to compromise their preferred way of doing things to chase how GNOME is evolving. Meanwhile, users of Windows or MacOS have very little choice but to continue adjusting to the ways Apple and Microsoft choose to evolve the desktop. Microsoft even uses DMCA to prevent users from doing customization.
Speaking as someone who abandoned the Microsoft stack for a much better development experience with free software, it’s worth remembering that free software has its own related failure mode:
https://www.jwz.org/doc/cadt.html
I mean, that failure mode is alive and well at the last place we both worked - hardly a problem restricted to OSS.
Ah that’s subtler - CADPM (Cascade of Attention-Deficit Product Managers) ;-P
Seriously though yes, it’s certainly not unique to free software.
I switched to Linux as my main computer at the almost same time as you did (if I recall it correctly) and never looked back.
I am using musl-linux + wayland on some of my computers which is little unstable combination with proprietary SW (firefox, chromium) and still, it is ten times better than anything I have ever used. Mostly for the reasons Nikita says in this article.
This article is the most refreshing read about computers since pandemic started. Great Nikita!
So I went and installed the Syncthing, and the one thing, that struck me is how good it is localized. On top of all the stuff, Nikita wrote.
I use SyncThing and I love it, for pretty much all the reasons listed in the article. However, I could write almost the same article about Jitsi Meet versus Zoom for videoconferencing.
I kind of roll my eyes whenever somebody sends me a Zoom meeting invite. I get that network effects are a thing and people have invested time and money into Zoom and it’s Good Enough, but Zoom’s deliberate anti-features frustrate me.
This definitely resonates with me.
There’s always this sense of grandiose importance that a lot of end-user software has for itself. It’s like it can’t just do the damn thing it was supposed to do and stay out of the way. Instead, we get:
Thus, when software appears to show even a lick of restraint, it ends up being this huge breath of fresh air from noxious cloud of “engagement.”
My use cases for Dropbox, in descending order of importance:
Syncthing looks great for 3. but I don’t know of a better alternative for 1, 2 or 4, unfortunately.
There are plenty of photo sharing alternatives, but I want my photos to be backed up to several places (you know, actual backups, where you keep every states of the archive over time so that even if you accidentally move / delete / corrupt then you can go back).
There are large file sharing services but they don’t seem as under my control or permanent. I don’t do this enough to care much though.
I think nextcloud can be used for the rest, but I’m not entirely happy with the code quality of nextcloud. They seem to be chasing features.
My use cases for Dropbox are similar to yours, with the addition of Dropbox’s “smart sync”. My laptops have tiny hard drives, whereas my Dropbox is almost two terabytes. Tbh could be that Syncthing has a similar feature, I know at least Resilio Sync does.
I like syncthing, I have used it before, but I’m on an iPhone now and there’s no client for it, so, too bad.
But what I really wanna say is that I’m glad this is not yet another masturbatory nostalgic rant about how things where great when computers where just shitty terminals running BASIC, or shitty CPUs running DOS, and how the web ruined everything and javascript and npm are the literal antichrist.
I also looked for an iOS client for Syncthing, and didn’t find one. I had a look at the protocol specification, and it looked like it should be quite doable to implement it as a Files-compatible thing. If only I had unlimited time to work on side projects…
Does syncthing support xattr? That is a big drawback of Dropbox, if you want to build a persistent tagging system for search
I love Syncthing, and using it across multiple devices, including Android phones, to sync most of my files.
For Android, I recommend (weirdly named) Syncthing-Fork, it’s some nice power saving features and sync conditions (e.g. so you can only sync select folders when you’re roaming). The only thing that annoys me is that Android OS doesn’t support symlinks, so I have to maintain separate Syncthing directories for Photos/Pictures/Downloads, etc.
For sync problems: I have a script that’s doing ‘heartbeats’ every hour, merely by writing timestamp into a special file (e.g.
/syncthing/.heartbeat/$DEVICE). This is running on every device (so each device has its own hearbeat file). In addition each device is checking for other devices’ heartbeats – if they fall behind (say) for a day, it shows error. So far that’s been mostly overly paranoid, and on all occasions the problems weren’t with Syncthing itself, but with misconfiguration (e.g. firewall issues, wrong sync conditions on the phone, etc). But I feel like this kind of functionality could be helpful in Syncthing by default, maybe I should contribute it.Even if it doesn’t get merged, publishing your script somewhere publicly visible could prove useful to someone!
I love syncthing but it’s not fun to use in my use case (I haven’t found a solution yet, I suppose it can be solved).
I want to run it on FreeNAS (so I need to rely on the plugin being updated, but at least you can easily replace the binary with the latest FreeBSD release..) but also expose this via SMB to my windows machine where I don’t want to run the sync daemon when I can have it on a network share and I’m only really changing my password manager file anyway… But this seems to be hard. Well maybe I will try the Windows sync app again, I heard it has been improved a lot in the last years, that’s when I originally set up this weird construct :P
Why is serving over SMB hard? Because of some FreeNAS things?
On my Fedora NAS I run Syncthing, serve it over SMB, access with zero effort with my Live ID authenticated windows logon as a breeze. (Mostly, sometimes explorer doesn’t find the shares, must ping the server for explorer to attempt a conncetion. No idea what causes this)
Not the serving but accessing the same files via SMB and locally, and their windows/unix volumes with ACLs.
While I too use syncthing, it still has to improve a bit. I have had notes in org-mode files killed without a trace or mbox file corrupted, and sometimes my computers don’t synchronise for days. Part of the reason is different versions running on various machines, but that’s unavoidable unless I have the same installation everywhere.
The lack of synced ignore files bothers me most. Luckily I had no data loss yet.
You talk about simplicity alot in this article, but to load it I had to download 12.9 MB worth of stuff. Surely a blog with a few screenshots does not need to consume that much bandwidth.
It’s images. It’s all images. There’s a 3MiB gif and a bunch of smaller PNGs that add up to that much data. A 1200x848 gif of that many frames is apparently 3MiB.
What happens if
Then you have a conflict. The conflict version is local to each machine and has to be fixed locally.
So don’t do that. It’s not magic.
Thanks. Does that mean you have to periodically check the website to ensure there are no (unintended) conflicts?
By the website you mean Syncthing web interface (it runs locally)?
But it’s actually not displaying the conflicts anywhere (unfortunately), so I’ve got an external script that checks for conflicted files every hour and shows a notification if there are any. Perhaps I should contribute and make it the default behaviour.
Yes, by “website” I meant the Syncthing web interface.
Not having conflict notifications is pretty much a bummer for me.