Interestingly, this issue can be solved quite easily: Use a single file per task, using UUIDs, timestamps, or similar as the filename. Now conflicts only appear when two systems edit the same task, and if the listing of tasks is implemented via a simple ls, the task will simply appear twice, making conflict resolution easy. I shall call this system todos/
Or conflicts appear when one system edits and another deletes, or when both edit whatever metadata file holds the sorting… I’m sure there are more scenarios, distributed systems have lots of crazy corner cases.
Also, you now probably have performance problems because every operation is O(N) and involves a file read.
If my TODO list gets so long that opening and reading one file per entry takes more than 10ms on a modern machine, maybe I should prune the list rather than optimising the storage format. ;)
You could also store each property of a task (due date, description, …) into its own file to make conflict resolution even more fine-grained, but now sync takes a lot of syscalls, which may be a problem. Also with either of those alternatives just an editor wouldn’t suffice as UI anymore.
I think an easier solution would be to treat edits as an append-only log of changes tagged with timestamps or UUID2s, which can then be sorted when merging conflicts. Or, even better, each device can have its own log, and then we can merge them all as the files sync into a master log.
Of course that’s a preferred solution (also coincidetally the one I’m using for a toy task-management tool I call rtask), but with the constraint of being “easily editable in any text editor” I think that’s out of the question :)
Could this be solved with a vcs like git? I have Working Copy on my iphone, git on my desktop and laptop, and can run portable git on other computers; any time I make a change, I could “git commit && git push”.
Slightly more cumbersome than auto-saving to dropbox, but a lot more resistant to messing up than other methods.
I don’t really think git’s conflict resolution will work that much better on the todo.txt format in particular, though it’s certainly a step up in safety.
Tangential to your point, butI believe dropbox actually uses a standard vcs under the hood. Last time I checked it was Mercurial, though that’s just scuttlebutt I’ve read around the web.
Interestingly, this issue can be solved quite easily: Use a single file per task, using UUIDs, timestamps, or similar as the filename. Now conflicts only appear when two systems edit the same task, and if the listing of tasks is implemented via a simple
ls, the task will simply appear twice, making conflict resolution easy. I shall call this systemtodos/Or conflicts appear when one system edits and another deletes, or when both edit whatever metadata file holds the sorting… I’m sure there are more scenarios, distributed systems have lots of crazy corner cases.
Also, you now probably have performance problems because every operation is O(N) and involves a file read.
Yup, that’s absolutely right. I’m sorry if my comment indicated that I thought my suggestion would solve the distributed-system problem :-)
If my TODO list gets so long that opening and reading one file per entry takes more than 10ms on a modern machine, maybe I should prune the list rather than optimising the storage format. ;)
Well, any algorithm scales if I delete most of the data.
You could also store each property of a task (due date, description, …) into its own file to make conflict resolution even more fine-grained, but now sync takes a lot of syscalls, which may be a problem. Also with either of those alternatives just an editor wouldn’t suffice as UI anymore.
I think an easier solution would be to treat edits as an append-only log of changes tagged with timestamps or UUID2s, which can then be sorted when merging conflicts. Or, even better, each device can have its own log, and then we can merge them all as the files sync into a master log.
If you want a really good solution, look at taskwarrior: https://taskwarrior.org/docs/design/protocol.html
Of course that’s a preferred solution (also coincidetally the one I’m using for a toy task-management tool I call rtask), but with the constraint of being “easily editable in any text editor” I think that’s out of the question :)
Sure it is- that’s just a separate layer. You can stage changes by editing files locally.
Thanks for fleshing out your comment into a larger post.
Youre welcome, though it could probably have been just as short without loosing any content.
[Comment removed by author]
It sounds like the gripe here is dropbox’s conflict resolution (or, lack thereof).
Dropbox does the best it can do, because it has no knowledge of the file’s format.
Could this be solved with a vcs like git? I have Working Copy on my iphone, git on my desktop and laptop, and can run portable git on other computers; any time I make a change, I could “git commit && git push”.
Slightly more cumbersome than auto-saving to dropbox, but a lot more resistant to messing up than other methods.
I don’t really think git’s conflict resolution will work that much better on the todo.txt format in particular, though it’s certainly a step up in safety.
Tangential to your point, butI believe dropbox actually uses a standard vcs under the hood. Last time I checked it was Mercurial, though that’s just scuttlebutt I’ve read around the web.
So you didn’t consider how Dropbox works when choosing it to solve a problem. And now you know better. Good for you, I guess?
The bigger problem is specifically merging arbitrary data. AFAIK it’s an unsolved one.
I didn’t create todo.txt. And I probably wouldn’t have written this post if I didn’t encounter people who try to use Dropbox that way.
Not the author, me. I didn’t know. And it is good me me to know better rather than learn it through bitter experience, yes.