Symbolic links. I’m not sure if it’s ever done, but technically you could have tag directories, and then symlink files into them? It would probably be a putid approach. More often, files are tagged, and the file-tag relations are stored in a database, and symlinks are used as described below.
I started doing this with my knowledge base! I have one Files folder that contains every single file, and everything outside that must be a symlink. It works really, really well. The main benefit is you can manipulate your taxonomy with ordinary shell scripting. Features I found really frustrating to do with a tag database got really easy with a symlinks:
If I want published to be a subtag of fiction, I just drop a symlink to published in the fiction folder. Then I can anything directly tagged fiction with ls tags/fiction and anything transitively tagged fiction with ls -R tags/fiction.
Smart tags are just tags with an associated “population” script. Like tags/csvs/populate.sh is just ls files/*.csv | xargs ln -s -t .¹
Actually the file is called files/populate_csvs.sh, but I can pick different names for symlinks so all the smart tags are populated by populate.
I can add tag autocompletion for add-tag --tag.
Searching for text across all files is just grep files/. Searching for text across all files tagged foo is just grep tags/foo/.
You can build more exotic relationships on top of this system, like Zettelkasten-style archive notes.
It’d probably work less well if you did it for your whole filesystem. It’s easier when it’s just a self-contained knowledge base.
¹ Prolly not exactly that, I’m translating from Powershell
This is not too dissimilar from the architecture of Nixos, where every meaningful configuration file in your system is stored with an opaque hash filename under the /nix directory and symlinked to a traditional linux file path, with those symlinks changing as required by system updates.
Hmm, that actually sounds quite functional. Thanks! Do you have some way of automatically updating the symlinks when files in the Files directory are renamed — or do you just avoid renaming the actual files as much as possible?
For now I’ve been avoiding renaming the actual files. I think at some point I’m going to replace all the filenames with 0001, 0002 etc and then create a second Names folder with one symlink for each file. Then if foobar.txt -> 0001 is in Names I’d make all the symlink tag names foobar.txt but have them actually link to 0001. I’m going to try to stay on “never rename files” for as long as possible though.
A lot of attention has fallen on symlinking files out of a Store a la Nix. In the last thread I mentioned git-annex, who’s index essentially provides this as a Content Addressed Store. It allows you to restructure your worktree into arbitrary hierarchies based on your tags with one-liners, as easily as if you were checking out a branch, and then to manipulate a file’s tags by just moving it around in these ephemeral tag trees! It fixes broken symlinks automatically, and even comes with a file watcher that can add, commit, and push in the background.
I mentioned some issues I had with storing large numbers of files in git, but also that there are config settings to address that and broader techniques that can be applied (it’d be kinda nice to represents directories as single store items without zipping them, accepting that their contents wouldn’t tagged or indexed in the same ways). One may also like to eg. ensure the daemon is always running and seeing everything, perhaps by pulling git-annex further down the software stack, towards the FUSE level that many existing options are operating at; I’m not sure what other benefits could be gained from that. I think git-annex offers a compelling model and existing implimentation that could be leveraged and adapted towards these use-cases, and do intend to revisit annexing my entire array’s worth of storage :p
I started doing this with my knowledge base! I have one
Filesfolder that contains every single file, and everything outside that must be a symlink. It works really, really well. The main benefit is you can manipulate your taxonomy with ordinary shell scripting. Features I found really frustrating to do with a tag database got really easy with a symlinks:publishedto be a subtag offiction, I just drop a symlink to published in the fiction folder. Then I can anything directly tagged fiction withls tags/fictionand anything transitively tagged fiction withls -R tags/fiction.tags/csvs/populate.shis justls files/*.csv | xargs ln -s -t .¹files/populate_csvs.sh, but I can pick different names for symlinks so all the smart tags are populated bypopulate.add-tag --tag.grep files/. Searching for text across all files tagged foo is justgrep tags/foo/.It’d probably work less well if you did it for your whole filesystem. It’s easier when it’s just a self-contained knowledge base.
¹ Prolly not exactly that, I’m translating from Powershell
This is not too dissimilar from the architecture of Nixos, where every meaningful configuration file in your system is stored with an opaque hash filename under the
/nixdirectory and symlinked to a traditional linux file path, with those symlinks changing as required by system updates.Hmm, that actually sounds quite functional. Thanks! Do you have some way of automatically updating the symlinks when files in the Files directory are renamed — or do you just avoid renaming the actual files as much as possible?
For now I’ve been avoiding renaming the actual files. I think at some point I’m going to replace all the filenames with
0001,0002etc and then create a secondNamesfolder with one symlink for each file. Then iffoobar.txt -> 0001is in Names I’d make all the symlink tag namesfoobar.txtbut have them actually link to0001. I’m going to try to stay on “never rename files” for as long as possible though.A lot of attention has fallen on symlinking files out of a Store a la Nix. In the last thread I mentioned git-annex, who’s index essentially provides this as a
Content Addressed Store. It allows you to restructure your worktree into arbitrary hierarchies based on your tags with one-liners, as easily as if you were checking out a branch, and then to manipulate a file’s tags by just moving it around in these ephemeral tag trees! It fixes broken symlinks automatically, and even comes with a file watcher that can add, commit, and push in the background.I mentioned some issues I had with storing large numbers of files in git, but also that there are config settings to address that and broader techniques that can be applied (it’d be kinda nice to represents directories as single store items without zipping them, accepting that their contents wouldn’t tagged or indexed in the same ways). One may also like to eg. ensure the daemon is always running and seeing everything, perhaps by pulling git-annex further down the software stack, towards the FUSE level that many existing options are operating at; I’m not sure what other benefits could be gained from that. I think git-annex offers a compelling model and existing implimentation that could be leveraged and adapted towards these use-cases, and do intend to revisit annexing my entire array’s worth of storage :p