Hello fellow crustaceans,
I’ve been considering a personal wiki for quite sometime and I’d like to put something together to start the new year. What I’m looking for is something to run on my little DigitalOcean personal instance (OpenBSD 6.2) where I can connect remotely, either via HTTP over SSH or HTTPS (basically, only for me + my devices). The idea is to store everything I can think of here… anti-theft code for my Honda radio, previous cell phone models, instruction for how to do that $thing, wines I like, what size cut off wheels the grinder takes… you get the idea :)
Right now I’m leaning toward MediaWiki with a SQLite or maybe MariaDB backing, it’s a little heavy and I’m not thrilled about the extras I’d probably need (ImageMagik) but since it’s not directly Internet facing it’s less of a concern.
I am interested if anyone else has attempted this before or if anyone has suggestions for alternative wiki software (I’m curious about TiddlyWiki). Basically I want wiki edit-ability, ability to upload graphics / files, etc.
Thanks!
I have had a personal MediaWiki/[mysql|mariadb] setup going since 2005. I honestly don’t know how I would live without it. pretty much every little side project, OS install, ROM build, woodworking project, recipe, brewing protocol, cocktail recipe, knitting pattern, and discography listen-through have been documented there. I doubt anyone other than me could possibly navigate it, but that seems only natural. the only headaches through the years have been the half dozen badly-broken upgrade nightmares I have encountered. at least twice I have had to restart and restore from backup via some very one-off perl scripts. still, it beats the shelves of notebooks that came before.
Yeah this is making me feel old, but I have had a personal wiki since late 2004… I wrote it myself in Python and I still use it every day. I just checked and there are 2,737 pages, which is a rate of 1 page created every 1.73 days. Of course I edit pages more than creating them.
All my work notes, side project ideas and notes, notes on videos and books that I’ve finished, that I want to check out, tax info, etc. goes there.
I’m a bit of an information hoarder but it doesn’t seem to be a bad thing on balance.
I think Saturday morning cartoons taught me knowledge is power, so I don’t see hoarding as a problem. Especially if it’s all organised. Can you shed some light on the details/ architecture of your solution?
Sure, I think it was a really helpful project, although as mentioned it’s old.
The code started out as a fork of a 50 or 100 line CGI script I found on the web. It was a toy wiki backed by the file system. Back in 2004, I ran it on shared hosting with a shared SSL certificate! I had never done any web programming before – so this was my introduction to the web!
It has evolved a lot since then, and right now it’s about ~7200 lines of Python, which includes a Python web framework I wrote. It’s now backed by sqlite, but not in a particularly nice way. It has a tiny bit of JavaScript that I also wrote myself. It is now a WSGI program, which even runs under this web server I wrote (which is not necessarily a good idea – I’ve used it as a testbed for some experiments in web infrastructure.)
I try to iterate on what I need personally. It never seems like enough, but I like using my own code, even if the UI is crappy. I think that low latency is more important than bells and whistles on the UI – and I consciously write it for low latency, which is not hard at all if you’re not using any third party libraries. It should be 50ms or less for every page load – if it’s not, there’s a bug somewhere.
I even wrote my own markup syntax – back in 2004 that was perhaps somewhat reasonable. Now I wish it was markdown, but I didn’t even start using markdown heavily until a year or two ago.
I most often use it as a bookmark manager. I have browser bookmark like this:
And then that opens up a form with a section for notes. And then I can type notes and append it to a given wiki page.
The wiki is very messy, but I try to refactor it from time to time. Hyperlinks are exactly the right abstraction for notes IMO. Some people use flat text files in vim or emacs, but I think hyperlinks are essential. When you have thousands of pages, that’s the only reasonable organization mechanism for notes. (I think emacs org-mode might have hyperlinks, but I’ve never used it.)
It is honestly a great memory augmentation device. I often look back at old projects and I’m shocked at how much I forgot and how much I’ve learned since then! (And most people think I have a good memory in general.)
Another note: I have a cron job that syncs the sqlite database and checks plain text into an hg rep every night. So I have a coarse history by day. I don’t have a history of every edit, and I don’t think that’s necessary anyway.
Awesome! That’s basically what I’m going for, a catch all for every little morsel of information. Do you host your own setup?
yep. originally ran on whatever desktop I had, but now lives on a dedicated little box I use for one-off stuff too custom or weird for freenas plugins.
I did this once. After some research I ended up using DokuWiki. It worked pretty well, I stuck with it for a few years. I liked how the database was just the filesystem, my backups were just tarballs of the data directory.
My main gripe, and the reason why I’ve moved over to google keep + workflowy, is that it was difficult to edit pages while using my phone.
That’s a good point about editing it on mobile, that might be a sore spot.
I had the same problem recently, I got pretty annoyed by the editor on my iPhone’s Safari so I’ve started a side project recently to sync Markdown on Dropbox into DokuWiki (https://github.com/milanaleksic/notesforlife). There are many very good mobile Markdown editors that can store into Dropbox… Now I’m happy since dokuwiki backup is trivial, it has search… everything I need
While I have personally not used it, is this not something orgmode (emacs) does?
Org could be one component of a solution for this, but on its own it lacks: a way to edit via mobile/other devices, any means of uploading images, a blessed rendering path (there are many ways to render/export org files into something for display).
For instance, one solution might be to use Org’s “publish” feature. You could render to HTML, push that to some web host somewhere with rsync (that handles viewing on other/mobile devices). For editing you could sync your org source files (and any org-rendered images via things like plantuml, as well as static images) with something like syncthing/git/Dropbox/Box/iCloud/OneDrive etc. in combination with a non-Emacs editing app like Beorg (iOS) or Orgzly (Android).
That would be a workable and powerful system, but I think we have to admit it’s not as simple to use as just clicking “edit” in a wiki page from something like dokuwiki/mediawiki :-)
I’ve found I don’t do any significant note editing on the phone - just capture.
So I use Google Photos + Orgzly + Syncthing + emacs. It used to be MobileOrg, and I started with org ~2005, so these files got bones.
I have been looking for something like beorg for a long time. Thanks!!
I love orgmode and use it on and off but last I looked sharing it was read-only and meant exporting the static document or running something (node, ruby) that parses the format on the fly.
OneNote onto a self-hosted SharePoint site with the Office Web Apps installed. Arguably overkill, but OneNote’s client does a lot of things right and OWA is handy if I don’t have OneNote on a desktop. (I believe any normal WebDAV site would work for sync. Also theoretically possible, is that LibreOffice Online connected to NextCloud uses the same protocol Office Web Apps does for connecting to SharePoint - see where I’m going with this?) The problem is the mobile clients want 365, which I don’t have or really want, since I’m not using MS infra and use perpetually licensed desktop Office…
MediaWiki I unfortunately have too much experience with - it’ll work, but you don’t want to see how the sausage is made.
edit: Another option is org-mode, plus sync and mobile clients. If you like Emacs, this is pretty great. If you’re like me and aren’t too hot for Emacs, not very good.
I use vim plus SyncThing. My wiki (and journal) contains sensitive stuff, so I avoid storing the data on publicly accessible servers. Also, no maintenance of some database.
The obvious downside of vim is the lack of rich text, especially I cannot insert graphics or other files.
I am a vim aficionado myself, but for this usecase I bit the bullet and use Emacs (+evil) with org-mode, it supports links, tables and graphics.
I do similar, but with SpiderOak for sync. I’m using vimwiki set up so wiki 1 is always machine-local and wiki 2 is global. I also have a variant of this config to let me open/edit
.gpg.wikifiles, for content I’d like that extra layer on. I just include the.gpgin the page path and vimwiki happily plays along. Vimwiki is a little more close at hand than web-based wikis for me – I especially love the quick diary shortcut.Buuuut it sounds like the OP is looking for something prettier and with seamless image inclusion. With vimwiki, file attachments are just relative links to stuff I’ve dropped in the sync dir, and you can include images in the markup, then have vimwiki set to regenerate HTML copies of your pages on write, and view the pretty version through those. I haven’t even touched the HTML features, personally, just mentioning that’s about the best it can do in this area.
I use the ZIM Desktop Wiki (http://zim-wiki.org/) for years now. Pretty good edit capabilities including images, tables and you can easily export HTML.
To sync between multiple machines I put all Notebooks into a git repo synced via SSH with my personal server.
Zim is really awesome, I’m using it both as a personal Wiki and as a Wiki for the dev team at the studio. The main drawback for me is that the markup is not markdown and that the default styling is a bit uninspiring. Before that, I was using Tomboy.
Right, the default style is ugly. I fixed this by adding a customized style.conf.
I’d highly recommend MoinMoin. I’ve been running Moin for over a decade as my personal wiki and it works spectacularly well.
When I was looking for a wiki the three primary considerations were cost of maintenance (must not require constant maintenance, be easy to setup, easy to backup), ease of use (from the UI), and extensibility (so that I could work my way around any deficiencies in the software). Moin did well on all of these critera. It’s a single standalone app that stores everything as files on the disk, no db server. The file layout is logical and easy to browse with
lsif you’re so inclined, no proprietary formats, and it’s easy to backup with a simple rsync or tar command. It’s got a decent web UI and a lot of macros that make information organization easy. It’s also got an API that works well, and for which I’ve built integration with VIM. It’s also written in Python which makes it easy to install in a virtualenv (no root access necessary) for isolation and it’s designed to be extended (though this can range from very easy to kinda painful depending on what you’re extending). It can also use the Python Xapian extension for fast full-test indexing.It’s also used by quite a few open source projects like Debian, Python, and GNOME to name a few.
Something worth checking during your wiki-hopping: table support. Whilst many backends support tables, some have non-simple syntax and others don’t support multi-line text inside table cells. I discovered this during a few wiki-hops.
Some random notes:
ikiwiki https://ikiwiki.info/
I gave up trying to install this on my first attempt after several hundred MB of perl deps.
The default page template (HTML/CSS structure) is much more complicated than it needs to be (ridiculous “IF HTML5” for every single tag, IIRC); I ended up writing my own much simpler one before even attempting to make some CSS.
pmwiki http://www.pmwiki.org/
Stores pages as plaintext by default BUT in a ‘all edit history merged into one file’ format. In other words the files look like crazy nested diff’s.
foswiki http://foswiki.org/
Looks pretty, comes with an instant-launch script. Uploading images is as simple as drag+drop onto the page whilst editing! (you need to enable the WYSIWYG editor for this).
Have not dabbled deeper than this. I’m surprised I have never read of this wiki anywhere on the web, it looks like a big project.
I use Gollum, which uses git for its database and it’s the same wiki GitHub uses.
I use it, or used it. It supports Markdown editing, creation/editing in wiki, file upload (well, I’m 90%).
Any particular reason for not using it anymore?
I’m am using DokuWiki which I host on a small Digital Ocean box. $5 a month for a personal wiki I can access from any machine. Seems good enough for my purposes and it’s possible to secure it well.
I use Gollum, the same wiki as GitHub. I chose it because it is git backed and uses Markdown formatting. https://github.com/gollum/gollum/blob/master/README.md
I’ve been very pleased with DokuWiki, despite my (hypocritical, see below) tendency to complain about the install instructions for different distributions and operating systems. So far my favorite setup has been DokuWiki running on OpenBSD. However, I completely failed to document the process and so I’m going to have to sit down and run through it again to contribute to the OpenBSD guide on the DW site.
Nth-ing dokuwiki, as setup is really simple and the content is stored as plain files.
One Note is also a great product for personal notebooks.
I’ve been using TiddlyWiki for years (spot checking… I have nearly a thousand entries going back to 2006). It used to be a single page javascript-based app and I kept the html file synced, now it’s also capable of running as a node.js based app. There are trade-offs for both but the node-based version is very convenient after you set it running behind nginx or such to handle security.
https://tiddlywiki.com/
I like gitit, but don’t use it for everything. I write notes for current projects and such in org-mode, and then if I think it will be of interest to me later (some new technique or tech I’ve learned, for example) I will write a short summary in the wiki or add to a previously-existing page.
Thanks to everyone that left a comment, this is amazing! I decided to start with Dokuwiki and see how that goes.
I use dokuwiki myself for long term storage, and google keep while mobile. Have a script set up to query google keep and add the content to my dokuwiki instance every night.
This seems like a nice solution… mind sharing the script somewhere?
Maybe take a look at Perkeep (né Camlistore)?
I’d love perkeep to support that, but it definitely doesn’t right now. Wouldn’t be that hard to add though…
I use a local install of MediaWiki with MariaDB and it’s much more useful than any note taking software I tried before.
Can anyone give me some examples of the sort of things you write on a personal wiki? I’ve always enjoyed the idea of having one, but at the same time I can’t maintain one for a long time. I feel like it’s too much effort, and that I can find whatever I want using Google anyway.
I’m very happy with Ōlelo (demo website), which is an an active fork of the old git-wiki. For quick things I use TiddlyWiki or the Seafile embedded wiki.
I started with a MediaWiki instance years ago but I had 2 problems: the fact that I started wanting simpler, text-file based storage (it doesn’t scale, but it’s nicer and good enough for personal content), and the fact that I wanted support for “metadata”… stuff like “tag my ‘project’ pages and then write a query that lists those pages in an index page”, which led me to SemanticMediaWiki and a bunch of other plugins.
Eventually, I started writing my own wiki engine (don’t we all?), based on text files (Markdown + some templating/front-matter/etc.), Mercurial or Git storage for revision control, some basic user permission support, etc. Similar to a bunch of other similar things, basically, but with just a few different or additional things that make it work for me. The nice thing with SCM-backed wikis is you can edit locally in your favourite editor and push to the remote repo instead of using the Web UI. Mobile editing isn’t solved though – the Web UI is responsive but it would probably be better with a dedicated app (there’s a REST API for that if I ever want to go there). Anyway, it’s called “Wikked”, see here: https://bolt80.com/wikked/
One nice thing is that it’s using an SQLite cache to not work things out of the file-system all the time, which makes it scale better than I expected. See some performance tests here: https://ludovic.chabant.com/devblog/2014/04/13/wikked-performance/
I don’t work on it very often because it works fine on my server, but it may require a bit of bug fixing for anybody else. For instance, I use it with a Mercurial repository, so the Git backend is probably not totally functional out of the box… ping me in this case, I’ll be glad to help!
Checking back on TiddlyWiki it looks like they added a bunch of stuff like search and semantic queries, which was one of a big reasons I started writing my own engine. That’s pretty awesome!
You can also consider to use a static website generator, such as Jekyll. The idea is to do not expose any dynamic content such as an administration panel that could lead to a potential vulnerability. You may not have the time to update a CMS in the future, and a static website won’t expose a vulnerability. Also you are free to design your website according to your needs, unlike with a CMS.
Obligatory link to the original wiki source in HyperPerl
private git repo on gitlab.
I’ve been using Sphinx with a custom theme and makefile to push it to a local intel nuc I use for dev and custom apps and have configured as my browsers default url for new tabs/windows. I just use notes on my iPhone for when I’m out and about. That way I’m forced to sit down and go through the notes to get them into the wiki if i need to.
Also let’s me have a central Sphinx site that links into project sphinx docs. So my default Sphinx page is a sort of directory for the main wiki slash quick links to more in-depth linked Sphinx projects docs.
I’ve gone through so many note taking and knowledge management apps/systems and not been happy with any of them (some after only a couple hours) and this ones served me pretty well for a couple years. I’ve got January off and might revisit the whole thing to see if I can tweak it a bit more.
There is a lot of engouement for Gopher on mastodon, so I’m thinking I might try to setup something like it for personal note keeping. For file uploads I can probably use sftp.
I’m also thinking of using Dat with the beaker browser. It’s pretty easy to add new pages and the likes, and as long as I dont share any urls it’s perfectly secure.