This post led me to stumble upon dogsheep, which looks really cool. I had my own “personal API builder” a while back, but it tried to do too many things and didn’t do any of them particularly well.
I like the idea of having small, independent databases backed by something stable like SQLite. I think I want something updates my personal datasets irregularly (~yearly) and that I can run some data analytics scripts on. The API aspect (i.e. building something callable from the public internet) doesn’t really appeal to me… but, still sounds like it’d be neat as a way to populate a personal website.
Glad there’s others thinking in this space! I’ll be looking forward to reading updates :)
Running analytics on your own data isn’t something I thought about but I like that idea. At the end of every year (for the past few) I like to reflect and more often than not I forget what transpired the first half of the year. Having analytics could help. Also, it’d be neat seeing your listening habits over time.
An added bonus of this is for tech-minded people like ourselves to physically see the value of our data. I think we all have a general idea but the existence of data we can see certainly quantifies it.
I was going to work on a series of posts called Facebook without Facebook. One of the things I did was use the ical feed so I didn’t have to visit Facebook to get my event invites. A few months ago, Facebook changed their API so you can’t get invited events; only events you’ve confirmed you’re interested in. There is no way to undo this change. Under the guise of privacy, it really made you actually visit their website.
At the time I just copied events from that feed to my main calendar (self-hosted in Radicale) in Thunderbird, but I was going to write a little tool that e-mail me event invites so I could just clip approve and add them. Now, if I want to do this, I’ve gotta write a scraper that works in my browser to get events (unless anyone has something that does this?)
The idea of a personal API that you can connect to other components (an IFTTT type system I guess) seems like a good idea, until you realize most services are going to be missing all the ways you can sanely interact with them to update that information; because they need you on their services watching ads.
Your last paragraph is unfortunately very true. Even services that provide good value would love to lock you in somehow.
To fully invest in an personal API, you’ve got to become comfortable with not using popular services and sometimes that’s not possible.
My wife sews for small pattern companies and the entirety of the sewing world is on Facebook. She’d prefer not to use it and she’s not alone in that feeling among her community. However, a free platform with group support that people are already signed up for? Can’t be beat.
an sqlite database that stores my music listening history, synced periodically from my music player with a cron job, managed via a database admin native app, and serving a few cool facts on a web page served by node-red (such as my most listened to artists and genres for the day and week).
Didn’t go very far though, but the idea is always in the backburner.
This reminds me of something one of my friends is working on! I haven’t heard any recent updates, but I really liked his demonstrations of what it could do.
I’m so glad I’m on this site. I’ve never heard of a “memex” but it sounds like what I’ve wanted for years. I’m gonna watch your friend’s talk later today.
Resonating with @jpscaletti I think this is an extension of the idea of data sovereignty through self-hosting. In my perfect world, every single piece of data about me is signed by me. I don’t care where the data lives or how the API for that works.
Obviously this isn’t achievable so the ‘personal API’ that puts a layer of service-specific workarounds on top of that is a great next-best and much more realistic solution.
It’d be amazing if IndieWeb was setup at the creation of the Internet but as it stands now, it’s too much to ask of the online services we use to open up our data to us. Fortunately (?) we can take matters into our own hands.
Urbit is written by a neofeudalist, and his work reflects that. The idea that he would ever deign to give the user more power when his whole philosophy goes against that is very naive. Decentralization with the goal of being coordinated and orchestrated by a wealthy or chosen few isn’t really what I had in mind with “decentralized”.
Urbit tried to do too much, imho. I did not enjoy the experience of writing Hoon at all so it didn’t help that it was necessary (at the time I did it) to participate.
The idea is pretty cool. Full distributedness of data etc etc. but I did not enjoy using it in practice. I wouldn’t recommend it, though perhaps it’s worth it as a store of ideas.
I also tried building the same idea. I also thought of sharing the recipes in a github repository, like 1.
Instead of sharing updates (which I rarely do) I was planning to build a personal data manager/updater.
Messaging part is also important for me for privacy reasons. i don’t like how facebook uses cookies to track everything, so I want to be able to track all messaging on Whatsapp and Facebook.
The biggest challenge was the things that does not give an API, like whatsapp.
I think the biggest problem was, it required too much work. I have around 500 websites in my password manager. As I mentioned before, a collaborative open source repository could have helped a lot…
This post led me to stumble upon dogsheep, which looks really cool. I had my own “personal API builder” a while back, but it tried to do too many things and didn’t do any of them particularly well.
I like the idea of having small, independent databases backed by something stable like SQLite. I think I want something updates my personal datasets irregularly (~yearly) and that I can run some data analytics scripts on. The API aspect (i.e. building something callable from the public internet) doesn’t really appeal to me… but, still sounds like it’d be neat as a way to populate a personal website.
Glad there’s others thinking in this space! I’ll be looking forward to reading updates :)
I’ll definitely update here once I make progress!
Running analytics on your own data isn’t something I thought about but I like that idea. At the end of every year (for the past few) I like to reflect and more often than not I forget what transpired the first half of the year. Having analytics could help. Also, it’d be neat seeing your listening habits over time.
An added bonus of this is for tech-minded people like ourselves to physically see the value of our data. I think we all have a general idea but the existence of data we can see certainly quantifies it.
All of these ideas are part of what some know as the “indieweb” https://indieweb.org/
Shameless plug: I’m using python package, i.e.
import myas my personal API; I’m in progress of writing up on it, you can see some examples here.I like this idea a lot. I might just do the same thing.
Good luck and have fun!
Please do, I’d love to see how you implement this.
I have a couple projects to take care of first but I cannot wait to get started on my personal API.
I was going to work on a series of posts called Facebook without Facebook. One of the things I did was use the ical feed so I didn’t have to visit Facebook to get my event invites. A few months ago, Facebook changed their API so you can’t get invited events; only events you’ve confirmed you’re interested in. There is no way to undo this change. Under the guise of privacy, it really made you actually visit their website.
At the time I just copied events from that feed to my main calendar (self-hosted in Radicale) in Thunderbird, but I was going to write a little tool that e-mail me event invites so I could just clip approve and add them. Now, if I want to do this, I’ve gotta write a scraper that works in my browser to get events (unless anyone has something that does this?)
The idea of a personal API that you can connect to other components (an IFTTT type system I guess) seems like a good idea, until you realize most services are going to be missing all the ways you can sanely interact with them to update that information; because they need you on their services watching ads.
Sigh
Your last paragraph is unfortunately very true. Even services that provide good value would love to lock you in somehow.
To fully invest in an personal API, you’ve got to become comfortable with not using popular services and sometimes that’s not possible.
My wife sews for small pattern companies and the entirety of the sewing world is on Facebook. She’d prefer not to use it and she’s not alone in that feeling among her community. However, a free platform with group support that people are already signed up for? Can’t be beat.
I’ve toyed with something similar before:
an sqlite database that stores my music listening history, synced periodically from my music player with a cron job, managed via a database admin native app, and serving a few cool facts on a web page served by node-red (such as my most listened to artists and genres for the day and week).
Didn’t go very far though, but the idea is always in the backburner.
This reminds me of something one of my friends is working on! I haven’t heard any recent updates, but I really liked his demonstrations of what it could do.
https://hyfen.net/memex/
I’m so glad I’m on this site. I’ve never heard of a “memex” but it sounds like what I’ve wanted for years. I’m gonna watch your friend’s talk later today.
I’d be really interested in providing either feedback or code if he’s still working on this.
Don’t forget about https://github.com/muesli/beehive An open source ifttt clone. I had it running on my raspberry for some time. 👍
Oh wow, this is incredible! The UX is better than IFTTT in some ways too. Thanks for sharing!
Resonating with @jpscaletti I think this is an extension of the idea of data sovereignty through self-hosting. In my perfect world, every single piece of data about me is signed by me. I don’t care where the data lives or how the API for that works.
Obviously this isn’t achievable so the ‘personal API’ that puts a layer of service-specific workarounds on top of that is a great next-best and much more realistic solution.
It’d be amazing if IndieWeb was setup at the creation of the Internet but as it stands now, it’s too much to ask of the online services we use to open up our data to us. Fortunately (?) we can take matters into our own hands.
This is one of the things that Urbit is trying to make possible.
Urbit is very confusing to me. I’d love to see posts from people actually using it, I think that’d make it easier to grok.
Urbit is written by a neofeudalist, and his work reflects that. The idea that he would ever deign to give the user more power when his whole philosophy goes against that is very naive. Decentralization with the goal of being coordinated and orchestrated by a wealthy or chosen few isn’t really what I had in mind with “decentralized”.
Urbit tried to do too much, imho. I did not enjoy the experience of writing Hoon at all so it didn’t help that it was necessary (at the time I did it) to participate.
The idea is pretty cool. Full distributedness of data etc etc. but I did not enjoy using it in practice. I wouldn’t recommend it, though perhaps it’s worth it as a store of ideas.
I also tried building the same idea. I also thought of sharing the recipes in a github repository, like 1.
Instead of sharing updates (which I rarely do) I was planning to build a personal data manager/updater.
Messaging part is also important for me for privacy reasons. i don’t like how facebook uses cookies to track everything, so I want to be able to track all messaging on Whatsapp and Facebook.
The biggest challenge was the things that does not give an API, like whatsapp.
I wonder if tools/scrapers like cheerio could help. Then again, you’re at the mercy of these platforms without APIs.
Yes, Cheerio was my primary tool.
I think the biggest problem was, it required too much work. I have around 500 websites in my password manager. As I mentioned before, a collaborative open source repository could have helped a lot…