I really dislike that the base protocol is all written in javascript. I know this a volunteer project, but it’s clearly aiming at becoming a basis for the decentralized internet of the future and not just a personal project. Right now starting a dat/hypercore project in another language is a lot harder than it should be, and that will imo kill the project.
I encourage people who don’t like Javascript to chip in there, or get something going in their preferred language. The more implementations, the merrier!
A lot of activity is also happening on https://github.com/datrs/ – after I stepped away from the project in 2019 activity quieted down for a while. But recently things have picked up again thanks to Frando & Bruno.
“… As the commandline tool is no longer a driving force for the development of the protocol, it is becoming increasingly clear that more fine-grained control and modularity is needed. Thus, the core maintainers of the protocol have decided to retire the ‘dat’ protocol and rename it to reflect it’s underlying component called ‘hypercore’. …”
Is there a more detailed list of dat protocol shortcomings in ‘fine-grained control and modularity’, and how they will be addressed? Or were the shortcomings in the cli tool itself?
lobste.rs has had a number of interesting articles/submissions that touched on IPFS and DAT, I am still slowly learning and figuring out which of those will ‘take off’ so to speak. Especially in ecosystems encompassing ‘GeoCities-like’ blog, discussion forums, and surrounding services, but with strong privacy needs, slow/unreliable internet connection and low compute resources.
The Beaker Browser seems like a very interesting development in the area of “Geocities-like”, with its builtin website editor and insta-sharing. Paired with e.g. a 100MB free tier for “pinning” via hashbase.io IIUC, or alternatively self-hosted seeding e.g. on a RPi I hope. Apparently they released an official beta of Beaker with some major improvements the same day they announced the Dat->Hyper restructuring. With those steps I now have kinda feeling like they outcompeted IPFS on user-friendliness front ATM. Super cool to watch such an awesome race! :)
At least in my readidng, it seems that IPFS sees browsing for information as ‘last mile’ as in, I guess cable/telephony companies see connecting to ‘residence’ as ‘last mile’. With that analogy, IPFS sees their HTTP gateways as a necessary component that would allow existing web browsers (with specific extentions) to plug-in into the IPFS ecosystem. [1]
While Hypercore’s Beaker appears to be a ‘File Manager’ with built-in Document Editor, Address Book, Disk Manager, web site builder (Profile Drives) [2]
What I do not understand though, if I somebody has say a hybrid citizen of the ecosystem: an SPA webapp (single page java-script app that needs access to person’s address book, and some remote none-p2p database) – how that would work?
I believe they are shortcomings with the CLI itself.
Dat and the CLI were initially informed by academic use: versioning & sharing datasets in academic settings (think sharing genome sequences or astronomy data between universities). But it became clear that the underlying protocols could be used for lots of different applications outside of academia. Personally I’m most excited about kafka-like databases built on hypercore such as kappa. Over the past few years the protocol has seen a lot of iteration, and the CLI was maintained by a different group than the protocol devs and didn’t really keep up.
With regards to kappa, it seems that a query requires to pre-create materialized views
(I am reading [1] )
It means that ad-hoc queries cannot be done efficiently (as it requires essentially creating a subset of the database data that the query needs).
Seems like a reasonable compromise in many cases for the real-world. But, perhaps not suited, for the ones were a developer does not anticipate ahead of time the specific queries.
I started exploring this area yesterday, and I noticed they apparently also issued a major beta release of Beaker Browser more or less the same day, with a host of interesting IMO features. I was kinda betting my Internet Pixie Points on IPFS really beforehand, but in my eyes the Dat/Hyper community suddenly outcompeted them on user-friendliness front as of now :D I find it so cool and uplifting to watch such a positive race in a fresh and people-oriented area, not yet spoiled by all the FAANG behemoths :) 🌼
So now dey don’t dis DAT, but it sounds like some Intel marketing BS instead. Who comes up with these silly names? Naming things was already a famously hard problem in computer science before all this cheesy sci-fi bling-lang became the dominant paradigm.
Old-man grumbling aside, I hope the rebranding and restructuring effort pays off for the project. This is a really promising technology that deserves much wider adoption.
I really dislike that the base protocol is all written in javascript. I know this a volunteer project, but it’s clearly aiming at becoming a basis for the decentralized internet of the future and not just a personal project. Right now starting a dat/hypercore project in another language is a lot harder than it should be, and that will imo kill the project.
This Rust implementation looks like it is under active development: https://github.com/Frando/hypercore-protocol-rs
I encourage people who don’t like Javascript to chip in there, or get something going in their preferred language. The more implementations, the merrier!
A lot of activity is also happening on https://github.com/datrs/ – after I stepped away from the project in 2019 activity quieted down for a while. But recently things have picked up again thanks to Frando & Bruno.
In January a milestone was achieved: datrs running on an Android phone talking to the JS impl on Windows. This is really exciting and the project is in very capable hands!
Is the protocol fairly stable nowadays? Or is it changing/evolving? How hard is it to keep up with upstream?
Is there a more detailed list of dat protocol shortcomings in ‘fine-grained control and modularity’, and how they will be addressed? Or were the shortcomings in the cli tool itself?
lobste.rs has had a number of interesting articles/submissions that touched on IPFS and DAT, I am still slowly learning and figuring out which of those will ‘take off’ so to speak. Especially in ecosystems encompassing ‘GeoCities-like’ blog, discussion forums, and surrounding services, but with strong privacy needs, slow/unreliable internet connection and low compute resources.
The Beaker Browser seems like a very interesting development in the area of “Geocities-like”, with its builtin website editor and insta-sharing. Paired with e.g. a 100MB free tier for “pinning” via hashbase.io IIUC, or alternatively self-hosted seeding e.g. on a RPi I hope. Apparently they released an official beta of Beaker with some major improvements the same day they announced the Dat->Hyper restructuring. With those steps I now have kinda feeling like they outcompeted IPFS on user-friendliness front ATM. Super cool to watch such an awesome race! :)
Yes it is interesting to see.
At least in my readidng, it seems that IPFS sees browsing for information as ‘last mile’ as in, I guess cable/telephony companies see connecting to ‘residence’ as ‘last mile’. With that analogy, IPFS sees their HTTP gateways as a necessary component that would allow existing web browsers (with specific extentions) to plug-in into the IPFS ecosystem. [1]
While Hypercore’s Beaker appears to be a ‘File Manager’ with built-in Document Editor, Address Book, Disk Manager, web site builder (Profile Drives) [2]
What I do not understand though, if I somebody has say a hybrid citizen of the ecosystem: an SPA webapp (single page java-script app that needs access to person’s address book, and some remote none-p2p database) – how that would work?
[1] https://blog.ipfs.io/2019-10-08-ipfs-browsers-update/ [2] https://docs.beakerbrowser.com/joining-the-social-network
I believe they are shortcomings with the CLI itself.
Dat and the CLI were initially informed by academic use: versioning & sharing datasets in academic settings (think sharing genome sequences or astronomy data between universities). But it became clear that the underlying protocols could be used for lots of different applications outside of academia. Personally I’m most excited about kafka-like databases built on hypercore such as kappa. Over the past few years the protocol has seen a lot of iteration, and the CLI was maintained by a different group than the protocol devs and didn’t really keep up.
Thx for the background.
With regards to kappa, it seems that a query requires to pre-create materialized views (I am reading [1] )
It means that ad-hoc queries cannot be done efficiently (as it requires essentially creating a subset of the database data that the query needs).
Seems like a reasonable compromise in many cases for the real-world. But, perhaps not suited, for the ones were a developer does not anticipate ahead of time the specific queries.
[1] https://github.com/kappa-db/kappa-view-query
I started exploring this area yesterday, and I noticed they apparently also issued a major beta release of Beaker Browser more or less the same day, with a host of interesting IMO features. I was kinda betting my Internet Pixie Points on IPFS really beforehand, but in my eyes the Dat/Hyper community suddenly outcompeted them on user-friendliness front as of now :D I find it so cool and uplifting to watch such a positive race in a fresh and people-oriented area, not yet spoiled by all the FAANG behemoths :) 🌼
So now dey don’t dis DAT, but it sounds like some Intel marketing BS instead. Who comes up with these silly names? Naming things was already a famously hard problem in computer science before all this cheesy sci-fi bling-lang became the dominant paradigm.
Old-man grumbling aside, I hope the rebranding and restructuring effort pays off for the project. This is a really promising technology that deserves much wider adoption.