we want to make it open source and easy to contribute to
i’m confused about the choice to run this on cloudflare workers. i understand that the author wants the code to be capable of running on more than a single platform. it just feels inevitable that the code gets tied up around proprietary cloudflare features.
my opinion: it’d have been better off as a monolith that runs on any old linux vps, not a bunch of newfangled stuff that targets a proprietary platform.
i like pijul, but this feels a little shortsighted to me
“Shortsighted” is a strong word here, I’ve been running a “your opinion”-style solution for years and know exactly why I can’t make it work.
“Proprietary Cloudflare Workers features” aren’t so unique that we can’t easily replicate them in a day of work (without the open source runtime, it would probably take a few days of work instead).
cloudflare is possibly the biggest threat to the free & open web, and hitching nest to it & claiming it’s for “ease of contribution” feels off optically. i have no doubt that you have good reasons, you’re much smarter than i - but intuitively, it feels wrong, and i believe this perception could be problematic. actually optimizing for contribution would mean making nest simple to run without a cloudflare account.
can i self-host nest without tying it to a proprietary platform? if the answer is no, it doesn’t feel very “open source”, even if the license says so.
I wouldn’t have done this without workerd and miniflare, two open source runtime environments to run Cloudlfare Workers scripts outside of Cloudflare.
At a higher level, the biggest threat to the free and open internet isn’t a company providing useful services and infrastructure, it is the absence of political will, public investment and regulations. Europe could have done that, we have some of the world experts in databases, CRDTs, replication, programming languages… I actually made contributions in some of these fields, mostly out of my own money and on my free time. There was also a time in Europe where we were not afraid of ambitious projects and regulations.
So, until Europe (or someone else: China?) invests in open and public cloud infrastructure and useful projects instead of what they’re doing (I won’t cite any example, but there are many), I’m fine using Cloudflare.
Cloudflare maintains two open source runtimes/simulators for cloudflare workers that you could probably use to self-host. The open source runtimes are workerd (based at least in part on the real code used at cloudflare) and miniflare (a simulator written in typescript).
This feels like an exaggeration. Cloudflare to my knowledge have only ever blocked one site for non-legal reasons, and they seemed to regret it. Unless cloudflare start disabling “DNS only” settings I can’t see the issue, could you enlighten me?
I don’t want to get into an argument about it, but it is untrue that Cloudflare has only blocked one site (except where compelled by law). Cloudflare kicked the fascism-promoting daily stormer and 8chan sites and kicked the hatespeech site kiwi farms.
Cloudflare also kicked switter.at, a mastodon instance for sex workers, which they claimed to be doing because of FOSTA, but they did this without warning and before any lawsuit was filed. In other cases cloudflare has fought censorship demands in the courts and won, and in the years following FOSTA it seems like courts are less willing to prosecute service providers than was originally thought, so cloudflare could probably have kept hosting switter.at, and could certainly have given them notice.
By the way, I am aware of these things with Cloudflare, and I do disagree with their responses (or delays, in the case of KiwiFarms) in 100% of the cases, but I’m willing to believe it is an irresponsible, rather than malevolent, behaviour, as is often the case in large organisations, especially if they’ve grown too fast.
I chose to work with CF anyway, because I don’t think that choice will influence Pijul’s future. If anything bad happens to us (censorship or otherwise) because of CF, it will mean that Pijul is big enough to be a problem, and that’ll prompt us to come up with a solution to keep going. I’m used to working with extremely limited setups and budgets, so that doesn’t scare me much.
Using a privacy-oriented machine setup and either on my real IP which is not in the West or using a commercial VPN, I am constantly needing to solve machine learning hCAPTCHAs to the point I peace out if I don’t feel it’s worth it because it’s exhausting to do multiple times a day.
IP-based blocking and the service itself may to to comply with US embargoes which can lights-out entire regions even if the folks living there have nothing to do with their ruling government.
There’s also the morale issue of pointing at a proprietary service as the path of leash setup resistance in this particular hosting case. Maybe we’ll get lucky and someone determined enough will make the NixOS module to where you can services.pijul.enable = true , but until then I bet most of the resources will recommend signing up with another publicity-traded, proprietary service.
Fortunately, no, I’ve never seen it with the current Nest at least. I stopped subscribing to a VPN though because of how many Cloudflare-fronted services prevented me from using them so I’ve not tested with them.
I know that Cloudflare gives me that option of “verifying” users, but that’s totally optional from what I’m seeing. I still dream of a public infrastructure that would provide the same thing, but it doesn’t exist yet.
Not yet, but I’ll probably share them when people start using it.
Without counting any external costs:
My goal with this serverless version is to provide a 100% reliable service, which the previous version could not possibly become from where it was, mostly because of the way PostgreSQL and Pijul have to work together (you can’t do a “join” between a Pijul branche and a PostgreSQL table, so you have to do lots of SQL requests, and have the servers close to your repos, which is hard to replicate). This will in turn allow me to sell “pro” account, which you can already buy in the beta version (nest.pijul.org) if you want to host private projects, so instead of having only bills to pay, I hope to stop losing (my own personal) money with that service.
FaaS means you don’t need to fix crashes, downtimes, servers, databases… It’s a nightmare to debug, especially if you’re like me and need to mix JS and WASM: no stack traces, no break points, manual debugging messages. But once you get past that, you can release confidently and sleep well at night. I guess the value of that increases as you age, so I can’t evaluate it properly.
The comparison between the provider bills is probably negligible next to these two parameters.
Are there any papers or posts that describe the current Pijul data model? I’d be interested in reading about the underlying format. I know it’s undergone some revisions over the past few years.
Is it just the KV store that would need to be replaced to self-host with workerd? Part of me is tempted to try to package this for Sandstorm, but any components that aren’t FOSS would of course be blockers.
Kenton was at one point excited about the idea of workers apps on Sandstorm, having started both projects. But up until now I haven’t seen any workers apps that seemed interesting to package.
Durable objects are really important for the Nest, but if you don’t require any distributed setup, they’re just fundamentally another layer of KV stores.
Their platform and setup is not at all how I would have done things, but the main value of CF Workers for me is not the software, it’s the scalability and reliability.
i’m confused about the choice to run this on cloudflare workers. i understand that the author wants the code to be capable of running on more than a single platform. it just feels inevitable that the code gets tied up around proprietary cloudflare features.
my opinion: it’d have been better off as a monolith that runs on any old linux vps, not a bunch of newfangled stuff that targets a proprietary platform.
i like pijul, but this feels a little shortsighted to me
“Shortsighted” is a strong word here, I’ve been running a “your opinion”-style solution for years and know exactly why I can’t make it work.
“Proprietary Cloudflare Workers features” aren’t so unique that we can’t easily replicate them in a day of work (without the open source runtime, it would probably take a few days of work instead).
cloudflare is possibly the biggest threat to the free & open web, and hitching nest to it & claiming it’s for “ease of contribution” feels off optically. i have no doubt that you have good reasons, you’re much smarter than i - but intuitively, it feels wrong, and i believe this perception could be problematic. actually optimizing for contribution would mean making nest simple to run without a cloudflare account.
can i self-host nest without tying it to a proprietary platform? if the answer is no, it doesn’t feel very “open source”, even if the license says so.
I respectfully disagree, for two reasons:
I wouldn’t have done this without workerd and miniflare, two open source runtime environments to run Cloudlfare Workers scripts outside of Cloudflare.
At a higher level, the biggest threat to the free and open internet isn’t a company providing useful services and infrastructure, it is the absence of political will, public investment and regulations. Europe could have done that, we have some of the world experts in databases, CRDTs, replication, programming languages… I actually made contributions in some of these fields, mostly out of my own money and on my free time. There was also a time in Europe where we were not afraid of ambitious projects and regulations.
So, until Europe (or someone else: China?) invests in open and public cloud infrastructure and useful projects instead of what they’re doing (I won’t cite any example, but there are many), I’m fine using Cloudflare.
Cloudflare maintains two open source runtimes/simulators for cloudflare workers that you could probably use to self-host. The open source runtimes are workerd (based at least in part on the real code used at cloudflare) and miniflare (a simulator written in typescript).
This feels like an exaggeration. Cloudflare to my knowledge have only ever blocked one site for non-legal reasons, and they seemed to regret it. Unless cloudflare start disabling “DNS only” settings I can’t see the issue, could you enlighten me?
I don’t want to get into an argument about it, but it is untrue that Cloudflare has only blocked one site (except where compelled by law). Cloudflare kicked the fascism-promoting daily stormer and 8chan sites and kicked the hatespeech site kiwi farms.
Cloudflare also kicked switter.at, a mastodon instance for sex workers, which they claimed to be doing because of FOSTA, but they did this without warning and before any lawsuit was filed. In other cases cloudflare has fought censorship demands in the courts and won, and in the years following FOSTA it seems like courts are less willing to prosecute service providers than was originally thought, so cloudflare could probably have kept hosting switter.at, and could certainly have given them notice.
By the way, I am aware of these things with Cloudflare, and I do disagree with their responses (or delays, in the case of KiwiFarms) in 100% of the cases, but I’m willing to believe it is an irresponsible, rather than malevolent, behaviour, as is often the case in large organisations, especially if they’ve grown too fast.
I chose to work with CF anyway, because I don’t think that choice will influence Pijul’s future. If anything bad happens to us (censorship or otherwise) because of CF, it will mean that Pijul is big enough to be a problem, and that’ll prompt us to come up with a solution to keep going. I’m used to working with extremely limited setups and budgets, so that doesn’t scare me much.
My apologies, I only remembered them dropping KF.
No apology necessary, but thanks for being polite anyway :)
Using a privacy-oriented machine setup and either on my real IP which is not in the West or using a commercial VPN, I am constantly needing to solve machine learning hCAPTCHAs to the point I peace out if I don’t feel it’s worth it because it’s exhausting to do multiple times a day.
IP-based blocking and the service itself may to to comply with US embargoes which can lights-out entire regions even if the folks living there have nothing to do with their ruling government.
There’s also the morale issue of pointing at a proprietary service as the path of leash setup resistance in this particular hosting case. Maybe we’ll get lucky and someone determined enough will make the NixOS module to where you can
services.pijul.enable = true, but until then I bet most of the resources will recommend signing up with another publicity-traded, proprietary service.Is this happening for the Nest though?
Fortunately, no, I’ve never seen it with the current Nest at least. I stopped subscribing to a VPN though because of how many Cloudflare-fronted services prevented me from using them so I’ve not tested with them.
I know that Cloudflare gives me that option of “verifying” users, but that’s totally optional from what I’m seeing. I still dream of a public infrastructure that would provide the same thing, but it doesn’t exist yet.
The answer is yes: https://github.com/cloudflare/workerd
Thanks for sharing! It’s cool that you’re hosting your database on top of the Cloudflare KV store.
You say this should be more financially sustainable for you. Do you have any numbers for that?
Not yet, but I’ll probably share them when people start using it.
Without counting any external costs:
My goal with this serverless version is to provide a 100% reliable service, which the previous version could not possibly become from where it was, mostly because of the way PostgreSQL and Pijul have to work together (you can’t do a “join” between a Pijul branche and a PostgreSQL table, so you have to do lots of SQL requests, and have the servers close to your repos, which is hard to replicate). This will in turn allow me to sell “pro” account, which you can already buy in the beta version (nest.pijul.org) if you want to host private projects, so instead of having only bills to pay, I hope to stop losing (my own personal) money with that service.
FaaS means you don’t need to fix crashes, downtimes, servers, databases… It’s a nightmare to debug, especially if you’re like me and need to mix JS and WASM: no stack traces, no break points, manual debugging messages. But once you get past that, you can release confidently and sleep well at night. I guess the value of that increases as you age, so I can’t evaluate it properly.
The comparison between the provider bills is probably negligible next to these two parameters.
Are there any papers or posts that describe the current Pijul data model? I’d be interested in reading about the underlying format. I know it’s undergone some revisions over the past few years.
No, not really. The underlying format is complicated, mostly because it relies on Sanakirja, which is itself complicated.
Is it just the KV store that would need to be replaced to self-host with workerd? Part of me is tempted to try to package this for Sandstorm, but any components that aren’t FOSS would of course be blockers.
Kenton was at one point excited about the idea of workers apps on Sandstorm, having started both projects. But up until now I haven’t seen any workers apps that seemed interesting to package.
Durable objects are really important for the Nest, but if you don’t require any distributed setup, they’re just fundamentally another layer of KV stores.
Their platform and setup is not at all how I would have done things, but the main value of CF Workers for me is not the software, it’s the scalability and reliability.
Someone pointed out to me that DO actually are in the FOSS version: https://github.com/cloudflare/workerd/pull/302
…so that’s exciting.