This offers some interesting funding ideas that I like, but I don’t think it identified any actual problem (that I agree with). The closest problem statements I could see were these:
As we have moved to more and more niche tools, it becomes harder to justify the time investment to become a contributor.
The other problem is the growing imbalance between producers and consumers. In the past, these were roughly in balance. Everyone put time and effort in to the Commons and everyone reaped the benefits. These days, very few people put in that effort and the vast majority simply benefit from those that do. This imbalance has become so ingrained that for a company to re-pay (in either time or money) even a small fraction of the value they derive from the Commons is almost unthinkable.
In the abstract, these seem like interesting problems. But is there hard evidence that this is causing a serious problem in the proliferation of free software?
It seems to me like the problem being described here is one of power imbalance. Namely, that there is a small set of contributors and a large group of users. You might find this inherently disturbing, but what are its real world implications? Should I, as a programmer who contributes to free software, feel bad that there are people using it that don’t give back to my project? (I certainly do not!)
In the end though, it is a bleak landscape right now.
And this is where I’m like: huh? Free software is flourishing. Compare the rise and proliferation of code sharing today with ten years ago. There are vast networks of online communities collaborating—in the open—on free software for the whole world to use.
What exactly is “bleak” about today? Is there some credible threat to free software that is looming in the shadows waiting to destroy the free sharing of code as we know it today?
The threat is the persistent and pervasive burnout amongst people working on projects that are OMG-level critical to the tech sector. A lot of people are starting to step back from major projects like Python, Postgres, Django, Ruby, etc, and that’s going to have an impact. Most of the people leaving are the ones that feel like what used to be a hobby is now a full-time, unpaid job. If we don’t figure out a better way to support those people, we’re going to have a bad time.
I just wanted to pop up a level.
I feel like I support the message you intended to convey in the OP: let’s work on helping to fund free software contributors. That is a noble goal that is hard to disagree with. I thought that part of your post was pretty good. The problem I’m having is with your framing; frankly, you come across as an alarmist with the idea that free software is going to be in huge huge trouble unless we figure out some sort of funding for free software contributors. It really put me off to be honest.
In the last 18 months we have seen some of the issues of a lack of funding - HeartBleed exemplified the problems that OpenSSL has been suffering for years from chronic under funding.
OpenBSD nearly ran out of money to cover the cost of its electricity usage.
I think there are plenty of examples of Open Source projects lacking a reasonable financial backing.
Unfortunately, I don’t have any bright ideas for solving this problem (but I do order OpenBSD CD’s twice a year :~])
Right… Maybe I misunderstood the OP. I wouldn’t have considered either of those projects as examples, because once they got into trouble, others stepped in to help out. To me, this seems like things are working great and that there’s no cause for alarm. From the OP’s tone/phrasing, I was expecting to hear about critical projects that had become completely defunct (none of OpenSSL nor OpenBSD nor PyPI fit that description).
Don’t you think it would have been much cheaper to prevent these problems than to scramble to fix them after the fact? Certainly, when people are burned out to the point of leaving a project, there is a huge transaction cost to someone else stepping in and getting up to speed, even if we assume that there will always be somebody willing and able to do so.
Of course! I’ve stated several times in this thread that I support more funding! What I don’t understand is the alarm.
The threat is the persistent and pervasive burnout amongst people working on projects that are OMG-level critical to the tech sector.
Who is responsible for this threat? Can you provide examples of critical open source projects that have become defunct (i.e., no longer useful) because of burnout?
A lot of people are starting to step back from major projects like Python, Postgres, Django, Ruby, etc, and that’s going to have an impact.
Can you elaborate? A lot of people take steps back from projects—not just major projects. Is there a particular reason why you think this is particularly bad today? And if people take a step back from these projects, is there some reason to believe that the slack won’t be picked up by other (new or old) contributors?
Most of the people leaving are the ones that feel like what used to be a hobby is now a full-time, unpaid job.
That seems like a perfectly legitimate reason to leave a project. Sometimes you lose your passion for a project. It happens, and not just in free software. Why is this a major threat to free software?
If we don’t figure out a better way to support those people, we’re going to have a bad time.
You really haven’t made a convincing case for why you think this is true. In particular, free software is flourishing in both quantity and quality, yet you seem to completely ignore this point.
Rubygems.org has had issues like ‘gems with native dependencies don’t install on windows’ open for 8+ months because it is effectively unmaintained due to the maintainers being burned out.
The critical vulnerability with YAML a few months back only happened because the maintainers had ‘investigate if that bug affects gemspecs’ on their TODO list but couldn’t find the time to do it.
AT&T deciding to get rid of their Ruby open source contributions has harmed Ruby and Rails immensely. The pernicious things about issues like the blog post is that you don’t realize they’re happening until they’ve already happened. It’s difficult to quantify in the moment.
I think that OSS looks really great on the surface, you see more projects than have ever existed in the past, more companies using it, and just a general greater acceptance across the board. However, much like a family that are running up tens of thousands of dollars of credit card debt in order to “keep up”, if you look below the surface at the “finacials” of OSS you’ll see that a frightening amount of really critical stuff is severely under-maintained if they are maintained at all.
This I think is what coderanger is speaking to when he’s talking about the landscape. This problem actually gets a lot worse the more popular OSS becomes if there isn’t also a large enough investment back into these projects by enough of their users. As OSS becomes more accepted it more people use it, as more people use it you have much more demand which places additional pressure of the maintainers of that software. They start getting more people submitting bug reports, more people demanding fixes, more people yelling at them when something doesn’t go their way and I think for a lot of maintainers the project that used to be fun to work on in their spare time starts to become something they dread touching because it brings with it feelings of guilt and anxiety and a constant need to be fighting fires.
In the end, a large numbers of projects, even well written projects, without contributors or maintainers is a pretty bad outcome if we push enough of them away.
I think that OSS looks really great on the surface, you see more projects than have ever existed in the past, more companies using it, and just a general greater acceptance across the board.
I’m having a difficult time understanding why these great attributes of free software are being qualified with “on the surface.” Why are these surface level qualities? The increase in acceptance, quantity and quality of free software don’t seem like surface level qualities to me. They seem like deep and entrenched improvement. I kindly ask you to compare the state of free software today with the state of free software ten years ago. At least from my perspective, the difference and improvement is astounding.
I otherwise take your point though. I totally get that a really important project (like PyPI!) is critical to maintain. What I don’t understand is the alarm. As you described in another comment, you eventually couldn’t keep up with PyPI any more and companies stepped in to fund it. If they disappear, and PyPI stops working, do we think that some other company won’t jump in and foot the bill? I certainly think someone would. That seems OK to me.
We have no reason to suspect anyone else would fund it, it took us years to work out the current deals keep things just on this side of failure. If Rackspace of Fastly pulled support tomorrow, we would have to start that all over again. We have contracts in place where possible to diffuse some of the risk, but it’s still a “nod and a handshake”-based mess. Rubygems is similar in a lot of ways, NPM has resources behind it from VCs, this is not a pretty picture.
That brings up a point that I don’t think was in your post: running infrastructure for some projects is a big chore, and typically getting funding for that is hard or impossible. Most projects I know run either out of basements or off a VPS somewhere, so if they get wildly popular they just fall over.
The best single example I can cite is Python packaging and its whole ecosystem. Just a few years ago, it was almost unusable due to years of neglect. PyPI was down frequently, pip was difficult to install, slow, and very insecure. While it’s not the only factor that made things better, a huge part of the improvement was due to one man (Donald Stufft) and the fact that he has had financial support in working first 50% on packaging via Rackspace and now 100% via HP. If he lost that funding, I have no doubt he would have to scale down his efforts and given what happened before I would call that a critical issue to the Python community. We have no backup plan, if HP’s generosity runs out the fallback is to just accept packaging being on a slow slide back in to the dark.
I can be more explicit, I was working on packaging prior to funding from Rackspace or HP and I was heading towards burn out pretty rapidly. I was forgoing spending time with my family or doing anything else to try and find time to work on it because, while I’m not the only person, I’m one of (if not the) primary driving force currently. The funding from Rackspace and now HP has given me the ability to dedicate time to it, without forgetting what my family looks like. You can look at OpenSSL and GPG for similar situations. There are countless tools at varying levels of critical-ness to the infrastructure of organizations (or to the internet as a whole) that have little to no funding.
This sounds more convincing. It would have helped me interpret your OP more charitably with these examples in your post.
I still don’t think these examples warrant the level of alarm in your OP. It sounds like the system, as is, is working great. Under funded critical projects are getting attention after we notice they need attention. I personally don’t see that as a major problem in and of itself.
That’s fair. It is hard to see this from the outside sometimes. As someone with friends in more or less every major FOSS project, all I hear is a sea of discontent and burnout. As dstufft pointed out though, this has stayed well hidden for years. I think the Python and DevOps communities in particular are making huge strides in it being okay to talk about burnout in public, but a lot of it is still in hidden backchannels (-dev IRC channels, private Slacks, contributor-only mailing lists, etc). The saving grace so far is that each time someone had flamed out, another has stepped up to replace them. That’s a terrible way to get forward progress though, especially when you see the massive value companies are extracting from our collective work.
I see. That’s interesting. When you frame it that way, it seems like one of the problems you’re trying to address is to make it OK to talk about burnout. That seems like a great goal.
That’s part of it, but it’s also important that we all start realizing how much of our critical infrastructure is maintained as a side project, not as something full-time. I’ve seen this happen to a bunch of projects, including some of my own.
For my own part, I basically end up telling people I accept patches to my projects I’m burned out on, but in a world where the conversation included them wanting to hire someone to do the work, I can think of a handful of people I could propose as potential contractors with the right expertise. In general though, people have an attitude that precludes this for some reason. Generally if the subject of paying for a feature they need comes up they leave upset.
A friend spent a period working on his project by soliciting donations, and it basically fizzled after about 18 months - the donations from companies dried up, and that was the end of the road. Now he’s got a corporate patron, and it’s fine, but that’s still entirely too rare.
It’s worth considering that today much of open source is created largely by people in a very substantial position of privilege. People like myself who can afford to be self-employed or not even get paid at all for extended periods of time. Some maintainers are those who got lucky with an employer who permits them to spend some amount of their time on open source work.
And the effect? We in a position of privilege gain yet more privilege. Because of my open source work (and the ability to do it), I get way more interest from eagerly-hiring companies than any of my friends without a Github repo. I gain more public respect and recognition because I can afford to do this. It gives me the luxury of being a lot more picky.
Free software may be flourishing, but who are the maintainers and contributors who are flourishing with it?
Having more options for funding open source enables a greater diversity of people to participate, and I think that’s a good thing for both software and people.
I’m having a hard time parsing your central message. You’re saying more funding is good. Great, I agree. I’m taking issue with this idea that free software is somehow in a boat load of trouble today. As the OP says, it’s a “bleak landscape right now.” Huh?
[EDIT] See some of my other comments for more explanation. :-)
I’m not arguing that the end of the world is here today. It’s a social issue, like race/gender/income inequality. By ideal societal standards, the FOSS ecosystem is not in good shape (“bleak landscape right now” are not my words, but I can understand the sentiment). Probably far worse than corporate IT in general, which is not great to begin with.
Even at the most regrettable and embarrassing times in our society’s history, even during slavery, our GDP continued to grow. It’s dangerous to ignore systemic problems just because the metrics are going up and to the right.
I don’t know how to respond to this. Our perceptions of reality are just way too different. Ideal societal standards? Corporate IT? Slavery? GDP? Income equality? Holy moly.
Ideal societal standards? Slavery? GDP? Income equality? Holy moly.
Hm? Are those questions or just mocking? :/
They are questions of a baffled reader. I was inquiring: how is free software “bleak”? What is justifying all this alarm?
Instead, i’m met with a comment that strolls into a whole bunch of seemingly unrelated topics. What more can i say? At a certain point, i have to acknowledge that we’re speaking way past one another and cut the conversation short.
At a certain point, i have to acknowledge that we’re speaking way past one another and cut the conversation short.
By the way, I’ve used your Go toml library, good work. I hope producing great FOSS work continues to be feasible for you. :)
Thanks. Me too! :-)
Thanks - exactly my own feelings on these topics, but I’ve tried and never been able to say them very well.