I grew up with Release Early, Release Often [0] mantra, first as way to take part of the FLOSS community, but also to try to gather feedback to grow my skills. The latter is mostly unsuccessful, I used to mostly receive indirect feedback e.g. other projects picking up the same ideas, and more recently direct feedback in the form of private or public e-mails, conversations on IRC, mailing lists or forums (: e.g. https://lobste.rs/ :)
There is many ways to release a project, from sharing a roadmap, to a binary that is ready to use, or a mere toot.
How do you balance a) reaching users, contributors, and receive feedback, and b) avoiding too much communication that may be dubbed pollution? And according to you, and your experience, what is the good or perfect balance of Release Early, Release Often?
There is a concept of Worse is Better that may be relevant. It was originally coined for software, but I see it mostly in videogames nowadays.
To quickly sketch my understanding of it, it comes down to the idea that the most widespread software will have glaring flaws, and in fact profits from having glaring flaws, because they catalyze debate which is equivalent to advertisement.
“You are without a doubt the worst pirate I’ve ever heard of.” “Ah, but you have heard of me.”
From it, we may derive some rules:
For instance, consider Minecraft: acceptable but non-impressive graphics, a simple core gameplay loop that gives room for modders to expand but is strongly fun in itself, and an unopinionated design philosophy that instead exposes tools and affordances to the player to use freely.
Disclaimer: I have not yet put these into practice. In fact, I’m right now probably holding off on releasing my current compiler a bit longer than I should. (Just one more feature though! It’s not ready!)
Your understanding of ‘worse is better’ is different to mine - see https://en.wikipedia.org/wiki/Worse_is_better
In particular:
I’m not sure where you got this from.
Yeah, sorry if I was unclear; I tried to point out that was my interpretation that doesn’t necessarily match the original idea. This is mostly based on my impression of what sort of media gets big fandoms; the need and ability to improve on the canon seems to be essential. Which I think generalizes. Your users need something to draw them to your software, but also things to debate and improve. I think this is also part of what made the original concept of worse-is-better work, even though the author didn’t acknowledge it.
There are two parts in this. “Early” is IMHO complex, and there is a risk of releasing too early. OTOH the “Often” part is IMHO simple and — if your build/test/release process is healthy — you can release as often as you want.
Cost of testing is the limiting factor for release frequency. If your test process is manual and expensive, you’ll be forced to release less often to amortize this cost. It usually pays off to automate testing, because there are many positives to frequent releases.
Software changes, and therefore bugs, are going to be written at a constant speed of your development. Therefore, release frequency doesn’t change total amount of issues you’ll have to deal with. Releases aren’t causing bugs, only expose them. If you release one change at a time and something breaks, it will be easy to narrow it, and probably easy to revert. OTOH if you release X changes at a time, any problem may require investigating X changes. You may have multiple issues to deal with at the same time, compounding complexity super-linearly. A revert will cost you losing all X changes, including good ones, which may be very undesirable.
Users typically dislike any changes. If you change one small thing at a time, you’ll “boil the frog”. If you release a major redesign, many users will automatically hate it, even if it’s objectively better.
There are sometimes external reasons that require urgent releases (security issues, legal requirements, breaking changes in external dependencies). If you release often, it’s just another release. If you release rarely, being forced to release mid cycle can be a major pain.
Releasing frequently is organizationally simpler. Every feature can be released whenever it’s ready. OTOH when there are rare fanfare releases, there’s a last-minute crunch to cram everything almost-finished into the release, which lowers quality and adds stress. Or releases may get delayed when a planned flagship feature isn’t ready yet, causing frustration and slowing down the whole product.
I think these are very good points and they match my experience. A couple of comments:
I think the biggest risk is too cry wolf (i.e., “it’s ready”) one too many times.
IMO, users like improvements (useful functionality, bug fixes, performance optimizations, etc). What they dislike is pointless release churn. And they hate backwards-incompatible changes.
I think this is a double-edged sword: “things that can be done any time are often done at no time”.
That’s certainly true. But I also find that nothing focuses attention as an approaching release day and we often do more in the last month before the release than in the preceding many. So lately I’ve been trying not to stress too much about a release date slippage as long as we are making progress. Paraphrasing recent research, “stress is only stressful if you stress about it.”
Your first question is essentially a content strategy question. The answer is providing a gradient of communication channels with varying degrees of frequency and “bakedness” of ideas, and then make all channels strictly opt-in.
Maintained relationships and private communication with early adopters, major contributors, and evangelists can be important channels for new ideas, ways to frame or improve ideas in progress, or get early feedback on prototypes
Chat channels, Forums, mailing lists, and even Twitter/mastodon can be a great way to get new ideas, roadmaps, or features for the most plugged-in parts of your community
A public (and notification subscribable) changelog is a great way to get changes in front of the early adopters crowd that mostly cares about new things they can try today
Once your changelog reaches a critical mass of cool stuff, publish a digest, maybe even do a marketing-version bump if applicable.
Longer-form blog posts, especially after-its-released retrospectives, can be a great way to reach people who have never heard of you, since that level of high-effort content can be reasonably posted to news aggregators
It sounds like you may be doing all or some of these already, but I think the key takeaway is that all of it is valuable to different groups of people, and all of them come to different levels of maintenance burden on you. Invest in the ones that you see as having the most impact/reward to effort ratio.
I haven’t thought about it from the point of view of marketing yourself or getting feedback. For me it’s equivalent to the metric from Accelerate of how long it takes before code is checked in before it reaches production. If the time from commit to production is a couple of days, then you can make steady changes. If getting code to production takes two months, then you will continuously rely on hot fixing and your whole flow will be a confused mess.
I think keeping a roadmap in a public place in your repository and referencing it in your README is a first step.
As far as user feedback, that all depends on the ‘openness’ of your project. Are decisions all made internally by your marketing department, or are the users part of that process?
You can set boundaries on your issue discussions. Boundaries can be time frames for the discussion, and concentrating on issue at hand. Its good to be clear at the beginning what you’re seeking. When creating a issue you could specify you are seeking feedback and a decision will be made by X, or after reaching consensus; whatever your group process is.
I personally think all discussions should happen as close to the code as possible, thankfully git(hub,lab) make this easy enough. The moment you start straying over to EXTERNALISSUETRACKER and some other tooling you start hitting user friction from ease of access to ease of finding relevant issues. (Not knocking those tools, they work, just from a OSS perspective they can create friction).
Reaching your users about new releases? That should be an automated process and just hit as many relevant systems as possible. Do your users follow your software on twitter? Then toot! Do you only do github? Then update your README and Release there. Etc.
RERO, I think its good to release MVP’s and build from there. Don’t prematurely build anything you don’t know you need, but do try to build your code in a maintainable way. Rather than waiting to release until feature X does a, b, and c; release it when it does a and get feedback on how its working. Refine the code, improve a, further iterate on b, and maybe c isn’t even needed anymore.
And of course, have the semantic versioning argument if your software has the potential to ‘break’ or ‘change’ apis. Set some boundaries there too, you can follow semver as specified or declare where your edges are and how you’ll handle them.
IMO ESR’s quote is an anachronism. Back then, the ecosystem was less complete so declaring a goal for a project implied building something that a large fraction of people wanted; and the vast bulk of open source users were also developers, so those people were available to help contribute in terms of architecture or direction.
Today, the ecosystem is more mature, so releasing a half-baked kernel will be ignored rather than embraced since many already exist. Users outnumber contributors, so community interaction takes the form of support requests and complaints.
Now, “release early” means having a very small number of users whose support burden is discouraging and are more likely to slow down progress than accelerate it.
“Release often” historically made sense as users typically installed release tarballs and built patches against those releases, then sent text patches over email. Frequent releases means trunk is closer to the contributions, making contributions easier. Today, it’s very unnatural to contribute a patch based on a release; pull requests are against trunk. The challenge has become ensuring that trunk is buildable, usable and testable because that’s what contributors are running.
Frequent releases imply feedback is more relevant to the code in trunk. But, does your new project have an auto updater? Is it in common repos? If not, how many of your users are actually upgrading every week? If they don’t, this is a theoretical benefit that’s less valuable in practice. The goal is to issue releases that people will actually deploy.
So my advice is to push for 1.0, being something that anyone can install, has documentation, people can find their way around, and adds substantial value despite not being fully complete. This provides opportunities for motivated contributors while minimizing the taxing part of support. Even with this, the huge number of projects relative to contributors mean your project may never get the community you want, so prepare for the implied rejection.
(Seriously: I have a project on Github with 1k stars. About 20 people have ever interacted with me, and 3 people have sent pull requests. Those changes are typically minor. The challenge today is how to have any contributor community at all.)