I would draw a distinction between the stale bots that are really aggressive and those that aren’t. I’ve seen stale bot configurations that wait a long time for inactivity, and when they do comment, give those subscribed a while to respond (I’ve seen 90 days which is lovely). Those aren’t too bad. But the more aggressive they become, the more they start to feel frustrating or like spam.
Also, most of the effects of people being forced to file duplicates come from stale bots locking issues. If you don’t lock the issue when you close it, then people can just comment and ask for it to be reopened.
Interesting that the discussion around this all sort of seems to assume that the maintainers are choosing not to fix bugs and therefore must be bad at running their projects.
That’s probably true pretty often. But I think it’s also often true that open-source projects have a small, more-or-less-fixed time budget for maintenance that does not expand at the same rate as the growth of the user base, whereas the volume of bug reports does expand along with the user base. Over the years I’ve frequently been surprised to learn that some piece of software I use constantly was the product of one person working solo and it wasn’t even their day job.
If I publish something I wrote in my spare time to scratch an itch, and it happens to get popular enough to generate more bug reports than I can handle in the few hours a week I’m able to spend on the thing, what are my options, other than quitting my job and working full-time on bug fixing? Do I delete the popular repo that people are finding useful enough to file lots of bug reports about, in the name of reducing the number of unhealthy projects in the world? Do I just let the bug list expand to infinity knowing I’ll never have time to clear the backlog? Do I spend my project time reviewing historical bug reports rather than working on the code?
In theory, of course, a project that gets bug reports should have lots of people volunteering to fix the bugs. But the quantity and quality of community contributions seems to vary dramatically from one project to the next.
It’s not about not fixing bugs. That’s expected, normal, fine. Maintainer does not owe me labour. Closing issues that aren’t fixed is just an FU to the users, though. Let the issue stay until someone fixes it, hopefully someone being affected by it.
Exactly this. I was raised in a world where SFTW or RTFM was an acceptable response, and so put effort into providing good issues. Having a bot drive by and close/lock my still open issue is a slap in the face with a wet fish.
I’m not a fan of simple auto-closing stale bots (roughly: I think the incentives are misaligned, and ~stale is a not-quite-right proxy of a few real problems, and it is trying to work around some limitations of using the GH issue system for triage)
but I am a big fan of having a way to punt unactionable issues back to the reporter, and let automation close the issue if the reporter never responds or return it to the triage queue if they do
That out of the way: in a busy long-term project, a stack of open bug reports comes with ongoing logistical overhead. GH’s UI/X exacerbates this, but some of it just comes with the territory.
This cost is an aggregate of: maintainer time spent managing the requests, the opportunity-cost of that time, the motivation it saps from the maintainers as they spend more time doing paperwork and less time making electrons dance, and the despair that comes from feeling obliged to play Sisyphus on an ever-growing mountain of reports (or the guilt that comes with having to ignore the reports).
Each person helping with bug triage will be paying a pretty similar cost unless you have tools/processes for de-duplicating logistical effort (unlikely for most volunteer efforts).
This cost is somewhat proportional to the amount of time each triager is able to invest. If you have 100 contributors doing 1 hour of triage a week, there’s going to be a lot more duplicate effort in the reports they read than if you have 2 people doing 50 hours of triage a week.
As the pile grows, efficiently pairing people with reports that are the best use of their time will get both harder and more important to your success.
At scale, the high-leverage maintainers most-likely to have the authority (whether literally the permission, or just the reputational capital) to go around closing reports manually will usually have more critical things to invest their time in. Those the task actually falls to aren’t as likely to be empowered to weed the garden.
Unless you’re willing to declare bug-bankruptcy or use automation (i.e., ideally #2 above–though obviously also a stale-bot), weeding out old dead-weight reports with the same care/consideration you’d usually run triage with can be (ironically?) a massive ongoing timesink in its own right.
Reports don’t have to be bad or obsolete to contribute to this cost. 10 or 20 brilliant suggestions may be an asset if you can find time to implement 1 a year for the next 5 years. 1000 brilliant suggestions may be an albatross if the extra logistical overhead fritters away the time and motivation you need to implement one.
It doesn’t, of course, apply to all reports equally. It’s better to have a single long-lived “darkmode pls” issue with gobs of comments and reactions than to deal with someone opening a new one every few days.
This. I am constantly surprised how many people dismiss the paralyzing effect of 2.4k open issues, even if they all make sense in some way. Resources are limited.
Yes. The issue trackers we have need the following features.
accept issues
comments on issues
occasional attachments to issues (need? Maybe not)
assigning people to issues
lifecycles as makes sense for your team and organization (open, closed, or on the other extreme: open, ready, in work, in testing, merged, in production, closed)
tags on issues for classifications
filtering on reporter, tag, assignee, and status, including default filtering.
You do that, you fit 99% of people’s needs.
Imagine if github had a tag “maintainers will not address” and the automatic filter (currently set to only show issues where their lifecycle is at “open”) only showed “open & not(tag:will-not-address-but-might-merge)”
People would be happier to tune that automatic filter for their own needs, and this would allow a tag to banish the issue from a maintainer’s headspace.
Mostly that there will be a certain number of old bugs that get fixed or rendered irrelevant as a side effect of some other change, and searching through the bug database will become less useful if a sizable portion of the bug reports reflect an obsolete reality.
I’ve occasionally run into this on existing projects, in fact: I will search for some error message I’m seeing, find a bug report with some discussion that includes workarounds people used, and only after wasting time trying those workarounds and digging into why they’re not working for me do I discover that the bug they were talking about is in a part of the code that no longer even exists.
Of course, in those kinds of cases, issues being auto-closed by a stale bot wouldn’t have helped much; I’d have still found them and probably still tried the obsolete workarounds. I don’t have a perfect answer for this either!
I haven’t heard a good case for a stale-bot. In the article it says “to reduce the number of open tickets”, but it doesn’t explain why that is a good thing – the number could be reduced to zero by not having a bug tracker at all!
Perhaps it is a UI thing, and Github should display the number of “active” open tickets, for some activity metric.
I turned a stale bot on for a popular project because I thought it would be a more honest communication of the likely fate of old GitHub reports - “this is stale and we aren’t planning to triage or address it further”. Few others perceived it that way and, after a few months, the bot was removed.
The triage work is substantial - we spend easily 120 hours a week on community question/support/defect/feature-request triage. I’d love to do more but of course there are fixed dollars and hours to allocate across many priorities.
Anyway, that was my reasoning, which turned out to be incorrect.
Automatically closing stale issues is a good sign that a project is unhealthy and doesn’t care about their issue tracker. Good project to avoid contributing to, usually.
That seems a bit harsh. People posting random non-issues can be a genuine issue for larger projects. People posting on long-since solved issues is also an issue, which tends to be >95% generic support or outright nonsense, and <5% useful.
I don’t care much for auto-close bots, but I understand why people use them. Managing all of this requires a significant amount of time.
I bet Angular had this exact problem; JavaScript tends to attract a lot of beginners and you’re forever cluttering the bug list with non-bugs unless you’re really diligent about maintaining this, and I can’t blame the maintainers on wanting to focus on actually maintaining the Angular project instead of guiding the endless stream of new users unfamiliar with Angular, JavaScript, etiquette, etc. It’s essentially the “Eternal September” problem.
Can’t both of those issues be solved, well, closing the issues manually?
I’d assume long-since solved issues should be closed because solved. For “junk” issues, generic support and whatnot, is it really better to just let them sit open for a week or two (or however long the bot takes) rather than just manually marking them as “offtopic/support/wtfisgoingonhere” and closing them?
I’d assume long-since solved issues should be closed because solved.
Yeah but people will comment on them. With this I meant the “lock bots” that lock issues after being closed for n days which prevents adding new comments.
As for manual closing/locking, sure, but that’s not “free” time-wise, and it can be emotionally draining. I don’t really want to tell people to ask their question somewhere else or that they’re making zero sense, but I also don’t necessarily want to provide mentoring to random newcomers as I got a life to lead and stuff to do. People can also get angry or even abusive about, no matter how nicely and gentle you phrase it (I’ve had that even with random strangers emailing me out of the blue because they saw me on Lobsters, Stack Overflow, GitHub, or wherever). It’s not super-common, but it sucks when it happens.
A bot just makes all of this easier and avoids the emotional drain. Is it the “chicken way out”? I suppose it is, like I said I don’t use it myself and generally just manually lock old issues and such if they attract a lot of comments, but I also never maintained a project the size of Angular, and I can see the reasons why people would use it.
I think the “emotional cost” of maintaining larger open source projects is often underestimated. Everyone is different and some people struggle with this more than others, but personally I find it hard. I want to be helpful, but that’s just not feasible or realistic beyond a certain scale so there’s some amount of (internal) tension there. It also leads to situations where you feel obligated to do things that you don’t really want to do, and this is how maintainers burn out.
In Bristol there are many homeless people asking you for money; walking to the city centre or Tesco’s (~2km) can easily mean you’ll be asked 3 or 4 times. Sitting out on the harbourside for dinner or a drink will net you about one homeless person every 30 mins or so on average. Before I lived there I never hesitated to give some change if I had any, because these kind of things are a fairly rare event in Eindhoven. But if you’re asked multiple times every day it just becomes unrealistic. I found it difficult, because I don’t want to say “no”, but I also can’t say “yes” all the time. One of the many reasons I was happy to leave that place.
Closed does not imply locked. I close inactive issues because when someone new looks at my repository I want them to see what’s currently being discussed — not every random bug or feature request everyone has ever considered. (Yes, the real problem is that GitHub’s issue UI sucks at sorting and displaying things that matter. Stale bots are just duct tape.)
In my experience, there are two pain points with stale bots:
Inactive maintainers whose stale bot is more active than they are. Meta: I tried to improve Probot’s stale bot and spent months fighting their bot while begging for a review.
Participants who can’t separate the “issue closed” and “bug is solved” semantics. For most projects closing an issue is much closer to “I’m not convinced that this is a discussion I’d like to highlight for people browsing the issue list”. Or even “I have limited energy and having 900 open issues with no activity for three years stresses me out”. I think we have this problem because the semantics of open/closed differ and it affects the UI. (Consider how much we’d argue about whether something is a bug/feature if GitHub only showed bugs by default.)
Yet, every issue I search on the web that comes up with a GitHub issue for exactly the issue I have which has been marked stale is closed for non-collaborators.
Do you happen to remember which bot is closing them? The default configurations for the stale bot and the stale GitHub Action don’t automatically lock issues, but it’s possible that everyone is configuring them the same way.
On the other hand, I could still see the “this issue is locked, please create a new one if you want to talk about this” being a justifiable option. (Personally, I would probably configure this for a longer period of time after closing.) There’s certainly a trade-off, as it raises the barrier to entry for necro-bumping, but it also means that each thread has less historical context baggage.
My main point is regarding the semantics of open/closed, in retrospect I probably should have lead with that.
The heart of the issue is that GH issues (and all of the various clones) are really really bad for triage. People are talking about the burden of thousands of un-triaged issues, but in my experience, they’re barely even useful for keeping track of a couple dozen issues, at least without a bunch of extra stuff layered on top.
Bots that auto-lock issues seem like a really bad solution to me. I have more experience with projects who’s bot will comment after a couple months, and then after some additional amount of time if there’s no activity, close the issue.
This seems like a better approach, but there are several issues I’m following where this just results in somebody immediately commenting “not stale” to keep it open, and on it goes. The bug still just sits there, open and unaddressed, but now there’s more accidental spam in the world.
I think the root of the problem is that the open/closed classification isn’t nuanced enough – it seems like it’s a good fit for support tickets, but that’s not really how people use FOSS issue trackers. It is possible for an issue to be unaddressed, unlikely to be addressed by the maintainers themselves anytime soon, and yet still useful to have a central place to keep track of discussion around the issue. A proper FOSS issue tracker should be able to reflect this state – valid, but low priority, and provide a way to simultaneously not clutter the maintainers’ dashboards with it but still let users who care about it discuss, find it when googling error messages, propose fixes etc.
Other folks in this thread have made some good suggestions: marking an issue as waiting on submitter, bouncing it back to triage when there’s activity, etc.
I would draw a distinction between the stale bots that are really aggressive and those that aren’t. I’ve seen stale bot configurations that wait a long time for inactivity, and when they do comment, give those subscribed a while to respond (I’ve seen 90 days which is lovely). Those aren’t too bad. But the more aggressive they become, the more they start to feel frustrating or like spam.
Also, most of the effects of people being forced to file duplicates come from stale bots locking issues. If you don’t lock the issue when you close it, then people can just comment and ask for it to be reopened.
(Edit: improve clarity)
Interesting that the discussion around this all sort of seems to assume that the maintainers are choosing not to fix bugs and therefore must be bad at running their projects.
That’s probably true pretty often. But I think it’s also often true that open-source projects have a small, more-or-less-fixed time budget for maintenance that does not expand at the same rate as the growth of the user base, whereas the volume of bug reports does expand along with the user base. Over the years I’ve frequently been surprised to learn that some piece of software I use constantly was the product of one person working solo and it wasn’t even their day job.
If I publish something I wrote in my spare time to scratch an itch, and it happens to get popular enough to generate more bug reports than I can handle in the few hours a week I’m able to spend on the thing, what are my options, other than quitting my job and working full-time on bug fixing? Do I delete the popular repo that people are finding useful enough to file lots of bug reports about, in the name of reducing the number of unhealthy projects in the world? Do I just let the bug list expand to infinity knowing I’ll never have time to clear the backlog? Do I spend my project time reviewing historical bug reports rather than working on the code?
In theory, of course, a project that gets bug reports should have lots of people volunteering to fix the bugs. But the quantity and quality of community contributions seems to vary dramatically from one project to the next.
It’s not about not fixing bugs. That’s expected, normal, fine. Maintainer does not owe me labour. Closing issues that aren’t fixed is just an FU to the users, though. Let the issue stay until someone fixes it, hopefully someone being affected by it.
Exactly this. I was raised in a world where SFTW or RTFM was an acceptable response, and so put effort into providing good issues. Having a bot drive by and close/lock my still open issue is a slap in the face with a wet fish.
I’ve never seen this acronym before? What does it mean?
It was meant to be STFW, but I can’t type.
Search The Fucking Web
(I had still never seen this one and had to look it up … by searching the web)
Ironically this becomes more difficult as the GitHub bug pages show up first and are locked/closed due to being old.
What’s the downside of letting the open bug list expand?
Eh. Caveats first:
I’m not a fan of simple auto-closing stale bots (roughly: I think the incentives are misaligned, and ~stale is a not-quite-right proxy of a few real problems, and it is trying to work around some limitations of using the GH issue system for triage)
but I am a big fan of having a way to punt unactionable issues back to the reporter, and let automation close the issue if the reporter never responds or return it to the triage queue if they do
That out of the way: in a busy long-term project, a stack of open bug reports comes with ongoing logistical overhead. GH’s UI/X exacerbates this, but some of it just comes with the territory.
Reports don’t have to be bad or obsolete to contribute to this cost. 10 or 20 brilliant suggestions may be an asset if you can find time to implement 1 a year for the next 5 years. 1000 brilliant suggestions may be an albatross if the extra logistical overhead fritters away the time and motivation you need to implement one.
It doesn’t, of course, apply to all reports equally. It’s better to have a single long-lived “darkmode pls” issue with gobs of comments and reactions than to deal with someone opening a new one every few days.
Similar to not cleaning up your house. Increased cognitive load and ugliness.
This. I am constantly surprised how many people dismiss the paralyzing effect of 2.4k open issues, even if they all make sense in some way. Resources are limited.
Wouldn’t auto-closing issues be like just hiding everything under the bed in this analogy?
This all looks like the consequence of bad tooling to me.
Yes. The issue trackers we have need the following features.
You do that, you fit 99% of people’s needs.
Imagine if github had a tag “maintainers will not address” and the automatic filter (currently set to only show issues where their lifecycle is at “open”) only showed “open & not(tag:will-not-address-but-might-merge)”
People would be happier to tune that automatic filter for their own needs, and this would allow a tag to banish the issue from a maintainer’s headspace.
Mostly that there will be a certain number of old bugs that get fixed or rendered irrelevant as a side effect of some other change, and searching through the bug database will become less useful if a sizable portion of the bug reports reflect an obsolete reality.
I’ve occasionally run into this on existing projects, in fact: I will search for some error message I’m seeing, find a bug report with some discussion that includes workarounds people used, and only after wasting time trying those workarounds and digging into why they’re not working for me do I discover that the bug they were talking about is in a part of the code that no longer even exists.
Of course, in those kinds of cases, issues being auto-closed by a stale bot wouldn’t have helped much; I’d have still found them and probably still tried the obsolete workarounds. I don’t have a perfect answer for this either!
I haven’t heard a good case for a stale-bot. In the article it says “to reduce the number of open tickets”, but it doesn’t explain why that is a good thing – the number could be reduced to zero by not having a bug tracker at all!
Perhaps it is a UI thing, and Github should display the number of “active” open tickets, for some activity metric.
I turned a stale bot on for a popular project because I thought it would be a more honest communication of the likely fate of old GitHub reports - “this is stale and we aren’t planning to triage or address it further”. Few others perceived it that way and, after a few months, the bot was removed.
The triage work is substantial - we spend easily 120 hours a week on community question/support/defect/feature-request triage. I’d love to do more but of course there are fixed dollars and hours to allocate across many priorities.
Anyway, that was my reasoning, which turned out to be incorrect.
Automatically closing stale issues is a good sign that a project is unhealthy and doesn’t care about their issue tracker. Good project to avoid contributing to, usually.
Automatically closing stale issues is a useful signal that the project follows the CADT development model. https://www.jwz.org/doc/cadt.html
That seems a bit harsh. People posting random non-issues can be a genuine issue for larger projects. People posting on long-since solved issues is also an issue, which tends to be >95% generic support or outright nonsense, and <5% useful.
I don’t care much for auto-close bots, but I understand why people use them. Managing all of this requires a significant amount of time.
I bet Angular had this exact problem; JavaScript tends to attract a lot of beginners and you’re forever cluttering the bug list with non-bugs unless you’re really diligent about maintaining this, and I can’t blame the maintainers on wanting to focus on actually maintaining the Angular project instead of guiding the endless stream of new users unfamiliar with Angular, JavaScript, etiquette, etc. It’s essentially the “Eternal September” problem.
Can’t both of those issues be solved, well, closing the issues manually?
I’d assume long-since solved issues should be closed because solved. For “junk” issues, generic support and whatnot, is it really better to just let them sit open for a week or two (or however long the bot takes) rather than just manually marking them as “offtopic/support/wtfisgoingonhere” and closing them?
Yeah but people will comment on them. With this I meant the “lock bots” that lock issues after being closed for n days which prevents adding new comments.
As for manual closing/locking, sure, but that’s not “free” time-wise, and it can be emotionally draining. I don’t really want to tell people to ask their question somewhere else or that they’re making zero sense, but I also don’t necessarily want to provide mentoring to random newcomers as I got a life to lead and stuff to do. People can also get angry or even abusive about, no matter how nicely and gentle you phrase it (I’ve had that even with random strangers emailing me out of the blue because they saw me on Lobsters, Stack Overflow, GitHub, or wherever). It’s not super-common, but it sucks when it happens.
A bot just makes all of this easier and avoids the emotional drain. Is it the “chicken way out”? I suppose it is, like I said I don’t use it myself and generally just manually lock old issues and such if they attract a lot of comments, but I also never maintained a project the size of Angular, and I can see the reasons why people would use it.
I think the “emotional cost” of maintaining larger open source projects is often underestimated. Everyone is different and some people struggle with this more than others, but personally I find it hard. I want to be helpful, but that’s just not feasible or realistic beyond a certain scale so there’s some amount of (internal) tension there. It also leads to situations where you feel obligated to do things that you don’t really want to do, and this is how maintainers burn out.
In Bristol there are many homeless people asking you for money; walking to the city centre or Tesco’s (~2km) can easily mean you’ll be asked 3 or 4 times. Sitting out on the harbourside for dinner or a drink will net you about one homeless person every 30 mins or so on average. Before I lived there I never hesitated to give some change if I had any, because these kind of things are a fairly rare event in Eindhoven. But if you’re asked multiple times every day it just becomes unrealistic. I found it difficult, because I don’t want to say “no”, but I also can’t say “yes” all the time. One of the many reasons I was happy to leave that place.
I’m curious, which projects do you maintain?
I see this is the ops world too. Not just devs.
I would also suggest that the existence of stale-bots implies deficits in Github’s Issue model and/or UI.
This.
“Open” and “closed” are completely arbitrary semantics that don’t map to whether the issue describes a problem that’s fixed in the default branch.
Closed does not imply locked. I close inactive issues because when someone new looks at my repository I want them to see what’s currently being discussed — not every random bug or feature request everyone has ever considered. (Yes, the real problem is that GitHub’s issue UI sucks at sorting and displaying things that matter. Stale bots are just duct tape.)
In my experience, there are two pain points with stale bots:
Yet, every issue I search on the web that comes up with a GitHub issue for exactly the issue I have which has been marked stale is closed for non-collaborators.
Do you happen to remember which bot is closing them? The default configurations for the stale bot and the stale GitHub Action don’t automatically lock issues, but it’s possible that everyone is configuring them the same way.
On the other hand, I could still see the “this issue is locked, please create a new one if you want to talk about this” being a justifiable option. (Personally, I would probably configure this for a longer period of time after closing.) There’s certainly a trade-off, as it raises the barrier to entry for necro-bumping, but it also means that each thread has less historical
contextbaggage.My main point is regarding the semantics of open/closed, in retrospect I probably should have lead with that.
It comes up most with hashicorp
https://github.com/hashicorp/terraform/issues/27827#issuecomment-804522619
The heart of the issue is that GH issues (and all of the various clones) are really really bad for triage. People are talking about the burden of thousands of un-triaged issues, but in my experience, they’re barely even useful for keeping track of a couple dozen issues, at least without a bunch of extra stuff layered on top.
Bots that auto-lock issues seem like a really bad solution to me. I have more experience with projects who’s bot will comment after a couple months, and then after some additional amount of time if there’s no activity, close the issue.
This seems like a better approach, but there are several issues I’m following where this just results in somebody immediately commenting “not stale” to keep it open, and on it goes. The bug still just sits there, open and unaddressed, but now there’s more accidental spam in the world.
I think the root of the problem is that the open/closed classification isn’t nuanced enough – it seems like it’s a good fit for support tickets, but that’s not really how people use FOSS issue trackers. It is possible for an issue to be unaddressed, unlikely to be addressed by the maintainers themselves anytime soon, and yet still useful to have a central place to keep track of discussion around the issue. A proper FOSS issue tracker should be able to reflect this state – valid, but low priority, and provide a way to simultaneously not clutter the maintainers’ dashboards with it but still let users who care about it discuss, find it when googling error messages, propose fixes etc.
Other folks in this thread have made some good suggestions: marking an issue as waiting on submitter, bouncing it back to triage when there’s activity, etc.
I just want to say that I hate stale bots, I just want to contribute if I found a solution to the bug or see the solution others have found