my partner submitted a patch to OpenBSD a few weeks ago, and he had to set up an entirely new mail client which didn’t mangle his email message to HTML-ise or do other things to it, so he could even make that one patch. That’s a barrier to entry that’s pretty high for somebody who may want to be a first-time contributor.
I think it was actually Gmail that was a barrier. And he also couldn’t do it from Apple Mail. It is just that the modern mail client has intentionally moved towards HTML
I am flabbergasted someone is able to send a patch to OpenBSD but is not able to set the web interface of GMail to sent plain-text emails. Or install Thunderbird, which might have been the solution, in that particular case.
I also never used git send-email, but I don’t think it is the barrier to become a kernel maintainer.
Actually, that might work as an efficient sieve to select the ones who want to be active contributors from those who are just superficial or ephemeral contributors.
In my opinion, although Github supposedly decreases the barrier of first-time contribution, it increases the load on the maintainers, which has to interact with a lot of low-quality implementation, reckless hacks and a torrent of issues which might not even be. That is my experience in the repositories I visit or contributed.
On the other hand, if someone knows how to install OpenBSD, use cvs (or git), program in C and navigate the kernel’s souce-code, I supposed they are capable of going to anysearchengine and find the answer on how to send plain-text email via the GMail interface.
GMail mangles your “plain-text” messages with hard-wraps and by playing with your indentation. You’d have to install and configure thunderbird or something.
Note that this is anecdotal hearsay. This person is saying their partner had to set things up… they may have misunderstood their partner or not realized their partner was exaggerating.
Also, one might expect the amount of patience and general debugging skill necessary to transfer rather well to the domain of email configuration.
It’s also possible that guiraldelli is assuming it was an excepted OpenBSD kernel patch, wheras we don’t know if the patch was accepted and we don’t know if it was a userland patch. It doesn’t take much skill to get a userland patch rejected.
I am flabbergasted someone is able to send a patch to OpenBSD but is not able to set the web interface of GMail to sent plain-text emails.
You can’t send patches, or indeed any formatted plain text emails through the gmail web interface. If you set gmail to plain text, gmail will mangle your emails. It will remove tabs and hardwrap the text. It’s hopeless.
Think about impediments in terms of probabilities and ease of access.
You wouldn’t believe how many people stop contributing because of tiny papercuts. There is a non-trivial amount of people who have the skills to make meaningful contributions, but lack the time.
Lobsters requires an invite. That’s a barrier to entry, so those who make it are more likely to make good contributions, and have already extended effort to comply with the social norms.
You may be willing to jump a barrier of entry for entertainment (that is lobsters) but not for free work for the benefit of others (because you have already done that work for yourself).
If the project wants to have contributors in the future it might want to consider using modern, user friendly technologies for contribution.
Linux is considering it, a discussion is started, which is positive (for the future of the project) in my opinion. It is a sign or responsive project management, where its conservatism does not hinder progress, only slows it to not jump for quickly fading fads.
Other communities are closing their microverse on themselves, where it is not enough to format your contribution in N<80 column plain text email without attachments with your contribution added to it via come custom non-standard inline encoding (like pasting the patch to the end of the line), but you may even need to be ceremonially accepted for contribution by the elders. Also some such projects still don’t use CI, or automated QA. These communities will die soon, actually are already dead, especially as they don’t have large corporations backing them, as someone getting paid usually accepts quite a few papercuts a job demands. This is why Linux could actually allow itself to be more retrograde than the BSD projects for example, which I think are even more contributor-unfriendly (especially ergonomically).
I tend to agree, however that discussion was started by someone with a very striking conflict of interest and therefore their opinion should be questioned and considered to be insincere at best and maleficent at worst.
That doesn’t matter, the discussion can be done despite that. I think a forge-style solution is the key.
Despite Microsoft having 2 “forge” type solutions in its offering (both quite usable in my experience, github being a de-facto standard) I still cannot see a conflict of interest in this topic. There are other software forges available. The current process is simply outdated.A custom solution could also be developed, if deemed necessary, as it was the case with git.
Pay attention my comment talks about maintainers, active contributors and superficial or ephemeral contributors.
From the article:
a problem recently raised by Linux kernel creator Linus Torvalds, that “it’s really hard to find maintainers.” Maintainers are the gatekeepers who determine what code ends up in the widely used open-source kernel, and who ensure its quality. It is part of a wider discussion about who will oversee Linux when the current team moves on.
And later on…
“We need to set up a better or a different or an additional way to view tools and the work that’s being done in the Linux project as we’re trying to bring in new contributors and maintain and sustain Linux in the future,” she told us.
Picking her words carefully, she said work is being done towards “moving from a more text-based, email-based, or not even moving from, but having a text-based, email-based patch system that can then also be represented in a way that developers who have grown up in the last five or ten years are more familiar with.
“Having that ability and that perspective into Linux is one of the ways we hope we can help bring newer developers into the kernel track.”
So I understood Sarah Novotny is addressing the problem of new contributors, not the one Linus Torvalds see, of new maintainers.
So, your comment that
how many people stop contributing because of tiny papercuts. There is a non-trivial amount of people who have the skills to make meaningful contributions, but lack the time
is not the problem the Linux Foundation has, but the one that Microsoft’s Sarah Novotny wants to solve. Those are two different problems, with two different solutions. On the surface, they might seem the same, but they are not. They might be correlated, but the solution for one problem does not necessarily mean it is the solution for the other.
Therefore, my argument still stands:
[having a plain-text email list as central communication and collaboration to the Linux’s kernel] might work as an efficient sieve to select the ones who want to be active contributors [i.e., maintainers] from those who are just superficial or ephemeral contributors.
If we are going to address, though, that it might hinder new contributors, then I tend to agree with you. :)
This is just my opinion but I think that while she’s totally missing the mark on finding Linux kernel module maintainers having anything at all to do with plain text email patch submission systems, the general issue she’s speaking to is one that has generated a lot of discussion recently and should probably not be ignored.
Also, in the spirit of the open source meritocracy, I’d prefer to let people’s actions and track records speak more loudly than which company they happen to work for, but then, lots of people consider my employer to be the new evil empire, so my objectivity in this matter could be suspect :)
So I understood Sarah Novotny is addressing the problem of new contributors, not the one Linus Torvalds see, of new maintainers.
Bingo!
My first thought after reading this was “Does this person ACTUALLY think that having to use plaintext E-mail is even statistically relevant to the problem of finding module maintainers for the Linux kernel?”
In a general sense I believe that the open source community’s reliance on venerable tools that are widely rejected by younger potential contributors is a huge problem.
Martin Wimpress of Ubuntu Desktop fame has spoken about this a lot recently, and has been advocating a new perspective for open source to increase engagement by embracing the communications technologies where the people are not where we would like them to be.
So he streams his project sessions on Youtube and runs a Discord, despite the fact that these platforms are inherently proprietary, and has reported substantial success at attracting an entirely new audience that would otherwise never have engaged.
is not the problem the Linux Foundation has, but the one that Microsoft’s Sarah Novotny wants to solve. Those are two different problems, with two different solutions. On the surface, they might seem the same, but they are not. They might be correlated, but the solution for one problem does not necessarily mean it is the solution for the other.
GKH even said that there are more than enough contributors in his last ama, so having contributors is “a priori” not a problem right now.
Also better tooling would be beneficial for everyone.
That is not necessarily true: the introduction of a tool requires a change of mindset and workflow of every single person already involved in the current process, as well as update of the current instructions and creation of new references.
I saw projects that took months to change simply a continuous integration tool (I have GHC in mind, except the ones in the companies I worked for). Here, we are talking about the workflow of many (hundreds for sure, but I estimate even thousands) maintainers and contributors.
If the problem is retrievability of the manual or specific information, than I think that should be addressed first. But that topic is not brought up in the article.
I am able to deal with email patches, but I hate doing it. I would not enjoying maintaining a project which accepts only email patches.
Actually, that might work as an efficient sieve to select the ones who want to be active contributors from those who are just superficial or ephemeral contributors.
Ephemeral contributors evolve to maintainers, but if that initial step is never made this doesn’t happen. I don’t know if this will increase the number of maintainers by 1%, 10%, or more, but I’m reasonably sure there will be some increase down the line.
Finding reliable maintainers is always hard by the way, for pretty much any project. I did some git author stats on various popular large open source projects, and almost all of them have quite a small group of regular maintainers and a long tail of ephemeral contributors. There’s nothing wrong with that as such, but if you’re really strapped for maintainers I think it’s a good idea to treat every contributor as a potential maintainer.
But like I said, just because you can deal with email patches doesn’t mean you like it.
GMail, even in plain-text mode, tends to mangle inline patches since it has no way of only auto-wrapping some lines and not others.
Not a major issue as you can attach a diff, but from a review-perspective I’ve personally found folks more inclined to even look at my patches if they’re small enough to be inlined into the message. I say this as someone having both submitted patches to OpenBSD for userland components in base as well as having had those patches reviewed and accepted.
I personally gave up fighting the oddities of GMail and shifted to using Emacs for sending/receiving to the OpenBSD lists. I agree with Novotny’s statement that “GMail [is] the barrier.” The whole point of inlining patches like this is the email body or eml file or whatever can be direct input to patch(1)…if GMail wraps lines of the diff, it’s broken the patch. Text mode or not.
Obviously, this doesn’t mean if the Linux kernel maintainers want to change their process that I don’t think they shouldn’t. (Especially if they take a lot of funding from the Foundation…and as a result, Microsoft/GitHub.) OpenBSD is probably always going to take the stance that contributions need to be accessible to those using just the base system…which means even users of mail(1) in my interpretation.
Good point…so basically GMail (the client) doesn’t really work with an email-only contribution system. It’s Gmail SMTP via mutt/emacs/thunderbird/etc. or bust.
I have found several Linux subtle kernel bugs related to PCIe/sysfs/vfio/NVMe hotplug. I fixed them locally in our builds, which will ultimately get published somewhere. I don’t know where, but it’s unlikely to ever get pushed out to the real maintainers.
The reason I don’t try to push them out properly? The damn email process.
I did jump through the hoops many years ago to configure everything. I tried it with gmail, and eventually got some Linux email program working well enough that I was able to complete some tutorial. It took me some non-negligible amount of time and frustration. But that was on an older system and I don’t have things set up properly anymore.
I don’t want to go through that again. I have too many things on my plate to figure out how to reconfigure everything and relearn the process. And then I won’t use it again for several months to a year and I’ll have to go through it all again. Sometimes I can get a coworker to push things out for me - and those have been accepted into the kernel successfully. But that means getting him to jump through the hoops for me and takes time away from what he should be doing.
So for now, the patches/fixes get pulled along with our local branch until they’re no longer needed.
I have no idea how dense they need to be to not be able to send text-only email from Apple Mail. It must be anecdotal, because I cannot believe that someone is smart enough to code anything, but dumb enough to be able to click Format > Change to text in the menu bar.
Corporate email is often based on an Outlook server, and it is becoming increasingly difficult to setup a regular email client that is relying on IMAP and SMTP to work with the corporate outlook server.
getting two factor authentication to work is a problem, and for Thunderbird to work with Outlook you need to either:
So yes, getting a plain text email client to work as a corporate developer is a problem, but you might as well blame Microsoft for putting up these barriers in their email server.
I work in a company where the email infrastructure is Office 365. After many interactions with the IT team, I made them whitelist DavMail, which is connected to my Thunderbird, in which I use exteditor to write my emails using, in my case, Neovim.
A friend of mine, who has a day-to-day job as contributor of a few FLOSS projects, had problems sending email from his company’s infrastructure (also Office 365) and decided to send patches with his private email. Last time we spoke about it, he was about to buy an Owl’s license.
Thus, I agree with you: Microsoft is the problem in here. And the article even touches that point:
We assumed that Outlook was to blame. Could Microsoft fix that instead? “The question always is fix it to whose standards, because we are focused much more on business and enterprise models of clients and customers. For them we fixed it to a more HTML-based model so it really depends on who your audience is and who your target is.”
It turned out, though, that this time Outlook was not guilty. “I think it was actually Gmail that was a barrier.[…]”
Good riddance to email based patching workflows. They’re fragile and clumsy; a forge is a much richer experience, but so much of the software development industry is really reactionary .
But the real problem is if any workflow gets forced upon a project because of its benefactors.
Several times in the last twenty years I’ve wanted to become a kernel contributor, but they’re so far away from being nice and supportive and helpful that I just won’t.
I think this is another case where technology will not solve your people problem.
The obvious solution would be to make GitHub (also Microsoft) work properly with mail based workflows of (other) projects instead of effectively forcing projects to go “all in” on GitHub. It’d be similar to sr.ht/SourceHut, but with a familiar web UI.
I think you can already set this up; Vim does this. I’m not entirely sure how they set it up, but issues/PRs get sent to the mailing list, and people can reply from there (while others reply via the web interface). It seems to work reasonably well for everyone.
Is it harder to send a plain-text email than to create a GitHub account (or equivalent,) create/upload SSH keys, learn non-git concepts (in addition to git concepts) like pull requests, and whatever else GH requires?
I think so, yes. The GitHub UI is fairly straightforward. Sending email patches isn’t, especially if you’re not very familiar with email, and there’s a long list of stuff you can do wrong.
Whether something is harder or easier to learn depends largely on your starting point and available educational resources.
I think we’ve reached a tipping point where A) more developers are familiar with github than with plaintext email, and B) better tutorials exist for github than for plaintext email.
GitHub is a propriety non-free software project controlled by Microsoft. A free software project like the Linux kernel should not treat the number of potential developers who are familiar with proprietary software-based workflows as relevant for running their free software project.
git-send-email is more scary than hard. It’s an interface that always makes me feel like I’m one keystroke away from doing something wrong/embarrassing and there’s no way back from sending an email.
In the Phabricator patch workflow or the GH/GL pull request workflow, there’s always a very clear preview of how everything would exactly look when it’s published.
--annotate can help you with that, letting you preview and edit emails before sending them. There is also --confirm=always that lets you confirm weather to send a message or not.
Well with the Github workflow, you cannot retract a pull request either. And if you push something bad and quickly git commit –amend ; git push -f your code, Github will show everyone that you force pushed.
I think that this is just one of the cases where Github is not intimidating because you know it, while the Git e-mail fow is intimidating because abeit it being around much longer, it’s less used.
We definitely need more and better guides for the mail flow (Drew has a nice start, we need more of those!)
I feel the same way - you don’t want to be the n00b who messes up and causes a maintainer to simply drop your patch on the floor without feedback.
However, this points to the need for lower-stakes projects that work with email-based workflows, so that new developers can acculturate to the workflow and then confidently take part in, for example, kernel development. In that sense, email-focused outfits like Sourcehut are a valuable feeder service.
I definately agree. As a security engineer it took me more that a day to setup git-send-email with my password stored encrypted. Even getting it to connect to the mail server (fastmail) was challenging because of poor documentation on TLS vs STARTTLS. I know many skilled programmers who wouldn’t bother setting this up to submit a patch, ever if they had already written it.
Not all barriers to entry are bad. Passwords, for example, prevent those who do not know and cannot guess them from accessing accounts. Likewise, requiring plain-text email (i.e., real email) discourages those who do not understand email from easily taking part in the kernel development process.
Is that bad? I don’t really think so. Although understanding of email is not a perfect proxy for understanding of the kernel, I imagine they are pretty highly correlated. Indeed, I would be cautious about running code from someone who cannot or does not configure his email client properly.
It’s a welcome barrier IMHO. If someone can’t configure an email client properly I would question whether they should be trusted with writing kernel code.
Yeah, why? I don’t know a lot of folks who are actively working on the Linux kernel, but they all do so natively in Linux (even if they still use a VM for testing kernel changes). But obviously my anecdote is just an anecdote, so I’m curious if you’ve experienced otherwise or ??
This was in TFA as well. Apple Mail absolutely does have that setting and I’ve been using it reliably for well over 15 years. The cut-down version on iOS doesn’t but that’s just one of the many ways in which it’s awful for anything other than quickly skimming emails and flagging some to reply to on a real computer (unreliable updating IMAP folders, sometimes deciding a folder is completely empty, no way of marking a folder as read, and so on). If you’re using mobile email for submitting patches, you’re probably doing many other things wrong.
If anything, the Linux kernel should consider sourcehut. It’s probably the only platform that works well with open standards and decentralised systems.
I think Sourcehut is an incredible product and speaks very well of Drew Devault’s technical and business acumen, but I’m not convinced that where Linux hosts its source code is its biggest problem to be worrying about.
I don’t think it’s the most pressing matter right now, but it is important to keep in mind that it was once and it was solved. Due to the whole BitKeeper mess, Torvald’s started working on git as a means to perform distributed version control - and mail was the cornerstone. Now, GitHub is lobbying to drift apart from email and use their non-standard and close issue management system. It does look familiar.
Alpine adopted sourcehut and it didn’t end well. Sourcehut’s extreme opinions on mail left package maintainers unable to send mail because Devault didn’t like the settings you sometimes couldn’t even change. After some developers left the project, they instead switched to GitLab.
Alpine seems to be a bit all over the place at the moment… Their repos look to be self-hosted cgit, their mailing lists are Sourcehut, their bug tracker is GitLab and their wiki is self-hosted Mediawiki.
Edit: ah, their cgit-hosted repos are a mirror of what’s in GitLab, so I guess that’s an interim measure.
This is an extremely uninformed and shitty take on what happened with Alpine Linux which doesn’t map at all onto reality. Keep your slander to yourself.
As someone with no stake in this, I went and had a look at the available evidence, and… maybe sourcehut needs to correct the record somewhere?
After looking over the alpine mailing lists, I got the distinct impression that some widely-used mail clients can’t send mail to sourcehut and some alpine contributors have refused to use it on that basis.
It never caused any developers to “leave the project”. The arguments were not being made in good faith, either. There was a subversive element with a personal vendetta against me who were pulling strings under the table to crank up the drama to 11. As a matter of fact, it was settled after an IRC discussion and the Alpine mailing lists (still running sourcehut) work fine with those mail clients now. All I pushed for was due consideration of the rationale behind the design decision, and during the discussion, a compromise was reached and implemented.
And they didn’t “switch” to GitLab from sourcehut, either. They were never using more than the mailing list features of sourcehut, and there was never a plan to start using them. The sourcehut mailing lists are still in use today, and the GitLab migration was planned in advance of that and completed for unrelated reasons.
There are persistent, awful comments from a handful of people who spread lies and rumors about what happened as if I attempted a hostile takeover of Alpine Linux. It’s complete horseshit, and incredibly offensive and hurtful, especially in light of how much time and effort I’ve devoted to improving Alpine Linux. I am a maintainer of many packages myself, and a contributor to many of their repositories!
Last time I had to deal with a mailing list, I literally just sent the patch as an attachment in an email. I don’t care, if they don’t want fixes they can ignore it and continue being ignorant of it. I’ve had this mutual feeling, especially with linux kernel development, for a long time.
I submitted a couple of patches where the maintainers have not responded to anything on their (low-volume) mailing list in about a month. I don’t even know if they’re even getting any of the emails, and I’m not sure what to do next. Moving away from email as the source of truth would at least solve that issue.
What is the future of Linux at Microsoft? “Linux at Microsoft continues to grow both as an investment space and a usage space,” said Novotny. “We have more than half of the Azure cores now are running Linux in some flavor.
“The teams that work on this are spending a lot of effort upstreaming any of the changes they find or make in order to make sure that we’re being good open source citizens. We have one Microsoft member on the technical advisory board for Linux. Sasha [Levin] has that role, and he and I collaborate on how do we help this move in a direction that is good for Linux,” she said.
Aside from the kerfuffle about email clients (which the savvy editors at El Reg no doubt selected as lede to get outrage clicks), this is a pretty amazing development. If you’d asked me 20 years ago whether MS would be actively contributing to the Linux kernel I’d have been incredulous.
I am flabbergasted someone is able to send a patch to OpenBSD but is not able to set the web interface of GMail to sent plain-text emails. Or install Thunderbird, which might have been the solution, in that particular case.
I also never used
git send-email, but I don’t think it is the barrier to become a kernel maintainer.Actually, that might work as an efficient sieve to select the ones who want to be active contributors from those who are just superficial or ephemeral contributors.
In my opinion, although Github supposedly decreases the barrier of first-time contribution, it increases the load on the maintainers, which has to interact with a lot of low-quality implementation, reckless hacks and a torrent of issues which might not even be. That is my experience in the repositories I visit or contributed.
Patching some component or other of OpenBSD is not a skill that transfers well to making some random application do what you want.
I agree with you.
On the other hand, if someone knows how to install OpenBSD, use
cvs(orgit), program in C and navigate the kernel’s souce-code, I supposed they are capable of going to any search engine and find the answer on how to send plain-text email via the GMail interface.GMail mangles your “plain-text” messages with hard-wraps and by playing with your indentation. You’d have to install and configure thunderbird or something.
One of the many reasons why I use Mutt.
Note that this is anecdotal hearsay. This person is saying their partner had to set things up… they may have misunderstood their partner or not realized their partner was exaggerating.
Also, one might expect the amount of patience and general debugging skill necessary to transfer rather well to the domain of email configuration.
It’s also possible that guiraldelli is assuming it was an excepted OpenBSD kernel patch, wheras we don’t know if the patch was accepted and we don’t know if it was a userland patch. It doesn’t take much skill to get a userland patch rejected.
You can’t send patches, or indeed any formatted plain text emails through the gmail web interface. If you set gmail to plain text, gmail will mangle your emails. It will remove tabs and hardwrap the text. It’s hopeless.
Think about impediments in terms of probabilities and ease of access.
You wouldn’t believe how many people stop contributing because of tiny papercuts. There is a non-trivial amount of people who have the skills to make meaningful contributions, but lack the time.
Lobsters requires an invite. That’s a barrier to entry, so those who make it are more likely to make good contributions, and have already extended effort to comply with the social norms.
You may be willing to jump a barrier of entry for entertainment (that is lobsters) but not for free work for the benefit of others (because you have already done that work for yourself).
If you want to save yourself compiling and patching your kernel every time there’s an update, you might want to submit it to the project maintainer.
If the project wants to have contributors in the future it might want to consider using modern, user friendly technologies for contribution.
Linux is considering it, a discussion is started, which is positive (for the future of the project) in my opinion. It is a sign or responsive project management, where its conservatism does not hinder progress, only slows it to not jump for quickly fading fads.
Other communities are closing their microverse on themselves, where it is not enough to format your contribution in N<80 column plain text email without attachments with your contribution added to it via come custom non-standard inline encoding (like pasting the patch to the end of the line), but you may even need to be ceremonially accepted for contribution by the elders. Also some such projects still don’t use CI, or automated QA. These communities will die soon, actually are already dead, especially as they don’t have large corporations backing them, as someone getting paid usually accepts quite a few papercuts a job demands. This is why Linux could actually allow itself to be more retrograde than the BSD projects for example, which I think are even more contributor-unfriendly (especially ergonomically).
I tend to agree, however that discussion was started by someone with a very striking conflict of interest and therefore their opinion should be questioned and considered to be insincere at best and maleficent at worst.
That doesn’t matter, the discussion can be done despite that. I think a forge-style solution is the key.
Despite Microsoft having 2 “forge” type solutions in its offering (both quite usable in my experience, github being a de-facto standard) I still cannot see a conflict of interest in this topic. There are other software forges available. The current process is simply outdated.A custom solution could also be developed, if deemed necessary, as it was the case with git.
[Comment removed by author]
Pay attention my comment talks about maintainers, active contributors and superficial or ephemeral contributors.
From the article:
And later on…
So I understood Sarah Novotny is addressing the problem of new contributors, not the one Linus Torvalds see, of new maintainers.
So, your comment that
is not the problem the Linux Foundation has, but the one that Microsoft’s Sarah Novotny wants to solve. Those are two different problems, with two different solutions. On the surface, they might seem the same, but they are not. They might be correlated, but the solution for one problem does not necessarily mean it is the solution for the other.
Therefore, my argument still stands:
If we are going to address, though, that it might hinder new contributors, then I tend to agree with you. :)
Let’s not let Microsoft decide how Linux is developed.
The waning popularity of their own, proprietary kernel is no excuse for telling other projects how they need to be run.
This is just my opinion but I think that while she’s totally missing the mark on finding Linux kernel module maintainers having anything at all to do with plain text email patch submission systems, the general issue she’s speaking to is one that has generated a lot of discussion recently and should probably not be ignored.
Also, in the spirit of the open source meritocracy, I’d prefer to let people’s actions and track records speak more loudly than which company they happen to work for, but then, lots of people consider my employer to be the new evil empire, so my objectivity in this matter could be suspect :)
[Comment from banned user removed]
Bingo!
My first thought after reading this was “Does this person ACTUALLY think that having to use plaintext E-mail is even statistically relevant to the problem of finding module maintainers for the Linux kernel?”
In a general sense I believe that the open source community’s reliance on venerable tools that are widely rejected by younger potential contributors is a huge problem.
Martin Wimpress of Ubuntu Desktop fame has spoken about this a lot recently, and has been advocating a new perspective for open source to increase engagement by embracing the communications technologies where the people are not where we would like them to be.
So he streams his project sessions on Youtube and runs a Discord, despite the fact that these platforms are inherently proprietary, and has reported substantial success at attracting an entirely new audience that would otherwise never have engaged.
GKH even said that there are more than enough contributors in his last ama, so having contributors is “a priori” not a problem right now.
Every maintainer was a new contributor at a point! Also better tooling would be beneficial for everyone.
That is true, but it is not the current problem: see /u/rmpr’s reply and his link to the Reddit’s AMA to verify that.
That is not necessarily true: the introduction of a tool requires a change of mindset and workflow of every single person already involved in the current process, as well as update of the current instructions and creation of new references.
I saw projects that took months to change simply a continuous integration tool (I have GHC in mind, except the ones in the companies I worked for). Here, we are talking about the workflow of many (hundreds for sure, but I estimate even thousands) maintainers and contributors.
As I already told before, RTFM [1] [2] does not take long for a single individual that wants to become a new contributor; in [1], there is even a session specially about the GMail’s Web GUI problem.
If the problem is retrievability of the manual or specific information, than I think that should be addressed first. But that topic is not brought up in the article.
I am able to deal with email patches, but I hate doing it. I would not enjoying maintaining a project which accepts only email patches.
Ephemeral contributors evolve to maintainers, but if that initial step is never made this doesn’t happen. I don’t know if this will increase the number of maintainers by 1%, 10%, or more, but I’m reasonably sure there will be some increase down the line.
Finding reliable maintainers is always hard by the way, for pretty much any project. I did some git author stats on various popular large open source projects, and almost all of them have quite a small group of regular maintainers and a long tail of ephemeral contributors. There’s nothing wrong with that as such, but if you’re really strapped for maintainers I think it’s a good idea to treat every contributor as a potential maintainer.
But like I said, just because you can deal with email patches doesn’t mean you like it.
GMail, even in plain-text mode, tends to mangle inline patches since it has no way of only auto-wrapping some lines and not others.
Not a major issue as you can attach a diff, but from a review-perspective I’ve personally found folks more inclined to even look at my patches if they’re small enough to be inlined into the message. I say this as someone having both submitted patches to OpenBSD for userland components in base as well as having had those patches reviewed and accepted.
I personally gave up fighting the oddities of GMail and shifted to using Emacs for sending/receiving to the OpenBSD lists. I agree with Novotny’s statement that “GMail [is] the barrier.” The whole point of inlining patches like this is the email body or eml file or whatever can be direct input to
patch(1)…if GMail wraps lines of the diff, it’s broken the patch. Text mode or not.Obviously, this doesn’t mean if the Linux kernel maintainers want to change their process that I don’t think they shouldn’t. (Especially if they take a lot of funding from the Foundation…and as a result, Microsoft/GitHub.) OpenBSD is probably always going to take the stance that contributions need to be accessible to those using just the base system…which means even users of
mail(1)in my interpretation.You can’t attach a diff – nearly all mailing lists remove attachments!
Heh, if you add
arcto the base system, you can adopt Phabricator without violating that principle :)Good point…so basically GMail (the client) doesn’t really work with an email-only contribution system. It’s Gmail SMTP via mutt/emacs/thunderbird/etc. or bust.
I have found several Linux subtle kernel bugs related to PCIe/sysfs/vfio/NVMe hotplug. I fixed them locally in our builds, which will ultimately get published somewhere. I don’t know where, but it’s unlikely to ever get pushed out to the real maintainers.
The reason I don’t try to push them out properly? The damn email process.
I did jump through the hoops many years ago to configure everything. I tried it with gmail, and eventually got some Linux email program working well enough that I was able to complete some tutorial. It took me some non-negligible amount of time and frustration. But that was on an older system and I don’t have things set up properly anymore.
I don’t want to go through that again. I have too many things on my plate to figure out how to reconfigure everything and relearn the process. And then I won’t use it again for several months to a year and I’ll have to go through it all again. Sometimes I can get a coworker to push things out for me - and those have been accepted into the kernel successfully. But that means getting him to jump through the hoops for me and takes time away from what he should be doing.
So for now, the patches/fixes get pulled along with our local branch until they’re no longer needed.
I have no idea how dense they need to be to not be able to send text-only email from Apple Mail. It must be anecdotal, because I cannot believe that someone is smart enough to code anything, but dumb enough to be able to click
Format > Change to textin the menu bar.Recommend adding this response to the story:
https://drewdevault.com/2020/08/27/Microsoft-plays-their-hand.html
Corporate email is often based on an Outlook server, and it is becoming increasingly difficult to setup a regular email client that is relying on IMAP and SMTP to work with the corporate outlook server.
getting two factor authentication to work is a problem, and for Thunderbird to work with Outlook you need to either:
So yes, getting a plain text email client to work as a corporate developer is a problem, but you might as well blame Microsoft for putting up these barriers in their email server.
I work in a company where the email infrastructure is Office 365. After many interactions with the IT team, I made them whitelist DavMail, which is connected to my Thunderbird, in which I use
exteditorto write my emails using, in my case, Neovim.A friend of mine, who has a day-to-day job as contributor of a few FLOSS projects, had problems sending email from his company’s infrastructure (also Office 365) and decided to send patches with his private email. Last time we spoke about it, he was about to buy an Owl’s license.
Thus, I agree with you: Microsoft is the problem in here. And the article even touches that point:
At last, I expect people contributing to the kernel to RTFM: there is a kernel.org’s page on email clients (for collaboration on the Linux’s kernel), describing problematic MUAs and even given basic configuration for some of them. And it is not the unique reference on the subject.
So much fuzz for something that does not even address the problem of lack of good maintainers for the kernel. 🤦
Good riddance to email based patching workflows. They’re fragile and clumsy; a forge is a much richer experience, but so much of the software development industry is really reactionary .
But the real problem is if any workflow gets forced upon a project because of its benefactors.
Several times in the last twenty years I’ve wanted to become a kernel contributor, but they’re so far away from being nice and supportive and helpful that I just won’t.
I think this is another case where technology will not solve your people problem.
The obvious solution would be to make GitHub (also Microsoft) work properly with mail based workflows of (other) projects instead of effectively forcing projects to go “all in” on GitHub. It’d be similar to sr.ht/SourceHut, but with a familiar web UI.
I think you can already set this up; Vim does this. I’m not entirely sure how they set it up, but issues/PRs get sent to the mailing list, and people can reply from there (while others reply via the web interface). It seems to work reasonably well for everyone.
It wouldn’t really be the same as sr.ht since that doesn’t support the PR-based flow.
translation: linux will have to move to github eventually, if they want to keep receiving microsoft money
For anyone wanting to learn how to send a patch via email, here’s a nice tutorial: https://git-send-email.io/
Is it harder to send a plain-text email than to create a GitHub account (or equivalent,) create/upload SSH keys, learn non-git concepts (in addition to git concepts) like pull requests, and whatever else GH requires?
I think so, yes. The GitHub UI is fairly straightforward. Sending email patches isn’t, especially if you’re not very familiar with email, and there’s a long list of stuff you can do wrong.
Whether something is harder or easier to learn depends largely on your starting point and available educational resources.
I think we’ve reached a tipping point where A) more developers are familiar with github than with plaintext email, and B) better tutorials exist for github than for plaintext email.
GitHub is a propriety non-free software project controlled by Microsoft. A free software project like the Linux kernel should not treat the number of potential developers who are familiar with proprietary software-based workflows as relevant for running their free software project.
Not sure what that has to do with the question I was answering, which was about the relative difficulty of different workflows.
git-send-emailis more scary than hard. It’s an interface that always makes me feel like I’m one keystroke away from doing something wrong/embarrassing and there’s no way back from sending an email.In the Phabricator patch workflow or the GH/GL pull request workflow, there’s always a very clear preview of how everything would exactly look when it’s published.
--annotatecan help you with that, letting you preview and edit emails before sending them. There is also--confirm=alwaysthat lets you confirm weather to send a message or not.Yes, I’ve used both of course. Can’t say it helps with the feeling.
Well with the Github workflow, you cannot retract a pull request either. And if you push something bad and quickly git commit –amend ; git push -f your code, Github will show everyone that you force pushed.
I think that this is just one of the cases where Github is not intimidating because you know it, while the Git e-mail fow is intimidating because abeit it being around much longer, it’s less used.
We definitely need more and better guides for the mail flow (Drew has a nice start, we need more of those!)
I feel the same way - you don’t want to be the n00b who messes up and causes a maintainer to simply drop your patch on the floor without feedback.
However, this points to the need for lower-stakes projects that work with email-based workflows, so that new developers can acculturate to the workflow and then confidently take part in, for example, kernel development. In that sense, email-focused outfits like Sourcehut are a valuable feeder service.
I definately agree. As a security engineer it took me more that a day to setup git-send-email with my password stored encrypted. Even getting it to connect to the mail server (fastmail) was challenging because of poor documentation on TLS vs STARTTLS. I know many skilled programmers who wouldn’t bother setting this up to submit a patch, ever if they had already written it.
Not all barriers to entry are bad. Passwords, for example, prevent those who do not know and cannot guess them from accessing accounts. Likewise, requiring plain-text email (i.e., real email) discourages those who do not understand email from easily taking part in the kernel development process.
Is that bad? I don’t really think so. Although understanding of email is not a perfect proxy for understanding of the kernel, I imagine they are pretty highly correlated. Indeed, I would be cautious about running code from someone who cannot or does not configure his email client properly.
It’s a welcome barrier IMHO. If someone can’t configure an email client properly I would question whether they should be trusted with writing kernel code.
You’re not supposed to “trust” totally new contributors either way. That’s why their code is reviewed by a tree of maintainers.
[Comment removed by author]
If someone is writing Linux Kernel code I’m pretty sure they have access to Linux, which does not run Apple Mail last time I checked.
I wouldn’t be surprised if a significant number of Linux kernel devs run macOS, and Linux inside a VM or some such.
Why do you think this is the case for a significant number of Linux kernel developers?
I don’t know if it is the case, just saying I wouldn’t be surprised if it was.
Yeah, why? I don’t know a lot of folks who are actively working on the Linux kernel, but they all do so natively in Linux (even if they still use a VM for testing kernel changes). But obviously my anecdote is just an anecdote, so I’m curious if you’ve experienced otherwise or ??
I’ve seen a lot of FreeBSD people working on their macbooks back in the day; I never really met many Linux (kernel) devs though.
This was in TFA as well. Apple Mail absolutely does have that setting and I’ve been using it reliably for well over 15 years. The cut-down version on iOS doesn’t but that’s just one of the many ways in which it’s awful for anything other than quickly skimming emails and flagging some to reply to on a real computer (unreliable updating IMAP folders, sometimes deciding a folder is completely empty, no way of marking a folder as read, and so on). If you’re using mobile email for submitting patches, you’re probably doing many other things wrong.
If anything, the Linux kernel should consider sourcehut. It’s probably the only platform that works well with open standards and decentralised systems.
I think Sourcehut is an incredible product and speaks very well of Drew Devault’s technical and business acumen, but I’m not convinced that where Linux hosts its source code is its biggest problem to be worrying about.
I don’t think it’s the most pressing matter right now, but it is important to keep in mind that it was once and it was solved. Due to the whole BitKeeper mess, Torvald’s started working on git as a means to perform distributed version control - and mail was the cornerstone. Now, GitHub is lobbying to drift apart from email and use their non-standard and close issue management system. It does look familiar.
Alpine adopted sourcehut and it didn’t end well. Sourcehut’s extreme opinions on mail left package maintainers unable to send mail because Devault didn’t like the settings you sometimes couldn’t even change. After some developers left the project, they instead switched to GitLab.
Source? What opinions?
Alpine seems to be a bit all over the place at the moment… Their repos look to be self-hosted cgit, their mailing lists are Sourcehut, their bug tracker is GitLab and their wiki is self-hosted Mediawiki.
Edit: ah, their cgit-hosted repos are a mirror of what’s in GitLab, so I guess that’s an interim measure.
This is an extremely uninformed and shitty take on what happened with Alpine Linux which doesn’t map at all onto reality. Keep your slander to yourself.
As someone with no stake in this, I went and had a look at the available evidence, and… maybe sourcehut needs to correct the record somewhere?
After looking over the alpine mailing lists, I got the distinct impression that some widely-used mail clients can’t send mail to sourcehut and some alpine contributors have refused to use it on that basis.
It never caused any developers to “leave the project”. The arguments were not being made in good faith, either. There was a subversive element with a personal vendetta against me who were pulling strings under the table to crank up the drama to 11. As a matter of fact, it was settled after an IRC discussion and the Alpine mailing lists (still running sourcehut) work fine with those mail clients now. All I pushed for was due consideration of the rationale behind the design decision, and during the discussion, a compromise was reached and implemented.
And they didn’t “switch” to GitLab from sourcehut, either. They were never using more than the mailing list features of sourcehut, and there was never a plan to start using them. The sourcehut mailing lists are still in use today, and the GitLab migration was planned in advance of that and completed for unrelated reasons.
There are persistent, awful comments from a handful of people who spread lies and rumors about what happened as if I attempted a hostile takeover of Alpine Linux. It’s complete horseshit, and incredibly offensive and hurtful, especially in light of how much time and effort I’ve devoted to improving Alpine Linux. I am a maintainer of many packages myself, and a contributor to many of their repositories!
Thanks, I appreciate the reply - I can only imagine this has all been quite draining, especially while trying to get a new product off the ground.
Last time I had to deal with a mailing list, I literally just sent the patch as an attachment in an email. I don’t care, if they don’t want fixes they can ignore it and continue being ignorant of it. I’ve had this mutual feeling, especially with linux kernel development, for a long time.
I submitted a couple of patches where the maintainers have not responded to anything on their (low-volume) mailing list in about a month. I don’t even know if they’re even getting any of the emails, and I’m not sure what to do next. Moving away from email as the source of truth would at least solve that issue.
I don’t think that move from email would change anything in this regard. You can ignore pull requests just as easily as emails.
Is it Microsoft Teams?
It’s Microsoft Teams isn’t it.
Aside from the kerfuffle about email clients (which the savvy editors at El Reg no doubt selected as lede to get outrage clicks), this is a pretty amazing development. If you’d asked me 20 years ago whether MS would be actively contributing to the Linux kernel I’d have been incredulous.
“Diffs happily accepted in winmail.dat format”
[Comment removed by author]