1. 28
  1.  

  2. 42

    I looked a bit at the git log of ElasticSearch and let some scripts loose on it; some of my findings:

    • From the 1,612 contributors, 161 used an @elasic.co or @elasticsearch.com email address in at least one of their commits, so it’s probably safe to assume they were Elastic employees. This accounts for 25,780 commits, or about 45%.

    • It seems a lot of people are committing with their personal address; for example Shay Banon, the ElasticSearch original author and top committer with 8% of al commits, uses his @gmail.com address. I looked at people’s GitHub profiles, and in a few rare cases used their homepage/Twitter/LinkedIn and some other e-stalking (as people move jobs) to determine if they work or have worked for Elastic.

      I counted a further 34 people as being Elastic employees this way, with 26,115 further commits. Bringing the total up to 195 authors with 51,895 commits, or about 91% of the total.

    • I only looked at people with 50 or more commits (just because this is boring work); from everyone with ≥50 commits I counted 3 committers from outside of elastic (all working for companies offering ElasticSearch-based services), and two where I wasn’t able to determine affiliation:

      • Amazon: 161 commits
      • FlaxSearch: 292
      • DataSolutions GmbH: 77
      • Unknown: 252 and 55
    • I did spotcheck a few people from the <50 commits list, and about half turned out to be working for elastic. This is a crude estimate, but adding half of the 4,852 remaining commits means that about 96% of Elastic is developed by Elastic employees.

    • 995 contributors have just one commit, I didn’t really check out those commits, but my experience is that a lot of the time when someone makes just one commit it’s for something trivial: fixing a typo, a small bug, some small documentation fix/addition, etc. While these contributions are certainly valuable, these are often the kind of trivial changes that don’t really impart copyright/authorship. So that would further boost that 96% number a bit.

    • This leaves us with an estimate of about 229 potential “community contributors” over a 11-year time period, of which about half have just 2 or 3 commits. That’s about two contributors/month on average.

    Counting by just commits is imperfect, of course (and so is counting by lines of code, there is no good way to do this). The script I used tries to “merge” authors if there is some overlap between the name/email addresses, but this may not be 100% accurate either. All of this is just a rough indication on who works on ElasticSearch.

    The full aggregate is over here, in case anyone is interested.


    What to make of this? Well, it seems that a large number of those 1,612 contributors (GitHub only lists those where the email address is linked to an account, so it’s a bit higher than the 1,573 listed in this article, although this may also be an artefact of the script) are actually Elastic employees, and that they write the overwhelming majority of the code (≥96%). In my opinion things are therefore considerably more nuanced than what is claimed in this article.

    What of those remaining committers? If they have objections against this license change then that’s fair enough, but, well, they did sign a CLA. Besides, all the old code still exists under the old license; it’s not like the change is retroactive. This is also why we still have illumos today, even though Oracle stopped releasing the source for Solaris under OpenSolaris over a decade ago.

    From a purely moral viewpoint, if I help you buid a fence in your backyard then do I “own” this fence or have a right to say how this fence is used? Or do I have a right to use your land? I don’t think I would. “Everyone owns it” is pretty simplistic IMO.

    This is an Oracle-level move.

    Oracle just stopped shipping source altogether. This is quite different.

    Many of those contributors were there because they believe in open source. Even those who work for Elastic as their employees, who had their copyright taken from them by their employer, work there because they believe in open source.

    I’m not so sure about that; I wouldn’t really mind, even if I had contributed to ES as a community contributor. I also noticed that quite a lot of the people I “e-stalked” to determine of they worked for Elastic has commits on their GitHub profiles only in the time period they worked for Elastic, and that it pretty much stopped after that.

    In my general experience, most of my coworkers I’ve worked with in various positions … most of them don’t really care all that much. Only a handful did, and even for those people their positions tend to be more nuanced (like mine).

    Elastic was not having their lunch eaten by Amazon. They cleared half a billion dollars last year.

    $428 million in revenue, yes (not quite half a billion), but with a $75 million net loss after expenses. It still seems some way from a healthy self-sustaining business.

    1. 2

      Thank you for the work you put into this very interesting analysis, but I would argue that the committing of code is the only way to contribute to a project. Another way is creating issues and engaging in discussions. This happens a lot more in open source projects than proprietary software and is one driving forces for the success of open source in my opinion.

    2. 17

      Here’s the summary of the licencing change:

      We are moving our Apache 2.0-licensed source code in Elasticsearch and Kibana to be dual licensed under Server Side Public License (SSPL) and the Elastic License, giving users the choice of which license to apply. This license change ensures our community and customers have free and open access to use, modify, redistribute, and collaborate on the code. It also [restricts] cloud service providers from offering Elasticsearch and Kibana as a service without contributing back

      I’m not clear on how this pairs with Drew’s summary of:

      Elastic is no longer open source [and] has spit in the face of every single one of 1,573 contributors

      The code that those folk wrote and contributed to still exists, right? Elastic is saying that further code that they themselves distribute, which includes code from those 1,573 contributors, will now be under some other licence. And I’ll grant that that’s not great, but Elastic isn’t sshing into all of your servers and deleting the code running on them. They aren’t retroactively undistributing or relicencing old code. They’re just releasing new code that they write under some other, less good, licence. Potential contributors going forward know that those are the terms going forward but the rug wasn’t pulled out from other them and they’re free to contribute to a fork instead, under whatever terms they like.

      I’m undoutedly missing all of the context so I feel like I’m missing out on what’s happening that’s so terrible.

      Yesterday I was offering beer at my bar for free and some folk kindly helped me develop the recipe and brew it. Today I’m going to start charging for it, and hey if you want to keep helping you’re welcome to but the old recipe’s still on the door outside if you want it. I don’t follow what’s worth being up in arms about.

      1. 7

        I was just trying to answer a similar comment in the HN thread when I got rate limited, so convenient that I can almost paste it into this thread instead :)

        The community is built around the upstream project, and has a lot invested in their continued existence as an open source entity. This move will force the community to start from scratch to build a new entity which continues to meet those guarantees - something Elastic plans to capitalize on by capturing more users on its paid offering while the open source community struggles to set up a new entity at the same level of sophistication from scratch.

        What Elastic done was within their rights, but it wasn’t right - so to speak.

        1. 5

          when I got rate limited

          Fun fact, HN does not actually have rate limiting for regular users. It’s only applied to a few users who are flagged by the mods as annoying, but where a ban would cause more trouble than it’s worth. Beyond that there’s the shadowban or insta-flagging of comments.

          If you get rate limited, it might be worth sending an email to the mods and asking what to change to get rid of it - that said, being rate limited on HN is something to be proud of ;)

          1. 13
            1. 9

              https://jcs.org/2012/06/13/hellbanned_from_hacker_news

              The last sentence of that post is the kicker.

              1. 4
              2. 6

                They have a clickthrough rate limit, and because everything is form-based every click is a new page load. You can hit it pretty easily by hitting the comments link on like 10 posts. I’m not sure sure on how aggressive it is though, or if that’s what Drew is talking about.

                1. 1

                  I can middle-click open the comments of the entire frontpage as fast as I can click and I don’t get rate limited. That said, the rate-limit for “troublesome” users applies not just to actions, but also to comments.

                  I do have enough karma to be able to downvote, though, so maybe that helps.

                  Your experience sounds like you’re on one of the lower ban levels.

                  1. 2

                    I wonder if it’s load-based. I was able to click through to the comments for nearly every article on the front page without issue. It may also depend on if you commented recently or something…

          2. 13

            Elastic never promised contributors that it would keep its project under the Apache License forever, only that what it did release under the Apache License would remain available under that license. Contributors never promised to maintain the code they contributed to whenever Elastic called. Users of the software didn’t promise to send patches back to the project. Everybody got what they got and gave what they gave, right then and there. That’s how standardized open software licensing has always worked to date. No promises, no service-level agreements, no warranty periods or contractually mandated update schedules, just code and permission to run with it.

            Kind of hard to argue with that.

            1. 10

              Minor nit:

              You cannot stop someone from making money from your software, but you can obligate them to share their improvements with everyone else, which you can incorporate back into the original product to make it more compelling for everyone. The GPL family of licenses is designed for this purpose.

              This is one of the most common misconceptions about the GPL. It does not require anyone to give changes back, it requires everyone to give changes forward. The GPL family is intended to enforce RMS’ view of software freedom: that anyone who receives a binary should be able to fix bugs in the source code used to create that binary and rebuild it. It is not intended to force anyone to give anything back to the original author because the original author already enjoys all of the freedoms that RMS considers important.

              This difference is what allows Google to take Linux, modify it extensively, and use it in their datacenters to run all of their back-end infrastructure. They are giving the source code to all of the recipients of the binary (i.e. Google).

              In many cases, the two are equivalent. Grsecurity, for example, tried to keep their patches to Linux for the exclusive use of their customers but the GPL explicitly allows someone to buy their product and then upstream their patches to mainline Linux. Most software; however, is not written on a COTS model and it’s completely fine under the GPL to take an existing project and build a product for a specific customer out of a modified version of it. You have to give the source code to your customer but they have no incentive to contribute it upstream if it gives them a competitive advantage. You can’t; however, prevent them from hiring someone else to work on it instead of you next time.

              There are some open source licenses that require you to submit changes upstream but the GPL is not one of them. Lawyers typically hate them even more than the GPL because checking compliance is incredibly hard.

              1. 8

                God I’m sick of hearing about how horrible Amazon is because they charge people to use the infrastructure they own, including the running of software EXPLICITLY MARKETED AND CREATED TO BE FREE TO USE.

                • If I run OpenFoo at home on my hardware without contributing, everybody’s happy
                • If I run OpenFoo on a server I rent instead, everybody’s happy
                • If I run OpenFoo on a server I rent that I pay somebody else to admin, everybody’s happy
                • If I run OpenFoo on a server I rent from Amazon, who pays somebody else to admin it, suddenly Amazon are worse than Hitler.

                This logic is ridiculous, and we should stop addressing these license changes as if “they’re probably not the best way to protect against those evil amazon guys”, but rather as a disingenuous cash grab by the organisations who make them and have until now pretended to be FOSS.

                (edit for style: I hate Markdown)

                1. 3

                  It is a bit weird indeed. The argument I saw can only be parsed if you believe software is a liability. It goes like this:

                  Because Foo is behind OpenFoo, it has to pouring effort to maintain OpenFoo in addition to provide the hosting service to make money. $AMZN can just use OpenFoo, wrap it up and make money without incur the maintenance cost.

                  It is weird because in this context, OpenFoo is a liability and the ownership provides no competitive advantage, just sinking cost and drag.

                  1. 1

                    IMO People who think software is a liability probably shouldn’t be in the software business, OSS or otherwise.

                    1. 4

                      I would argue the opposite - that it’s precisely people who think that software is a liability that should be writing it, because they’ll be inclined to write as little of it as possible as is necessary to achieve the desired result (as opposed to people who think that software is an asset, who will have far fewer inhibitions on writing way more than is actually necessary).

                      Engineering is about accomplishing a task, not building a particular tool. Software engineering is exactly the same way. If your goal is to build a tool or write a program, then you’re probably an artist, not an engineer.

                      Moreover, code is a liability, at least as long as you’re actually using it - more code leads to more bugs, you need to host the running instances and store the source somewhere, you have to continually keep on top of new libraries/dependency updates/security fixes even if you’re adding zero new functionality to your code. As a recent Lobsters submission (which I unfortunately can’t find at the moment) stated, good code can become bad code over time without you even touching it (e.g. because a bug was found, or something it interacts with changes (web browsers change over time, but so do hardware and networking protocols)).

                      Edit: Beyond whether or not you have code at all (ignoring how much), the more code you have, the higher all of this overhead is, in addition to things like super-linear costs of testing (more interactions between different components) and onboarding (due to super-linear complexity) if your application isn’t architected well.

                2. 8

                  Elasticsearch belongs to its 1,573 contributors, who retain their copyright

                  Point taken about not signing CLAs, but how many of those you think worked for Elastic? You can check the stats here… I don’t see much here that indicates elasticsearch was significantly community-run. And as another commenter already said, re-publishing code under a different license does not make the old code unavailable. It only applies to future patches, so to say.

                  But if you choose to make it FOSS, that means something, and you have the moral obligation to uphold.

                  I don’t think most people publishing open source software share the same ideals so please don’t hold them accountable to those. Quite honestly if FOSS mostly benefits existing monopolies I’d rather not have FOSS.

                  1. 8

                    As this is the expected denoumentd of the Elasticsearch license change, might be worth folding into https://lobste.rs/s/qtsjh1/elasticsearch_does_not_belong_elastic for context.

                    1. 1

                      Seems like a good call, I hadn’t realised we’d linked those stories. Thanks!

                      I think that’s something a mod would need to do, if appropriate, right?

                      1. 3

                        Yes, it has to be done by the Lone Ranger Mod[1], @pushcx

                        [1] so far

                        1. 4

                          Hey, we still have Irene!

                          1. 2

                            I can’t believe I forgot that. Apologies.

                    2. 6

                      (Re: Righteous, Expedient, Wrong merged story):

                      The same people who fight against Twitter/WhatsApp/etc. decentralisation will also argue that “it’s not Open Source, as this one central authority has not declared it to be”. Hmkay.

                      The claim that ElasticSearch is a “proprietary product” really made me choke in my coffee when I read it this morning. Because yeah, ES is now exactly like Oracle or Windows! No difference at all! 🙄

                      1. 4

                        Eh, this is something I’ve always wondered: if I release software under the AGPL/GPLV3, does that bind only the licensees and not me, given that I own the copyright? I could develop proprietary enterprise features and not be obliged to release their sources, right, but not be bound by the license. This is allowed, right? So in theory I could have the following business model:

                        1. The software is free software under AGPL
                        2. I register a trademark for the software, so others can’t redistribute the software under the same name without explicit permissions
                        3. I offer closed source proprietary “enterprise features” that don’t really respect the AGPL, but I don’t have to
                        4. I can also offer commercial proprietary licenses to users that don’t want to license under the AGPL

                        Right, this is a bit lawful evil, and it wouldn’t be my ideal business model. I would prefer offering services and support (hosted SaaS offerings etc.) instead of this but the model above is definitely easier to make money with, if it’s legal.

                        1. 10

                          Eh, this is something I’ve always wondered: if I release software under the AGPL/GPLV3, does that bind only the licensees and not me, given that I own the copyright? I could develop proprietary enterprise features and not be obliged to release their sources, right, but not be bound by the license. This is allowed, right?

                          Yes, but there’s a big catch: if you have accepted contributions from third parties who licensed their changes under the GPL and retain copyright, those changes virally “infect” your own code within the project, too. So you can distribute a project under a GPL-family license and change the license terms at your pleasure, if and only if you never accept outside contributions. And that’s a good thing IMO.

                          1. 1

                            Yes, right, that’s the caveat. Suppose I get contributors to sign some sort of CLA granting me the copyright? Again, lawful evil, but that would be the way around it?

                            For this very reason, CLAs are evil incarnate, because such contributors would essentially be working for me for free.

                            The only “CLAs” I’ve ever signed are the FSF ones. The FSF is probably the only entity in the world to which I’d gladly transfer my copyright.

                        2. 3

                          There’s a paragraph in this article which frames what Elastic are doing as ‘relicensing under copyleft’:

                          If you didn’t want your work “embedded in a proprietary product”, you should have picked a copyleft license that covered the key use cases, or switched to one before releasing new and valuable features or fixes. Exactly what Elastic is doing now.

                          I think the above is disingenuous, I seriously doubt anyone would be kicking up a fuss if Elastic had relicensed under AGPL-3.

                          What concerns me is the (I assume deliberately) vague wording in the definition of what constitutes Offering the Program as a Service section of the new license, where ‘offering a service’ is defined as:

                          offering a service the value of which entirely or primarily derives from the value of the Program or modified version

                          The above is the sort of sentence that feels likely to involve expensive lawyers and a long court case should the author of the software in question decide that they feel your business infringes.

                          It’s not hard to dream up hypotheticals where your value-add as a SaaS business comes from normalizing some domain specific dataset into an Elastic cluster and providing access to that search functionality.

                          If Elastic Inc come calling for you to either give them their pound of flesh, or open source your entire IP, can you afford the legal bill when you have to argue that the value of your offering ‘derives primarily’ from the mangling you do to get it into ElasticSearch, rather than the business value of being able to search for it?

                          I won’t pretend to understand the vitriol directed at the OSI in this article, as I clearly missed that particular set of drama when it happened.

                          1. 3

                            I do not fully understand the difference between SSPL and AGPL, can someone explain it to me? Is AGPL not considered open source? I think this may be the question to Drew?

                            If I understand correctly SSPL would force Amazon to open source anything that touches ElasticSearch eg. their entire platform? Is that right? So that would mean that SSPL violates freedom #0.

                            1. 5

                              You can check which licenses are considered legitimately open source here: https://opensource.org/licenses/alphabetical

                              Here’s a post by the OSI clarifying about the SSPL: https://opensource.org/node/1099

                              And here’s the definition of ‘open source’: https://opensource.org/osd

                              1. 4

                                I’m unsure that the OSI can be treated as honest brokers here. Open Source means what it says, not what OSI say it means, they can try to be a gatekeeper if they want, but noone’s obliged to take them too seriously.

                                Take this spew of nonsense (for example):

                                What a company may not do is claim or imply that software under a license that has not been approved by the Open Source Initiative, much less a license that does not meet the Open Source Definition, is open source software. It’s deception, plain and simple, to claim that the software has all the benefits and promises of open source when it does not.

                                That’s just incorrect, both factually and legally. Saying “license mets the open source definition” or “license is approved by the OSI” would both be deception. Saying “I believe this license is open source” is not.

                                1. 1

                                  “I believe X” is mostly not a testable assertion, also it could be true while X is wrong. It is also not that interesting to know what MongoDB and Elastic are believing, it is interesting what they did. Thus: What could happen if one were to use software under the SSPL? Do MongoDB or Elastic use software they license under the SSPL?

                                  A license like SSPL that nobody can comply with is not a license for you. Thus claiming its an open source license is deceptive. See https://lobste.rs/s/t9kcgy/righteous_expedient_wrong#c_swk45k .

                                2. 1

                                  Maybe I’m missing something, but I still don’t see any details in that posts, or the links from it, what disqualifies the SSPL (how does it differ from AGPL?)

                                  1. 1

                                    The post links to a mailing list thread which has further discussion about why the SSPL was not accepted as an open source license.

                                  2. 1

                                    Thank you, this clears up my initial misunderstanding of what SSPL requires.

                                3. 3

                                  I’m biased, I just led the effort to get an enterprise license of ELK into our org. We’re on GCP.

                                  This looks a lot like: amazon bad, open source good (read: elastic) good.

                                  How do you see this? Interested to hear outside ideas

                                  1. 1

                                    Elastic’s elasticsearch isn’t open source any longer, while Amazon’s is…. so…

                                    not saying Amazon’s not bad, but it looks to be better than elastic from where I’m sitting

                                  2. 3

                                    A license that you can’t comply with isn’t a license. This article entirely ignores that.

                                    The SSPL is not a license that is practical to comply with while running the software as a service. Neither MongoDB nor Elastic themselves are able to comply with their own license. Nor are they working on being able to.

                                    Nobody can currently run a SASS DB offering with only open source. There will be software in your USB-C cable, your network card, your ethernet switch or something like that. You can’t buy a version of every hardware needed, that works only with open source software.

                                    Yes, the reaction from OSI also doesn’t lead to a practical open source license that plugs the SASS-wraps-FOSS loophole. People with power in OSI have been against stronger copyleft.

                                    1. 1

                                      Power granted is power exercised, it’s only a matter of when.

                                      1. 1

                                        I’d like to see some in depth analysis on this topic to accompany the various ideological posts we often see floating around. My sense is that the commercial potential and capital of OSS is more often converted and captured by 3rd parties than the actual creators and maintainers given the scale of the industry, though I could be wrong. Send me some studies on this if you know of any.

                                        It’s quite common for people other than you to make money from your free and open source software works. Some will incorporate them into their own products to sell, some will develop an expertise with it and sell their skills as a consultant, some will re-package it in an easy-to-use fashion and charge people for the service. Others might come up with even more creative ways to monetize the software, like writing books about it. It will create wealth for everyone, not just the original authors. And if you want it to create wealth for you, you are responsible for figuring out how. Building a business requires more work than just writing the software.

                                        This all makes sense up until “it will create wealth for everyone”. In this example, the creator had provided free labor, and others have capitalized on it. That’s not the same thing as wealth for everyone. Creating a business around an open source project only works for certain kinds of technologies. If I write a cross-platform library for storing application data - which isn’t exactly fertile ground for monetization -, and Facebook uses it in Messenger, and then contributes back a couple features and a bug fix, and some programming influencer makes money off of it at a tech conference, earning me a few hundred stars on GitHub, it’s not a mystery as to who is richer as a result. There’s definitely something I gain in return, e.g. code contributions and online reputation, but given the relative scale of value, I’m not sure if I would describe it as “karmic justice”.

                                        1. 1

                                          (Re: Open source means surrendering your monopoly over commercial exploitation story):

                                          Definition of exploit; transitive verb

                                          1 : to make productive use of : utilize
                                              // exploiting your talents
                                              // exploit your opponent’s weakness

                                          2 : to make use of meanly or unfairly for one’s own advantage
                                              // exploiting migrant farm workers

                                          “Exploitation” according to definition 1 is okay. “Exploitation” according to definition 2 … probably a bit less so.

                                          This is why we have the (A)GPL and, yes, the SPPL, which is just another copyleft license. While copyleft doesn’t address commercial exploitation (definition 1) directly, it does address exploitation (definition 2) by ensuring it’s a reasonable fair deal for everyone.

                                          1. -2

                                            It’s funny that it seems that engineers are so smart, but so naive sometimes.