1. 1

    For anyone interested in the topic of managing ML teams, I recommend Andrew Ng’s: http://www.mlyearning.org

    1. 11

      This article would do well to actually define what event sourcing is.

      1. 2

        Agreed. Reading this article, I get the impression that the author’s pain comes from a misunderstanding of how and when to use projections vs. the log. But since he doesn’t explain it, I’m only left guessing.

        1. 0

          Would it? I thought the intended audience were people who had heard all the introductions and sales material for event sourcing, and hadn’t yet heard the potential downsides.

          1. 1

            It would at least for the author’s own clarity. As I’ve said in my other comment this article conflates event sourcing with a variety of things that event sourcing is used with.

            1. 1

              Having a basic definition as to what it is would help people who haven’t yet heard of event sourcing, but might in the future be exposed to the sales material/hype/etc. I would count myself in this category, for example.

          1. 2

            I think the article is missing another point regarding the lack of adversaries. Nobody blames the aircraft designer when an airliner is shot down with military weapons.

            On a separate note I find the fact that we don’t know how to build safe software even when we’re given the resources is shameful. The government could spend a lot of money on a safe digital voting platform, but we still wouldn’t trust the outcome.

            1. 4

              On a separate note I find the fact that we don’t know how to build safe software even when we’re given the resources is shameful. The government could spend a lot of money on a safe digital voting platform, but we still wouldn’t trust the outcome.

              That’s not true, though. Dijkstra’s method (1960’s onward), VDM (1970’s), Cleanroom (1980’s), B-Method (1980’s), and Correct-by-Construction (maybe 1990’s) helped teams deliver software with very, low, defect rates. Companies some even issued warranties for their code. They were usually on commercial schedules without time and staff to apply heaviest methods, like full proofs of correctness. So, they aren’t even the upper bound. They just applied known-good methods to design and analyze the software with bug counts orders of magnitude less than most software. CbC also used SPARK Ada to prove the most critical parts free of code-level errors. Rockwell-Collins and Galois are doing it for hardware/software combinations.

              So, we do know how to make low-defect software. Those with the time and money don’t care enough to spend it due to no liability. The FOSS people just don’t want to do it. Most don’t care about safety or security that much. The ones that claim to want safe and secure projects usually use the most unsafe methods claiming it’s a programmer’s responsibility to juggle it all in their head. This is unique mindset compared to most fields where they adopt every tool they can afford to boost productivity and lower risks. The biggest uptake of these methods are in regulated markets (i.e. safety-critical, smartcards) and a subset of CompSci that pushes state-of-the-art for safety and security. Even in safety-critical, the uptake is still minimal since they have a different mindset about how to do correctness which is [justifiably] slow to change.

              So, we know what works, most software avoids it due to no liability, and regulated markets do stuff like it due to regulations. That tells me we need regulations and/or court liability for any commercial software that has errors we can prevent using hardly any effort and/or automated means. That includes FOSS but only for commercial use. That will as a side effect force FOSS-using companies to invest in its quality. Like what happened in DO-178C market, you’ll see piles of pre-certified components and tools making safety easier pop up to reduce costs of projects. Also, it’s funny writers like OP talking about this never bring up success of DO-178C regulations. Probably didn’t know it existed.

              1. 3

                that we don’t know how to build safe software even when we’re given the resources is shameful

                I feel like that’s a bit like saying ‘that we don’t know how to factorise prime numbers quickly is shameful’. There’s no reason to assume that it is possible to have safe digital voting that preserves all the things we need in a voting system, namely that you can be sure your vote was counted, you can’t prove how you voted, you can be sure that nobody’s vote was counted twice, you can be sure that nobody voted twice, you can be sure nobody else knows what you voted for, etc.

                The government could spend a lot of money on a safe digital voting platform, but we still wouldn’t trust the outcome.

                Essentially, no, they couldn’t. The problem of voting systems can’t just be solved by throwing money at them. Digital voting is fundamentally not something that will ever be possible, for the simple reason that someone can stand over you while you vote, defeating the whole purpose of secret voting.

                1. 3

                  Now I see how I’ve driven my argument through a cliff by touching on digital voting, since the subject has theoretical problems. What I meant was, if it were something that had a solution, we wouldn’t trust the thing those people who’ve won the contract build. The argument is, if the government decided to spare the resources and build a 250-story-high mega sky scraper, I’d expect them to find the right people to build it. The thing probably wouldn’t collapse after all. But I’d never expect them to build something of that quality, if the mega artifact in question were software. We simply don’t have the established practices for building provably (and independently checkably) safe software, in any case even if there were people who could build that artifact, I’m pretty sure those ones wouldn’t get the contract, because we don’t have the means to identify those people either.

                  1. 2

                    But they can build highly reliable software at NASA/JPL etc. I think some of this may be the reliance on in-house or near-in-house engineering rather than contractors. The Fed contracting system weakness is particularly bad for software, although they have spectacular failures in other areas too.

                  2. 2

                    Of course voting by paper ballot has the same problem. You have just proved that voting cannot be possible.

                    1. 2

                      Paper ballots definitely don’t have the same problems. With paper ballots, you are sure your vote was counted, you can’t prove how you voted, you are sure that nobody vote was counted twice, you are sure nobody voted twice, and you are sure nobody can see what you voted for. The system is designed to sort these issues out, quite well.

                      1. 2

                        Paper ballot: physical presence at the scene of the crime for a few thousand votes.

                        Electronic voting: might be able to get with it, one person can sway an entire country. You only need access to a few machines at any point before the event.

                        The risk/reward for an attacker heavily favors paper ballots. Nothing is absolutely safe, there are people who disrupted paper voting too. They got a lot less for their actions and the rest of the system was safe.

                        1. 1

                          the problem with paper voting is that it is now usually tabulated by machine. But your points are well taken. Needs a better design than anyone has demonstrated yet to get the same level of security with purely electronic systems. There are tradeoffs, however. For example. Travis Country TX uses electronic voting to enable any county voter to vote at any polling place - the electronic system is able to produce the many different ballot configurations on demand which vary by location. This is especially useful because we have one of the most gerrymandered maps in the world imposed on us.

                      2. 2

                        for the simple reason that someone can stand over you while you vote, defeating the whole purpose of secret voting.

                        You could have a duress code or something for that. The reason it’s better to use paper is there’s simply fewer attacks that are possible and more people are mentally capable of auditing it. Less risk, more democratic.

                        1. 1

                          A duress code that they could see you putting in.

                          There are fewer attacks possible, for example you can actually have secret voting.

                          1. 1

                            Duress codes would be unique to the voter with the system showing the vote was tallied. Obviously. Otherwise, it wouldnt work.

                            1. 1

                              How would you guarantee the secret delivery of the duress code? Whatever you use for duress code delivery, why not use that for vote delivery? And repeat.

                              1. 1

                                Off top of head, they get it during registration. They dont bring it with them: they memorize it voting day and destroy it. If they cant, they can bring it with them labelled in an obfuscated way they understand but not the crooks. Some people might for whatever reason have the code on them. That case will range from ineffective (attackers see code) to effective (most not carrying it deterred manipulation).

                        2. 2

                          Digital voting is fundamentally not something that will ever be possible

                          I understand what you’re getting at, but keep in mind that I read this sentence while also doing my due diligence research on who I’ll be voting for in the upcoming Estonian parliamentary election from my home in Australia. I don’t mean to suggest that it’s unflawed or perfect, but digital voting is not only “not possible”, but has been implemented and in use for more than a decade in some countries.

                          1. 1

                            The article you link to has a large criticism section, and nothing in it seems to explain how they’ve got past any of the issues of online voting.

                            Firstly, there’s absolutely no way that your vote can be anonymous if it’s clearly recorded, which it has to be in order to be able to be erased by the same person making a subsequent vote.

                            Secondly, they haven’t solved the problem that such a system destroys the most important attribute in a voting system, above all other attributes, which is to have a secret ballot. If someone can watch you vote, then they can force you to vote a certain way. They can blackmail you, they can bribe you, etc. That’s antithetical to a fair voting system.

                            1. 1

                              I’m responding only to the claim that digital voting is impossible by pointing out that it is happening now and has been happening for decades. Thanks!

                              1. 1

                                I said that secure digital voting is impossible.

                                1. 2

                                  I said that secure digital voting is impossible.

                                  Digital voting is fundamentally not something that will ever be possible

                                  ¯\_(ツ)_/¯

                                  While I’m here, though: no, it’s not provably anonymous. We do need to trust the procedure that after vote deduplication, the remaining encrypted votes are passed onto the systems with the actual keys for decryption without any identifying data. This is why I said it’s not unflawed or perfect. At the end of the day, we do have to put some trust in the government running the show, rather than in cryptography. Not ideal, but it still does effect digital voting. You can’t just stick your head in the sand and shout that digital voting is impossible while it’s happening, even if it’s flawed.

                                  But at the same time, due to this property, your other point is moot. The ability to recast your vote means blackmail, bribes, etc. cannot be committed effectively.

                                  1. 1

                                    I thought it was pretty obvious that digital voting that isn’t secure was not really digital voting.

                                    But at the same time, due to this property, your other point is moot. The ability to recast your vote means blackmail, bribes, etc. cannot be committed effectively.

                                    Wrong. Someone can stand over you until the voting period is over. e.g. an abused spouse.

                                    1. 2

                                      It’s hard to coerce a vote at scale - i.e. enough to swing an election.

                                      Individual abuses of voter’s rights are to be deplored, but in the grand scale of things (electoral integrity) don’t really register.

                                      1. 2

                                        It’s hard to coerce a vote at scale. But even given theoretical flaws in a paper voting system, exploiting those flaws usually requires thousands of people to collaborate, and even one breaking their silence means the whole scheme is undone.

                                        But if there’s a security issue with digital voting, then the entire system is probably COMPLETELY broken and cannot be trusted at all, and it can be all done from another country by a foreign government, in secret, or even by a single person.

                                        1. 1

                                          It does with smaller elections, esp with low participation. People have lost city and county elections by a tiny number of votes. In once case, a politician was said to have lost just by not voting for himself because he was too busy getting others’ votes. I didn’t verify that but would be hilarious.

                                        2. 1

                                          I thought it was pretty obvious that digital voting that isn’t secure was not really digital voting.

                                          This leads to the claim that 31.5% of voters in the 2017 Estonian municipal elections gave their vote online, but that they did not vote digitally.

                                          Wrong. Someone can stand over you until the voting period is over. e.g. an abused spouse.

                                          They would need to stand over you at all times, including preventing you from attending a physical polling booth location. This is probably an unavoidable consequence.

                          1. 15

                            Welp, guess Github just lost my business

                            (because the one thing I was paying for is now free)

                            1. 13

                              I’d wager that most of GitHub’s income comes from companies who host their stuff there.

                              I just checked, and we pay $1,182/month to GitHub. The few people that had $7/month subscription probably isn’t really worth it, as private repos are free on a number of other providers.

                              In the long run, it might even be beneficial for GitHub’s income, as the more people that use GitHub for their personal stuff, the more likely they’ll be to use/recommend it for their business, too. Atlassian does something similar with their pricing for most products: it’s very cheap for small teams (<10 people), and then a massive price-hike if you’ve got more people.

                              1. 5

                                Most of GitHub’s income comes from enterprise users. This is generally true of every company with an enterprise offering, but we know it from a 2016 Bloomberg article (analysis). It’s reasonable to estimate GitHub’s personal plan was a single-digit percent of its revenue and only marginally profitable because of support.

                              2. 5

                                Usually works out the other way. That’s funny.

                                1. 4

                                  Maybe it will work the other way around: Everyone moves to MSGithub now, and next year they start charging for private repos. :-)

                                  1. 7

                                    But everyone (basically) is already on MSGitHub. If anything this seems like an attempt to build goodwill. Maybe they’re a little concerned about the growth of alternatives as well. But I doubt they were making very much off of individuals paying for five private repos. The big money is in the corporate contracts.

                                    1. 1

                                      Good thinking. :)

                                1. 1

                                  If I understand your goals correctly, this is exactly what rsnapshot does: https://rsnapshot.org

                                  1. 1

                                    Not looking for a backup system as much I’m looking for a backup format.

                                    1. 2

                                      Perkeep might do what you’re after; the format is well documented.

                                  1. 3

                                    So the game is to try and input a C program that generates idiomatic rust code.

                                    1. 6

                                      Or the opposite: submit compiling C code that doesn’t compile in Rust (or is otherwise broken) so that we can fix the translator to handle that case.

                                      See the known limitations page: https://github.com/immunant/c2rust/wiki/Known-Limitations-of-Translation