This is really exciting to me. I definitely love the mercurial workflow with a dag of commits that individually get reviewed and submitted. Generally the goal is not to have a deep dag when possible, but overall, it is a much nicer flow than adhoc GitHub branches and PRs.
I wonder how well or poorly this would fit with some open source work I do. I assume it wouldn’t work to be the only person using git-branchless or similar tools. From a first look, it seems to match the workflow I find best, but not at all sure about integration.
I really like the mercurial workflow too, and I’ve been chasing a similar experience in git.
git-branchless is the closest thing I’ve found, but if you need to use branches (e.g. to push PRs to Github), it can be a bit clunky. With Gerrit, git-branchless gets you really close to the mercurial flow
I used to do this but it would almost always get messy. If changes are necessary after code review on a PR early in the chain (and you should assume there will be), you need to rebase or merge the earlier one into any PRs that depend on it. It’s often not possible to do this cleanly.
Nope. Hard disagree. You need to rebase. Only rebase, no merge, never merge in this situation. It just fucks up the tree and make hard to figure out what should go where. I only ever merge in one direction, and rebase in the other.
I actually 100% agree, and would only ever rebase in this situation. I only added “or merge” in anticipation of some people who get weird about rebasing anything ever. :P
git-branchless handles this well (author here). You just check out the old commit and amend it and run git restack. This updates all descendant commits and branches. It won’t start merge conflict resolution unless you pass --merge, so you can decide to resolve conflicts either now or later. You can also use git-branchless’s amend tool to do both in one step.
You literally can’t do this with vanilla git or you’ll go crazy. Ben was underselling how hard it is in his post.
I currently use git-branchless thanks to @arxanas hard work. The only thing it’s missing is the ability to sync a branch of pull requests up to Bit Bucket but I’m not expect any tool to get that anytime soon ;(
Interesting, I haven’t noticed this in my experience. But I definitely see the risk with stacking commits that fundamental changes in the base commits would invalidate assumptions used to build later ones
Not sure I follow, but if the early commit has already been merged, and then you realize you have to make a fix before the next one goes in, then those just rebase on the fix – why is that a problem? What is the alternative?
The problem is with thrashing between several PRs and keeping track of which ones need updating and in what order (i.e. if you have a dep chain of 3+ PRs). The other problem is dealing with conflicts, which is just annoying. In my experience you’re just trading one kind of mental overhead for another.
The alternative is to just work on something else while the base PR is under review. IMO this is a social/collaboration problem, not a technical one. :)
I would say the solution is small batch sizes and a fast enough PR turnaround time. If you have those things, you can keep working on the next dependent PRs, and even if the reviewer requests larger changes than you anticipated, the rebasing and conflict resolution work should never be unmanageable.
Said another way, batch sizes are too big, or reviews happen too slowly, if the ripple effects of review scare you to the point of pausing work.
Well yeah, you’d need to rebase one, merge it, then the next, etc. My comment above about the social aspect is relevant here: everything will be fine if your PRs are small and getting reviewed quickly. If that’s not happening, you’re hosed anyway. This isn’t a tech problem, and you don’t need these other services on top of git to solve it.
I’m not sure I understand your comment. I think we might be talking about different things. If your version control system doesn’t support you continuing to work by building up changes on top of changes already being reviewed you will not create small, granular code reviews. It’s not a social problem that I’m talking about, it’s very much a tooling problem?
Dev A submits small PR X, and dev B begins reviewing.
During review, dev A continues work on Y and Z, branching off still unmerged X.
B requests changes to X
A switches back to X, makes changes, pushes them.
A rebases Y and Z on X.
B approves and merges X into main.
A rebases Y and Z on main, and now submits Y as a new PR.
etc…
My claim is that this works perfectly fine in vanilla git, and rebasing is quite easy, so long as we have small PRs moving along quickly. I guess my question to you is: If you disagree with this claim, what is the situation that breaks it and why do I not encounter it? My guess is that the answer would have to be differences in team size or culture, and not tooling.
This is what I’m disagreeing with. This doesn’t work well and requires a lot of manual work.
If you rebase Z on X you bring Y’s commits but not the Y branch pointer. If you rebase Y on X you bring neither the Z commits nor the Z branch pointer. You can force move the branch pointer but this doesn’t scale to a lot of stacked branches, since you’re now doing work PER branch, is generally messy and error prone, and is a pain in the neck. People are often surprised by this behavior but if you go and play with it you’ll quickly see what I mean.
To make stacked branches/PRs work, your vcs must make amending an earlier commit a single command, not one or two commands per descendant branch.
I never have these problem and I do what I decribed almost every day, though tbf I generally am only “1 deep” because the “X” will get merged within a day. But this is what I was saying originally: if your “stack” is so deep you have culture problem. Imo, at that point you change the workflow so you have smaller PRs and a shallow queue.
If you rebase Y on X you bring neither the Z commits nor the Z branch pointer.
Again, this is never a problem in practice for me, because the pain I’ll feel when I later rebase Z only depends on how far the work has drifted / how many conflicts I have to deal with.
but if you go and play with it you’ll quickly see what I mean.
I just played with it in a sample repo and I still don’t get it.
To make stacked branches/PRs work, your vcs must make amending an earlier commit a single command, not one or two commands per descendant branch.
I don’t follow the connection here. In the workflow I use, unless I am cleaning my own history, I am only ever rebasing and fixing conflicts – not amending an earlier commit.
though tbf I generally am only “1 deep” because the “X” will get merged within a day
Your vcs shouldn’t dictate how small your changes can reasonably be or how much time you can take for code review. I sometimes write 5 changes and publish 5 code reviews that build on each other in a single day. Maybe you don’t do that, but it’s not reasonable to say “just get code reviews faster.” git fails fast, small work that builds on earlier work. If that isn’t your workflow (maybe your changes all take 1+ days to do) then that’s cool but it just means you do different work.
And code review can’t really take 1 day for everybody, even if the changes are published slowly. Sometimes code is controversial or hard and needs to have multiple iterations.
I don’t follow the connection here. In the workflow I use, unless I am cleaning my own history, I am only ever rebasing and fixing conflicts – not amending an earlier commit.
The main use case for “cleaning history” is that you get code review feedback and need to amend earlier commits. It’s a pretty straightforward use case.
If git fails for you, fair enough. I don’t claim that there is no situation where vanilla git is not enough, only that I have never encountered it using git regularly for a long time.
The main use case for “cleaning history” is that you get code review feedback and need to amend earlier commits. It’s a pretty straightforward use case.
I still don’t understand your original comment. Is your complaint just that a reviewer asks for a change on a non-head commit in your PR, and in vanilla git you have to rebase -i to do it, so therefore you need another tool to make that easier?
I don’t claim that there is no situation where vanilla git is not enough, only that I have never encountered it using git regularly for a long time.
And that’s totally fair! I’m glad git serves you well. But it sounds like you don’t use stacked PRs, or as you said, your stack is never really more than one branch deep.
Is your complaint just that a reviewer asks for a change on a non-head commit in your PR,
yep!
and in vanilla git you have to rebase -i to do it, so therefore you need another tool to make that easier?
rebase -i cannot handle a stack of branches. It only works on a single branch. If you are rebasing a stack of branches you will not end with the intermediate branch pointers pointing at your rebased code. So therefore I need another tool to make that easier.
If branch A is in code review and I need to amend Commit 2 to make a requested change, there is no way to do this in vanilla git without manually rebasing every stacked branch separately after the amend operation. With other version control systems like Mercurial, no matter how many stacked commits I have the amend is exactly either one or two commands. It scales with the number of stacked commits, or in other words an amend is always O(1) operations, unlike git which is O(# branches). Let me know if that still doesn’t make sense… this comments section is a hard place to explain complicated things.
We also decided to split up feature PRs at work into smaller reviewable units. This is often a bit of a pain when changes needed to be made to the lowest PR in the DAG, since then you need to rebase all the dependent PRs. This will get especially tricky if PRs are squashed on merge. But, having small PRs to review is still a good thing :)
Critical feature: for a reviewer to see the size of a PR at a glance.
If the only thing you know about a review is that you got an email saying that you have a PR pending, you have no way to tell the size except looking at the PR, which is already the biggest hurdle.
That’s why I don’t bother with small PRs at work; they’re easier to review, but they have no way to signal that they’re easier to review, so they have a disadvantage competing for reviewer time.
I use the highly sophisticated solution of putting [easy] in the PR title ;). This works well when most of the commits in the stack are small things like refactoring, and only one or two are substantial.
This is a great point. A couple tools (Gerrit is one of them) display t-shirt sizes of PRs, which I like. (Not sure how good the heuristic is that they use, but I find it easier to judge at a glance between “Small” and “Extra Large” than by looking at diff line count)
What about if someone creates a PR notification service that hits your inbox with all the PRs grouped? Can view in a ‘single PR view’ and also presented as a DAG.
Indeed. I don’t find emails in my inbox, I data mine with gmail search. But seriously, gmail should get good enough with AI to detect and bump up emails it knows you want to read even better than now. I see my gmail will put emails I sent in to the top of my inbox after a few days that need attention to follow up. I indicated I would do just that in the email I had originally sent. I’m terrible at following up. Value add. Conceivable it should know a PR is really important, if you’re obviously using lobste.rs and whatnot. Definitely prefer if all that power wasn’t in Googles hands and some other email service could do that.
Oh I’m not even referring to gmail specifically. At work, where I’m forced to use Outlook, I receive thousands of messages a day where I need to set up almost 100 rules to make email manageable.
Sounds like the problem isn’t the notification system but that it is overloaded. Email gives you the tools to handle this like filters and subaddressing.
When I received many hundreds of emails a day (admittedly not thousands) my best rule was to prioritize anything that mentioned my actual email, so removing mailing lists from the priority triage. That and unsubscribing from junk was sufficient for dealing with a lot of info on a daily basis. Of course you can always do something more explicit if necessary.
I have a filter which marks emails read when I’m not in the To or CC. This lets me subscribe to high-volume mailing lists without having to sift through everything.
I was surprised not to see any discussion about reviewing the description/commit message.
This is a feature that I really miss in GitLab and GitHub as the merge request description is behind an issue number after merging occurs and those tools allow you to merge stacks of commits at a time which makes the problem even worse.
Google’s Critique and Gerrit handle this fairly well. The description of the merge request is the description of the merge request so it is upfront and center. However there is still no way to explicitly review the commit message, just top-level comments that reference it with English.
The only review system that I am aware of that handles this well is emailing patches where it is as simple to comment on the description as it is to comment on any file in the diff.
That’s a great point – I updated the post with a few things in this vein.
I also miss not having PR description match commit message in Github/GitLab. While I initially found it strange, I’ve become a fan of Gerrit’s handling of this – it allows the commit message itself to be reviewed (and commented/suggested upon), which is handy.
I yearn for the ability to review commit messages in GitHub. I love that about Gerrit. It really enabled enforcement of the idea that a commit message should be meaningful and tell a story about the patch.
If you’re ok with fewer / less precise measurements, you can also slap some cheap sensors onto the i2c pins on a raspberry pi. CCS811 does eCO2 and VOC and has cheap breakout boards for example. It’s fun if you’re up for just a little bit of soldering.
Just be patient with the setup. The initialisation is really temperamental and there are two “status” commands with similar output - you’d think both work in all modes, but no…
Cloud service providers like AWS and Azure leverage verification software such as the TLA+ model checker to achieve the same goal, but whereas those solutions typically verify a high level system design, Stateright is able to verify the underlying system implementation in addition to the design (along with providing other unique benefits explained in the “Comparison with TLA+” chapter).
This seems really promising, though I haven’t delved into the programming model yet. The TLA+ Comparison page looks interesting too
This seems really promising, though I haven’t delved into the programming model yet. The TLA+ Comparison page looks interesting too
I asked him about it over on the Orange Site, and he said it works by tying all of the actors to a test orchestrator. So they can run in production, but they can also run in a special “test” mode which (I think?) exposes all of the things they can do, and then the model checker can exhaustively explore the behavior space.
I guess it would be hard to know for sure if all bugs were caught, but IME the test suite was pretty rigorous. They’re more in the vein of “systems tests” or integration tests since they’re timing-dependent and opaque to the implementation.
I wouldn’t consider my toy implementation to be production ready, but the provided test suite was rigorous enough to show me where I messed up
Why would they? If you make your blog public, for everyone to read, then other people can talk about it. And categorize it. And link to it. That is basically how the internet works.
tr.v. cen·sored, cen·sor·ing, cen·sors
To examine and expurgate.
There’s a subtler meaning in there around “but it isn’t the government doing it”, but given the size of Google, Cloudflare, Facebook, and others who have successfully walled-in the public square, it is pretty disingenuous to pretend like there isn’t at least something going on there.
It’s completely reasonable for people to be concerned and want to learn how to host their own services, and mocking them for attempted independence seems to me to be both short-sighted and a defection against the hacker spirit.
Also on that topic, I’m reminded of the part in chapter 6 of Hackers by Steven Levy, where the MIT AI Lab hackers hated Multics in part because of its fine-grained usage accounting. Kind of like AWS and similar services, no?
(So yes, the fact that my current project is all-in on AWS causes me cognitive dissonance. Not sure how to resolve it though. Multi-AZ deployments with automated recovery from instance failures are certainly good for peace of mind.)
Think about what infrastructure changes you would need to make in order not to be fully dependent on AWS, and then make them. Even if you don’t switch away from AWS immediately, being prepared to do so will make it easier on you if they do decide for whatever reason to deplatformed you, or if a competing cloud provider starts offering a better deal.
What is it when all the infrastructure providers, payment processors, banks, and social media platforms all decide to stop doing business with you?
So glad my beliefs are currently in vogue with whatever you call that collective, whatever their non-censorship is, I’m glad I’m not being subjected to it.
I followed this saga at the time, and my impression was that AWS bent over backwards to accommodate this service. It was only after those responsible failed to moderate the statements made by their users that violated the ToS they had willingly agreed to that service was suspended - not terminated.
The site is back online. The service is not, which really makes one wonder how much this is a genuine wish to offer a free-speech platform and how much it was an attempt to soak a well-healed backer for a lot of money.
That writer seems to think any action is justified as long as it’s done by private companies in a competitive free market. Under that assumption, of course a breach of contract is more than enough reason to suspend service.
But if we care about who actually has power in society and how communication is shaped by different actors, this cold comfort. If AWS “bends over backwards” to offer a service they are being paid for, until the risk of a PR crisis makes it not worth it for them, they are still wielding unaccountable power to limit who gets to speak.
Masnick’s position is more nuanced than you summarize it, but the idea of private parties competing in a free market is one that has served the US economy well for a long time.
As for AWS “wielding unaccountable power”, they’re far from a monopoly. Oracle is gunning aggressively for their business, as mentioned in other threads on this very page, and in the article I linked.
And whose speech are we talking about? The users of the site are free to create accounts elsewhere, and many have surely done so. What’s left is the limited speech rights of the service to make money hosting these users. This right has to be weighed against AWS’ rights to make money providing cloud computing to many other customers, all of whom AWS is aware can change providers if AWS allows toxic actors on its service.
If the service’s business model was to make money hosting speech that was banned elsewhere, it bordered on criminal negligence not to take the risk of being suspended into account, and making plans to shift hosting providers accordingly. Again, this points to this being a grift rather than a sound business idea.
I think it is perverse to give any weight to a large company’s “right” to make money in a certain way, when weighed against issues that affect the mass of society. But I suspect you and I disagree fundamentally about this, so there’s probably no use trying to find agreement.
Another thing we will probably disagree about: the potential for competition among a small group of companies does not amount to accountability. People have very little say over what these companies do.
haha agreed, this the second time I’ve seen someone cite the “First they came for” poem to defend groups of people actively “coming for” me and my loved ones.
Always feel weird when this kind of content use proto-fascism and right-wing extremists as an example for why we need to fight oligarch censorship. Of all the victims of censorship, those are the one I could not care less.
No. Not when those being censored are the very same people that would want to be the “They” in the poem. The poem is that we shouldn’t stay idle while some group is trying to take advantage over other groups. I feel like what happened right now is that someone actually did speak up…
And good for them, let them rot in jail for all I care. I don’t mean that we should stand idle while states and corporations consolidate their power. Let’s speak about how HUAC or oligarchs can use their power monopoly and use it against the rest of the population. Let’s discuss about how we can fight against these crackdown in the point of view of freedom and privacy, not about how a right-wing extremist community should have done better.
Who is discussing how a right-wing extremist community “should have done better”? You lost me there.
I guess you don’t value the legal and social norm of free speech as such, and take no issue if that norm is violated to target racist groups. You either don’t think that makes it easier to target non-racist groups, or you don’t care.
Who is discussing how a right-wing extremist community “should have done better”?
Taken as-is from the article:
Parler’s epic fail: A crash course on running your own servers with a shoestring budget
I argue that your chances of survival are much better this way, and Parler is foolish for not going this route. We can do better.
Parler was cut off by their cloud hosting provider, Amazon. Where do we go from here? What can a freedom-minded person do to avoid censorship by tech oligarchs?
Hosting your own content is exactly the method by which you work around private platform companies refusing to do business with you for political reasons.
I have been using Obsidian to keep track of what I learn, and it has been working great for me so far. I love the fact that I can quickly link concepts together without too much hassle.
+1 for Obsidian. I like that it’s all local, and in plain markdown.
I setup a small script to automatically checkin changes to a git repo, for syncing.
ReadDir is optional so that ordinary files (full of bytes) don’t have to implement dummy ReadDir methods. It would be a bit odd to omit ReadDir on an actual directory, because then any traversal tools won’t work. For example the hypothetical key-value store without ReadDir will not work with fs.Walk (or fs.Glob).
Most code shouldn’t need to define things like readDirStatFS or even refer to the ReadDirFS and StatFS interfaces. It should instead use the helper functions fs.ReadDir and fs.Stat, which use those interfaces internally as appropriate. Code that instead requires, say, the readDirStatFS interface will not work with valid FS implementations that don’t provide a direct fs.Stat method (fs.Stat calls fs.Open+f.Stat+f.Close in that case).
fstest.TestFS does more than just assert that a few files exist. It walks the entire file tree in the file system you give it, checking that all the various methods it can find are well-behaved and diagnosing a bunch of common mistakes that file system implementers might make. For example it opens every file it can find and checks that Read+Seek and ReadAt give consistent results. And lots more. So if you write your own FS implementation, one good test you should write is a test that constructs an instance of the new FS and then passes it to fstest.TestFS for inspection.
One of the most common mistakes you might make in an FS implementation would be to mess up ReadDir on Open(”.”) somehow so that it returns no files at all. If you did this, then TestFS’s walk of the entire file system would find nothing to test, so nothing would fail any tests, and the overall TestFS call would incorrectly pass. The list of files passed to TestFS removes this failure mode by saying “look, when you do the walk looking for things to test, make sure you at least see this list of files, or else ReadDir is broken”.
Again, thanks for taking the time to write the post. It was fun to read.
I’m refactoring my Among Us league website so that I can move it off glitch.me which has been unreliable lately. It’s an Express app so I think I can wrap it somehow and deploy the whole thing to Netlify and have it run with serverless. Basically this tutorial.
I’m also learning more chess after ‘finishing’ my chess engine ♞
The best way to give a question rigor is to create it from a rigorous algorithm.
Yes, I skipped this line, so while reading I was wondering if I actually really suck at Python… I thought I knew what I was doing, until I was stumped by these questions.
This paper is an extended version of an earlier conference paper (Mokhovet al., 2018).The key changes compared to the earlier version are: (i) we added further clarifications and examples to §3, in particular, §3.8 is entirely new; (ii) §4 and §5 are based on the material from the conference paper but have been substantially expanded to include further details and examples, as well as completely new material such as §5.2.2; (iii) §7 is completely new; (iv) §8.1 and §§8.6-8.9 are almost entirely new, and §8.3 has been revised. The new material focuses on our experience and various important practical considerations, hence justifying the “and Practice” part of the paper title
+1 for Kleppmann’s Designing Data-Intensive Applications book. It’s really the best thing I’ve read on distributed systems: it has a good breadth of knowledge, and has enough depth that you can start searching out more detailed information for topics you’re interested in learning more about
This is really exciting to me. I definitely love the mercurial workflow with a dag of commits that individually get reviewed and submitted. Generally the goal is not to have a deep dag when possible, but overall, it is a much nicer flow than adhoc GitHub branches and PRs.
I wonder how well or poorly this would fit with some open source work I do. I assume it wouldn’t work to be the only person using git-branchless or similar tools. From a first look, it seems to match the workflow I find best, but not at all sure about integration.
I really like the mercurial workflow too, and I’ve been chasing a similar experience in git.
git-branchlessis the closest thing I’ve found, but if you need to use branches (e.g. to push PRs to Github), it can be a bit clunky. With Gerrit, git-branchless gets you really close to the mercurial flowgit-stack should probably link here instead https://github.com/gitext-rs/git-stack
Thanks! I missed that it’d moved (or maybe I linked to a fork?)
I used to do this but it would almost always get messy. If changes are necessary after code review on a PR early in the chain (and you should assume there will be), you need to rebase or merge the earlier one into any PRs that depend on it. It’s often not possible to do this cleanly.
Nope. Hard disagree. You need to rebase. Only rebase, no merge, never merge in this situation. It just fucks up the tree and make hard to figure out what should go where. I only ever merge in one direction, and rebase in the other.
I actually 100% agree, and would only ever rebase in this situation. I only added “or merge” in anticipation of some people who get weird about rebasing anything ever. :P
git-branchlesshandles this well (author here). You just check out the old commit and amend it and rungit restack. This updates all descendant commits and branches. It won’t start merge conflict resolution unless you pass--merge, so you can decide to resolve conflicts either now or later. You can also usegit-branchless’s amend tool to do both in one step.You literally can’t do this with vanilla git or you’ll go crazy. Ben was underselling how hard it is in his post.
I currently use git-branchless thanks to @arxanas hard work. The only thing it’s missing is the ability to sync a branch of pull requests up to Bit Bucket but I’m not expect any tool to get that anytime soon ;(
Interesting, I haven’t noticed this in my experience. But I definitely see the risk with stacking commits that fundamental changes in the base commits would invalidate assumptions used to build later ones
Not sure I follow, but if the early commit has already been merged, and then you realize you have to make a fix before the next one goes in, then those just rebase on the fix – why is that a problem? What is the alternative?
The problem is with thrashing between several PRs and keeping track of which ones need updating and in what order (i.e. if you have a dep chain of 3+ PRs). The other problem is dealing with conflicts, which is just annoying. In my experience you’re just trading one kind of mental overhead for another.
The alternative is to just work on something else while the base PR is under review. IMO this is a social/collaboration problem, not a technical one. :)
Gotcha.
Completely agree.
I would say the solution is small batch sizes and a fast enough PR turnaround time. If you have those things, you can keep working on the next dependent PRs, and even if the reviewer requests larger changes than you anticipated, the rebasing and conflict resolution work should never be unmanageable.
Said another way, batch sizes are too big, or reviews happen too slowly, if the ripple effects of review scare you to the point of pausing work.
You cannot rebase a stack of multiple branches in git with one command. Try it… crazy shit happens.
Well yeah, you’d need to rebase one, merge it, then the next, etc. My comment above about the social aspect is relevant here: everything will be fine if your PRs are small and getting reviewed quickly. If that’s not happening, you’re hosed anyway. This isn’t a tech problem, and you don’t need these other services on top of git to solve it.
I’m not sure I understand your comment. I think we might be talking about different things. If your version control system doesn’t support you continuing to work by building up changes on top of changes already being reviewed you will not create small, granular code reviews. It’s not a social problem that I’m talking about, it’s very much a tooling problem?
We might be. Example to clarify:
My claim is that this works perfectly fine in vanilla git, and rebasing is quite easy, so long as we have small PRs moving along quickly. I guess my question to you is: If you disagree with this claim, what is the situation that breaks it and why do I not encounter it? My guess is that the answer would have to be differences in team size or culture, and not tooling.
“ A rebases Y and Z on X.”
This is what I’m disagreeing with. This doesn’t work well and requires a lot of manual work.
If you rebase Z on X you bring Y’s commits but not the Y branch pointer. If you rebase Y on X you bring neither the Z commits nor the Z branch pointer. You can force move the branch pointer but this doesn’t scale to a lot of stacked branches, since you’re now doing work PER branch, is generally messy and error prone, and is a pain in the neck. People are often surprised by this behavior but if you go and play with it you’ll quickly see what I mean.
To make stacked branches/PRs work, your vcs must make amending an earlier commit a single command, not one or two commands per descendant branch.
I never have these problem and I do what I decribed almost every day, though tbf I generally am only “1 deep” because the “X” will get merged within a day. But this is what I was saying originally: if your “stack” is so deep you have culture problem. Imo, at that point you change the workflow so you have smaller PRs and a shallow queue.
Again, this is never a problem in practice for me, because the pain I’ll feel when I later rebase Z only depends on how far the work has drifted / how many conflicts I have to deal with.
I just played with it in a sample repo and I still don’t get it.
I don’t follow the connection here. In the workflow I use, unless I am cleaning my own history, I am only ever rebasing and fixing conflicts – not amending an earlier commit.
Your vcs shouldn’t dictate how small your changes can reasonably be or how much time you can take for code review. I sometimes write 5 changes and publish 5 code reviews that build on each other in a single day. Maybe you don’t do that, but it’s not reasonable to say “just get code reviews faster.” git fails fast, small work that builds on earlier work. If that isn’t your workflow (maybe your changes all take 1+ days to do) then that’s cool but it just means you do different work.
And code review can’t really take 1 day for everybody, even if the changes are published slowly. Sometimes code is controversial or hard and needs to have multiple iterations.
The main use case for “cleaning history” is that you get code review feedback and need to amend earlier commits. It’s a pretty straightforward use case.
If git fails for you, fair enough. I don’t claim that there is no situation where vanilla git is not enough, only that I have never encountered it using git regularly for a long time.
I still don’t understand your original comment. Is your complaint just that a reviewer asks for a change on a non-head commit in your PR, and in vanilla git you have to
rebase -ito do it, so therefore you need another tool to make that easier?And that’s totally fair! I’m glad git serves you well. But it sounds like you don’t use stacked PRs, or as you said, your stack is never really more than one branch deep.
yep!
rebase -icannot handle a stack of branches. It only works on a single branch. If you are rebasing a stack of branches you will not end with the intermediate branch pointers pointing at your rebased code. So therefore I need another tool to make that easier.If I have these commits/branches:
If
branch Ais in code review and I need to amendCommit 2to make a requested change, there is no way to do this in vanilla git without manually rebasing every stacked branch separately after the amend operation. With other version control systems like Mercurial, no matter how many stacked commits I have the amend is exactly either one or two commands. It scales with the number of stacked commits, or in other words an amend is always O(1) operations, unlike git which is O(# branches). Let me know if that still doesn’t make sense… this comments section is a hard place to explain complicated things.Ah, ok. Your last example clears it up for me. Thanks for explaining. Good to know about if I find myself in that situation in the future.
We also decided to split up feature PRs at work into smaller reviewable units. This is often a bit of a pain when changes needed to be made to the lowest PR in the DAG, since then you need to rebase all the dependent PRs. This will get especially tricky if PRs are squashed on merge. But, having small PRs to review is still a good thing :)
Critical feature: for a reviewer to see the size of a PR at a glance.
If the only thing you know about a review is that you got an email saying that you have a PR pending, you have no way to tell the size except looking at the PR, which is already the biggest hurdle.
That’s why I don’t bother with small PRs at work; they’re easier to review, but they have no way to signal that they’re easier to review, so they have a disadvantage competing for reviewer time.
I use the highly sophisticated solution of putting
[easy]in the PR title ;). This works well when most of the commits in the stack are small things like refactoring, and only one or two are substantial.This is a great point. A couple tools (Gerrit is one of them) display t-shirt sizes of PRs, which I like. (Not sure how good the heuristic is that they use, but I find it easier to judge at a glance between “Small” and “Extra Large” than by looking at diff line count)
What about if someone creates a PR notification service that hits your inbox with all the PRs grouped? Can view in a ‘single PR view’ and also presented as a DAG.
Email is a terrible notification system. I already receive literal thousands of messages per day.
Indeed. I don’t find emails in my inbox, I data mine with gmail search. But seriously, gmail should get good enough with AI to detect and bump up emails it knows you want to read even better than now. I see my gmail will put emails I sent in to the top of my inbox after a few days that need attention to follow up. I indicated I would do just that in the email I had originally sent. I’m terrible at following up. Value add. Conceivable it should know a PR is really important, if you’re obviously using lobste.rs and whatnot. Definitely prefer if all that power wasn’t in Googles hands and some other email service could do that.
Oh I’m not even referring to gmail specifically. At work, where I’m forced to use Outlook, I receive thousands of messages a day where I need to set up almost 100 rules to make email manageable.
Sounds like the problem isn’t the notification system but that it is overloaded. Email gives you the tools to handle this like filters and subaddressing.
When I received many hundreds of emails a day (admittedly not thousands) my best rule was to prioritize anything that mentioned my actual email, so removing mailing lists from the priority triage. That and unsubscribing from junk was sufficient for dealing with a lot of info on a daily basis. Of course you can always do something more explicit if necessary.
I have a filter which marks emails read when I’m not in the To or CC. This lets me subscribe to high-volume mailing lists without having to sift through everything.
Author here, glad you found this post interesting!
I was surprised not to see any discussion about reviewing the description/commit message.
This is a feature that I really miss in GitLab and GitHub as the merge request description is behind an issue number after merging occurs and those tools allow you to merge stacks of commits at a time which makes the problem even worse.
Google’s Critique and Gerrit handle this fairly well. The description of the merge request is the description of the merge request so it is upfront and center. However there is still no way to explicitly review the commit message, just top-level comments that reference it with English.
The only review system that I am aware of that handles this well is emailing patches where it is as simple to comment on the description as it is to comment on any file in the diff.
That’s a great point – I updated the post with a few things in this vein.
I also miss not having PR description match commit message in Github/GitLab. While I initially found it strange, I’ve become a fan of Gerrit’s handling of this – it allows the commit message itself to be reviewed (and commented/suggested upon), which is handy.
If the PR only has one commit GitHub pre-fills the PR Title and Body with the commit message. It is a step in Gerrit’s direction at least.
I yearn for the ability to review commit messages in GitHub. I love that about Gerrit. It really enabled enforcement of the idea that a commit message should be meaningful and tell a story about the patch.
If you’re ok with fewer / less precise measurements, you can also slap some cheap sensors onto the i2c pins on a raspberry pi. CCS811 does eCO2 and VOC and has cheap breakout boards for example. It’s fun if you’re up for just a little bit of soldering.
Neat! I hadn’t heard of that sensor before
Just be patient with the setup. The initialisation is really temperamental and there are two “status” commands with similar output - you’d think both work in all modes, but no…
This seems really promising, though I haven’t delved into the programming model yet. The TLA+ Comparison page looks interesting too
I asked him about it over on the Orange Site, and he said it works by tying all of the actors to a test orchestrator. So they can run in production, but they can also run in a special “test” mode which (I think?) exposes all of the things they can do, and then the model checker can exhaustively explore the behavior space.
Interesting post! Were all the bugs caught by MIT test suite?
I guess it would be hard to know for sure if all bugs were caught, but IME the test suite was pretty rigorous. They’re more in the vein of “systems tests” or integration tests since they’re timing-dependent and opaque to the implementation.
I wouldn’t consider my toy implementation to be production ready, but the provided test suite was rigorous enough to show me where I messed up
Does the owner of this site notify blogs that they’re included and give them an option to opt-out?
Why would they? If you make your blog public, for everyone to read, then other people can talk about it. And categorize it. And link to it. That is basically how the internet works.
They are selling their dataset, which might rub some folks the wrong way (without an opt-out) https://blogsurf.io/data
That is actually quite a neat way to monitize a site like this. No problem with manually adding 30 feeds, or pay 5 to get the opml file… Clever
I use a Kinesis Freestyle Pro, and a custom built mechanical keyboard. Both are split ergonomic keyboards
Out of interest do you have a preference for the othrolinear Kinesis or your staggered custom build?
Nearly stopped reading here, but I’ll happily post it again and again: It’s not censorship if one company chooses to not do business with you.
That’s actually still censorship:
There’s a subtler meaning in there around “but it isn’t the government doing it”, but given the size of Google, Cloudflare, Facebook, and others who have successfully walled-in the public square, it is pretty disingenuous to pretend like there isn’t at least something going on there.
For the folks going “hah, so what if it happens to people I don’t like?”–remember that time Tumblr’s ban hurt LGBTQ+ folks? Remember the various pro-BLM folks Twitter banned? Pepperidge Farm remembers.
It’s completely reasonable for people to be concerned and want to learn how to host their own services, and mocking them for attempted independence seems to me to be both short-sighted and a defection against the hacker spirit.
Also on that topic, I’m reminded of the part in chapter 6 of Hackers by Steven Levy, where the MIT AI Lab hackers hated Multics in part because of its fine-grained usage accounting. Kind of like AWS and similar services, no?
(So yes, the fact that my current project is all-in on AWS causes me cognitive dissonance. Not sure how to resolve it though. Multi-AZ deployments with automated recovery from instance failures are certainly good for peace of mind.)
Think about what infrastructure changes you would need to make in order not to be fully dependent on AWS, and then make them. Even if you don’t switch away from AWS immediately, being prepared to do so will make it easier on you if they do decide for whatever reason to deplatformed you, or if a competing cloud provider starts offering a better deal.
maybe it is if the handful of companies that are powerful than most countries decide not to do business with you.
Regardless of that highly charged political question, I think fighting oligarchy is worthwhile in itself.
What is it when all the infrastructure providers, payment processors, banks, and social media platforms all decide to stop doing business with you?
So glad my beliefs are currently in vogue with whatever you call that collective, whatever their non-censorship is, I’m glad I’m not being subjected to it.
I followed this saga at the time, and my impression was that AWS bent over backwards to accommodate this service. It was only after those responsible failed to moderate the statements made by their users that violated the ToS they had willingly agreed to that service was suspended - not terminated.
The site is back online. The service is not, which really makes one wonder how much this is a genuine wish to offer a free-speech platform and how much it was an attempt to soak a well-healed backer for a lot of money.
In what way did they bend over backwards? Was there even a court order telling them to take the site down?
AWS gave them multiple chances to implement effective moderation.
There was no court order, the issue was a breach of contract (the ToS).
I can recommend Techdirt’s coverage of the issue, with this opinion piece as a good starting point https://www.techdirt.com/articles/20210115/00240746061/few-more-thoughts-total-deplatforming-parler-infrastructure-content-moderation.shtml.
https://www.techdirt.com/search-g.php?q=parler
That writer seems to think any action is justified as long as it’s done by private companies in a competitive free market. Under that assumption, of course a breach of contract is more than enough reason to suspend service.
But if we care about who actually has power in society and how communication is shaped by different actors, this cold comfort. If AWS “bends over backwards” to offer a service they are being paid for, until the risk of a PR crisis makes it not worth it for them, they are still wielding unaccountable power to limit who gets to speak.
Masnick’s position is more nuanced than you summarize it, but the idea of private parties competing in a free market is one that has served the US economy well for a long time.
As for AWS “wielding unaccountable power”, they’re far from a monopoly. Oracle is gunning aggressively for their business, as mentioned in other threads on this very page, and in the article I linked.
And whose speech are we talking about? The users of the site are free to create accounts elsewhere, and many have surely done so. What’s left is the limited speech rights of the service to make money hosting these users. This right has to be weighed against AWS’ rights to make money providing cloud computing to many other customers, all of whom AWS is aware can change providers if AWS allows toxic actors on its service.
If the service’s business model was to make money hosting speech that was banned elsewhere, it bordered on criminal negligence not to take the risk of being suspended into account, and making plans to shift hosting providers accordingly. Again, this points to this being a grift rather than a sound business idea.
I think it is perverse to give any weight to a large company’s “right” to make money in a certain way, when weighed against issues that affect the mass of society. But I suspect you and I disagree fundamentally about this, so there’s probably no use trying to find agreement.
Another thing we will probably disagree about: the potential for competition among a small group of companies does not amount to accountability. People have very little say over what these companies do.
Agreed. i appreciate the meat of the article but I can definitely do without the edgy quote at the beginning.
Ditto with the mentions to Parler (including in the title)
haha agreed, this the second time I’ve seen someone cite the “First they came for” poem to defend groups of people actively “coming for” me and my loved ones.
Always feel weird when this kind of content use proto-fascism and right-wing extremists as an example for why we need to fight oligarch censorship. Of all the victims of censorship, those are the one I could not care less.
I think that’s rather the point of the poem, isn’t it?
No. Not when those being censored are the very same people that would want to be the “They” in the poem. The poem is that we shouldn’t stay idle while some group is trying to take advantage over other groups. I feel like what happened right now is that someone actually did speak up…
Plenty of people in Weimar Germany felt that way about the communists.
Not really, at least in my reading. It’s not about the dangers of a “couldn’t care less” mindset but rather one of cowardice.
HUAC was first used to jail Nazi sympathizers
And good for them, let them rot in jail for all I care. I don’t mean that we should stand idle while states and corporations consolidate their power. Let’s speak about how HUAC or oligarchs can use their power monopoly and use it against the rest of the population. Let’s discuss about how we can fight against these crackdown in the point of view of freedom and privacy, not about how a right-wing extremist community should have done better.
Who is discussing how a right-wing extremist community “should have done better”? You lost me there.
I guess you don’t value the legal and social norm of free speech as such, and take no issue if that norm is violated to target racist groups. You either don’t think that makes it easier to target non-racist groups, or you don’t care.
Taken as-is from the article:
ah
whos coming for you
Hosting your own content is exactly the method by which you work around private platform companies refusing to do business with you for political reasons.
I have been using Obsidian to keep track of what I learn, and it has been working great for me so far. I love the fact that I can quickly link concepts together without too much hassle.
+1 for Obsidian. I like that it’s all local, and in plain markdown. I setup a small script to automatically checkin changes to a git repo, for syncing.
Nice post, thanks. A couple clarifications:
os.File is not an interface
ReadDir is optional so that ordinary files (full of bytes) don’t have to implement dummy ReadDir methods. It would be a bit odd to omit ReadDir on an actual directory, because then any traversal tools won’t work. For example the hypothetical key-value store without ReadDir will not work with fs.Walk (or fs.Glob).
Most code shouldn’t need to define things like readDirStatFS or even refer to the ReadDirFS and StatFS interfaces. It should instead use the helper functions fs.ReadDir and fs.Stat, which use those interfaces internally as appropriate. Code that instead requires, say, the readDirStatFS interface will not work with valid FS implementations that don’t provide a direct fs.Stat method (fs.Stat calls fs.Open+f.Stat+f.Close in that case).
fstest.TestFS does more than just assert that a few files exist. It walks the entire file tree in the file system you give it, checking that all the various methods it can find are well-behaved and diagnosing a bunch of common mistakes that file system implementers might make. For example it opens every file it can find and checks that Read+Seek and ReadAt give consistent results. And lots more. So if you write your own FS implementation, one good test you should write is a test that constructs an instance of the new FS and then passes it to fstest.TestFS for inspection.
One of the most common mistakes you might make in an FS implementation would be to mess up ReadDir on Open(”.”) somehow so that it returns no files at all. If you did this, then TestFS’s walk of the entire file system would find nothing to test, so nothing would fail any tests, and the overall TestFS call would incorrectly pass. The list of files passed to TestFS removes this failure mode by saying “look, when you do the walk looking for things to test, make sure you at least see this list of files, or else ReadDir is broken”.
Again, thanks for taking the time to write the post. It was fun to read.
Thanks for the response! I updated the original post to include these clarifications.
I’m refactoring my Among Us league website so that I can move it off glitch.me which has been unreliable lately. It’s an Express app so I think I can wrap it somehow and deploy the whole thing to Netlify and have it run with serverless. Basically this tutorial.
I’m also learning more chess after ‘finishing’ my chess engine ♞
I’ve enjoyed both posts you wrote recently on the Among Us league and chess engine!
I’d appreciate a followup if you get your project working on Netlify – I also have a couple glitch projects I’d like to migrate
I just finished refactoring the Express application from Glitch to Netlify. (On Glitch, I had some static files in
public/and a few API routes.)It took around two hours and I don’t have a super-clean diff to show but I can write some notes :)
I used these resources:
The steps I took:
I created a
netlify.tomlfile:I installed the following with npm:
netlify-lambda,serverless-http, andencoding.I added a script to the package.json file:
"build": "netlify-lambda build src"I moved my express app into
src/app.jsand changed the exports to:Finally, I changed all the API calls from the static files so they start with:
.netlify/functions/app/DM me on Twitter/Email if you get stuck!
Thanks! :)
Will do. I should at least be able to send you a before and after GitHub diff.
Oh cool. Just wondering about how glitch has been unreliable, is it a downtime issue or other?
They had a little downtime recently but the main motivations for swapping to Netlify are:
I quite like glitch, especially as a learning platform but I’d like to have more control over the infrastructure
It helps to read the intro here! :^) I skimmed to the questions without reading the intro first, and was thoroughly confused.
I was thoroughly confused even after reading the intro. The whole article made little sense.
Yes, I skipped this line, so while reading I was wondering if I actually really suck at Python… I thought I knew what I was doing, until I was stumped by these questions.
Great post! I’ve been thinking of doing something similar – this seems like a neat approach.
As a meta question, does your site have an RSS (or similar) feed?
Thanks! Yes, it’s buried in the HTML metadata: https://shazow.net/index.xml
I should turn off the reformatting by Netlify so that it’s more readable.
This link 404’s for me. Here’s a mirror I found: https://www.microsoft.com/en-us/research/uploads/prod/2020/04/build-systems-jfp.pdf
Hm this paper is now 55 pages? The one from 2018 was 29 pages.
https://www.microsoft.com/en-us/research/uploads/prod/2018/03/build-systems.pdf
It’s kind of weird that they slightly modified the title and added more content, and the 3 authors are the same.
I’d be curious for a summary of the “diff” …
There is a summary by the end of section 1:
Previous discussions: 1, 2
Awesome book about distributed systems by Martin Kleppmann - http://dataintensive.net/
Awesome Distributed Systems
Testing of distributed systems - collection of links to interesting resources about distributed systems testing.
An introduction to distributed systems is actually an outline of a paid course by Aphyr, but it can be useful too.
+1 for Kleppmann’s Designing Data-Intensive Applications book. It’s really the best thing I’ve read on distributed systems: it has a good breadth of knowledge, and has enough depth that you can start searching out more detailed information for topics you’re interested in learning more about
another big +1 for Kleppmann’s. One of the best books in my library. I keep getting back to it all the time.