Threads for alerque

  1. 40

    The interface of Git and its underlying data models are two very different things, that are best treated separately.

    The interface is pretty bad. If I wasn’t so used to it I would be fairly desperate for an alternative. I don’t care much for the staging area, I don’t like to have to clean up my working directory every time I need to switch branches, and I don’t like how easy it is to lose commit from a detached HEAD (though there’s always git reflog I guess).

    The underlying data model however is pretty good. We can probably ditch the staging area, but apart from that, viewing the history of a repository as a directed graph of snapshots is nice. Captures everything we need. Sure patches have to be derived from those snapshots, but we care less about the patches than we care about the various versions we saved. If there’s one thing we need to get right, it’s those snapshots. You get reproducible builds & test from them, not from patches. So I think Patches are secondary. I used to love DARCS, but I think patch theory was probably the wrong choice.

    Now one thing Git really really doesn’t like is large binary files. Especially if we keep changing them. But then that’s just a compression problem. Let the data model pretend there’s a blob for each version of that huge file, even though in fact the software is automatically compressing & decompressing things under the hood.

    1. 62

      What’s wrong with the staging area? I use it all the time to break big changes into multiple commits and smaller changes. I’d hate to see it removed just because a few people don’t find it useful.

      1. 27

        Absolutely, I would feel like I’m missing a limb without the staging area. I understand that it’s conceptually difficult at first, but imo it’s extremely worth the cost.

        1. 7

          Do you actually use it, or do you just do git commit -p, which only happens to use the staging area as an implementation detail?

          And how do you test the code you’re committing? How do you make sure that the staged hunks aren’t missing another hunk that, for example, changes the signature the function you’re calling? It’s a serious slowdown in workflow to need to wait for CI rounds, stash and rebase to get a clean commit, and push again.

          1. 25

            Do you actually use it

            Yes.

            And how do you test the code you’re committing?

            rebase with --exec

            1. 12

              I git add -p to the staging area and then diff it before generating the commit. I guess that could be done without a staging area using a different workflow but I don’t see the benefit (even if I have to check git status for the command every time I need to unstage something (-: )

              As for testing, since I’m usually using Github I use the PR as the base unit that needs to pass a test (via squash merges, the horror I know). My commits within a branch often don’t pass tests; I use commits to break things up into sections of functionality for my own benefit going back later.

              1. 7

                Just to add on, the real place where the staging area shines is with git reset -p. You can reset part of a commit, amend the commit, and then create a new commit with your (original) changes or continue editing. The staging area becomes more useful the more you do commit surgery.

                1. 2

                  Meh, you don’t need a staging area for that (or anything). hg uncommit -i (for --interactive) does quite the same thing, and because it has no artificial staging/commit split it gets to use the clear verb.

                2. 2

                  I guess that could be done without a staging area using a different workflow but I don’t see the benefit

                  I don’t see the cost.

                  My commits within a branch often don’t pass tests;

                  If you ever need to git bisect, you may come to regret that. I almost never use git bisect, but for the few times I did need it it was a life saver, and passing tests greatly facilitate it.

                  1. 9

                    I bisect every so often, but on the squashed PR commits on main, not individual commits within a PR branch. I’ve never needed to do that to diagnose a bug. If you have big PRs, don’t squash, or don’t use a PR-based workflow, that’s different of course. I agree with the general sentiment that all commits on main should pass tests for the purposes of bisection.

                3. 3

                  I use git gui for committing, (the built in git gui command) which let’s you pick by line not just hunks. Normally the things I’m excluding are stuff like enabling debug flags, or just extra logging, so it’s not really difficult to make sure it’s correct. Not saying I never push bad code, but I can’t recall an instance where I pushed bad code because of that so use the index to choose parts of my unfinished work to save in a stash (git stash –keep-index), and sometimes if I’m doing something risky and iterative I’ll periodically add things to the staging area as I go so I can have some way to get back to the last known good point without actually making a bunch of commits ( I could rebase after, yeah but meh).

                  It being just an implementation detail in most of that is a fair point though.

                  1. 2

                    I personally run the regression test (which I wrote) to test changes.

                    Then I have to wait for the code review (which in my experience has never stopped a bug going through; when I have found bugs, in code reviews, it was always “out of scope for the work, so don’t fix it”) before checking it in. I’m dreading the day when CI is actually implemented as it would slow down an already glacial process [1].

                    Also, I should mention I don’t work on web stuff at all (thank God I got out of that industry).

                    [1] Our customer is the Oligarchic Cell Phone Company, which has a sprint of years, not days or weeks, with veto power over when we deploy changes.

                  2. 5

                    Author of the Jujutsu VCS mentioned in the article here. I tried to document at https://github.com/martinvonz/jj/blob/main/docs/git-comparison.md#the-index why I think users don’t actually need the index as much as they think.

                    I missed the staging area for at most a few weeks after I switched from Git to Mercurial many years ago. Now I miss Mercurial’s tools for splitting commits etc. much more whenever I use Git.

                    1. 1

                      Thanks for the write up. From what I read it seems like with Jujutsu if I have some WIP of which I want to commit half and continue experimenting with the other half I would need to commit it all across two commits. After that my continuing WIP would be split across two places: the second commit and the working file changes. Is that right? If so, is there any way to tag that WIP commit as do-not-push?

                      1. 3

                        Not quite. Every time you run a command, the working copy is snapshotted and becomes a real commit, amending the precis working-copy commit. The changes in the working copy are thus treated just like any other commit. The corresponding think to git commit -p is jj split, which creates two stacked commits from the previous working-copy commit, and the second commit (the child) is what you continue to edit in the working copy.

                        Your follow-up question still applies (to both commits instead of the single commit you seemed to imagine). There’s not yet any way of marking the working copy as do-not-push. Maybe we’ll copy Mercurial’s “phase” concept, but we haven’t decided yet.

                  3. 8

                    Way I see it, the staging area is a piece of state needed specifically for a command line interface. I use it too, for the exact reason you do. But I could do the same by committing it directly. Compare the possible workflows. Currently we do:

                    # most of the time
                    git add .
                    git commit
                    
                    # piecemeal
                    git add -p .
                    # review changes
                    git commit
                    

                    Without a staging area, we could instead do that:

                    # most of the time
                    git commit
                    
                    # piecemeal
                    git commit -p
                    # review changes
                    git reset HEAD~ # if the changes are no good
                    

                    And I’m not even talking about a possible GUI for the incremental making of several commits.

                    1. 7

                      Personally I use git add -p all of the time. I’ve simply been burned by the other way too many times. What I want is not to save commands but to have simple commands that work for me in every situation. I enjoy the patch selection phase. More often than not it is what triggers my memory of a TODO item I forgot to jot down, etc. The patch selection is the same as reviewing the diff I’m about to push but it lets me do it incrementally so that when I’m (inevitably) interrupted I don’t have to remember my place.

                      From your example workflows it seems like you’re interested in avoiding multiple commands. Perhaps you could use git commit -a most of the time? Or maybe add a commit-all alias?

                      1. 1

                        Never got around to write that alias, and if I’m being honest I quite often git diff --cached to see what I’ve added before I actually commit it.

                        I do need something that feels like a staging area. I was mostly wondering whether that staging area really needed to be implemented differently than an ordinary commit. Originally I believed commits were enough, until someone pointed out pre-commit hooks. Still, I wonder why the staging area isn’t at least a pointer to a tree object. It would have been more orthogonal, and likely require less effort to implement. I’m curious what Linus was thinking.

                        1. 2

                          Very honourable to revise your opinion in the face of new evidence, but I’m curious to know what would happen if you broadened the scope of your challenge with “and what workflow truly requires pre-commit hooks?”!

                          1. 1

                            Hmm, that’s a tough one. Strictly speaking, none. But I can see the benefits.

                            Take Monocypher for instance: now it’s pretty stable, and though it is very easy for me to type make test every time I modify 3 characters, in practice I may want to make sure I don’t forget to do it before I commit anything. But even then there are 2 alternatives:

                            • Running tests on the server (but it’s better suited to a PR model, and I’m almost the only committer).
                            • Having a pre push hook. That way my local commits don’t need the hook, and I could go back to using the most recent one as a staging area.
                        2. 1

                          I use git add -p all the time, but only because Magit makes it so easy. If I had an equally easy interface to something like hg split or jj split, I don’t think I’d care about the lack of an index/staging area.

                        3. 6

                          # most of the time

                          git add .

                          Do you actually add your entire working directory most of the time? Unless I’ve just initialized a repository I essentially never do that.

                          Here’s something I do do all the time, because my mind doesn’t work in a red-green-refactor way:

                          Get a bug report

                          Fix bug in foo_controller

                          Once the bug is fixed, I finally understand it well enough to write an automated regression test around it, so go do that in foo_controller_spec

                          Run test suite to ensure I didn’t break anything and that my new test is green

                          Add foo_controller and foo_controller_spec to staging area

                          Revert working copy (but not staged copy!) of foo_controller (but not it’s spec)

                          Run test suite again and ensure I have exactly one red test (the new regression test). If yes, commit the stage.

                          If no, debug spec against old controller until I understand why it’s not red, get it red, pull staged controller back to working area, make sure it’s green.

                          Yeah, I could probably simulate this by committing halfway through and then doing some bullshit with cherry-picks from older commits and in some cases reverting the top commit but, like, why? What would I gain from limiting myself to just this awkward commit dance as the only way of working? That’s just leaving me to cobble together a workflow that’s had a powerful abstraction taken away from it, just to satisfy some dogmatic “the commit is the only abstraction I’m willing to allow” instinct.

                          1. 4

                            Do you actually add your entire working directory most of the time?

                            Yes. And when I get a bug report, I tend to first reproduce the bug, then write a failing test, then fix the code.

                            Revert working copy (but not staged copy!) of foo_controller (but not it’s spec)

                            Sounds useful. How do you do that?

                            1. 7

                              Revert working copy (but not staged copy!) of foo_controller (but not it’s spec)

                              Sounds useful. How do you do that?

                              You can checkout a file into your working copy from any commit.

                              1. 6

                                Yes. And when I get a bug report, I tend to first reproduce the bug, then write a failing test, then fix the code.

                                Right, but that was just one example. Everything in your working copy should always be committed at all times? I’m almost never in that state. Either I’ve got other edits in progress that I intend to form into later commits, or I’ve got edits on disk that I never intend to commit but in files that should not be git ignored (because I still intend to merge upstream changes into them).

                                I always want to be intentionally forming every part of a commit, basically.

                                Sounds useful. How do you do that?

                                git add foo_controller <other files>; git restore -s HEAD foo_controller

                                and then

                                git restore foo_controller will copy the staged version back into the working set.

                            2. 1

                              TBH, I have no idea what “git add -p” does off hand (I use Magit), and I’ve never used staging like that.

                              I had a great example use of staging come up just yesterday. I’m working in a feature branch, and we’ve given QA a build to test what we have so far. They found a bug with views, and it was an easy fix (we didn’t copy attributes over when copying a view).

                              So I switched over to views.cpp and made the change. I built, tested that specific view change, and in Magit I staged that specific change in views.cpp. Then I commited, pushed it, and kicked off a pipeline build to give to QA.

                              I also use staging all the time if I refactor while working on new code or fixing bugs. Say I’m working on “foo()”, but while doing so I refactor “bar()” and “baz()”. With staging, I can isolate the changes to “bar()” and “baz()” in their own commits, which is handy for debugging later, giving the changes to other people without pulling in all of my changes, etc.

                              Overall, it’s trivial to ignore staging if you don’t want it, but it would be a lot of work to simulate it if it weren’t a feature.

                            3. 6

                              What’s wrong with the staging area? I use it all the time to break big changes into multiple commits and smaller changes.

                              I’m sure you do – that’s how it was meant to be used. But you might as well use commits as the staging area – it’s easy to commit and squash. This has the benefit that you can work with your whole commit stack at the same time. I don’t know what problem the staging area solves that isn’t better solved with commits. And yet, the mere existence of this unnecessary feature – this implicitly modified invisible state that comes and crashes your next commit – adds cognitive load: Commands like git mv, git rm and git checkout pollutes the state, then git diff hides it, and finally, git commit --amend accidentally invites it into the topmost commit.

                              The combo of being not useful and a constant stumbling block makes it bad.

                              1. 3

                                I don’t know what problem the staging area solves that isn’t better solved with commits.

                                If I’ve committed too much work in a single commit how would I use commits to split that commit into two commits?

                                1. 4

                                  Using e.g. hg split or jj split. The former has a text-based interface similar to git commit -p as well as a curses-based TUI. The latter lets you use e.g. Meld or vimdiff to edit the diff in a temporary directory and then rewrites the commit and all descendants when you’re done.

                                  1. 3

                                    That temporary directory sounds a lot like the index – a temporary place where changes to the working copy can be batched. Am I right to infer here that the benefit you find in having a second working copy in a temp directory because it works better with some other tools that expect to work files?

                                    1. 1

                                      The temporary directory is much more temporary than the index - it only exists while you split the commit. For example, if you’re splitting a commit that modifies 5 files, then the temporary directory will have only 2*5 files (for before and after). Does that clarify?

                                      The same solution for selecting part of the changes in a commit is used by jj amend -i (move into parent of specified commit, from working-copy commit by default), jj move -i --from <rev> --to <rev> (move changes between arbitrary commits) etc.

                                  2. 2

                                    I use git revise. Interactive revise is just like interactive rebase, except that it has is a cut subcommand. This can be used to split a commit by selecting and editing hunks like git commit -p.

                                    Before git-revise, I used to manually undo part of the commit, commit that, then revert it, and then sqash the undo-commit into the commit to be split. The revert-commit then contains the split-off changes.

                                  3. 3

                                    I don’t know, I find it useful. Maybe if git built in mercurials “place changes into commit that isn’t the most recent” amend thing then I might have an easier time doing things but just staging up relevant changes in a patch-based flow is pretty straightforward and helpful IMO

                                    I wonder if this would be as controversial if patching was the default

                                  4. 6

                                    What purpose does it serve that wouldn’t also be served by first-class rollback and an easier way of collapsing changesets on their way upstream? I find that most of the benefits of smaller changesets disappear when they don’t have commit messages, and when using the staging area for this you can only rollback one step without having to get into the hairy parts of git.

                                    1. 3

                                      The staging area is difficult to work with until you understand what’s happening under the hood. In most version control systems, an object under version control would be in one of a handful of states: either the object has been cataloged and stored in its current state, or it hasn’t. From a DWIM standpoint for a new git user, would catalog and store the object in its current state. With the stage, you can stage, and change, stage again, and change again. I’ve used this myself to logically group commits so I agree with you that it’s useful. But I do see how it breaks peoples DWIM view on how git works.

                                      Also, If I stage, and then change, is there a way to have git restore the file as I staged it if I haven’t committed?

                                      1. 7

                                        Also, If I stage, and then change, is there a way to have git restore the file as I staged it if I haven’t committed?

                                        Git restore .

                                        1. 3

                                          I’ve implemented git from scratch. I still find the staging area difficult to use effectively in practice.

                                        2. 1

                                          Try testing your staged changes atomically before you commit. You can’t.

                                          A better design would have been an easy way to unstage, similar to git stash but with range support.

                                          1. 5

                                            You mean git stash --keep-index?

                                            1. 3

                                              Interesting, that would solve the problem. I’m surprised I’ve not come across that before.

                                              In terms of “what’s wrong with the staging area”, what I was suggesting would work better is to have the whole thing work in reverse. So all untracked files are “staged” by default and you would explicitly un-stage anything you don’t want to commit. Firstly this works better for the 90% use-case, and compared to this workaround it’s a single step rather than 2 steps for the 10% case where you don’t want to commit all your changes yet.

                                              The fundamental problem with the staging area is that it’s an additional, hidden state that the final committed state has to pass through. But that means that your commits do not necessarily represent a state that the filesystem was previously in, which is supposed to be a fundamental guarantee. The fact that you have to explicitly stash anything to put the staging area into a knowable state is a bit of a hack. It solves a problem that shouldn’t exist.

                                              1. 2

                                                The way I was taught this, the way I’ve taught this to others, and the way it’s represented in at least some guis is not compatible.

                                                I mean, sure, you can have staged and unstaged changes in a file and need to figure it out for testing, or unstage parts, but mostly it’s edit -> stage -> commit -> push.

                                                That feels, to me and to newbies who barely know what version control is, like a logical additive flow. Tons of cases you stage everything and commit so it’s a very small operation.

                                                The biggest gripe may be devs who forget to add files in the proper commit, which makes bisect hard. Your case may solve that for sure, but I find it a special case of bad guis and sloppy devs who do that. Also at some point the fs layout gets fewer new files.

                                                1. 2

                                                  Except that in a completely linear flow the distinction between edit and stage serves no purpose. At best it creates an extra step for no reason and at worst it is confusing and/or dangerous to anyone who doesn’t fully understand the state their working copy is in. You can bypass the middle state with git add .; git commit and a lot of new developers do exactly that, but all that does is pretend the staging state doesn’t exist.

                                                  Staging would serve a purpose if it meant something similar to pushing a branch to CI before a merge, where you have isolated the branch state and can be assured that it has passed all required tests before it goes anywhere permanent. But the staging area actually does the opposite of that, by creating a hidden state that cannot be tested directly.

                                                  As you say, all it takes is one mistake and you end up with a bad commit that breaks bisect later. That’s not just a problem of developers being forgetful, it’s the bad design of the staging area that makes this likely to happen by default.

                                                  1. 1

                                                    I think I sort of agree but do not completely concur.

                                                    Glossing over the staging can be fine in some projects and dev sloppiness is IMO a bigger problem than an additive flow for clean commits.

                                                    These are societal per-project issues - what’s the practice or policy or mandate - and thus they could be upheld by anything, even using the undo buffer for clean commits like back in the day. Which isn’t to say you never gotta do trickery like that with Git, just that it’s a flow that feels natural and undo trickery less common.

                                                    Skimming the other comments, maybe jj is more like your suggestion, and I wouldn’t mind “a better Git”, but I can’t be bothered when eg. gitless iirc dropped the staging and would make clean commits feel like 2003.

                                            2. 2

                                              If git stash --keep-index doesn’t do what you want the you could help further the conversation by elaborating on what you want.

                                              1. 1
                                            3. 16

                                              The underlying data model however is pretty good. We can probably ditch the staging area,

                                              Absolutely not. The staging area was a godsend coming from Subversion – it’s my favorite part of git bar none.

                                              1. 4

                                                Everyone seem to suppose I would like to ditch the workflows enabled by the staging area. I really don’t. I’m quite sure there ways to keep those workflows without using a staging area. If there aren’t well… I can always admit I was wrong.

                                                1. 9

                                                  Well, what I prize being able to do is to build up a commit piecemeal out of some but not all of the changes in my working directory, in an incremental rather than all-in-one-go fashion (ie. I should be able to form the commit over time and I should be able to modify a file, move it’s state into the “pending commit” and continue to modify the file further without impacting the pending commit). It must be possible for any commit coming out of this workflow to both not contain everything in my working area, and to contain things no longer in my working area. It must be possible to diff my working area against the pending commit and against the last actual commit (separately), and to diff the pending commit against the last actual commit.

                                                  You could call it something else if you wanted but a rose by any other name etc. A “staging area” is a supremely natural metaphor for what I want to work with in my workflow, so replacing it hardly seems desirable to me.

                                                  1. 2

                                                    How about making the pending commit an actual commit? And then adding the porcelain necessary to treat it like a staging area? Stuff like git commit -p foo if you want to add changes piecemeal.

                                                    1. 11

                                                      No. That’s cool too and is what tools like git revise and git absorb enable, but making it an actual commit would have other drawbacks: it would imply it has a commit message and passes pre-commit hooks and things like that. The staging area is useful precisely for what it does now—help you build up the pieces necessary to make a commit. As such it implies you don’t have everything together to make a commit out of it. As soon as I do I commit, then if necessary --ammend, --edit, or git revise later. If you don’t make use of workflows that use staging then feel free to use tooling that bypasses it for you, but don’t try to take it away from the rest of us.

                                                      1. 9

                                                        pre-commit hooks

                                                        Oh, totally missed that one. Probably because I’ve never used it (instead i rely on CI or manually pushing a button). Still, that’s the strongest argument so far, and I have no good solution that doesn’t involve an actual staging area there. I guess it’s time to change my mind.

                                                        1. 2

                                                          I think the final word is not said. These tools could also run hooks. It may be that new hooks need to be defined.

                                                          Here is one feature request: run git hooks on new commit

                                                          1. 1

                                                            I think you missed the point, my argument is that the staging area is useful as a place to stage stuff before things like commit related hooks get run. I don’t want tools like git revise to run precommit hooks. When I use git revise the commit has already been made and presumably passed precommit phase.

                                                            1. 1

                                                              For the problem that git revise “bypasses” the commit hook when using it to split a commit, I meant the commit hook (not precommit hook).

                                                              I get that the staging area lets you assemble a commit before you can run the commit hook. But if this was possible to do statelessly (which would only be an improvement), you could do without it. And for other reasons, git would be so much better without this footgun:

                                                              Normally, you can look at git diff and commit what you see with git commit -a. But if the staging area is clobbered, which you might have forgot, you also have invisible state that sneaks in!

                                                              1.  

                                                                Normally, you can look at git diff and commit what you see with git commit -a.

                                                                Normally I do nothing of the kind. I might have used git commit -a a couple times in the last 5 years (and I make dozens to hundreds of commits per day). The stattefullness of the staging area is exactly what benefits my workflow and not the part I would be trying to eliminate. The majority of the time I stage things I’m working on from my editor one hunk at a time. The difference between my current buffer and the last git commit is highlighted and after I make some progress I start adding related hunks and shaping them into commits. I might fiddle around with a couple things in the current file, then when I like it stage up pieces into a couple different commits.

                                                                The most aggressive I’d get is occasionally (once a month?) coming up with a use for git commit -u.

                                                                A stateless version of staging that “lets you assemble a commit” sounds like an oxymoron to me. I have no idea what you think that would even look like, but a state that is neither the full contents of the current file system nor yet a commit is exactly what I want.

                                                          2. 1

                                                            Why not allow an empty commit message, and skip the commit hooks if a message hasn’t been set yet?

                                                            1. 1

                                                              Why deliberately make a mess of things? Why make a discreet concept of a “commit” into something else with multiple possible states? Why not just use staging like it is now? I see no benefit to jurry rigging more states on top of a working one. If the point is to simplify the tooling you won’t get there by overloading one clean concept with an indefinite state and contextual markers like “if commit message empty then this is not a real commit”.

                                                              1. 1

                                                                Empty commit message is how you abort a commit

                                                                1. 1

                                                                  With the current UI.

                                                                  When discussing changes, there’s the possibility of things changing.

                                                            2. 5

                                                              Again, what’s the benefit?

                                                              Sure, you could awkwardly simulate a staging area like this. The porcelain would have to juggle a whole bunch of shit to avoid breaking anytime you merge a bunch of changes after adding something to the fake “stage”, pull in 300 new commits, and then decide you want to unstage something, so the replacement of the dedicated abstraction seems likely to leak and introduce merge conflict resolution where you didn’t previously have to worry about it, but maybe with enough magic you could do it.

                                                              But what’s the point? To me it’s like saying that I could awkwardly simulate if, while and for with goto, or simulate basically everything with enough NANDs. You’re not wrong, but what’s in it for me? Why am I supposed to like this any better than having a variety of fit-for-purpose abstractions? It just feels like I’d be tying one hand behind my back so there can be one less abstraction, without explain why having N-1 abstractions is even more desirable than having N.

                                                              Seems like an “a foolish consistency is the hobgoblin of little minds” desire than anything beneficial, really.

                                                              1. 1

                                                                Again, what’s the benefit?

                                                                Simplicity of implementation. Implementing the staging area like a commit, or at least like a pointer to a tree object, would likely make the underlying data model simpler. I wonder why the staging area was implemented the way it is.

                                                                At the interface level however I’ve had to change my mind because of pre-commit hooks. When all you have is commits, and some tests are automatically launched every time you commit anything, it’s pretty hard to add stuff piecemeal.

                                                                1. 3

                                                                  Yes, simplicity of implementation and UI. https://github.com/martinvonz/jj (mentioned in the article) makes the working copy (not the staging area) an actual commit. That does make the implementation quite a lot simpler. You also get backups of the working copy that way.

                                                                  1. 1

                                                                    Simplicity of implementation.

                                                                    No offence but, why would I give a shit about this? git is a tool I use to enable me to get other work done, it’s not something I’m reimplementing. If “making the implementation simpler” means my day-to-day workflows get materially more unpleasant, the simplicity of the implementation can take a long walk off a short pier for all I care.

                                                                    It’s not just pre-commit hooks that get materially worse with this. “Staging” something would then have to have a commit message, I would effectively have to branch off of head before doing every single “staging” commit in order to be able to still merge another branch and then rebase it back on top of everything without fucking about in the reflog to move my now-burried-in-the-past stage commit forward, etc, etc. “It would make the implementation simpler” would be a really poor excuse for a user hostile change.

                                                                    1. 3

                                                                      If “making the implementation simpler” means my day-to-day workflows get materially more unpleasant, the simplicity of the implementation can take a long walk off a short pier for all I care.

                                                                      I agree. Users shouldn’t have to care about the implementation (except for minor effects like a simpler implementation resulting in fewer bugs). But I don’t understand why your workflows would be materially more unpleasant. I think they would actually be more pleasant. Mercurial users very rarely miss the staging area. I was a git developer (mostly working on git rebase) a long time ago, so I consider myself a (former) git power user. I never miss the staging area when I use Mercurial.

                                                                      “Staging” something would then have to have a commit message

                                                                      Why? I think the topic of this thread is about what can be done differently, so why would the new tool require a commit message? I agree that it’s useful if the tool lets you provide a message, but I don’t think it needs to be required.

                                                                      I would effectively have to branch off of head before doing every single “staging” commit in order to be able to still merge another branch and then rebase it back on top of everything without fucking about in the reflog to move my now-burried-in-the-past stage commit forward

                                                                      I don’t follow. Are you saying you’re currently doing the following?

                                                                      git add -p
                                                                      git merge <another branch>
                                                                      git rebase <another branch>
                                                                      

                                                                      I don’t see why the new tool would bury the staging commit in the past. That’s not what happens with Jujutsu/jj anyway. Since the working copy is just like any other commit there, you can simply merge the other branch with it and then rebase the whole stack onto the other branch after.

                                                                      I’ve tried to explain a bit about this at https://github.com/martinvonz/jj/blob/main/docs/git-comparison.md#the-index. Does that help clarify?

                                                                      1. 1

                                                                        Mercurial users very rarely miss the staging area.

                                                                        Well, I’m not them. As somebody who was forced to use Mercurial for a bit and hated every second of it, I missed the hell out of it, personally (and if memory serves, there was later at least one inevitably-nonstandard Mercurial plugin to paper over this weakness, so I don’t think I was the only person missing it).

                                                                        I’ve talked about my workflow elsewhere in this thread, I’m not really interested in rehashing it, but suffice to say I lean on the index for all kinds of things.

                                                                        Are you saying you’re currently doing the following? git add -p git merge

                                                                        I’m saying that any number of times I start putting together a commit by staging things on Friday afternoon, come back on Monday, pull in latest from main, and continue working on forming a commit.

                                                                        If I had to (manually, we’re discussing among other things the assertion that you could eliminate the stage because it’s pointless, and you could “just” commit whenever you want to stage and revert the commit whenever they want to unstage ) commit things on Friday, forget I’d done so on Monday, pull in 300 commits from main, and then whoops I want to revert a commit 301 commits back so now I get to back out the merge and etc etc, this is all just a giant pain in the ass to even type out.

                                                                        Does that help clarify?

                                                                        I’m honestly not interested in reading it, or in what “Jujutsu” does, as I’m really happy with git and totally uninterested in replacing it. All I was discussing in this thread with Loup-Vaillant was the usefulness of the stage as an abstraction and my disinterest in seeing it removed under an attitude of “well you could just manually make commits when you would want to stage things, instead”.

                                                                        1. 2

                                                                          I’m honestly not interested in reading it, or in what “Jujutsu” does

                                                                          Too bad, this link you’re refusing to read is highly relevant to this thread. Here’s a teaser:

                                                                          As a Git power-user, you may think that you need the power of the index to commit only part of the working copy. However, Jujutsu provides commands for more directly achieving most use cases you’re used to using Git’s index for.

                                                                          1. 0

                                                                            What “jujutsu” does under the hood has nothing whatsoever to do with this asinine claim of yours, which is the scenario I was objecting to: https://lobste.rs/s/yi97jn/is_it_time_look_past_git#c_k6w2ut

                                                                            At this point I’ve had enough of you showing up in my inbox with these poorly informed, bad faith responses. Enough.

                                                                            1. 1

                                                                              I was claiming that the workflows we have with the staging area, we could achieve without. And Jujutsu here has ways to do exactly that. It has everything to do with the scenario you were objecting to.

                                                                              Also, this page (and what I cited specifically) is not about what jujutsu does under the hood, it’s about its user interface.

                                                                              1. 1

                                                                                I’ve made it clear that I’m tired of interacting with you. Enough already.

                                                                      2. 1

                                                                        No offence but, why would I give a shit about [simplicity of implementation]?

                                                                        It’s because people don’t give a shit that we have bloated (and often slow) software.

                                                                        1. 0

                                                                          And it’s because of developers with their heads stuck so far up their asses that they prioritize their implementation simplicity over the user experience that so much software is actively user-hostile.

                                                                          Let’s end this little interaction here, shall we.

                                                          3. 15

                                                            Sublime Merge is the ideal git client for me. It doesn’t pretend it’s not git like all other GUI clients I’ve used so you don’t have to learn something new and you don’t unlearn git. It uses simple git commands and shows them to you. Most of git’s day-to-day problems go away if you can just see what you’re doing (including what you’ve mentioned).

                                                            CLI doesn’t cut it for projects of today’s size. A new git won’t fix that. The state of a repository doesn’t fit in a terminal and it doesn’t fit in my brain. Sublime Merge shows it just right.

                                                            1. 5

                                                              I like GitUp for the same reasons. Just let me see what I’m doing… and Undo! Since it’s free, it’s easy to get coworkers to try it.

                                                              1. 4

                                                                I didn’t know about GitUp but I have become a big fan of gitui as of late.

                                                                1. 2

                                                                  I’ll check that out, thank you!

                                                              2. 2

                                                                I use Fork for the same purpose and the staging area has never been a problem since it is visible and diffable at any time, and that’s how you compose your commits.

                                                              3. 6

                                                                See Game of Trees for an alternative to the git tool that interacts with normal git repositories.

                                                                Have to agree with others about the value of the staging area though! It’s the One Big Thing I missed while using Mercurial.

                                                                1. 5

                                                                  Well, on the one hand people could long for a better way to store the conflict resolutions to reuse them better on future merges.

                                                                  On the other hand, of all approaches to DAG-of-commits, Git’s model is plain worse than the older/parallel ones. Git is basically intended to lose valuable information about intent. The original target branch of the commit often tells as much as the commit message… but it is only available in reflog… auto-GCed and impossible to sync.

                                                                  1. 10

                                                                    Half of my branches are called werwerdsdffsd. I absolutely don’t want them permanently burned in the history. These scars from work-in-progress annoyed me in Mercurial.

                                                                    1. 9

                                                                      Honestly I have completely the opposite feeling. Back in the days before git crushed the world, I used Mercurial quite a lot and I liked that Mercurial had both the ephemeral “throw away after use” model (bookmarks) and the permanent-part-of-your-repository-history model (branches). They serve different purposes, and both are useful and important to have. Git only has one and mostly likes to pretend that the other is awful and horrible and nobody should ever want it, but any long-lived project is going to end up with major refactoring or rewrites or big integrations that they’ll want to keep some kind of “here’s how we did it” record to easily point to, and that’s precisely where the heavyweight branch shines.

                                                                      And apparently I wrote this same argument in more detail around 12 years ago.

                                                                      1. 1

                                                                        ffs_please_stop_refactoring_and_review_this_pr8

                                                                      2. 2

                                                                        This is a very good point. It would be interesting to tag and attach information to a group of related commits. I’m curious of the linux kernel workflows. If everything is an emailed patch, maybe features are done one commit at a time.

                                                                        1. 2

                                                                          If you go further, there are many directions to extend what you can store and query in the repository! And of course they are useful. But even the data Git forces you to have (unlike, by the way, many other DVCSes where if you do not want a meaningful name you can just have multiple heads in parallel inside a branch) could be used better.

                                                                        2. 2

                                                                          I can’t imagine a scenario where the original branch point of a feature would ever matter, but I am constantly sifting through untidy merge histories that obscure the intent.

                                                                          Tending to your commit history with intentionality communicates to reviewers what is important, and removes what isn’t.

                                                                          1. 1

                                                                            It is not about the point a branch started from. It is about which of the recurring branches the commit was in. Was it in quick-fix-train branch or in update-major-dependency-X branch?

                                                                            1. 2

                                                                              The reason why this isn’t common is because of GitHub more than Git. They don’t provide a way to use merge commits that isn’t a nightmare.

                                                                              When I was release managing by hand, my preferred approach was rebasing the branch off HEAD but retaining the merge commit, so that the branch commits were visually grouped together and the branch name was retained in the history. Git can do this easily.

                                                                        3. 5

                                                                          I never understood the hate for Git’s CLI. You can learn 99% of what you need to know on a daily basis in a few hours. That’s not a bad time investment for a pivotal tool that you use multiple times every day. I don’t expect a daily driver tool to be intuitive, I expect it to be rock-solid, predictable, and powerful.

                                                                          1. 9

                                                                            This is a false dichotomy: it can be both (as Mercurial is). Moreover, while it’s true that you can learn the basics to get by with in a few hours, it causes constant low-level mental overhead to remember how different commands interact, what the flag is in this command vs. that command, etc.—and never mind that the man pages are all written for people thinking in terms of the internals, instead of for general users. (That this is a common failing of man pages does not make it any less a problem for git!)

                                                                            One way of saying it: git has effectively zero progressive disclosure of complexity. That makes it a continual source of paper cuts at minimum unless you’ve managed to actually fully internalize not only a correct mental model for it but in many cases the actual implementation mechanics on which it works.

                                                                            1. 3

                                                                              Its manpages are worthy of a parody: https://git-man-page-generator.lokaltog.net

                                                                            2. 2

                                                                              Its predecessors CVS and svn had much more intuitive commands (even if they were was clumsy to use in other ways). DARCS has been mentioned many times as being much more easy to use as well. People migrating from those tools really had a hard time, especially because git changed the meanings of some commands, like checkout.

                                                                              Then there were some other tools that came up around the same time or shortly after git but didn’t get the popularity of git like hg and bzr, which were much more pleasant to use as well.

                                                                              1. 2

                                                                                I think the issues people have are less about the CLI itself and more about how it interfaces with the (for some developers) complex and hard to understand concepts at hand.

                                                                                Take rebase for example. Once you grok what it is, it’s easy, but trying to explain the concept of replaying commits on top of others to someone used to old school tools like CVS or Subversion can be a challenge, especially when they REALLY DEEPLY don’t care and see this as an impediment to getting their work done.

                                                                                I’m a former release engineer, so I see the value in the magic Git brings to the table, but it can be a harder sell for some :)

                                                                              2. 5

                                                                                The interface is pretty bad.

                                                                                I would argue that this is one of the main reasons for git’s success. The CLI is so bad that people were motivated to look for tools to avoid using it. Some of them were motivated to write tools to avoid using it. There’s a much richer set of local GUI and web tools than I’ve seen for any other revision control system and this was true even when git was still quite new.

                                                                                I never used a GUI with CVS or Subversion, but I wanted to as soon as I started touching the git command line. I wanted features like PRs and web-based code review, because I didn’t want to merge things locally. I’ve subsequently learned a lot about how to use the git CLI and tend to use it for a lot of tasks. If it had been as good as, say, Mercurial’s from the start then I never would have adopted things like gitx / gitg and GitHub and it’s those things that make the git ecosystem a pleasant place to be.

                                                                                1. 4

                                                                                  The interface of Git and its underlying data models are two very different things, that are best treated separately.

                                                                                  Yes a thousand times this! :) Git’s data model has been a quantum leap for people who need to manage source code at scale. Speaking as a former release engineer, I used to be the poor schmoe who used to have to conduct Merge Day, where a branch gets merged back to main.

                                                                                  There was exactly one thing you could always guarantee about merge day: There Will Be Blood.

                                                                                  So let’s talk about looking past git’s god awful interface, but keep the amazing nubbins intact and doing the nearly miraculous work they do so well :)

                                                                                  And I don’t just mean throwing a GUI on top either. Let’s rethink the platonic ideal for how developers would want their workflow to look in 2022. Focus on the common case. Let the ascetics floating on a cloud of pure intellect script their perfect custom solutions, but make life better for the “cold dark matter” developers which are legion.

                                                                                  1. 2

                                                                                    I would say that you simultaneously give credit where it is not due (there were multiple DVCSes before Git, and approximately every one had a better data model, and then there are things that Subversion still has better than everyone else, somehow), and ignore the part that actually made your life easier — the efforts of pushing Git down people’s throat, done by Linus Torvalds, spending orders of magnitude more of his time on this than on getting things right beyond basic workability in Git.

                                                                                    1. 1

                                                                                      Not a DVCS expert here, so would you please consider enlightening me? Which earlier DVCS were forgotten?

                                                                                      My impressions of Mercurial and Bazaar are that they were SL-O-O-W, but they’re just anecdotal impressions.

                                                                                      1. 3

                                                                                        Well, Bazaar is technically earlies. Monotone is significantly earlier. Monotone has quite interesting and nicely decoupled data model where the commit DAG is just one thing; changelog, author — and branches get the same treatment — are not parts of a commit, but separately stored claims about a commit, and this claim system is extensible and queriable. And of course Git was about Linus Torvalds speedrunning implementation of the parts of BitKeeper he really really needed.

                                                                                        It might be that in the old days running on Python limited speed of both Mercurial and Bazaar. Rumour has it that the Monotone version Torvalds found too slow was indeed a performance regression (they had one particularly slow release at around that time; Monotone is not in Python)

                                                                                        Note that one part of things making Git fast is that enables some optimisations that systems like Monotone make optional (it is quite optimistic about how quickly you can decide that the file must not have been modified, for example). Another is that it was originally only intended to be FS-safe on ext3… and then everyone forgot to care, so now it is quite likely to break the repository in case of unclean shutdown mid-operation. Yes, I have damaged repositories that way to a state where I could not find advice on how to avoid re-cloning to get even partially working repository.

                                                                                        As of Subversion, it has narrow checkouts which are a great feature, and DVCSes could also have them, but I don’t think anyone properly has them. You kind of can hack something with remote-automate in Monotone, but probably flakily.

                                                                                  2. 4

                                                                                    Let the data model pretend there’s a blob for each version of that huge file, even though in fact the software is automatically compressing & decompressing things under the hood.

                                                                                    Ironically, that’s part of the performance problem – compressing the packfiles tends to be where things hurt.

                                                                                    Still, this is definitely a solvable problem.

                                                                                    1. 2

                                                                                      I used to love DARCS, but I think patch theory was probably the wrong choice.

                                                                                      I have created and maintains official test suite for pijul, i am the happiest user ever.

                                                                                      1. 2

                                                                                        Hmm, knowing you I’m sure you’ve tested it to death.

                                                                                        I guess they got rid of the exponential conflict resolution that plagued DARCS? If so perhaps I should give patch theory another go. Git ended up winning the war before I got around to actually study patch theory, maybe it is sounder than I thought.

                                                                                        1. 1

                                                                                          Pijul is a completely different thing than Darcs, the current state of a repository in Pijul is actually a special instance of a CRDT, which is exactly what you want for a version control system.

                                                                                          Git is also a CRDT, but HEAD isn’t (unlike in Pijul), the CRDT in Git is the entire history, and that is not a very useful property.

                                                                                        2. 1

                                                                                          Best test suite ever. Thanks again, and again, and again for that. It also helped debug Sanakirja, a database engine used as the foundation of Pijul, but usable in other contexts.

                                                                                        3. 2

                                                                                          There are git-compatible alternatives that keep the underlying model and change the interface. The most prominent of these is probably gitless.

                                                                                          1. 1

                                                                                            I’ve been using git entirely via UI because of that. Much better overview, much more intuitive, less unwanted side effects.

                                                                                            1. 1

                                                                                              You can’t describe Git without discussing rebase and merge: these are the two most common operations in Git, yet they don’t satisfy any interesting mathematical property such as associativity or symmetry:

                                                                                              • Associativity is when you want to merge your commits one by one from a remote branch. This should intuitively be the same as merging the remote HEAD, but Git manages to make it different sometimes. When that happens, your lines can be shuffled around more or less randomly.

                                                                                              • Symmetry means that merging A and B is the same as merging B and A. Two coauthors doing the same conflictless merge might end up with different results. This is one of the main benefits of GitHub: merges are never done concurrently when you use a central server.

                                                                                              1. 1

                                                                                                Well, at least this is not the fault of the data model: if you have all the snapshots, you can deduce all the patches. It’s the operations themselves that need fixing.

                                                                                                1. 1

                                                                                                  My point is that this is a common misconception: no datastructure is ever relevant without considering the common operations we want to run on it.

                                                                                                  For Git repos, you can deduce all the patches indeed, but merge and rebase can’t be fixed while keeping a reasonable performance, since the merge problem Git tries to solve is the wrong one (“merge the HEADs, knowing their youngest common ancestor”). That problem cannot have enough information to satisfy basic intuitive properties.

                                                                                                  The only way to fix it is to fetch the entire sequence of commits from the common ancestor. This is certainly doable in Git, but merges become O(n) in time complexity, where n is the size of history.

                                                                                                  The good news is, this is possible. The price to pay is a slightly more complex datastructure, slightly harder to implement (but manageable). Obviously, the downside is that it can’t be consistent with Git, since we need more information. On the bright side, it’s been implemented: https://pijul.org

                                                                                                  1. 1

                                                                                                    no datastructure is ever relevant without considering the common operations we want to run on it.

                                                                                                    Agreed. Now, how often do we actually merge stuff, and how far is the common ancestor in practice?

                                                                                                    My understanding of the usage of version control is that merging two big branches (with an old common ancestor) is rare. Far more often we merge (or rebase) work units with just a couple commits. Even more often than that we have one commit that’s late, so we just pull in the latest change then merge or rebase that one commit. And there are the checkout operations, which in some cases can occur most frequently. While a patch model would no doubt facilitate merges, it may not be worth the cost of making other, arguably more frequent operations, slower.

                                                                                                    (Of course, my argument is moot until we actually measure. But remember that Git won in no small part because of its performance.)

                                                                                                    1. 2

                                                                                                      I agree with all that, except that:

                                                                                                      • the only proper modelling of conflicts, merges and rebases/cherry-picking I know of (Pijul) can’t rely on common ancestors only, because rebases can make some future merges more complex than a simple 3-way merge problem.

                                                                                                      • I know many engineers are fascinated by Git’s speed, but the algorithm running on the CPU is almost never the bottleneck: the operator’s brain is usually much slower than the CPU in any modern version control system (even Darcs has fixed its exponential merge). Conflicts do happen, so do cherry-picks and rebases. They aren’t rare in large projects, and can be extremely confusing without proper tools. Making these algorithms fast is IMHO much more important from a cost perspective than gaining 10% on a operation already taking less than 0.1 second. I won’t deny the facts though: if Pijul isn’t used more in industry, it could be partly because that opinion isn’t widely shared.

                                                                                                      • some common algorithmic operations in Git are slower than in Pijul (pijul credit is much faster than git blame on large instances), and most operations are comparable in speed. One thing where Git is faster is browsing old history: the datastructures are ready in Pijul, but I haven’t implemented the operations yet (I promised I would do that as soon as this is needed by a real project).

                                                                                            1.  

                                                                                              This sounds like a great initiative — all except the unfortunate choice of names! In spite of the good intentions of many and having done much good along the way, at this point in history PGP is an absolute fiasco. I can only hope the boondoggle of legacy protocols and differing implementations in its namesake does not forebode anything for this project.

                                                                                              1. 1
                                                                                                1. 1

                                                                                                  I really hope to see this get upstreamed someday. I currently depend a lot on remake, but sandboxing would be a great tool to have too. When all the fun tools are their own forks it makes it really hard to use both feature sets!

                                                                                                  1. 2

                                                                                                    TIL about .EXTRA_PREREQS. I mean landlock make sounds great too and I’ll play around with it, but I can put .EXTRA_PREQS into production and start stripping out $(filter-out ...) clutter in Makefiles today!

                                                                                                    1. 6

                                                                                                      This article conflates ‘alignment’, ‘formatting’, and ‘indentation’.

                                                                                                      Without covering the differences it doesn’t bring an light to the conversation. Yes editorconfig is cool and I recommend using it too, but since the article doesn’t even ask the right questions it cannot be trusted to have the only true answer.

                                                                                                      1. 4

                                                                                                        Imagine “tabs only” for lisp:

                                                                                                        ((foo)
                                                                                                                (bar))
                                                                                                        

                                                                                                        😭

                                                                                                        1. 3

                                                                                                          You could set your tab size to 1. I’m not saying you should, but you could. And with the EditorConfig in place per the article that would stick across many editors and even code hosting sites.

                                                                                                          1. 1

                                                                                                            If you don’t like how big your tabs are, you are free to make them smaller. That’s kind of the whole point

                                                                                                            1. 1

                                                                                                              Lisp alignment offsets are variable, unlike traditional indentation. So no single fixed number can work.

                                                                                                              You’d need something like Graavgard’s elastic tab stops.

                                                                                                              1. 0

                                                                                                                Or write lisp with sensible indentation… Most lisp uses whitespace as obfuscation I find…

                                                                                                        1. 4

                                                                                                          If you really want durable storage for a vault of important files, check out git annex. It handles storing large files in Git repositories in a long term manageable way—keeping distributed records of other checkouts and what checkouts have what files, maintaining at least N copies of files on other checkouts if you want lighten up a filesystem by deleting some files from a checkout, etc. It also handles faulty data quite gracefully and helps you copy in corrupted blobs from whichever other repositories have copies.

                                                                                                          1. 17

                                                                                                            I know everyone loves to write stuff in rust, but why not use unix tools you already have?

                                                                                                            find . -type f | xargs -n 1 -P 4 shasum -a 512 > files

                                                                                                            some time later

                                                                                                            find . -type f | xargs -n 1 -P 4 shasum -a 512 > files2

                                                                                                            and finally you can use diff to find differences. Probably best to throw in a sort after the find for better diffing.

                                                                                                            It is all already there, right on your system.

                                                                                                            1. 13

                                                                                                              You need a sort, then you need a diff that doesn’t show additions only deletions and changes, then you need the file2→file shuffling at the end, and pretty soon you have yourself a chunky script—and you haven’t even gotten to pretty-printing a progress bar. I’m a shell scripting junkie but this comment is not fair to what the tool in this post is actually doing.

                                                                                                              1. 3

                                                                                                                Okay, this was a 5 minute hack. I can spend 10 more minutes on it and it does everything the tool does (minus the porgress bar). The point is that the unix philosophy tells us that you should combine the tools you have to build higher level tools and this is perfectly doable in no time for the problem at hand.

                                                                                                                1. 18

                                                                                                                  OK, but maybe one can interpret this as the “tools you have” being Rust packages, and you combine them in Rust? Then you get the benefit of using a modern language with data types and stuff, instead of a gnarly shell with WTF syntax* whose only data type is byte-streams.

                                                                                                                  * I know there are shell enthusiasts here, but really, if it didn’t exist and anyone announced it as a new programming language they’d be laughed out of town IMO.

                                                                                                                2. 2

                                                                                                                  With Relational pipes:

                                                                                                                  find -type f -print0 | relpipe-in-filesystem --file path --streamlet hash --relation files_1 > ../files_1.rp
                                                                                                                  # do some changes
                                                                                                                  find -type f -print0 | relpipe-in-filesystem --file path --streamlet hash --relation files_2 > ../files_2.rp
                                                                                                                  cat ../files_*.rp | relpipe-tr-sql --relation 'diff' "SELECT f1.path, f1.sha256 AS old_hash, f2.sha256 AS new_hash FROM files_1 AS f1 LEFT JOIN files_2 AS f2 ON (f1.path = f2.path) WHERE f1.sha256 <> f2.sha256 OR f2.path IS NULL" | relpipe-out-tabular
                                                                                                                  

                                                                                                                  And get result like this (I removed bash-completion.sh, modified CLIParser.h and added a new file that is not listed):

                                                                                                                  diff:
                                                                                                                   ╭──────────────────────┬──────────────────────────────────────────────────────────────────┬──────────────────────────────────────────────────────────────────╮
                                                                                                                   │ path        (string) │ old_hash                                                (string) │ new_hash                                                (string) │
                                                                                                                   ├──────────────────────┼──────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────┤
                                                                                                                   │ ./bash-completion.sh │ 3f8c20eb917302c01a029711e3868665185c314d4a8d32cc2dfe1091521402c8 │                                                                  │
                                                                                                                   │ ./src/CLIParser.h    │ ba75414ced3163ce2b4a31f070a601d27258750806843bb18464e5efb5bc71fd │ 3f64b56674587f3b4130d531d46b547a44c18523abf0c4a3c09696277a4de6f0 │
                                                                                                                   ╰──────────────────────┴──────────────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────╯
                                                                                                                  Record count: 2
                                                                                                                  

                                                                                                                  I use this to catalogize some removable/remote media and search them offline.

                                                                                                                  1. 1

                                                                                                                    git diff --no-index --diff-filter=MD. I use git diff --no-index quite a lot (as an alias) as I like it over the plain diff command.

                                                                                                                    1. 1

                                                                                                                      I love myself a good progress bar ! :D

                                                                                                                    2. 11

                                                                                                                      I know everyone loves to write stuff in rust, but why not use unix tools you already have?

                                                                                                                      For me personally it was few different things.

                                                                                                                      • I like Rust, we don’t use it at work much so trying to not forget how to use it by writing small programs like those.
                                                                                                                      • speed - the computations are heavily parallelised, which was easier for me to write in Rust than using e.g. gnu parallel
                                                                                                                      • cannot stress enough how I love progress bars, and writing a decent one in a shell scripting language exceeds my allocated brain points :D
                                                                                                                      1. 7

                                                                                                                        find piped to xargs is a great example of something I just never use anymore, now that I have a Rewrite It In Rust™ version (fd). Nice and simple:

                                                                                                                            fd -t f -x shasum -a 512
                                                                                                                        

                                                                                                                        Let’s see how it compares to your version . . . Whoops! Your version doesn’t work at all! It breaks with lots of perfectly valid filenames. To fix that, I have to add -0 and -print0, so it’s even more awkward to use, and it’s still slower than fd.

                                                                                                                        I don’t use too many of the Rewrite It In Rust™ tools myself, but fd and rg are really nice.

                                                                                                                        As an aside, you should also probably get out of the habit of running find . and redirecting it to a file in the current directory, since find is likely feeding its output back into itself. When I run your find, I end up with a checksum for files which isn’t right anymore by the time find finishes running. It’s probably not what you wanted.

                                                                                                                        1. 1

                                                                                                                          Yeah. I also avoid xargs, and use find -exec instead. I definitely like the single character flags in fd, though.

                                                                                                                          1. 1

                                                                                                                            I think the difference is find -exec alone isn’t parallel, you have to add something like xargs to get parallelism. Whereas as fd -x is parallel by itself.

                                                                                                                        2. 4

                                                                                                                          Because these tools straight up don’t work properly, unless you heavily invest in learning them. Your solution, for example, breaks if there are spaces in filenames.

                                                                                                                          $ echo 1 > '1 2'
                                                                                                                          $ echo 3 > '3 4'
                                                                                                                          $ find . -type f | xargs -n 1 -P 4 shasum -a 512 > files
                                                                                                                          shasum: ./1: No such file or directory
                                                                                                                          shasum: 2: No such file or directory
                                                                                                                          shasum: ./3: No such file or directory
                                                                                                                          shasum: 4: No such file or directory
                                                                                                                          

                                                                                                                          This one will work:

                                                                                                                          find . -type f -print0 | xargs -0 -n 1 -P 4 shasum -a 512
                                                                                                                          
                                                                                                                          1. 1

                                                                                                                            It’s definitely faster to write this down in bash than.. Java. But I’d definitely be scared to not mess up quotation, special file names and what else shellcheck tries to catch. And then there is the issue remembering what this actually did in some weeks.

                                                                                                                            1. 1

                                                                                                                              I’d suggest aide, sort (case-insensitive), then comm.

                                                                                                                            1. 5
                                                                                                                              1. Copilot has a vim plugin? Wow where did that come from — it is going to be a pain to setup and unlikely to work well.
                                                                                                                              2. Hey that was actually pretty painless and goes vrrrum! When did GitHub get decent VIM coders on board for their closed source projects?
                                                                                                                              3. Oh it’s written by tpope, of course it works.

                                                                                                                              Bonus: Tim Pope works for Microsoft‽

                                                                                                                              1. 2

                                                                                                                                Has anybody benchmarked this against other options for similar results such as CSVKit’s csvsql or the Zed project’s zq query tool? Less SQL like options like the AWK variant that has direct CSV support and duckdb or others would be interesting to see benchmarked too.

                                                                                                                                1. 1

                                                                                                                                  There are links to benchmarks in this post. :)

                                                                                                                                  1. 1

                                                                                                                                    Only 1 of the 4 I listed is benchmarked or even mentioned in any of the links I see.

                                                                                                                                1. 2

                                                                                                                                  I couldn’t find any documentation for the remote access. VS Code’s depends on running an entire copy of the JavaScript engine and any binaries that plugins use on the remote side, which makes it hard to support on other platforms. Is this a more structured approach?

                                                                                                                                  1. 2

                                                                                                                                    I believe the situation is similar for lapce, but much simplified because the editor itself is a single binary with everything baked it, so the remote dependency chain should be a bit lighter/less convoluted.

                                                                                                                                    By the way I don’ think this is a design flaw. There are such things as remote file systems you can use with any editor to work on remote files, the point of a remote editor is that it runs the editor remotely. These two things are distinctly different and it makes sense you need the dependencies/plugins remotely for a remote editor instance, that’s the whole point. Also it means you don’t need them locally, which is also part of the point.

                                                                                                                                  1. 7

                                                                                                                                    Flagged; no code, no source, just a signup page. Lobsters is not your advertising channel.

                                                                                                                                    1. 4

                                                                                                                                      I’m not advertising anything, I have no affiliation, only interest—there is lots to discuss here as far as concepts go and with the recent Atom sunset announcement and Lapce getting discussed both here on lobste.rs this seemed an interesting discovery and something folks might want to watch.

                                                                                                                                      1. 3

                                                                                                                                        Lapce is OSS; this is vaporware.

                                                                                                                                        1. 1

                                                                                                                                          A fair point—and I’m sure a large part of why this post has +8 -6; it is tantalizing but not yet here.

                                                                                                                                          I’ve been digging in to lapce and there is a lot of promise, but it also still a bit squishy still. No plugins, no extensibility, little configurability, etc.

                                                                                                                                          I also don’t know if Zed materializes what the license model will be. The dev talk video said they planned on open sourcing the GUI toolkit, but I haven’t spotted any hints on whether the editor itself will be FOSS.

                                                                                                                                    1. 1

                                                                                                                                      Flagging as “already posted”

                                                                                                                                      Suggest folding this into rkn9s6: https://lobste.rs/s/rkn9s6/introducing_zed_lightning_fast

                                                                                                                                      1. 1

                                                                                                                                        Folding is fine with me, but the other link is getting spam flags because it “doesn’t have code”. The announcement blog post and related site stuff is interesting but light on detail—most of the public information seems to be in this video really so maybe better to fold this direction.

                                                                                                                                        1. 2

                                                                                                                                          Fair enough. I prefer text-based submissions as the “primary” when merging articles, but the decision is for @pushcx to make.

                                                                                                                                      1. 2

                                                                                                                                        I think one of the stand-out features of Atom was Teletype. Apparently some of the devs that worked on Atom and Teletype are making another effort at a better collaborative editing experience, see zed.dev (on lobste.rs).

                                                                                                                                        1. 1

                                                                                                                                          Vscode has that these days with “liveshare”, we have encountered a few bugs (particularly with terminals, and undo), but it generally works well enough.

                                                                                                                                          1. 1

                                                                                                                                            Any way to use “liveshare” to pair-program across editors, e.g. from VSCode to NeoVIM per this question? I’ve been looking for a way for me to stay in NeoVIM and let other folks follow along and chime in via a GUI editor for a while. So far the quest has been elusive.

                                                                                                                                            1. 1

                                                                                                                                              Strictly speaking I think the answer is yes, because it supports both VSCode and actual Visual Studio (I haven’t tried it with Visual Studio, but the internet tells me it works even cross editor).

                                                                                                                                              But I believe the entire extension is proprietary and I don’t think there is a way for anyone but microsoft to add support for another editor. I suspect it would be a non-trivial undertaking as well, because presumably they need to use some standardized CRDT for representing the buffer behind the scenes. The cursor position and text selection presumably also both need to use that CRDT (both because it shows them to the other users, and because other uses editing things need to not screw them up).

                                                                                                                                              One of the people (or maybe both? Not sure about the other) who I’ve been using this with uses vim keybindings in VSCode, and that works as well as vim keybindings in VSCode ever work. I’m sure that’s not the solution you are looking for, but it might be an acceptable workaround?

                                                                                                                                        1. 22

                                                                                                                                          Welp, we all saw this coming. Microsoft already has two editors (VS and VS Code), keeping a third around was unsustainable.

                                                                                                                                          (Though I am curious what are the reasons for continuing to use Atom instead of VS Code. By all accounts, as well as in my own experience, it’s far slower and clunkier than its counterparts.)

                                                                                                                                          EDIT: the founder of Atom, nathansobo, commented on the related HN and r/programming discussions that they are working on a new editor focused on speed and real-time collaboration, Zed.

                                                                                                                                          1. 18

                                                                                                                                            (Though I am curious what are the reasons for continuing to use Atom instead of VS Code. By all accounts, as well as in my own experience, it’s far slower and clunkier than its counterparts.)

                                                                                                                                            It’s slower, but depending on what features you heavily rely on, I’d honestly call VSCode the clunkier of the 2. For something I’m in and out of dozens of times a day, the Project-wide Find (and Find-and-Replace) experience in VSCode is terrible compared to Atom’s, and hasn’t seen significant improvement in years. There’s a couple of different “make VSCode’s search work like Atom/Sublime” plugins out there, but they’re mostly broken in my experience.

                                                                                                                                            Tons of other things are similar – the settings UI is a lot more pleasant in Atom than the weird way it works in VSCode. Atom just generally traded off a focus on speed in favor of a much more polished experience, generally.

                                                                                                                                            All that said, I saw the writing on the wall and went back to Sublime last year. It’s not as polished as Atom, but less of a jankfest than VSCode, and its search interface was what Atom’s was based on anyways. Bonus points for not setting my battery on fire by running an entire webbrowser just to draw text on a screen.

                                                                                                                                            1. 9

                                                                                                                                              Interesting to read this point of view. But a bit baffling, I should say. I used atom a little bit back in the day, but found it too hip and it was very sluggish, compared to gedit, grant, scribes, sublime, etc.

                                                                                                                                              When visual studio code came, it felt snappier, launched quicker, and personally I found it to be more pleasant to use. The UI was more focus on being functional than visually slick. Git integration was done just the way I like it. As well as other rminor but I portant details such as mru file switching. Built in terminal. Split screen.

                                                                                                                                              This is very much an opinion. But I find VSCode to be just a well designed and executed product. With pragmatism taking the central role. Kind of Microsoft showing that it is still capable of releasing useful software.

                                                                                                                                              1. 8

                                                                                                                                                How Microsoft manages the feature requests and bugs on the public vscode issue tracker never fails to impress me. There is a ton of professionalism shown towards a free and open source product. Product owners clearly communicate status and engage feedback.

                                                                                                                                                1. 5

                                                                                                                                                  When visual studio code came, it felt snappier, launched quicker, and personally I found it to be more pleasant to use. The UI was more focus on being functional than visually slick.

                                                                                                                                                  Very much different tastes. I booted it up after posting this, and still find it incredibly grating to use. They’ve screwed up the page “weight” or scrolling speed (on the Mac, at least), so that a sweep of my fingers on the trackpad when I’m scrolling a file is much, much faster than it is in any other app on the system (whereas scrolling in Sublime feels like scrolling in Safari feels like scrolling in the Terminal feels like, etc). It gives the whole thing a floaty, jittery feeling and screws up my years of scrolling muscle memory (cynically, I half wonder if they use a faster-than-system-scroll-speed to perpetuate the whole “VSCode is fast” marketing, because the rest of it, from the app startup to doing a search, is noticeably laggy)

                                                                                                                                                  Like the preference pane’s spastic sidebar flying open and closed by itself as you scroll down the unbelievably long single preference page (which the floatyness of the scrolling in general exacerbates) it just behaves like nothing else on the system, and that makes the whole experience grating and weird compared to everything else. It’s like nails on a chalkboard, just a constant low-level stream of annoyances, to me.

                                                                                                                                                  But if it’s your cup of tea, all power to you. Editor monocultures are the last thing we need.

                                                                                                                                                  1. 6

                                                                                                                                                    The thing I dislike the most about VSCode vs other editors is that it uses the Microsoft method of text selection. So if you hover over a character and click + drag the cursor, it won’t highlight the character that you are hovering over. Whereas in a macOS text field/editor it does.

                                                                                                                                                    So in VSCode when I am forced to use it, I tend to miss-select a bunch of text all the time making it an extremely frustrating experience.

                                                                                                                                                    The same issue exists in other interfaces that try to emulate some sort of text selection, the AWS console for example has a way to use AWS SSM to connect to a remote system and get a terminal in your browser. It has the same selection behavior.

                                                                                                                                                    When applications don’t match the behavior of the OS they are running on it becomes an incredibly jarring experience.

                                                                                                                                                    1. 1

                                                                                                                                                      So if you hover over a character and click + drag the cursor, it won’t highlight the character that you are hovering over. Whereas in a macOS text field/editor it does.

                                                                                                                                                      I agree that inconsistency is frustrating, but I cannot reproduce the inconsistency you describe in macOS 10.14 Mojave. Whether I’m using VS Code, TextEdit, or Finder, when I hover over a character, then click and drag, the selection starts from whichever side of the character the mouse cursor was closest to. I never see, for example, the selection start at the right side of an ‘O’ if I position my cursor on the left side of the ‘O’ before dragging.

                                                                                                                                                    2. 2

                                                                                                                                                      But if it’s your cup of tea, all power to you. Editor monocultures are the last thing we need.

                                                                                                                                                      I agree 100%. I’m a long time vim user, but have switched to using both vscode and emacs. I interchange between the two, and am interested in newer projects too such as helix. All monocultures are bad and have awful side effects. That said, I think editors are personal enough that vscode will never fully dominate.

                                                                                                                                                      I keep coming back to vscode for multiple reasons including: multiple selection, remote ssh sessions, and relatively simple configuration of everything including plugins. I’ve never noticed significant performance issues in vscode, while I have spent countless hours learning vim and emacs even to do basic configuration. That has taken away from development time. I don’t regret that time, but it is a trade off decision everyone has to make.

                                                                                                                                                      If I may ask, what do you use?

                                                                                                                                                      1. 1

                                                                                                                                                        If I may ask, what do you use?

                                                                                                                                                        I went back to Sublime when it seemed the writing was on the wall for Atom, and I’m currently pretty happy with it – much of the same basic features, and at least my battery life is a lot better. I’ve flirted with Emacs for years, and I’m pretty proficient with elisp, but I always hit a point where the constant mental friction of switching between “one-set-of-platform-wide-conventions-and-keystrokes” and “special-set-of-conventions-and-keystrokes-just-for-emacs” wears me out.

                                                                                                                                                      2. 2

                                                                                                                                                        That’s exactly my experience. There’s a very clear lag even while typing. I’m also encountering so many rendering bugs. For example, code warnings show up inline in a very long floating horizontal bar and it’s extremely difficult to scroll through it to read the entire message. The embedded terminals are full of display bugs and characters suddenly start flying all over the place while I type for no obvious reason.

                                                                                                                                                        What I found really impressive is the the ability to quickly launch a dev Docker container and do all of the work inside it. That’s the only reason I’m using it. It’s sad though that the core editing experience is so subpar compared to sublime, IntelliJ, Xcode and even TextEdit.

                                                                                                                                                        Atom was just as bad though in my experience.

                                                                                                                                                        1. 1

                                                                                                                                                          Now that you mention it, you’re right, editors in VS Code do seem to scroll faster than other scrollable areas. I looked into this and found an existing bug report and a workaround.

                                                                                                                                                          Bug report: After 1.66 update scroll speed is faster, submitted 2022-03-31

                                                                                                                                                          Workaround

                                                                                                                                                          Add this to your settings.json:

                                                                                                                                                            // Adjust scroll sensitivity to match macOS-native scroll surfaces, by my estimation
                                                                                                                                                            // These settings might become unnecessary after this scrolling speed bug is fixed: https://github.com/microsoft/vscode/issues/146403
                                                                                                                                                            "editor.mouseWheelScrollSensitivity": 0.5,
                                                                                                                                                            "workbench.list.mouseWheelScrollSensitivity": 0.5,
                                                                                                                                                          
                                                                                                                                                        2. 1

                                                                                                                                                          I used atom a little bit back in the day, but found it too hip and it was very sluggish, compared to gedit, grant, scribes, sublime, etc.

                                                                                                                                                          I think back in the day, it was very slugish. But because of those early fails, I still don’t want to touch VS Code, even though I know it’s everywhere. At $DAY_JOB we use IntelliJ so I use that after hours as well (yeah, yeah, I know, talk about slugishness - but the important parts are fast for me), and for quick and dirty stuff, I rather pick Sublime. But yeah, I don’t know where it’ll go in the future, I know I’ll keep my vim scripts up to date in any case.

                                                                                                                                                      3. 4

                                                                                                                                                        Muscle memory? I mean, if it still works for you, why incur switching costs?

                                                                                                                                                        1. 2

                                                                                                                                                          A reason to keep using Atom instead of VS Code could be the use of telemetry in the latter. Although I’m not sure if Atom lacks this ‘feature’.

                                                                                                                                                          1. 2

                                                                                                                                                            I’m not a VS Code fan, but just FYI Atom does have/had telemetry. If that’s something that bothers you also note there are builds of VS Code that do not (see vscodium).

                                                                                                                                                        1. 9

                                                                                                                                                          Wow, this looks like a great start. Some rough edges and incomplete, but great underpinnings.

                                                                                                                                                          Ironic that it hit lobste.rs the same day as the Atom sunset announcement and in spite of being familiar with Xi and Druid this is the first I’ve heard of it.

                                                                                                                                                          1. 6

                                                                                                                                                            I’m grateful they announced this rather than just letting it die in ignominy. It has been getting more and more decrepit for some time and holding back the Electron ecosystem for some time. The ambiguity about whether or not anything was going to get fixed or updating was worse than just knowing there won’t be a future for it.

                                                                                                                                                            I was never a user myself (neovimer), but it has been my go-to recommendation for non-programmers needing to hack on my projects for a while now. It served that purpose well for many years, but the lack of maintenance has been showing through and I’ve found myself writing hacks to help people I support work around it’s problems.

                                                                                                                                                            1. 2

                                                                                                                                                              I can understand using it if you’re familiar with it, but I’m curious what drove you to keep recommending it to newcomers instead of switching to the very similar but better maintained vscode?

                                                                                                                                                              1. 7

                                                                                                                                                                Because the UI is much less intimidating out of the gate and as far as keeping non-programmers comfortable without understanding what they are looking at it was easy to configure for use in advanced projects while keeping the UI to a minimum. VSCode is conceptually very similar but presents a lot more complexity within easy reach out of the box.

                                                                                                                                                                Secondarily, I’m familiar with walking people through setup of the editor and relevant plugins they would need for the projects I manage—projects coordinated between a handful of programmers and a whole host of content authors (think: writers, translators, copy-editors, etc. working with Markdown in Git repos). Of course I’ll be looking for and getting a workflow going for them in some other editor now which is quite a project for me, but as I said it is at least nice to know that’s the way this is headed.

                                                                                                                                                            1. 15

                                                                                                                                                              Dear projects: Release your stuff.

                                                                                                                                                              As a distro packager (Arch Linux & others) I’m frequently put in a jam by upstream projects that refuse to make patch releases for Known Broken Stuffⓡ because their Next Big Thing™ isn’t ready to ship yet. That leaves us distros cherry picking patches so that we can keep shipping their project as a package.

                                                                                                                                                              I’m happy to stick to stable releases — as long as your stable releases build against a current environment like the latest interpreter for your language and the latest of whatever else you depend on. If it doesn’t then you (as a project dev) doom distros to make a choice between either dropping your package or shipping unreleased code.

                                                                                                                                                              This isn’t a problem for projects that stay on top of their respective language ecosystem and have a sufficient release cadence. This is only a regular problem for projects that fail one to do of those two things.

                                                                                                                                                              1. 1

                                                                                                                                                                Given that this is mostly a sign of a bad project maybe the best course of action would either be abandoning such projects or joining them to make sure that stuff gets out. After all it means that you are putting crooks onto it, with the user not evening knowing it’s in such a bad state and if the maintainer leaves (real life emergencies, etc.) it’s likely to all fall apart.

                                                                                                                                                                It’s good to be vocal about it, so users actually know how well things go. I started digging a bit into how many patches important software has to avoid them, because they can bite you, especially when some major upgrade comes out and maintainers have to spend a lot of time to adapt or completely replace all the patches.

                                                                                                                                                                It of course gets worse with containers and such hiding that it’s all a mess and based on horribly outdated ecosystems, etc.

                                                                                                                                                                Anyways, thanks four your work. As a former package maintainer myself I know how hard it can be to make things that should and seem to be easy work. It feels like package maintainers and software porters are the silent heroes, that nobody talks about. Maybe a Package Maintainer day would be apt. :)

                                                                                                                                                                1. 8

                                                                                                                                                                  If distros dropped every upstream project with a “bad project” smell we wouldn’t have any users — distros appeal to users specifically by having a wide swath of things that work out of the box. Every distro will have distinctive approaches and specialties beyond that, but distros with few packages or lots of broken packages out of the box never survive long no matter how slick their other selling points.

                                                                                                                                                                  If I joined every upstream project with a “bad project” smell I would never get anything done. I already barely have a life.

                                                                                                                                                                  I already do contribute patches upstream any time I have to fix build issues just to get something packaged. My main beef here is with the plethora of projects that have fixes for know issues but don’t release them because it is extra work to make patch releases when they have other things in their main branch that “aren’t ready for release”. Instead they force distro packagers to go through the extra work of cherry picking. This is especially egregious for language ecosystem updates like Python. I can’t tell you how many projects say “support for python x.y will be in the next release” when they have patches in their tree that fix it, but proceed to delay the actual next release indefinitely because some feature they are also working on (or dreaming about) that they want to land in the next changelog so they have something besides dep bumps to show for themselves.

                                                                                                                                                                  As for being a hero, I appreciate the sentiment but count me out of anything apt ;-) ;-)

                                                                                                                                                                  1. 1

                                                                                                                                                                    If distros dropped every upstream project with a “bad project” smell we wouldn’t have any users — distros appeal to users specifically by having a wide swath of things that work out of the box.

                                                                                                                                                                    Sure, but there should also be some visibility for users to know what’s hanging on a thin thread. Few highly active maintainers that have to add huge amounts of patches that upstream for one reason or another doesn’t integrate are a bad sign and honestly when wanting to run a stable system I’d like to avoid that.

                                                                                                                                                                    Even “good” projects end up in non-nice situations sometimes. So it’s more about whether contributions increasing portability are welcome or not.

                                                                                                                                                                    Of course from the project’s author perspective they might want to either only officially support certain systems or not at all.

                                                                                                                                                                    If these two sentiments diverge and a distro/OS/OS’s architecture/… only supports a piece of software for a single person as a user that can be really important to be aware of, because it means that this maintainer getting frustrated, a change of interest, family, sickness, frustration, etc. can mean that one might need to take over, if that’s reasonably possible. Even worse when as mentioned earlier this happens when a project has a major release or adding a new dependency that might not end up being supported.

                                                                                                                                                              1. 6

                                                                                                                                                                Emacs does not need to be a VSCode clone in a TUI.

                                                                                                                                                                What it does need is some ants in its pants—it needs a NeoEmacs fork with some energy behind it.

                                                                                                                                                                It needs something to force a rethink of defaults and such. NeoVIM did this for VIM—even users that have stayed on the original VIM project have benefited, but for the NeoVIM fork itself being able to drop legacy cruft only beneficial to 30 year old systems has enabled all sorts of new features that were being blocked. (Case in paint, I’m writing this Lobste.rs post in a full blown NeoVIM driven text area with 100% of my terminal editor environment nestled nicely in the browser.)

                                                                                                                                                                1. 2

                                                                                                                                                                  This is pretty exciting - https://emacs-ng.github.io/emacs-ng/

                                                                                                                                                                  1. 1

                                                                                                                                                                    How, if at all, does this compare to:

                                                                                                                                                                    https://github.com/commercial-emacs/commercial-emacs

                                                                                                                                                                    …?

                                                                                                                                                                    (Not an Emacs user, for reasons I can give elsewhere.)

                                                                                                                                                                    1. 1

                                                                                                                                                                      That does actually look promising! As a 30 year vimmer I probably won’t be using it much myself, but I will keep an eye on it. That’s the kind of project that could even improve VIM just from having the progress and diversity on the other side of the fence. I’ll take NeoVIM’s Lua over emacs-ng’s Javascript personally, but I think both are a win. The progress in a canvas and just being able to have more than one front end is cool too.

                                                                                                                                                                  1. 17

                                                                                                                                                                    If there’s no Emacs implementation, does it actually exist?

                                                                                                                                                                    1. 3

                                                                                                                                                                      No vim either, clearly this is vaporware. At best it is a UI concept such as you might find on displays in SciFi moves…