1. 102

Please don’t post it to HN yet. I want to do it tomorrow, at some reasonable time, so that it has a chance to be visible. Posting it there now would insta-flop it for >100%, and people who might find it interesting and useful wouldn’t even have an idea they may have missed something.

I just wanted to post it here already, because I can’t resist finally sharing it with you, as I’ve just put the final touches! :) Still warm from the oven ;)

Hope you find it useful!
Bless you.

edit: I don’t have a Mac to test it, so no idea if it would work out of the box. You’re welcome to try and go get it, and hopefully tell me! TIA

edit 2: Ok, it’s getting late here in my timezone, so I’m going to get some sleep. Will be back tomorrow to answer your posts — and to hopefully try launching it on HN on my early afternoon, so as to have a chance to catch both the EU and US timezones. Thanks for all the initial good words, have fun with it hopefully, and good luck with your lifes regardless! :)

  1.  

  2. 14

    This looks great, but is the approach of “try this command over and over” dangerous for pipelines that may contain “non-idempotent” operations?

    Specifically, and for example, I use the “perl -pi.bak -e …stuff… filename.glob “ pattern often, to search-and-replace text in files, and while figuring out the regex to use in there, I think this would try over and over, … and if my filename.glob was too broad (i use “*” too often), my directory would quickly fill up with files like filename.bak.bak.bak.bak.bak.

    I realize I’ll probably get a “don’t do that, then” comments… but isn’t there a case where you’d want to have a “don’t ‘hit return’, until the user signals that it’s ok to make an attempt to execute? Does this already exist in there and I missed it?

    1. 3

      Who am I to tell anyone “don’t do”! :)

      Actually, my initial idea was in fact to have such a “run/pause” feature. But then I thought I could maybe take a risk and go without it? It could even work! :) So it’s certainly not alien to me; however, I’m also not very sure now: would it then bring enough value over just writing the stuff on a normal shell prompt? What do you think? I’m genuinely curious; I’m deep enough in the trenches now, that I’ve already lost some end-user perspective, so your input would be very valuable to me here and now. Also, and maybe especially: would you think the -i.bak option would make any sense with the pause/run feature? Given that you would probably want to pause-run more than once anyway, so you might still get some .bak.bak.baks?… or do I not see something?… This is certainly interesting…

      1. 3

        original commenter here: I was thinking about this some more, and I bet you can get tricky using capabilities(7) or other containment mechanisms (on Linux, at least) to render the filesystem upon which you’re noodling to be immutable.

        1. 2

          Oh, wow; now that would be seriously amazing hack if it works… Thank you good sir for the idea, it totally made my eyes wide open… I wonder if it’s really possible, and how much work it would require…

          1. 2

            A more complex project to write, but perhaps a FUSE driver that act as writable snapshot overlay which doesn’t commit to disk. write / unlink “works” but only affects virtual filesystem nodes.

      2. 2

        Yeah, you’d need to be especially careful with find -exec or xargs when the operation might involve copying/modifying/deleting files.

      3. 8

        This is fantastic. Just watching the screencast is getting my creative juices flowing. What would a shell that always had this look like? We’d need some way to handle destructive commands like rm. Or commands that don’t just take time to run but also are expensive to rerun.

        1. 9

          <3 :)

          Actually, after I imagined it, I found it kinda hard to believe nobody seems to have done it earlier?… I think the pieces were mostly all there since the dawn of Unix, or before still.

          For me, Luna is the logical next step. But I got the idea before I learnt about them, and Luna is still a bit too early to be usable in this use case, so I couldn’t restrain myself to wait any longer. But I would totally love to concede to them in the long term, and I’m looking forward to a future where Luna reigns…

          As to rm… I was kinda scared after my first idea, that it would be dangerous… but then, it’s hard to accidentally type rm, and you need to give it some params anyway… So, in the end, I couldn’t think of any actual commands that would be more-than-typically risky here. That said, I have an idea for a means to protect against some simplest accidents. But, uh, I have really many ideas for up, and I really wanted to share as soon as possible, in some useful enough form… I myself find it hard to live without it at my hand’s reach, now that I know it exists.

          edit: Personally, I’m kinda stoked about one more idea I mentioned in the readme: of capturing stdout of already running processes… like if you do something, then be surprised it takes longer than expected… not wanting to kill it now and restart with a pipe… just run a capture command after the fact, maybe plug it in to up, and go on like nothing happened… :) this could be a totally separate tool. In fact, AFAIK there are already some like this, but I think they may need some refreshing…

          edit 2: as to rm… uh, oh, now I think of it, a tool is not really Unixy if you can’t potentially hurt yourself with it, amirite? ;P

          1. 7

            Actually, after I imagined it, I found it kinda hard to believe nobody seems to have done it earlier?… I think the pieces were mostly all there since the dawn of Unix, or before still.

            It was “done” before. Pipecut which seems to never have been released. The author wanted to “clean up some code first” and there you go. :( Perfection is the enemy of something or other.

            Glad someone actually finished something like this.

            1. 3

              I remember watching the video for that (with slides) thinking it was a great idea. Not sure what happened to it, but I’m glad somebody came up with the good idea and is moving forward with it.

              Note to @akavel, please go through pipecuts ideas (even if you don’t use them). One of the things I remember is that he had thought out a lot of the design, so you can figure out some of the design decisions even faster.

              1. 1

                Oh, awesome, thanks a ton! I didn’t notice the links to slides and video when glancing through their website, and your kind and thoughtful recommendation makes me really want to check them. Thanks!

              2. 1

                At this time, the author wants to clean up some portions of the code … The code will be published at code.google.com/p/pipecut

                Wow —haven’t seen a Google Code link in a while 😄

                1. 1

                  Oh, interesting, thanks! I’ll add it as “prior art” in the readme then.

                2. 2

                  ’m kinda stoked about one more idea I mentioned in the readme: of capturing stdout of already running processes… like if you do something, then be surprised it takes longer than expected… not wanting to kill it now and restart with a pipe… just run a capture command after the fact, maybe plug it in to up, and go on like nothing happened… :)

                  https://github.com/nelhage/reptyr

                  1. 2

                    Yes :) Thanks! Also potentially: neercs, injcode. I just haven’t found time to research them enough yet to learn how to make them cooperate best…

              3. 5

                Would it make sense to implement this as a hook in fzf? Then fzf’s UI (including “preview” feature, which makes sense here) and keybindings code can be reused.

                1. 1

                  Wow, that would make this truly excellent.

                  1. 1

                    Hmm! I know about fzf’s existence, but never actually used it yet, so I’m not very familiar with how it works. Could you point me out to where the hooks mechanism is described/implemented? I don’t seem to be able to find such keyword in the readme…?

                    To tell the truth, I actually considered trying to build up in fzf at an early point after my vision materialized! I even tried to see whether fzf doesn’t already have such functionality :) I think I became overwhelmed with the fzf’s guide however, such that I wasn’t sure if my would be able to fit in, and even if it did, whether I wouldn’t get lost in the fzf codebase… I would be interested in getting some help here with someone knowledgeable enough in fzf’s inner workings. I might try to reach to fzf’s author, maybe by creating an issue with a proposal at some later time {note to self: TODO} — but please do chime in earlier if you can help.

                    That said, I do also have some ideas where I would like to try to take up further, and I’m also not really sure if they are compatible with fzf in any way… meaning, I’m afraid they could tear fzf apart anyway if I tried to apply them there… Hmm! — but maybe I could leave the up repo to myself for prototyping, and just try porting what’s possible to fzf! Ok, that sounds like a reasonable plan to me at this point. Thanks!

                  2. 4

                    I’ve daydreamed of something like this for a while. Great job, akavel!

                    1. 1

                      <3 Really happy to hear, and that I could materialize your dreams! :)

                    2. 4

                      Update: posted to HN just now. You’re welcome to upvote it there for an initial boost of visibility if you like. Hope this won’t trigger their “voting ring detector” or something.

                      edit: Eh, flopped on HN anyway. Pity, but I had to be prepared for this, it’s always a roulette there. Probably better for my mental health this way :P I’m even more happy then that I could share it with you here on lobste.rs, got so many awesome comments with valuable discussion and kind words, and first of all, a chance to tell you about it and hopefully make your work, and life through this, a teeny tiny bit better/easier with it. Ah, by the way: I don’t have an active twitter account, so if you find some people sharing this there (or elsewhere), or some custom screenshots/screencasts they did with it, I’d be very happy if you could let me know via a private message on lobste.rs! But no pressure about it. TIA

                      1. 2

                        Yes, I think the direct link excludes your vote. Better go to “https://news.ycombinator.com/show” and manually up-vote “Show HN: up – a tool for writing Linux pipes with instant live preview”. Good luck!

                        PS: Worst case you might get reselected by the mods or you can I think somehow ask mods to resubmit your story.

                        Seems it got resubmitted https://news.ycombinator.com/item?id=18292712

                        1. 1

                          Hahaha, lol, the storm unwinds indeed, if with a small delay :D Funny to watch it happen :) Thanks a lot for the pingback! :) And quite some interesting ideas and thoughts there, too. Oooh, will be a lot to digest after the dust settles down. I’m starting to have second thoughts if this won’t result in some files lost accidentally indeed for someone :( Eh, hope not, and let’s hope it doesn’t take me too long to release a new version with at least a pausing functionality…

                          edit: And I’m still surprised with the amount of positivity and interest I’m getting even there! It really feels warm in the heart to read every and one post, here, there, everywhere, from people who wanted such a tool, and now discover it exists (even if they sometimes weren’t even aware of that wanting!)… so that I could be helpful for them… uh, just hoping now that I won’t be getting too many “give me back my lost files” issues on github starting a week from today, till the end times…

                      2. 2

                        This looks like the tool I never knew I wanted. Great job man, I’m going to take a look at getting it building on BSD tomorrow.

                        1. 2

                          Thanks! Please do give me a note about your findings. One of the reasons I didn’t go for GPL was to make it more comfortable for potential integration with *BSDs…

                          1. 1

                            I ended up trying to compile it before bed, and it just worked. Haven’t found any bugs thus far. You should be able to just cross pile and have it work.

                            1. 2

                              Cool, thanks! You mean you tested the basic working of the TUI interface, correct? Also, which BSD — FreeBSD, NetBSD, OpenBSD, DragonFly?…

                              1. 1

                                TUI works as it should, I tested on FreeBSD 12.0-BETA1

                                1. 2

                                  Thanks! :) FYI, I’ve released a new version a few hours ago, and uploaded binaries for all the BSDs I could see supported :) Only I compiled them with CGO_ENABLED=0; hope this works ok.

                                  1. 1

                                    I just fetched the FreeBSD binary and it’s working fine here. Thanks.

                        2. 2

                          This is the tool I never knew that I needed but I absolutely love the idea. Will have to play around with it tomorrow, will try it on macOS as well.

                          1. 2

                            <3

                            Actually, this made me think that I don’t remember now what originally inspired me to start envisioning it. Pity, I’d love to be able to thanks someone concrete. But maybe it has just grown on a variety of distinct bits and pieces…

                          2. 1

                            would be neat to have a set of whitelisted commands it could run, to avoid the issue of a dangerous command being a prefix of the one you want.

                            1. 1

                              Looks really neat!

                              Apropos of nothing, what’s the font used in the GIF? Looks like Go Mono.

                              1. 2

                                Thanks! :)

                                Go Mono it is :) You may, or may not, want to check https://github.com/akavel/asciinema2gif. (No readme or otherwise yet; I plan to hopefully write it and announce at some later time. Enough releases for one day! That takes some effort to do… I wrote the asciinema2gif app over the last two evenings, as I couldn’t fathom and accept in my heart that all the asciinema2gif converters I could find wanted to download and use phantomjs — i.e., to capture screenshots from a browser… please, for converting a json file to a series of text-based gif frames?!? Fortunately, I quickly found that Go has libs for animated gifs, asciicast parsing, ANSI sequence parsing, and knew about freetype beforehand, so I was set to go.)

                              2. 1

                                I want something like this but for Haskell/ghci functions.

                                1. 1

                                  Have you tried IHaskell?

                                    1. 1

                                      Umm, what’s this?

                                      1. 4

                                        I modified up to execute ghci instead of bash.

                                        1. 1

                                          Ah, lol, super cool! :) You should probably replace the | in the prompt with a λ then, right? ;P Hm, do you think adding a flag to choose a custom binary/commandline as an interpreter would generalize over your approach? If yes, would you maybe care to submit a Feature Request issue on GH? Or even a PR if you fancy ;) Also, I’m curious what you think: would just writing ghci -something "...your code..." be similarly powerful, if slightly more cumbersome? I was thinking of writing lua -e "..." some time ago… Also, of pluggable “subinterpreters”… just not clear if this wouldn’t take it too far in the realm of writing a custom editor… I’m kinda deep in the trenches now, so don’t have a good view from a distance as to what is most comfortable from an end-user perspective, like yours now…

                                          1. 1

                                            The command is something like:

                                            ghci \
                                              -XOverloadedStrings \
                                              -e ':m + Data.Aeson' \
                                              -e ':m + Control.Lens' \
                                              -e ':m + Data.Aeson.Lens' \
                                              -e "interact ($COMMAND)"
                                            
                                            1. 1

                                              Uh, oh; that’s somewhat of a mouthful. Hmmm, I’ll have to think about this more. Thanks for the reply, I’m letting myself copy it to the issue you submitted on the up repo, so that it’s easier to track for me. I’d be super grateful if you (or someone else) could also submit an example shebang counterpart to this — is it possible to write one?

                                2. 1

                                  As the post on HN points out, doing this with any command that has side-effects will blow both your feet off in short order.

                                  At the same time it looks like a fantastic tool for constructing text pipelines - the interactivity is lovely.

                                  Maybe you could white-list side-effect free commands, or add arguments to known commands that would make them side-effect free? The user could be asked whether an unknown command was safe at runtime for anything new. As it is, I’d be afraid to touch this - a couple of wayward keystrokes could be devastating!