The only problem with lots of custom aliases (or custom keybindings in other programs like editors), is that the muscle memory burns you every time you have to work on a remote machine. I used to go custom-to-the-max with my config, but I’ve gradually shifted back to fewer and fewer aliases except for the most prevalent build/version control commands I run dozens of times each day.
When I need to remote into machines where I can’t set up my shell profile for whatever reason, I just config ssh to run my preferred shell setup commands (aliases, etc) as I’m connecting.
Yeah, single session only. There are a bunch of different ways to skin this cat — LocalCommand and RemoteCommand along with ForceTTY in ssh_config can help.
Conceptually you want to do something like (syntax probably wrong, I’m on my phone)
which you could parameterize with a shell function or set up via LocalCommand and RemoteCommand above, or skip the temp file entirely with clever use of an env variable to slurp the rc file in and feed it into the remote bash (with a heredoc or SendEnv/SetEnv)
Nope. At least I’m not aware of any Linux distro installing it by default.
But being installed by default is IMHO totally overrated. The main point is that it is available in many Linux distribution’s repos without having to add 3rd party repos—at least in Debian and all derivatives like Devuan, Kali oder Ubuntu.
I understand, but it’s not the same. If I don’t have a shell regularly there, and not my own dotfiles, I likely want to avoid installing and removing system packages on other people’s systems. When stuff breaks, I want the minimum amount of blame :)
Ok, granted. Working as a system administrator it’s usually me who has to fix things anyway. And it happens only very, very seldom that something breaks just because you install a commandline tool. (Saying this with about 25 years of Linux system administration experience.)
Only zutils can theoretically have an impact as it renames commandline system tools and replaces them with wrappers. But so far in the past decade, I’ve never seen any system break due to zutils. (I only swa things not working properly because it was not installed. But that was mostly because I’m so used to it that I take it as given that zutils is installed. :-)
Yep, different role. I did some freelance work a long ago, and learned on (fortunately) my predecessor’s mistake: they hired me to do some work, because I guess someone before me updated some stuff, and that broke… probably PHP version? Anyway, their shop didn’t work any more and they were bleeding money till I fixed it. It was one of my early freelance jobs, so that confirmed the age-old BOFH mantra of if it ain’t broke, don’t fix it. So given time, I would always explicitly ask permission to do this or that or install the other, if needed.
But I went a different route anyway, so even though I am still better than average, I think, I’m neither good nor professional. But I think old habits die hard, so that’s why I’m saying “if this stuff isn’t there by default, you’ll just have to learn your tar switches” :)
This is not a good thing to have in your muscle memory. You should pause before you extract an archive. It may or may not have a top-level directory and depending on its origin it might be a bomb.
But universal extractor might come in handy. I like libarchive. It’s used by bsdtar.
There’s no reason it won’t happen on GNU/Linux (although I’ve never tried it myself).
It relates to the power of data formats and being able to perform relatively powerful computation when viewing them as a language. Relevant: http://langsec.org/
These kinds of attacks are fundamental and don’t have to do with a particular operating system.
It’s not really corrupted – it’s a valid zip file! It derives from the computational properties of the zip format itself. It’s hard to get rid of the bug without getting rid of features people use.
I mean one obvious thing you could do is hard-code a check – is the compression rate greater than a million? i.e. look for a 10 KB zip file that expands to 10GB. And then abort. But you can do that on Windows or Linux!
depends on the “this”. Windows may be more vulnerable to some specific harmful consequences but there’s nothing to stop, for instance, empty files named for every word of the dictionary getting sprayed all over your working directory and being an incredible annoyance to clean up, on any OS.
dtrx doesn’t protect against zip bombs (the ones where the inflated file size is huge), does it? It looks like it just extracts archives to a new directory each time, which is not sufficient.
I use bsdtar for this on Arch Linux. It’s the libarchive backed tar implementation from FreeBSD. It handles all formats you usually encounter. I have it aliased to tar for muscle memory reasons.
alias ecc='emacsclient -c'
alias eccn='emacsclient -c -n'
alias ecn='emacsclient -n'
alias ecnw='emacsclient -c -nw'
(I should use them more often. I actually forgot about them until this comment. :-)
Another set of aliases which I consider life hacks of this kind, i.e. which I use all the time:
Git related
alias amend='git commit --amend'
alias ga='git annex'
alias gap='git add -p'
APT-related or otherwise debian-specific
alias acs='apt-cache show --no-all-versions'
alias acp='apt-cache policy'
alias acr='apt-cache rdepends'
alias ac/='apt-cache search'
alias acn='apt-cache search -n'
alias af/='apt-file search'
alias afl='apt-file list'
alias acsrc='apt-cache showsrc'
# Use like "btsmutt 123456" to open a Debian bug report in mutt
alias btsmutt='bts --mbox show'
(Aliases with a slash in their name IIRC don’t work in Bash, but they do work in Zsh. So I mostly don’t care about that detail.)
And yeah, in the meanwhile I could replace apt-get and apt-cache in most of these with just apt. I’d though still keep the alias names due to muscle memory…
SSH related
# My "get me a remote screen session and stay connected no matter how" command
function asc() {
# Set the title to something more obvious, e.g. the expanded
# alias, eh, function
print -Pn "\e]0;%n@%m: autossh -t $* 'screen -RdU'\a";
# For the following hack for getting ssh-agent to work inside the
# reattached screen session, see
# http://samrowe.com/wordpress/ssh-agent-and-gnu-screen/ and
# http://www.stderr.nl/Blog/Software/Mutt/MuttRemoteAttachments.html?seemore=y
autossh -x -A -t "$@" 'screen -RdU'
}
compdef asc=ssh
(Granted, it’s a bit too complex for an alias and that compdef thingy is zsh-specific, but not required.)
Thanks. Interesting to see. I used to have many equivalents to these. Over time, they have gradually dissapeared to Emacs flows (ie. magit). If invoking shell utils, I now use dwim wrappers for password protecting pdfs, creating macOS icons, and all others here.
Learning how to use magit is still on my TODO list. I bascially do all git stuff on the commandline and use emacsclient to edit files in the already running Emacs instance from the commandline.
And thanks for these links. I think I should read quite some of your Emacs/DWIM related postings. :-)
The thing to say about atool is that it includes the aunpack command which fits the article’s description, is prevalent and is muscle memory safe: It recognizes file formats and it defuses archive bombs (creates a directory named after the archive if the archive lacks a toplevel directory).
That’s not what my understanding of a zip bomb is, I always assumed it referred to archives which produce unreasonably massive output. See https://en.wikipedia.org/wiki/Zip_bomb
I think it depends on where you come from what the prevalent problem is. I’m more used to “archive bomb” meaning “tar bomb”. It is unfortunate that an insignificant file format difference means so different things:
Tar bomb: Archive containing multiple toplevel files
I’ve been using unar which seems similar to atool. But I’m really hoping to find something better as I need to unpack zstandard time to time. Any suggestions?
Regarding the “would love to hear of others”: In Debian there are also the packages unp, zutils and patool:
unp is what I actually use as “DWIM unzip” command, because unzip is heavily tied to this annoying ZIP format used on Windows all the time. ;-)
And then there’s also zutils which replaces zcat, zgrep and friends which generic wrappers which also support bzip2, xz, lzma and other compression formats. The cool thing here is that this way you can zgrep through files with different compression formats in one go. And you never have to think anymore about if that one command is named bz2grep, bzgrep2 or just bzgrep. :-)
Another one I’m aware of, but never really used it, is patool.
EDIT: When looking through my shell aliases, I noticed that I was actually using patool quite often, because atool offers adiff but not an agrep. So I built it via alias agrep='patool search'. But I’m only using this for multi-file archive formats. Because for just compressed single file archives, there’s the zgrep from zutils which I use way more often.
Oh, and I of course also use atool, but mostly only one of its commands, namely als.
There seem to be some terrible aliasing in the GIFs on my screen. I’d recommend using one of those terminal stream recorders instead, resulting in animated SVGs.
The only problem with lots of custom aliases (or custom keybindings in other programs like editors), is that the muscle memory burns you every time you have to work on a remote machine. I used to go custom-to-the-max with my config, but I’ve gradually shifted back to fewer and fewer aliases except for the most prevalent build/version control commands I run dozens of times each day.
When I need to remote into machines where I can’t set up my shell profile for whatever reason, I just config ssh to run my preferred shell setup commands (aliases, etc) as I’m connecting.
My tools work for me, I don’t work for my tools.
You mean, could single session only? Care to share that lifehack? I’m assuming something in ssh_config?
Yeah, single session only. There are a bunch of different ways to skin this cat — LocalCommand and RemoteCommand along with ForceTTY in ssh_config can help.
Conceptually you want to do something like (syntax probably wrong, I’m on my phone)
scp .mypreferedremoterc me@remote:.tmprc; ssh -t me@remote “bash —rcfile ~/.tmprc -l; rm .tmprc”
which you could parameterize with a shell function or set up via LocalCommand and RemoteCommand above, or skip the temp file entirely with clever use of an env variable to slurp the rc file in and feed it into the remote bash (with a heredoc or SendEnv/SetEnv)
every time i have to work on a remote machine i do the commands through ssh or write a script to do it for me.
naming a meta-archive-extracter, “atool” doesn’t help either. OP used
unzip
for this but it is overloaded.uncompress
also is taken.What word would you guys use for aliasing it?
I use extract as a function that just calls the right whatever based on the filename.
I think prezto comes with
x
alias, and I like it a lot. It’s burns easily into the muscle memory.To defeat muscle memory when changing tools, I make sure the muscle memory command fails:
alias unzip = “echo ‘use atool’”
It doesn’t take many times to break the muscle memory. Then I remove the alias.
Is
atool
there by default on Linux boxes?Nope. At least I’m not aware of any Linux distro installing it by default.
But being installed by default is IMHO totally overrated. The main point is that it is available in many Linux distribution’s repos without having to add 3rd party repos—at least in Debian and all derivatives like Devuan, Kali oder Ubuntu.
I understand, but it’s not the same. If I don’t have a shell regularly there, and not my own dotfiles, I likely want to avoid installing and removing system packages on other people’s systems. When stuff breaks, I want the minimum amount of blame :)
Not that this is not a useful tool.
Ok, granted. Working as a system administrator it’s usually me who has to fix things anyway. And it happens only very, very seldom that something breaks just because you install a commandline tool. (Saying this with about 25 years of Linux system administration experience.)
Only
zutils
can theoretically have an impact as it renames commandline system tools and replaces them with wrappers. But so far in the past decade, I’ve never seen any system break due tozutils
. (I only swa things not working properly because it was not installed. But that was mostly because I’m so used to it that I take it as given that zutils is installed. :-)Yep, different role. I did some freelance work a long ago, and learned on (fortunately) my predecessor’s mistake: they hired me to do some work, because I guess someone before me updated some stuff, and that broke… probably PHP version? Anyway, their shop didn’t work any more and they were bleeding money till I fixed it. It was one of my early freelance jobs, so that confirmed the age-old BOFH mantra of if it ain’t broke, don’t fix it. So given time, I would always explicitly ask permission to do this or that or install the other, if needed.
But I went a different route anyway, so even though I am still better than average, I think, I’m neither good nor professional. But I think old habits die hard, so that’s why I’m saying “if this stuff isn’t there by default, you’ll just have to learn your tar switches” :)
Note that this doesn’t apply for eshell as the OP is using: If you cd to a remote machine in eshell, your aliases are still available.
Command history and completion suggestions have really helped me avoid new aliases.
This is not a good thing to have in your muscle memory. You should pause before you extract an archive. It may or may not have a top-level directory and depending on its origin it might be a bomb.
But universal extractor might come in handy. I like libarchive. It’s used by
bsdtar
.please elaborate…I didn’t know extracting an archive may be harmful!
Zip bomb!
https://en.wikipedia.org/wiki/Zip_bomb
this can’t happen in GNU/Linux OSes right?
There’s no reason it won’t happen on GNU/Linux (although I’ve never tried it myself).
It relates to the power of data formats and being able to perform relatively powerful computation when viewing them as a language. Relevant: http://langsec.org/
These kinds of attacks are fundamental and don’t have to do with a particular operating system.
Related but distinct: https://research.swtch.com/zip
I mean doesn’t the
zip
packaged by distros alert on potential corrupt archives?Depends on how they’re corrupted :-). And if you think zip bombs can’t get you, we also have tar bombs!
It’s not really corrupted – it’s a valid zip file! It derives from the computational properties of the zip format itself. It’s hard to get rid of the bug without getting rid of features people use.
I mean one obvious thing you could do is hard-code a check – is the compression rate greater than a million? i.e. look for a 10 KB zip file that expands to 10GB. And then abort. But you can do that on Windows or Linux!
depends on the “this”. Windows may be more vulnerable to some specific harmful consequences but there’s nothing to stop, for instance, empty files named for every word of the dictionary getting sprayed all over your working directory and being an incredible annoyance to clean up, on any OS.
Just use dtrx, no more bombs ever again
dtrx doesn’t protect against zip bombs (the ones where the inflated file size is huge), does it? It looks like it just extracts archives to a new directory each time, which is not sufficient.
I use
bsdtar
for this on Arch Linux. It’s the libarchive backedtar
implementation from FreeBSD. It handles all formats you usually encounter. I have it aliased totar
for muscle memory reasons.Thanks. TIL about bsdtar.
This is the reason I have alias vim=‘emacsclient -nc’
And relatedly the reason that I install dtrx on every machine I touch.
Lol. I had the same vim alias but memory willingly moved over to ec (emacsclient alias).
Thanks. TIL about dtrx.
Yep. And I have some more of that sort:
(I should use them more often. I actually forgot about them until this comment. :-)
Another set of aliases which I consider life hacks of this kind, i.e. which I use all the time:
Git related APT-related or otherwise debian-specific(Aliases with a slash in their name IIRC don’t work in Bash, but they do work in Zsh. So I mostly don’t care about that detail.)
And yeah, in the meanwhile I could replace
SSH relatedapt-get
andapt-cache
in most of these with justapt
. I’d though still keep the alias names due to muscle memory…(Granted, it’s a bit too complex for an alias and that
Otherscompdef
thingy is zsh-specific, but not required.)Thanks. Interesting to see. I used to have many equivalents to these. Over time, they have gradually dissapeared to Emacs flows (ie. magit). If invoking shell utils, I now use dwim wrappers for password protecting pdfs, creating macOS icons, and all others here.
Learning how to use magit is still on my TODO list. I bascially do all git stuff on the commandline and use
emacsclient
to edit files in the already running Emacs instance from the commandline.And thanks for these links. I think I should read quite some of your Emacs/DWIM related postings. :-)
https://github.com/mholt/archiver is an alternative, written in pure Go so a self-contained binary
atool
supports many more formats.Thank you! Didn’t know about it.
The thing to say about
atool
is that it includes theaunpack
command which fits the article’s description, is prevalent and is muscle memory safe: It recognizes file formats and it defuses archive bombs (creates a directory named after the archive if the archive lacks a toplevel directory).That’s not what my understanding of a zip bomb is, I always assumed it referred to archives which produce unreasonably massive output. See https://en.wikipedia.org/wiki/Zip_bomb
I think it depends on where you come from what the prevalent problem is. I’m more used to “archive bomb” meaning “tar bomb”. It is unfortunate that an insignificant file format difference means so different things:
I’ve been using
unar
which seems similar toatool
. But I’m really hoping to find something better as I need to unpack zstandard time to time. Any suggestions?You could probably fork atool to add support for the zstd binary fairly easily.
Turns out there’s the fork https://github.com/solsticedhiver/atool which claims the upstream maintainer is unreachable. Oh well.
Nice. I hope it’s useful to you!
Regarding the “would love to hear of others”: In Debian there are also the packages
unp
,zutils
andpatool
:unp is what I actually use as “DWIM unzip” command, because
unzip
is heavily tied to this annoying ZIP format used on Windows all the time. ;-)And then there’s also zutils which replaces
zcat
,zgrep
and friends which generic wrappers which also supportbzip2
,xz
,lzma
and other compression formats. The cool thing here is that this way you canzgrep
through files with different compression formats in one go. And you never have to think anymore about if that one command is namedbz2grep
,bzgrep2
or justbzgrep
. :-)Another one I’m aware of, but never really used it, is patool.
EDIT: When looking through my shell aliases, I noticed that I was actually using
patool
quite often, becauseatool
offersadiff
but not anagrep
. So I built it viaalias agrep='patool search'
. But I’m only using this for multi-file archive formats. Because for just compressed single file archives, there’s thezgrep
fromzutils
which I use way more often.Oh, and I of course also use
atool
, but mostly only one of its commands, namelyals
.HTH
Thanks for these. They look great. I’ll add them to the post also. All new to me.
Didn’t know about zutils, but big fan of the approach. ripgrep-all does a great job here. Their readme shows how diverse the set can be:
My alternative option is to install The Unarchiver.app on macOS and then just use
open foo.zip
and it will usually do what I need =)Neat trick!
There seem to be some terrible aliasing in the GIFs on my screen. I’d recommend using one of those terminal stream recorders instead, resulting in animated SVGs.
ah good idea. thank you. is there one you used before?
Haven’t used any of them myself, but the first one that pops into my mind is asciinema
Solve this problem with a simple shell function:
The
aunpack
alias is very easy to remember already.Not sure how I missed this existing alias. Thank you.