Sad to see the GNU project leaning on Posix to justify removing useful functionality. GNU’s not Unix, after all, while Posix is a committee-designed bare minimum created to uphold the illusion of interoperability between different Unix dialects, almost none of which are used anymore.
And where people do still use old Unix, making them magically conform to some new POSIX spec isn’t possible. I still use Solaris regularly and have many scripts that use egrep -q for a silent grep because that option combination was portable.
When I want something to be portable, I care about the actual implementations I need to deal with and not what some spec written after the OS was released says it ought to do.
I am particularly baffled by this because it seems to have no… real maintainership consequences of any kind. I’ve been trying to figure out why this change was made because, like many Unix users, I have decades’ (fsck I’m old…) worth of scripts, and knowing the idioms behind your tools is useful in these cases. My scripts seem to work okay (heh, not so dirty after all!) but you never know. And I’m still at a loss about how to explain it in terms other than pettiness.
egrep and fgrep aren’t POSIX-mandated, but grep implements a lot of things that aren’t POSIX. Maybe the maintainers don’t want to deal with bugs introduced by using egrep or fgrep – which I can understand, but why not just EWONTFIX them? I mean, this has been working fine practically forever, and any weird cornercases could’ve long been documented in man erm info pages.
This all seems stupid, but I want to use this opportunity to reiterate something not-stupid: an eco-system that can’t handle people adding new warnings is utterly broken.
The constraint seems to be that you can raise warnings to ask your users to make changes, but you can’t raise warnings to users of your users. Some thoughts on that:
We use libraries and tools for their abstraction, but if your library or tool is transmitting warnings about your code to its consumers, it’s not a very good abstraction.
If only we could add constraints to a tool like, “not to be used in published tools.” You may use egrep and fgrep in tools for yourself, but it’s a bad idea to publish such tools. If only people would avoid using – and definitely building more layers atop – scripts by others that are held together with baling wire.
If only we had communication channels between zones of ownership. To be able to talk to direct users of your tool without talking to their users, that would be something.
Did gcc really add -W all in one version and -Wall all in another version? Perhaps they should have called them -W2.8, -W10.2, etc.?
This all seems stupid, but I want to use this opportunity to reiterate something not-stupid: an eco-system that can’t handle people adding new warnings is utterly broken.
This is certainly correct but grep, like all Unix tools, doesn’t really have the concept of warnings. At best, there’s a split between conventional data (stdout) and diagnostic output (stderr), although IIRC there’s not a very rigorous distinction between what conventional data and diagnostic output is supposed to mean and there are some practical differences between the two streams anyway (e.g. buffered vs. unbuffered).
It’s not a good environment but it’s the environment that grep is written for, for better or for worse…
Yeah, if it was a web server or a daemon or something, who cares, emit more warnings, but the point of (e|f)?grep is to make pipelines of redirected output. You just can’t assume that messing with stderr is okay in that scenario.
egrep means [0-9]+ works like in Perl/Python/JS/PCRE/every language, not [0-9]\+ like GNU grep.
egrep syntax is consistent with bash [[ $x =~ $pat ]], awk and sed –regexp-extended (GNU extension)
These are POSIX extended regular expressions. Awk uses them too.
fgrep is useful when I want to search for source code containing operators, without worrying about escaping
This style makes regular expressions and grep a lot easier to remember! I want to remember 2 regex syntaxes (shell and every language), not 3 (shell, awk, every language) !
This change should be reverted; there is no point to needless breakage
Again you wouldn’t remove grep –long-flags because it’s not POSIX
Yes, egrep is a shorthand for grep -E and fgrep for grep -F. You haven’t lost anything. You can even make aliases or script wrappers if you want to use that syntax. But the point of this decision is that if you’re writing a script which is meant to be portable, you should use grep’s flags.
In grep-2.8 GNU added an echo "$cmd: warning: $cmd is obsolescent; using @grep@ @option@" >&2 in the upstream {e,f}grep scripts [ https://git.savannah.gnu.org/cgit/grep.git/commit/?id=a9515624709865d480e3142fd959bccd1c9372d1 ]
In Slackware at least suddenly many script were spamming the terminal with this arguably useless warning.
Ultimately Slackware decided to patch upstream (something rarely done) to undo this change:
Wed Sep 7 18:40:44 UTC 2022
[…]a/grep-3.8-x86_64-2.txz: Rebuilt.
Folks, I rarely veto upstream, but I'm not going to entertain this nonsense.
The egrep and fgrep commands were part of Unix since the 70s, continue to be
included with the BSDs, and frankly, aren't hurting anything. GNU grep
declared them deprecated in 2007 and when they were changed into shell
scripts around 8 years ago I figured that's where it would end. I can see no
logical justification to have these scripts start making noise and then to
eventually pull the rug out from under any code that might be using them, so
I've placed non-noisy versions of them into the package sources and will be
installing those during the build. Given that the -F and -E options are part
of the POSIX standard, these scripts will continue to work fine. That said,
we will be continuing to change our own code over to the recommended syntax
to avoid the minimal overhead incurred compared to using grep directly.
I would hope that the other distros follow suit. I don’t like the change in behavior.
While I understand the desire to deprecate “old” functionality, it is just inescapable that egrep and fgrep have existed for many decades, and have come to be relied upon by generations of shell script programmers. And now scripts that have worked fine for decades will now break in unexpected ways.
I’ll make it a point not to use egrep in the future (never used fgrep). But, realistically, what real harm was there in leaving the old /bin/egrep and fgrep scripts? Can anyone actually argue that they were some kind of maintenance burden? Yes, they each take up a sector on the drive when installed, but that marginal cost is very low.
The presence of extra commands should always conform to POSIX? i.e. a system with XYZgrep is still POSIX compliant?
I believe the argument goes like this:
POSIX specifies the behavior of grep -E and grep -F, but does not specify egrep or fgrep.
Therefore: if you use grep -E or grep -F, and it behaves in a way that deviates from POSIX’s specification, it is the grep implementation’s bug to deal with. But if you use egrep or fgrep, and it doesn’t behave the way POSIX grep -E/grep -F are specified, it is your bug to deal with, because POSIX didn’t actually guarantee egrep or fgrep would exist or behave in any particular way.
Therefore: the responsible thing to do as a programmer, if you want the bug to be someone else’s to deal with, is always to use grep -E/grep -F, never egrep/fgrep.
Therefore: programmers should be reminded to use grep -E/grep -F.
Honestly, I find it very hard to get worked up much about this, given that it is just a warning. I also very much do not subscribe to the “warnings are part of your public API” school of thought.
Sad to see the GNU project leaning on Posix to justify removing useful functionality. GNU’s not Unix, after all, while Posix is a committee-designed bare minimum created to uphold the illusion of interoperability between different Unix dialects, almost none of which are used anymore.
The POSIX argument doesn’t make any sense … would you remove support for
grep --long-flagsbecause it’s not POSIX?POSIX is a subset of useful functionality; this is baffling
And where people do still use old Unix, making them magically conform to some new POSIX spec isn’t possible. I still use Solaris regularly and have many scripts that use
egrep -qfor a silent grep because that option combination was portable.When I want something to be portable, I care about the actual implementations I need to deal with and not what some spec written after the OS was released says it ought to do.
I am particularly baffled by this because it seems to have no… real maintainership consequences of any kind. I’ve been trying to figure out why this change was made because, like many Unix users, I have decades’ (fsck I’m old…) worth of scripts, and knowing the idioms behind your tools is useful in these cases. My scripts seem to work okay (heh, not so dirty after all!) but you never know. And I’m still at a loss about how to explain it in terms other than pettiness.
egrepandfgreparen’t POSIX-mandated, butgrepimplements a lot of things that aren’t POSIX. Maybe the maintainers don’t want to deal with bugs introduced by usingegreporfgrep– which I can understand, but why not justEWONTFIXthem? I mean, this has been working fine practically forever, and any weird cornercases could’ve long been documented inmanerminfopages.This all seems stupid, but I want to use this opportunity to reiterate something not-stupid: an eco-system that can’t handle people adding new warnings is utterly broken.
The constraint seems to be that you can raise warnings to ask your users to make changes, but you can’t raise warnings to users of your users. Some thoughts on that:
We use libraries and tools for their abstraction, but if your library or tool is transmitting warnings about your code to its consumers, it’s not a very good abstraction.
If only we could add constraints to a tool like, “not to be used in published tools.” You may use
egrepandfgrepin tools for yourself, but it’s a bad idea to publish such tools. If only people would avoid using – and definitely building more layers atop – scripts by others that are held together with baling wire.If only we had communication channels between zones of ownership. To be able to talk to direct users of your tool without talking to their users, that would be something.
Did
gccreally add-Wall in one version and-Wallall in another version? Perhaps they should have called them-W2.8,-W10.2, etc.?This is certainly correct but
grep, like all Unix tools, doesn’t really have the concept of warnings. At best, there’s a split between conventional data (stdout) and diagnostic output (stderr), although IIRC there’s not a very rigorous distinction between what conventional data and diagnostic output is supposed to mean and there are some practical differences between the two streams anyway (e.g. buffered vs. unbuffered).It’s not a good environment but it’s the environment that
grepis written for, for better or for worse…Yeah, if it was a web server or a daemon or something, who cares, emit more warnings, but the point of (e|f)?grep is to make pipelines of redirected output. You just can’t assume that messing with stderr is okay in that scenario.
I would agree with the statement that “the cron(8) ecosystem is utterly broken”. We still don’t have
?timespecs.BTW I only use egrep and fgrep !!!
egrepmeans[0-9]+works like in Perl/Python/JS/PCRE/every language, not[0-9]\+like GNU grep.fgrepis useful when I want to search for source code containing operators, without worrying about escapingThis style makes regular expressions and grep a lot easier to remember! I want to remember 2 regex syntaxes (shell and every language), not 3 (shell, awk, every language) !
This change should be reverted; there is no point to needless breakage
Again you wouldn’t remove grep –long-flags because it’s not POSIX
Yes,
egrepis a shorthand forgrep -Eandfgrepforgrep -F. You haven’t lost anything. You can even make aliases or script wrappers if you want to use that syntax. But the point of this decision is that if you’re writing a script which is meant to be portable, you should use grep’s flags.I understand that, but what I don’t understand is why anyone, especially GNU grep maintainers, would think this would lead to portable shell scripts
Again, why not remove
grep --extended-regexptoo? You’re supposed to usegrep -E, idiotI don’t understand what’s going on here …
The presence of extra commands should always conform to POSIX? i.e. a system with XYZgrep is still POSIX compliant?
Then, from https://www.phoronix.com/news/GNU-Grep-3.8-Stop-egrep-fgrep
It’s already like this on my Ubuntu machine:
So this feels like a non-announcement for most people … ? It’s some cruft in upstream GNU grep removed that doesn’t affect anyone?
I can’t imagine that anyone would actually remove those 1 line shell scripts.
In grep-2.8 GNU added an
echo "$cmd: warning: $cmd is obsolescent; using @grep@ @option@" >&2in the upstream {e,f}grep scripts [ https://git.savannah.gnu.org/cgit/grep.git/commit/?id=a9515624709865d480e3142fd959bccd1c9372d1 ]In Slackware at least suddenly many script were spamming the terminal with this arguably useless warning.
Ultimately Slackware decided to patch upstream (something rarely done) to undo this change:
This is just dumb? I can’t believe anyone would spend time on this
Having 1 line shell script wrappers causes absolutely no maintenance burden.
I would hope that the other distros follow suit. I don’t like the change in behavior.
While I understand the desire to deprecate “old” functionality, it is just inescapable that
egrepandfgrephave existed for many decades, and have come to be relied upon by generations of shell script programmers. And now scripts that have worked fine for decades will now break in unexpected ways.I’ll make it a point not to use
egrepin the future (never usedfgrep). But, realistically, what real harm was there in leaving the old/bin/egrepandfgrepscripts? Can anyone actually argue that they were some kind of maintenance burden? Yes, they each take up a sector on the drive when installed, but that marginal cost is very low.I believe the argument goes like this:
grep -Eandgrep -F, but does not specifyegreporfgrep.grep -Eorgrep -F, and it behaves in a way that deviates from POSIX’s specification, it is thegrepimplementation’s bug to deal with. But if you useegreporfgrep, and it doesn’t behave the way POSIXgrep -E/grep -Fare specified, it is your bug to deal with, because POSIX didn’t actually guaranteeegreporfgrepwould exist or behave in any particular way.grep -E/grep -F, neveregrep/fgrep.grep -E/grep -F.Honestly, I find it very hard to get worked up much about this, given that it is just a warning. I also very much do not subscribe to the “warnings are part of your public API” school of thought.