While writing a post on practical shell patterns I had a couple of patterns that used
I had to drop those patterns from the post, though, because as soon as I thought about them I got lost in all the
grep flags I wanted to talk about. I realised
grep deserved its own post.
grep is one of the most universal and commonly-used commands on the command line. I count about 50 flags you can use on it in my man page. So which ones are the ones you should know about for everyday use?
I tried to find out.
I started by asking on Twitter whether the five flags I have ‘under my fingers’ and use 99% of the time are the ones others also use.
It turns out experience varies widely on what the 5 most-used are.
Here are the results, in no particular order, of my researches.
I’ve tried to categorize them to make it easier to digest. The categories are:
- ABC – The Context Ones
- What To Match?
- What To Report?
- What To Read?
- Honourable Mention
If you think any are missing, let me know in the comments below.
ABC – The Context Ones
These arguments give you more context around your match.
I hadn’t included these in my top five, but as soon as I was reminded of them,
-C got right back under my fingertips.
I call them the ‘ABC flags’ to help me remember them.
Each of these gives a specified context around your grep’d line.
-A gives you lines after the match,
-B gives you lines before the match, and
-C (for ‘context’) gives you both the before and after lines.
$ mkdir grepflags && cd grepflags $ cat > afile <<EOF a b c d e EOF $ grep -A2 c afile c d e $ grep -B2 c afile a b c $ grep -C1 c afile b c d
This is especially handy for going through configuration files, where the ‘before’ context can give you useful information about where the thing your matching sits within a wider context.
What To Match?
These flags relate to altering what you match and don’t match with your grep.
This flag ignores the case of the match. Very handy and routine for me to use to avoid not missing matches I might want to see (I often grep through large amounts of plain text prose).
$ cat > afile <<EOF SHOUT shout let it all out EOF $ grep shout afile shout $ grep -i shout afile SHOUT shout
This matches any lines that don’t match the regular expression (inverts), for example:
$ touch README.md $ IFS=$'\n' # avoid problems with filenames with spaces in it $ for f in $(ls | grep -v README) > do echo "top of: $f" > head $f > done SHOUT shout let it all out
which outputs the heads of all files in the local folder except any files with
README in their names.
-w flag only matches ‘whole-word’ matches, ignoring cases where submitted words are part of longer words.
This is a useful flag to narrow down your matches, and also especially useful when searching through prose:
$ cat > afile <<EOF na NaNa na-na-na na_na_na hey jude EOF $ grep na afile na na-na-na na_na_na $ grep -w na afile na na-na-na $ grep -vwi na afile NaNa na_na_na hey jude
You might be wondering what characters are considered part of a word. The manual tells us that ‘Word-constituent characters are letters, digits, and the underscore.’ This is useful to know if you’re searching code where word separators in identifiers might switch between dashes and underscore. You can see this above with the
What To Report?
grep flags offer choices about how the output you see is rendered.
grep -h suppresses the prefixing of filenames on output. An example is demonstrated below:
$ rm -f afile $ cat > afile1 << EOF a EOF $ cp afile1 afile2 $ grep a * afile1:a afile2:a $ grep -h a * a a
This is particularly useful if you want to process the matching lines without the filename spoiling the input. Compare these to the output without the
$ grep -h a * | uniq a $ grep -h a * | uniq -c 2
This outputs only the text specified by your regular expression. One match is output per line, but multiple matches may be made per line.
This can result in more matches than lines, as in the example below, where you look for words that end lines that end in ‘ay’, and then any words with the letter ‘e’ in them (but not at the start or the end of the word).
$ rm -f afile1 afile2 $ cat > afile << EOF Yesterday All my troubles seemed so far away Now it looks as though they're here to stay Oh I believe In yesterday EOF $ grep -o ' [^ ]*ay$' afile away stay yesterday $ grep -o ' [^ ]*e[^ ]*' afile troubles seemed they're here believe yesterday
If you’re fighting through a blizzard of output and want to focus only on which files your matches are in rather than the matches themselves, then using this flag will show you where you might want to look:
$ cat > afile << EOF a a EOF $ cp afile afile2 $ cp afile afile3grep -l $ grep a * afile:a afile:a afile2:a afile2:a afile3:a afile3:a $ grep -l a * afile afile2 afile3
What To Read?
These flags change which files
grep will look at.
A very popular flag, this flag recurses through the filesystem looking for matches.
$ grep -r securityagent /etc
This one is my favourite, as it’s incredibly useful, and not so well known, despite being widely applicable.
If you’ve ever been desperate to find where a string is referenced in your filesystem (usually as root) and run something like this:
$ grep -rnwi specialconfig /
then you won’t have failed to notice that it can take a good while. This is partly because it’s looking at every file from the root, whether it’s a binary or not.
-I flag only considers text files. This radically speeds up recursive greps. Here we run the same command twice (to ensure it’s not only slow the first time due to OS file cacheing), then run the command with the extra flag, and see a nearly 50% speedup.
$ time sudo grep -rnwi specialconfig / 2>/dev/null sudo grep -rnwi specialconfig / 418.01s user 382.19s system 70% cpu 19:03.09 total $ time sudo grep -rnwi specialconfig / sudo grep -rnwi specialconfig / 434.19s user 411.62s system 70% cpu 19:56.25 total $ time sudo grep -rnwiI specialconfig / sudo grep -rnwiI specialconfig / 33.54s user 322.64s system 52% cpu 11:19.03 total
There are many other
grep flags, but I’ll just add one honourable mention at the end here.
I spent an embarrassingly long time trying to get regular expressions with + signs in them to work in grep before I realised that default grep didn’t support so-called ‘extended’ regular expressions.
-E you can use those regular expressions just as the regexp gods intended.
$ cat > afile <<< aaaaaa+bc $ grep -o 'a+b' a # + is treated literally a+b $ grep -o 'aa*b' a # a workaround aaaaaaaaaab $ grep -oE 'a+b' a # extended regexp aaaaaaaaaab
If you like this, you might like one of my books:
Learn Bash the Hard Way
Learn Git the Hard Way
Learn Terraform the Hard Way
If you enjoyed this, then please consider buying me a coffee to encourage me to do more.
7 thoughts on “grep Flags – The Good Stuff”
-e # for OR-ing — still miffed there’s no good order-independent AND equivalent and more and more using awk in pipes for that reason (awk ‘/a/ && /b/’ is just so much easier than -e ‘a.*b’ -e ‘b.*a’ the more complex a and b get ; and once you need fields from the grepped lines anyway…)
-v # more and more often in combination with many -e
-E # mostly when I need something involving ‘|’ but more complex than just using two ‘-e’s
-C / -A / -B # -C was a revelation when, after literally years of “-A n -B n”, I (re-)read the man page properly :-D
PS: just discovered your Blog; off to read some more :-)
Nice! Here’s one you missed: -F (fixed strings), turns off regex matching and just matches straight text, which can be MUCH faster
I switched over to ack a while back and never looked back.
I’m surprised -q didn’t make the list. Often I’m just looking to see if there is any match present in a file. In which case, grep -q can stop after the first hit. This should result in better speed. Otherwise, grep will merrily continue to search for all matches which is often unnecessary.
I usually use -l for that.