Document not found (404)
This URL is invalid, sorry. Please use the navigation bar or search to continue.
Exercise solutions
cat and tac
1) The given sample data has empty lines at the start and end of the input. Also, there are multiple empty lines between the paragraphs. How would you get the output shown below?
# note that there's an empty line at the end of the output
+$ printf '\n\n\ndragon\n\n\n\nunicorn\nbee\n\n\n' | cat -sb
+
+ 1 dragon
+
+ 2 unicorn
+ 3 bee
+
+2) Pass appropriate arguments to the cat command to get the output shown below.
$ cat greeting.txt
+Hi there
+Have a nice day
+
+$ echo '42 apples and 100 bananas' | cat - greeting.txt
+42 apples and 100 bananas
+Hi there
+Have a nice day
+3) What does the -v option of the cat command do?
Displays nonprinting characters using the caret notation.
4) Which options of the cat command do the following stand in for?
-eoption is equivalent to-vE-toption is equivalent to-vT-Aoption is equivalent to-vET
5) Will the two commands shown below produce the same output? If not, why not?
$ cat fruits.txt ip.txt | tac
+
+$ tac fruits.txt ip.txt
+No. The first command concatenates the input files before reversing the content linewise. With the second command, each file content will be reversed separately.
6) Reverse the contents of blocks.txt file as shown below, considering ---- as the separator.
$ cat blocks.txt
+----
+apple--banana
+mango---fig
+----
+3.14
+-42
+1000
+----
+sky blue
+dark green
+----
+hi hello
+
+$ tac -bs '----' blocks.txt
+----
+hi hello
+----
+sky blue
+dark green
+----
+3.14
+-42
+1000
+----
+apple--banana
+mango---fig
+7) For the blocks.txt file, write solutions to display only the last such group and last two groups.
# can also use: tac -bs '----' blocks.txt | awk '/----/ && ++c==2{exit} 1'
+$ tac blocks.txt | sed '/----/q' | tac
+----
+hi hello
+
+$ tac -bs '----' blocks.txt | awk '/----/ && ++c==3{exit} 1' | tac -bs '----'
+----
+sky blue
+dark green
+----
+hi hello
+8) Reverse the contents of items.txt as shown below. Consider digits at the start of lines as the separator.
$ cat items.txt
+1) fruits
+apple 5
+banana 10
+2) colors
+green
+sky blue
+3) magical beasts
+dragon 3
+unicorn 42
+
+$ tac -brs '^[0-9]' items.txt
+3) magical beasts
+dragon 3
+unicorn 42
+2) colors
+green
+sky blue
+1) fruits
+apple 5
+banana 10
+head and tail
1) Use appropriate commands and shell features to get the output shown below.
$ printf 'carpet\njeep\nbus\n'
+carpet
+jeep
+bus
+
+# use the above 'printf' command for input data
+$ c=$(printf 'carpet\njeep\nbus\n' | head -c3)
+$ echo "$c"
+car
+2) How would you display all the input lines except the first one?
$ printf 'apple\nfig\ncarpet\njeep\nbus\n' | tail -n +2
+fig
+carpet
+jeep
+bus
+3) Which command would you use to get the output shown below?
$ cat fruits.txt
+banana
+papaya
+mango
+$ cat blocks.txt
+----
+apple--banana
+mango---fig
+----
+3.14
+-42
+1000
+----
+sky blue
+dark green
+----
+hi hello
+
+$ head -n2 fruits.txt blocks.txt
+==> fruits.txt <==
+banana
+papaya
+
+==> blocks.txt <==
+----
+apple--banana
+4) Use a combination of head and tail commands to get the 11th to 14th characters from the given input.
# can also use: tail -c +11 | head -c4
+$ printf 'apple\nfig\ncarpet\njeep\nbus\n' | head -c14 | tail -c +11
+carp
+5) Extract the starting six bytes from the input files ip.txt and fruits.txt.
$ head -q -c6 ip.txt fruits.txt
+it is banana
+6) Extract the last six bytes from the input files fruits.txt and ip.txt.
$ tail -q -c6 fruits.txt ip.txt
+mango
+erish
+7) For the input file ip.txt, display except the last 5 lines.
$ head -n -5 ip.txt
+it is a warm and cozy day
+listen to what I say
+go play in the park
+come back before the sky turns dark
+8) Display the third line from the given stdin data. Consider the NUL character as the line separator.
$ printf 'apple\0fig\0carpet\0jeep\0bus\0' | head -z -n3 | tail -z -n1
+carpet
+tr
1) What's wrong with the following command?
$ echo 'apple#banana#cherry' | tr # :
+tr: missing operand
+Try 'tr --help' for more information.
+
+$ echo 'apple#banana#cherry' | tr '#' ':'
+apple:banana:cherry
+As a good practice, always quote the arguments passed to the tr command to avoid conflict with shell metacharacters. Unless of course, you need the shell to interpret them.
2) Retain only alphabets, digits and whitespace characters.
$ printf 'Apple_42 cool,blue\tDragon:army\n' | tr -dc '[:alnum:][:space:]'
+Apple42 coolblue Dragonarmy
+3) Similar to rot13, figure out a way to shift digits such that the same logic can be used both ways.
$ echo '4780 89073' | tr '0-9' '5-90-4'
+9235 34528
+
+$ echo '9235 34528' | tr '0-9' '5-90-4'
+4780 89073
+4) Figure out the logic based on the given input and output data. Hint: use two ranges for the first set and only 6 characters in the second set.
$ echo 'apple banana cherry damson etrog' | tr 'a-ep-z' '12345X'
+1XXl5 21n1n1 3h5XXX 41mXon 5XXog
+5) Which option would you use to truncate the first set so that it matches the length of the second set?
The -t option is needed for this.
6) What does the * notation do in the second set?
The [c*n] notation repeats a character c by n times. You can specify n in decimal or octal formats. If n is omitted, the character c is repeated as many times as needed to equalize the length of the sets.
7) Change : to - and ; to the newline character.
$ echo 'tea:coffee;brown:teal;dragon:unicorn' | tr ':;' '-\n'
+tea-coffee
+brown-teal
+dragon-unicorn
+8) Convert all characters to * except digit and newline characters.
$ echo 'ajsd45_sdg2Khnf4v_54as' | tr -c '0-9\n' '*'
+****45****2****4**54**
+9) Change consecutive repeated punctuation characters to a single punctuation character.
$ echo '""hi..."", good morning!!!!' | tr -s '[:punct:]'
+"hi.", good morning!
+10) Figure out the logic based on the given input and output data.
$ echo 'Aapple noon banana!!!!!' | tr -cs 'a-z\n' ':'
+:apple:noon:banana:
+11) The books.txt file has items separated by one or more : characters. Change this separator to a single newline character as shown below.
$ cat books.txt
+Cradle:::Mage Errant::The Weirkey Chronicles
+Mother of Learning::Eight:::::Dear Spellbook:Ascendant
+Mark of the Fool:Super Powereds:::Ends of Magic
+
+$ <books.txt tr -s ':' '\n'
+Cradle
+Mage Errant
+The Weirkey Chronicles
+Mother of Learning
+Eight
+Dear Spellbook
+Ascendant
+Mark of the Fool
+Super Powereds
+Ends of Magic
+cut
1) Display only the third field.
$ printf 'tea\tcoffee\tchocolate\tfruit\n' | cut -f3
+chocolate
+2) Display the second and fifth fields. Consider , as the field separator.
$ echo 'tea,coffee,chocolate,ice cream,fruit' | cut -d, -f2,5
+coffee,fruit
+3) Why does the below command not work as expected? What other tools can you use in such cases?
cut ignores all repeated fields and the output is always presented in the ascending order.
# not working as expected
+$ echo 'apple,banana,cherry,fig' | cut -d, -f3,1,3
+apple,cherry
+
+# expected output
+$ echo 'apple,banana,cherry,fig' | awk -F, -v OFS=, '{print $3, $1, $3}'
+cherry,apple,cherry
+4) Display except the second field in the format shown below. Can you construct two different solutions?
# solution 1
+$ echo 'apple,banana,cherry,fig' | cut -d, --output-delimiter=' ' -f1,3-
+apple cherry fig
+
+# solution 2
+$ echo '2,3,4,5,6,7,8' | cut -d, --output-delimiter=' ' --complement -f2
+2 4 5 6 7 8
+5) Extract the first three characters from the input lines as shown below. Can you also use the head command for this purpose? If not, why not?
$ printf 'apple\nbanana\ncherry\nfig\n' | cut -c-3
+app
+ban
+che
+fig
+head cannot be used because it acts on the input as a whole, whereas cut works line wise.
6) Display only the first and third fields of the scores.csv input file, with tab as the output field separator.
$ cat scores.csv
+Name,Maths,Physics,Chemistry
+Ith,100,100,100
+Cy,97,98,95
+Lin,78,83,80
+
+$ cut -d, --output-delimiter=$'\t' -f1,3 scores.csv
+Name Physics
+Ith 100
+Cy 98
+Lin 83
+7) The given input data uses one or more : characters as the field separator. Assume that no field content will have the : character. Display except the second field, with : as the output field separator.
$ cat books.txt
+Cradle:::Mage Errant::The Weirkey Chronicles
+Mother of Learning::Eight:::::Dear Spellbook:Ascendant
+Mark of the Fool:Super Powereds:::Ends of Magic
+
+$ <books.txt tr -s ':' | cut --complement -d':' --output-delimiter=' : ' -f2
+Cradle : The Weirkey Chronicles
+Mother of Learning : Dear Spellbook : Ascendant
+Mark of the Fool : Ends of Magic
+8) Which option would you use to not display lines that do not contain the input delimiter character?
You can use the -s option to suppress such lines.
9) Modify the command to get the expected output shown below.
$ printf 'apple\nbanana\ncherry\n' | cut -c-3 --output-delimiter=:
+app
+ban
+che
+
+$ printf 'apple\nbanana\ncherry\n' | cut -c1,2,3 --output-delimiter=:
+a:p:p
+b:a:n
+c:h:e
+10) Figure out the logic based on the given input and output data.
$ printf 'apple\0fig\0carpet\0jeep\0' | cut -z --complement -c-2 | cat -v
+ple^@g^@rpet^@ep^@
+seq
1) Generate numbers from 42 to 45 in ascending order.
$ seq 42 45
+42
+43
+44
+45
+2) Why does the command shown below produce no output?
You have to explicitly provide a negative step value to generate numbers in descending order.
# no output
+$ seq 45 42
+
+# expected output
+$ seq 45 -1 42
+45
+44
+43
+42
+3) Generate numbers from 25 to 10 in descending order, with a step value of 5.
$ seq 25 -5 10
+25
+20
+15
+10
+4) Is the sequence shown below possible to generate with seq? If so, how?
$ seq -w -s, 01.5 6
+01.5,02.5,03.5,04.5,05.5
+5) Modify the command shown below to customize the output numbering format.
$ seq 30.14 3.36 40.72
+30.14
+33.50
+36.86
+40.22
+
+$ seq -f'%.3e' 30.14 3.36 40.72
+3.014e+01
+3.350e+01
+3.686e+01
+4.022e+01
+shuf
1) What's wrong with the given command?
shuf doesn't accept multiple input files. You can use cat to concatenate them first.
$ shuf --random-source=greeting.txt fruits.txt books.txt
+shuf: extra operand ‘books.txt’
+Try 'shuf --help' for more information.
+
+# expected output
+$ cat fruits.txt books.txt | shuf --random-source=greeting.txt
+banana
+Cradle:::Mage Errant::The Weirkey Chronicles
+Mother of Learning::Eight:::::Dear Spellbook:Ascendant
+papaya
+Mark of the Fool:Super Powereds:::Ends of Magic
+mango
+2) What do the -r and -n options do? Why are they often used together?
The -r option helps if you want to allow input lines to be repeated. This option is usually paired with -n to limit the number of lines in the output. Otherwise, shuf -r will produce output lines indefinitely.
3) What does the following command do?
The -e option is useful to specify multiple input lines as arguments to the command.
$ shuf -e apple banana cherry fig mango
+cherry
+banana
+mango
+fig
+apple
+4) Which option would you use to generate random numbers? Given an example.
The -i option helps generate random positive integers.
$ shuf -n3 -i 100-200
+128
+177
+193
+5) How would you generate 5 random numbers between 0.125 and 0.789 with a step value of 0.023?
$ seq 0.125 0.023 0.789 | shuf -n5
+0.378
+0.631
+0.447
+0.746
+0.723
+paste
1) What's the default delimiter character added by the paste command? Which option would you use to customize this separator?
Tab. You can use the -d option to change the delimiter between the columns.
2) Will the following two commands produce equivalent output? If not, why not?
$ paste -d, <(seq 3) <(printf '%s\n' item_{1..3})
+1,item_1
+2,item_2
+3,item_3
+
+$ printf '%s\n' {1..3},item_{1..3}
+1,item_1
+1,item_2
+1,item_3
+2,item_1
+2,item_2
+2,item_3
+3,item_1
+3,item_2
+3,item_3
+The outputs are not equivalent because brace expansion creates all combinations when multiple braces are used.
3) Combine the two data sources as shown below.
$ printf '1)\n2)\n3)'
+1)
+2)
+3)
+$ cat fruits.txt
+banana
+papaya
+mango
+
+$ paste -d '' <(printf '1)\n2)\n3)') fruits.txt
+1)banana
+2)papaya
+3)mango
+4) Interleave the contents of fruits.txt and books.txt.
$ paste -d'\n' fruits.txt books.txt
+banana
+Cradle:::Mage Errant::The Weirkey Chronicles
+papaya
+Mother of Learning::Eight:::::Dear Spellbook:Ascendant
+mango
+Mark of the Fool:Super Powereds:::Ends of Magic
+5) Generate numbers 1 to 9 in two different formats as shown below.
$ seq 9 | paste -d: - - -
+1:2:3
+4:5:6
+7:8:9
+
+$ paste -d' : ' <(seq 3) /dev/null /dev/null <(seq 4 6) /dev/null /dev/null <(seq 7 9)
+1 : 4 : 7
+2 : 5 : 8
+3 : 6 : 9
+6) Combine the contents of fruits.txt and colors.txt as shown below.
$ cat fruits.txt
+banana
+papaya
+mango
+$ cat colors.txt
+deep blue
+light orange
+blue delight
+
+$ paste -d'\n' fruits.txt colors.txt | paste -sd,
+banana,deep blue,papaya,light orange,mango,blue delight
+pr
1) What does the -t option do?
The -t option turns off the pagination features like headers and trailers.
2) Generate numbers 1 to 16 in two different formats as shown below.
$ seq -w 16 | pr -4ats,
+01,02,03,04
+05,06,07,08
+09,10,11,12
+13,14,15,16
+
+$ seq -w 16 | pr -4ts,
+01,05,09,13
+02,06,10,14
+03,07,11,15
+04,08,12,16
+3) How'd you solve the issue shown below?
$ seq 100 | pr -37ats,
+pr: page width too narrow
+
+$ seq 100 | pr -J -w73 -37ats,
+1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37
+38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74
+75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100
+(N-1)*length(separator) + N is the minimum page width you need, where N is the number of columns required. So, for 37 columns and a separator of length 1, you'll need a minimum width of 73. -J option ensures input lines aren't truncated.
4) Combine the contents of fruits.txt and colors.txt in two different formats as shown below.
$ cat fruits.txt
+banana
+papaya
+mango
+$ cat colors.txt
+deep blue
+light orange
+blue delight
+
+$ pr -mts' : ' fruits.txt colors.txt
+banana : deep blue
+papaya : light orange
+mango : blue delight
+
+$ pr -n:2 -mts, fruits.txt colors.txt
+ 1:banana,deep blue
+ 2:papaya,light orange
+ 3:mango,blue delight
+5) What does the -d option do?
You can use the -d option to double space the input contents. That is, every newline character is doubled.
fold and fmt
1) What's the default wrap length of the fold and fmt commands?
80 bytes and 93% of 75 columns respectively.
2) Fold the given stdin data at 9 bytes.
$ echo 'hi hello, how are you?' | fold -w9
+hi hello,
+ how are
+you?
+3) Figure out the logic based on the given input and output data using the fold command.
$ cat ip.txt
+it is a warm and cozy day
+listen to what I say
+go play in the park
+come back before the sky turns dark
+
+There are so many delights to cherish
+Apple, Banana and Cherry
+Bread, Butter and Jelly
+Try them all before you perish
+
+$ head -n2 ip.txt | fold -sw10
+it is a
+warm and
+cozy day
+listen to
+what I say
+4) What does the fold -b option do?
The -b option will cause fold to treat tab, backspace, and carriage return characters as if they were a single byte character.
5) How'd you get the expected output shown below?
# wrong output
+$ echo 'fig appleseed mango pomegranate' | fold -sw7
+fig
+applese
+ed
+mango
+pomegra
+nate
+
+# expected output
+$ echo 'fig appleseed mango pomegranate' | fmt -w7
+fig
+appleseed
+mango
+pomegranate
+6) What do the options -s and -u of the fmt command do?
By default, the fmt command joins lines together that are shorter than the specified width. The -s option will disable this behavior.
The -u option changes multiple spaces to a single space. Excess spacing between sentences will be changed to two spaces.
sort
1) Default sort doesn't work for numbers. Which option would you use to get the expected output shown below?
$ printf '100\n10\n20\n3000\n2.45\n' | sort -n
+2.45
+10
+20
+100
+3000
+2) Which sort option will help you ignore case? LC_ALL=C is used here to avoid differences due to locale.
$ printf 'Super\nover\nRUNE\ntea\n' | LC_ALL=C sort -f
+over
+RUNE
+Super
+tea
+3) The -n option doesn't work for all sorts of numbers. Which sort option would you use to get the expected output shown below?
# wrong output
+$ printf '+120\n-1.53\n3.14e+4\n42.1e-2' | sort -n
+-1.53
++120
+3.14e+4
+42.1e-2
+
+# expected output
+$ printf '+120\n-1.53\n3.14e+4\n42.1e-2' | sort -g
+-1.53
+42.1e-2
++120
+3.14e+4
+4) What do the -V and -h options do?
The -V option is useful when you have a mix of alphabets and digits. It also helps when you want to treat digits after a decimal point as whole numbers, for example 1.10 should be greater than 1.2.
Commands like du (disk usage) have the -h and --si options to display numbers with SI suffixes like k, K, M, G and so on. In such cases, you can use sort -h to order them.
5) Is there a difference between shuf and sort -R?
The sort -R option will display the output in random order. Unlike shuf, this option will always place identical lines next to each other due to the implementation.
6) Sort the scores.csv file numerically in ascending order using the contents of the second field. Header line should be preserved as the first line as shown below.
$ cat scores.csv
+Name,Maths,Physics,Chemistry
+Ith,100,100,100
+Cy,97,98,95
+Lin,78,83,80
+
+$ (sed -u '1q' ; sort -t, -k2,2n) < scores.csv
+Name,Maths,Physics,Chemistry
+Lin,78,83,80
+Cy,97,98,95
+Ith,100,100,100
+7) Sort the contents of duplicates.csv by the fourth column numbers in descending order. Retain only the first copy of lines with the same number.
$ cat duplicates.csv
+brown,toy,bread,42
+dark red,ruby,rose,111
+blue,ruby,water,333
+dark red,sky,rose,555
+yellow,toy,flower,333
+white,sky,bread,111
+light red,purse,rose,333
+
+$ sort -t, -k4,4nr -u duplicates.csv
+dark red,sky,rose,555
+blue,ruby,water,333
+dark red,ruby,rose,111
+brown,toy,bread,42
+8) Sort the contents of duplicates.csv by the third column item. Use the fourth column numbers as the tie-breaker.
$ sort -t, -k3,3 -k4,4n duplicates.csv
+brown,toy,bread,42
+white,sky,bread,111
+yellow,toy,flower,333
+dark red,ruby,rose,111
+light red,purse,rose,333
+dark red,sky,rose,555
+blue,ruby,water,333
+9) What does the -s option provide?
The -s option is useful to retain the original order of input lines when two or more lines are deemed equal.
10) Sort the given input based on the numbers inside the brackets.
$ printf '(-3.14)\n[45]\n(12.5)\n{14093}' | sort -k1.2n
+(-3.14)
+(12.5)
+[45]
+{14093}
+11) What do the -c, -C and -m options do?
The -c option helps you spot the first unsorted entry in the given input. The uppercase -C option is similar but only affects the exit status. Note that these options will not work for multiple inputs.
The -m option is useful if you have one or more sorted input files and need a single sorted output file. This would be faster than normal sorting.
uniq
1) Will uniq throw an error if the input is not sorted? What do you think will be the output for the following input?
uniq doesn't necessarily require the input to be sorted. Adjacent lines are used for comparison purposes.
$ printf 'red\nred\nred\ngreen\nred\nblue\nblue' | uniq
+red
+green
+red
+blue
+2) Are there differences between sort -u file and sort file | uniq?
Yes. For example, you may need to sort based on some specific criteria and then identify duplicates based on the entire line contents. Here's an example:
# can't use sort -n -u here
+$ printf '2 balls\n13 pens\n2 pins\n13 pens\n' | sort -n | uniq
+2 balls
+2 pins
+13 pens
+3) What are the differences between sort -u and uniq -u options, if any?
sort -u retains the first copy of duplicates that are deemed to be equal. uniq -u retains only the unique copies (i.e. not even a single copy of the duplicates will be part of the output).
4) Filter the third column items from duplicates.csv. Construct three solutions to display only unique items, duplicate items and all duplicates.
$ cat duplicates.csv
+brown,toy,bread,42
+dark red,ruby,rose,111
+blue,ruby,water,333
+dark red,sky,rose,555
+yellow,toy,flower,333
+white,sky,bread,111
+light red,purse,rose,333
+
+# unique
+$ cut -d, -f3 duplicates.csv | sort | uniq -u
+flower
+water
+
+# duplicates
+$ cut -d, -f3 duplicates.csv | sort | uniq -d
+bread
+rose
+
+# all duplicates
+$ cut -d, -f3 duplicates.csv | sort | uniq -D
+bread
+bread
+rose
+rose
+rose
+5) What does the --group option do? What customization features are available?
The --group options allows you to visually separate groups of similar lines with an empty line. This option can accept four values — separate, prepend, append and both. The default is separate, which adds a newline character between the groups. prepend will add a newline before the first group as well and append will add a newline after the last group. both combines the prepend and append behavior.
6) Count the number of times input lines are repeated and display the results in the format shown below.
$ s='brown\nbrown\nbrown\ngreen\nbrown\nblue\nblue'
+$ printf '%b' "$s" | sort | uniq -c | sort -n
+ 1 green
+ 2 blue
+ 4 brown
+7) For the input file f1.txt, retain only unique entries based on the first two characters of each line. For example, abcd and ab12 should be considered as duplicates and neither of them will be part of the output.
$ cat f1.txt
+3) cherry
+1) apple
+2) banana
+1) almond
+4) mango
+2) berry
+3) chocolate
+1) apple
+5) cherry
+
+$ sort f1.txt | uniq -u -w2
+4) mango
+5) cherry
+8) For the input file f1.txt, display only the duplicate items without considering the first two characters of each line. For example, abcd and 12cd should be considered as duplicates. Assume that the third character of each line is always a space character.
$ sort -k2 f1.txt | uniq -d -f1
+1) apple
+3) cherry
+9) What does the -s option do?
The -s option allows you to skip the first N characters (calculated as bytes).
10) Filter only unique lines, but ignore differences due to case.
$ printf 'cat\nbat\nCAT\nCar\nBat\nmat\nMat' | sort -f | uniq -iu
+Car
+comm
1) Get the common lines between the s1.txt and s2.txt files. Assume that their contents are already sorted.
$ paste s1.txt s2.txt
+apple banana
+coffee coffee
+fig eclair
+honey fig
+mango honey
+pasta milk
+sugar tea
+tea yeast
+
+$ comm -12 s1.txt s2.txt
+coffee
+fig
+honey
+tea
+2) Display lines present in s1.txt but not s2.txt and vice versa.
# lines unique to the first file
+$ comm -23 s1.txt s2.txt
+apple
+mango
+pasta
+sugar
+
+# lines unique to the second file
+$ comm -13 s1.txt s2.txt
+banana
+eclair
+milk
+yeast
+3) Display lines unique to the s1.txt file and the common lines when compared to the s2.txt file. Use ==> to separate the output columns.
$ comm -2 --output-delimiter='==>' s1.txt s2.txt
+apple
+==>coffee
+==>fig
+==>honey
+mango
+pasta
+sugar
+==>tea
+4) What does the --total option do?
Gives you the count of lines for each of the three columns.
5) Will the comm command fail if there are repeated lines in the input files? If not, what'd be the expected output for the command shown below?
The number of duplicate lines in the common column will be minimum of the duplicate occurrences between the two files. Rest of the duplicate lines, if any, will be considered as unique to the file having the excess lines.
$ cat s3.txt
+apple
+apple
+guava
+honey
+tea
+tea
+tea
+
+$ comm -23 s3.txt s1.txt
+apple
+guava
+tea
+tea
+join
Assume that the input files are already sorted for these exercises.
1) Use appropriate options to get the expected outputs shown below.
# no output
+$ join <(printf 'apple 2\nfig 5') <(printf 'Fig 10\nmango 4')
+
+# expected output 1
+$ join -i <(printf 'apple 2\nfig 5') <(printf 'Fig 10\nmango 4')
+fig 5 10
+
+# expected output 2
+$ join -i -a1 -a2 <(printf 'apple 2\nfig 5') <(printf 'Fig 10\nmango 4')
+apple 2
+fig 5 10
+mango 4
+2) Use the join command to display only the non-matching lines based on the first field.
$ cat j1.txt
+apple 2
+fig 5
+lemon 10
+tomato 22
+$ cat j2.txt
+almond 33
+fig 115
+mango 20
+pista 42
+
+# first field items present in j1.txt but not j2.txt
+$ join -v1 j1.txt j2.txt
+apple 2
+lemon 10
+tomato 22
+
+# first field items present in j2.txt but not j1.txt
+$ join -v2 j1.txt j2.txt
+almond 33
+mango 20
+pista 42
+3) Filter lines from j1.txt and j2.txt that match the items from s1.txt.
$ cat s1.txt
+apple
+coffee
+fig
+honey
+mango
+pasta
+sugar
+tea
+
+# note that sort -m is used since the input files are already sorted
+$ join s1.txt <(sort -m j1.txt j2.txt)
+apple 2
+fig 115
+fig 5
+mango 20
+4) Join the marks_1.csv and marks_2.csv files to get the expected output shown below.
$ cat marks_1.csv
+Name,Biology,Programming
+Er,92,77
+Ith,100,100
+Lin,92,100
+Sil,86,98
+$ cat marks_2.csv
+Name,Maths,Physics,Chemistry
+Cy,97,98,95
+Ith,100,100,100
+Lin,78,83,80
+
+$ join -t, --header marks_1.csv marks_2.csv
+Name,Biology,Programming,Maths,Physics,Chemistry
+Ith,100,100,100,100,100
+Lin,92,100,78,83,80
+5) By default, the first field is used to combine the lines. Which options are helpful if you want to change the key field to be used for joining?
You can use -1 and -2 options followed by a field number to specify a different field number. You can use the -j option if the field number is the same for both the files.
6) Join the marks_1.csv and marks_2.csv files to get the expected output with specific fields as shown below.
$ join -t, --header -o 1.1,1.3,2.2,1.2 marks_1.csv marks_2.csv
+Name,Programming,Maths,Biology
+Ith,100,100,100
+Lin,100,78,92
+7) Join the marks_1.csv and marks_2.csv files to get the expected output shown below. Use 50 as the filler data.
$ join -t, --header -o auto -a1 -a2 -e '50' marks_1.csv marks_2.csv
+Name,Biology,Programming,Maths,Physics,Chemistry
+Cy,50,50,97,98,95
+Er,92,77,50,50,50
+Ith,100,100,100,100,100
+Lin,92,100,78,83,80
+Sil,86,98,50,50,50
+8) When you use the -o auto option, what'd happen to the extra fields compared to those in the first lines of the input data?
If you use auto as the argument for the -o option, first line of both the input files will be used to determine the number of output fields. If the other lines have extra fields, they will be discarded.
9) From the input files j3.txt and j4.txt, filter only the lines are unique — i.e. lines that are not common to these files. Assume that the input files do not have duplicate entries.
$ cat j3.txt
+almond
+apple pie
+cold coffee
+honey
+mango shake
+pasta
+sugar
+tea
+$ cat j4.txt
+apple
+banana shake
+coffee
+fig
+honey
+mango shake
+milk
+tea
+yeast
+
+$ join -t '' -v1 -v2 j3.txt j4.txt
+almond
+apple
+apple pie
+banana shake
+coffee
+cold coffee
+fig
+milk
+pasta
+sugar
+yeast
+10) From the input files j3.txt and j4.txt, filter only the lines are common to these files.
$ join -t '' j3.txt j4.txt
+honey
+mango shake
+tea
+nl
1) nl and cat -n are always equivalent for numbering lines. True or False?
True if there are no empty lines in the input data. cat -b and nl are always equivalent.
2) What does the -n option do?
You can use the -n option to customize the number formatting. The available styles are:
rnright justified with space fillers (default)rzright justified with leading zeroslnleft justified with space fillers
3) Use nl to produce the two expected outputs shown below.
$ cat greeting.txt
+Hi there
+Have a nice day
+
+# expected output 1
+$ nl -w3 -n'rz' greeting.txt
+001 Hi there
+002 Have a nice day
+
+# expected output 2
+$ nl -w3 -n'rz' -s') ' greeting.txt
+001) Hi there
+002) Have a nice day
+4) Figure out the logic based on the given input and output data.
$ cat s1.txt
+apple
+coffee
+fig
+honey
+mango
+pasta
+sugar
+tea
+
+$ nl -w2 -s'. ' -v15 -i-2 s1.txt
+15. apple
+13. coffee
+11. fig
+ 9. honey
+ 7. mango
+ 5. pasta
+ 3. sugar
+ 1. tea
+5) What are the three types of sections supported by nl?
nl recognizes three types of sections with the following default patterns:
\:\:\:as header\:\:as body\:as footer
These special lines will be replaced with an empty line after numbering. The numbering will be reset at the start of every section unless the -p option is used.
6) Only number the lines that start with ---- in the format shown below.
$ cat blocks.txt
+----
+apple--banana
+mango---fig
+----
+3.14
+-42
+1000
+----
+sky blue
+dark green
+----
+hi hello
+
+$ nl -w2 -s') ' -bp'^----' blocks.txt
+ 1) ----
+ apple--banana
+ mango---fig
+ 2) ----
+ 3.14
+ -42
+ 1000
+ 3) ----
+ sky blue
+ dark green
+ 4) ----
+ hi hello
+7) For the blocks.txt file, determine the logic to produce the expected output shown below.
$ nl -w1 -s'. ' -d'--' blocks.txt
+
+1. apple--banana
+2. mango---fig
+
+1. 3.14
+2. -42
+3. 1000
+
+1. sky blue
+2. dark green
+
+1. hi hello
+8) What does the -l option do?
The -l option controls how many consecutive empty lines should be considered as a single entry. Only the last empty line of such groupings will be numbered.
9) Figure out the logic based on the given input and output data.
$ cat all_sections.txt
+\:\:\:
+Header
+teal
+\:\:
+Hi there
+How are you
+\:\:
+banana
+papaya
+mango
+\:
+Footer
+
+$ nl -p -w2 -s') ' -ha all_sections.txt
+
+ 1) Header
+ 2) teal
+
+ 3) Hi there
+ 4) How are you
+
+ 5) banana
+ 6) papaya
+ 7) mango
+
+ Footer
+wc
1) Save the number of lines in the greeting.txt input file to the lines shell variable.
$ lines=$(wc -l <greeting.txt)
+$ echo "$lines"
+2
+2) What do you think will be the output of the following command?
Word count is based on whitespace separation.
$ echo 'dragons:2 ; unicorns:10' | wc -w
+3
+3) Use appropriate options and arguments to get the output as shown below. Also, why is the line count showing as 2 instead of 3 for the stdin data?
Line count is based on the number of newline characters. So, if the last line of the input doesn't end with the newline character, it won't be counted.
$ printf 'apple\nbanana\ncherry' | wc -lc greeting.txt -
+ 2 25 greeting.txt
+ 2 19 -
+ 4 44 total
+4) Use appropriate options and arguments to get the output shown below.
$ printf 'greeting.txt\0scores.csv' | wc --files0-from=-
+2 6 25 greeting.txt
+4 4 70 scores.csv
+6 10 95 total
+5) What is the difference between wc -c and wc -m options? And which option would you use to get the longest line length?
The -c option is useful to get the byte count. Use the -m option instead of -c if the input has multibyte characters.
# byte count
+$ printf 'αλεπού' | wc -c
+12
+
+# character count
+$ printf 'αλεπού' | wc -m
+6
+You can use the -L option to report the length of the longest line in the input (excluding the newline character of a line).
6) Calculate the number of comma separated words from the scores.csv file.
$ cat scores.csv
+Name,Maths,Physics,Chemistry
+Ith,100,100,100
+Cy,97,98,95
+Lin,78,83,80
+
+$ <scores.csv tr ',' ' ' | wc -w
+16
+split
1) Split the s1.txt file 3 lines at a time.
$ split -l3 s1.txt
+
+$ head xa?
+==> xaa <==
+apple
+coffee
+fig
+
+==> xab <==
+honey
+mango
+pasta
+
+==> xac <==
+sugar
+tea
+
+$ rm xa?
+2) Use appropriate options to get the output shown below.
$ echo 'apple,banana,cherry,dates' | split -t, -l1
+
+$ head xa?
+==> xaa <==
+apple,
+==> xab <==
+banana,
+==> xac <==
+cherry,
+==> xad <==
+dates
+
+$ rm xa?
+3) What do the -b and -C options do?
The -b option allows you to split the input by the number of bytes. This option also accepts suffixes such as K for 1024 bytes, KB for 1000 bytes, M for 1024 * 1024 bytes and so on.
The -C option is similar to the -b option, but it will try to break on line boundaries if possible. The break will happen before the given byte limit. If a line exceeds the given limit, it will be broken down into multiple parts.
4) Display the 2nd chunk of the ip.txt file after splitting it 4 times as shown below.
$ split -nl/2/4 ip.txt
+come back before the sky turns dark
+
+There are so many delights to cherish
+5) What does the r prefix do when used with the -n option?
This creates output files with interleaved lines.
6) Split the ip.txt file 2 lines at a time. Customize the output filenames as shown below.
$ split -l2 -a1 -d --additional-suffix='.txt' ip.txt ip_
+
+$ head ip_*
+==> ip_0.txt <==
+it is a warm and cozy day
+listen to what I say
+
+==> ip_1.txt <==
+go play in the park
+come back before the sky turns dark
+
+==> ip_2.txt <==
+
+There are so many delights to cherish
+
+==> ip_3.txt <==
+Apple, Banana and Cherry
+Bread, Butter and Jelly
+
+==> ip_4.txt <==
+Try them all before you perish
+
+$ rm ip_*
+7) Which option would you use to prevent empty files in the output?
The -e option prevents empty files in the output.
8) Split the items.txt file 5 lines at a time. Additionally, remove lines starting with a digit character as shown below.
$ cat items.txt
+1) fruits
+apple 5
+banana 10
+2) colors
+green
+sky blue
+3) magical beasts
+dragon 3
+unicorn 42
+
+$ split -l5 --filter='grep -v "^[0-9]" > $FILE' items.txt
+
+$ head xa?
+==> xaa <==
+apple 5
+banana 10
+green
+
+==> xab <==
+sky blue
+dragon 3
+unicorn 42
+
+$ rm xa?
+csplit
1) Split the blocks.txt file such that the first 7 lines are in the first file and the rest are in the second file as shown below.
$ csplit -q blocks.txt 8
+
+$ head xx*
+==> xx00 <==
+----
+apple--banana
+mango---fig
+----
+3.14
+-42
+1000
+
+==> xx01 <==
+----
+sky blue
+dark green
+----
+hi hello
+
+$ rm xx*
+2) Split the input file items.txt such that the text before a line containing colors is part of the first file and the rest are part of the second file as shown below.
$ csplit -q items.txt '/colors/'
+
+$ head xx*
+==> xx00 <==
+1) fruits
+apple 5
+banana 10
+
+==> xx01 <==
+2) colors
+green
+sky blue
+3) magical beasts
+dragon 3
+unicorn 42
+
+$ rm xx*
+3) Split the input file items.txt such that the line containing magical and all the lines that come after are part of the single output file.
$ csplit -q items.txt '%magical%'
+
+$ cat xx00
+3) magical beasts
+dragon 3
+unicorn 42
+
+$ rm xx00
+4) Split the input file items.txt such that the line containing colors as well the line that comes after are part of the first output file.
$ csplit -q items.txt '/colors/2'
+
+$ head xx*
+==> xx00 <==
+1) fruits
+apple 5
+banana 10
+2) colors
+green
+
+==> xx01 <==
+sky blue
+3) magical beasts
+dragon 3
+unicorn 42
+
+$ rm xx*
+5) Split the input file items.txt on the line that comes before a line containing magical. Generate only a single output file as shown below.
$ csplit -q items.txt '%magical%-1'
+
+$ cat xx00
+sky blue
+3) magical beasts
+dragon 3
+unicorn 42
+
+$ rm xx00
+6) Split the input file blocks.txt on the 4th occurrence of a line starting with the - character. Generate only a single output file as shown below.
$ csplit -q blocks.txt '%^-%' '{3}'
+
+$ cat xx00
+----
+sky blue
+dark green
+----
+hi hello
+
+$ rm xx00
+7) For the input file blocks.txt, determine the logic to produce the expected output shown below.
$ csplit -qz --suppress-matched blocks.txt '/----/' '{*}'
+
+$ head xx*
+==> xx00 <==
+apple--banana
+mango---fig
+
+==> xx01 <==
+3.14
+-42
+1000
+
+==> xx02 <==
+sky blue
+dark green
+
+==> xx03 <==
+hi hello
+
+$ rm xx*
+8) What does the -k option do?
By default, csplit will remove the created output files if there's an error or a signal that causes the command to stop. You can use the -k option to keep such files. One use case is line number based splitting with the {*} modifier.
$ seq 7 | csplit -q - 4 '{*}'
+csplit: ‘4’: line number out of range on repetition 1
+$ ls xx*
+ls: cannot access 'xx*': No such file or directory
+
+# -k option will allow you to retain the created files
+$ seq 7 | csplit -qk - 4 '{*}'
+csplit: ‘4’: line number out of range on repetition 1
+$ head xx*
+==> xx00 <==
+1
+2
+3
+
+==> xx01 <==
+4
+5
+6
+7
+
+$ rm xx*
+9) Split the books.txt file on every line as shown below.
# can also use: split -l1 -d -a1 books.txt row_
+$ csplit -qkz -f'row_' -n1 books.txt 1 '{*}'
+csplit: ‘1’: line number out of range on repetition 3
+
+$ head row_*
+==> row_0 <==
+Cradle:::Mage Errant::The Weirkey Chronicles
+
+==> row_1 <==
+Mother of Learning::Eight:::::Dear Spellbook:Ascendant
+
+==> row_2 <==
+Mark of the Fool:Super Powereds:::Ends of Magic
+
+$ rm row_*
+10) Split the items.txt file on lines starting with a digit character. Matching lines shouldn't be part of the output and the files should be named group_0.txt, group_1.txt and so on.
$ csplit -qz --suppress-matched -q -f'group_' -b'%d.txt' items.txt '/^[0-9]/' '{*}'
+
+$ head group_*
+==> group_0.txt <==
+apple 5
+banana 10
+
+==> group_1.txt <==
+green
+sky blue
+
+==> group_2.txt <==
+dragon 3
+unicorn 42
+
+$ rm group_*
+expand and unexpand
1) The items.txt file has space separated words. Convert the spaces to be aligned at 10 column widths as shown below.
$ cat items.txt
+1) fruits
+apple 5
+banana 10
+2) colors
+green
+sky blue
+3) magical beasts
+dragon 3
+unicorn 42
+
+$ <items.txt tr ' ' '\t' | expand -t 10
+1) fruits
+apple 5
+banana 10
+2) colors
+green
+sky blue
+3) magical beasts
+dragon 3
+unicorn 42
+2) What does the expand -i option do?
The -i option converts only the tab characters present at the start of a line. The first occurrence of a character that is not tab or space characters will stop the expansion.
3) Expand the first tab character to stop at the 10th column and the second one at the 16th column. Rest of the tabs should be converted to a single space character.
$ printf 'app\tfix\tjoy\tmap\ttap\n' | expand -t 10,16
+app fix joy map tap
+
+$ printf 'appleseed\tfig\tjoy\n' | expand -t 10,16
+appleseed fig joy
+
+$ printf 'a\tb\tc\td\te\n' | expand -t 10,16
+a b c d e
+4) Will the following code give back the original input? If not, is there an option that can help?
By default, the unexpand command converts only the initial blank characters (space or tab) to tabs. The -a option will allow you to convert all sequences of two or more blanks at tab boundaries.
$ printf 'a\tb\tc\n' | expand | unexpand
+a b c
+
+$ printf 'a\tb\tc\n' | expand | unexpand | cat -T
+a b c
+
+$ printf 'a\tb\tc\n' | expand | unexpand -a | cat -T
+a^Ib^Ic
+5) How do the + and / prefix modifiers affect the -t option?
If you prefix a / character to the last width, the remaining tab characters will use multiple of this position instead of a single space default.
If you use + instead of / as the prefix for the last width, the multiple calculation will use the second last width as an offset.
basename and dirname
1) Is the following command valid? If so, what would be the output?
Yes, it is valid. Multiple slashes will be considered as a single slash. Any trailing slashes will be removed before determining the portion to be extracted.
$ basename -s.txt ~///test.txt///
+test
+2) Given the file path in the shell variable p, how'd you obtain the outputs shown below?
$ p='~/projects/square_tictactoe/python/game.py'
+$ dirname $(dirname "$p")
+~/projects/square_tictactoe
+
+$ p='/backups/jan_2021.tar.gz'
+$ dirname $(dirname "$p")
+/
+3) What would be the output of the basename command if the input has no leading directory component or only has the / character?
If there's no leading directory component or if slash alone is the input, the argument will be returned as is after removing any trailing slashes.
$ basename filename.txt
+filename.txt
+$ basename /
+/
+4) For the paths stored in the shell variable p, how'd you obtain the outputs shown below?
$ p='/a/b/ip.txt /c/d/e/f/op.txt'
+
+# expected output 1
+$ basename -s'.txt' $p
+ip
+op
+
+# expected output 2
+$ dirname $p
+/a/b
+/c/d/e/f
+5) Given the file path in the shell variable p, how'd you obtain the outputs shown below?
$ p='~/projects/python/square_tictactoe/game.py'
+$ basename $(dirname "$p")
+square_tictactoe
+
+$ p='/backups/aug_2024/ip.tar.gz'
+$ basename $(dirname "$p")
+aug_2024
+
If the last input line doesn't end with a newline, the output will also not have that newline character.
All the exercises are also collated together in one place at 
