CMD List [10]

1.1         sed

A stream editor.

 

1)      The sed command includes many features for selecting lines to be modified and making changes only to the selected lines.

2)      save sed output. e.g. sed ‘some-sed-commands’ input-file > outfile

3)      Regular Expressions refer Regular Expressions.

 

Option

-e	Only print line number.
-n	Not print line. Default print all lines (include edited and not edited)
-p	Only print edited line.
-f	Read from file rather than from command line. Use sed script. e.g. sed -f myscript.sed input_file

 

Pattern

n	n is line number, search line n.
n,m	From line n to m, e.g. 2,5 represent from line 2 to line 5
/pattern/	Search pattern
/pattern1/pattern2/augu.	Search both 2 patterns & - remember the match pattern. E.g. sed –n ‘s/nurse/”hello” &/p’ quote.txt. # origin: … nurse…, result: … hello nurse … augu. g – Global replacement, e.g. sed ‘s/\$//g’ quote.txt
/pattern/,n	Search pattern in line n
n,/pattern/	As above.
n,m!	Not search line n and m.
p	Print match line.
=	Display line number
d	Remove line
c\	Replace text.
s	Replace mode.
r	Read text from another file. E.g. sed ‘/company./r file.txt’ quote.txt
w	Write file
q	Exit pattern if pattern matched
n	Concatenate.
$	Last line.

 

e.g.

sed  "s/happy/enchanted/g" chap1  >chap1.new

Replaces each occurrence of the word happy found in the file chap1 with the word enchanted. Without the g character, replaces only the first occurrence of the word happy on a line.

 

sed –n ‘/\$/’p quote.txt  # \$ stands for $

sed –n ‘1,$’p quote.txt  # print full file

sed 's/.txt/.id/' quote.txt  # replace .txt by .id

grep xxx /etc/passwd | sed -e ‘s/Taylor/Tailor/g;s/:/ /g’

cat xxx | head -15 | sed ‘/eiyo/,/rpm/d’

sed 's;INSERT INTO;TERMINATE INTO;'

 

1.2         awk

More powerful than sed, a real text programming kit – awk.

Finds lines in files that match a pattern and performs specified actions on those lines.

nawk – support more which written by original author;

gawk – support more which created by free software foundation GNU (may not exist in some system).

 

Option

-f	Read from file rather than from command line.
-Fc	c as separator between fields (default space)

 

Pattern

awk ‘{command}’	Command format.
$n	Field n. ($0 is the entire line)
“xxx”	The constant.
NF	how many fields are on a line (for the blank field cases)
$NF	always the value of the last field on the line (for the blank field cases)
NR	Record count.
END	When pointer at the end of records.

 

Command

print

Display all lines.

 

Symbol

= =+ -= *= /= %= ^=	Assignment.
?:	C conditional expression (nawk and gawk).
||	Logical OR.
&&	Logical AND.
~ !~	Match regular expression and negation.
< <= > >= != ==	Relational operators.
(blank)	Concatenation.
+ -	Addition, subtraction.
* / %	Multiplication, division, and modulus.
+ - !	Unary plus and minus, and logical negation.
^	Exponentiation.
++ --	Increment and decrement, either prefix or postfix.
$	Field reference.

 

String Function (awk)

Option or argument	Function
gsub(r,s,t)	For nawk. Globally substitutes s for each match of the regular expression r in the string t. Returns the number of substitutions. If t is not supplied, defaults to $0.
index(s,t)	Returns position of substring t in string s or zero if not present. (offset start from 1)
length(s)	Returns length of string s or length of $0 if no string is supplied.
match(s,r)	For nawk. Returns either the position in s where the regular expression r begins, or 0 if no occurrences are found. Sets the values of RSTART and RLENGTH.
split(s,a,sep)	Parses string s into elements of array a using field separator sep; returns number of elements. If sep is not supplied, FS is used. Array splitting works the same way as field splitting. (array start from [1])
sprintf("fmt",expr)	Uses printf format specification for expr. e.g. sprintf(“%c”,i)
sub(r,s,t)	For nawk. Substitutes s for first match of the regular expression r in the string t. Returns 1 if successful; 0 otherwise. If t is not supplied, defaults to $0.
substr(s,p) substr(s,p,n)	Returns substring of string s at beginning position p up to a maximum length of n. If n is not supplied, the rest of the string from p is used.
tolower(s)	For gawk. Translates all uppercase characters in string s to lowercase and returns the new string.
toupper(s)	For gawk. Translates all lowercase characters in string s to uppercase and returns the new string.