Very useful handy 1 liners needed to be successful, good Administrator. Its very handy to troubleshoot issue Smartly.
Sed (stream editor): Sed id a stream editor is used to perform basic text transformations on an input stream (a file or input from a pipeline).
AWK (Aho, Weinberger, and Kernighan): is a programming language designed for text processing and typically used as a data extraction and reporting tool. It is a standard feature of most Unix-like operating system
Perl (Practical Extraction and Report Language):
Perl is a powerful text processing facility provider that makes report processing easier.
Below few one liner we can use in day to day Linux shell programming and some string management:
1) find and Replace in file
perl -pi -e 's/first/second/g' file1 file2 file3
2) find and replace only on line matching word "single"
perl -pi -e 's/first/second/g if /single/' file
perl -ne 'print if $a{$_}++' file
3) Print the N records after some pattern:
awk 'c&&c--;/pattern/{c=N}' file
4) Kill all process matching pattern
kill -9 $(ps -ef|grep search_string|grep -v grep|awk -F " " '{ print $2 }')
kill -9 `ps -ef |awk ' /search_string/ {print $2} '`
5) delete ALL blank lines from a file (same as "grep '.' ")
awk NF
awk '/./'
6) remove duplicate, nonconsecutive lines
awk '!a[$0]++' # most concise script
awk '!($0 in a){a[$0];print}' # fast
7) print the line immediately after a regexp, but not the line
# containing the regexp
sed -n '/regexp/{n;p;}'
7) print 1 line of context before and after regexp, with line number
# indicating where the regexp occurred (similar to "grep -A1 -B1")
sed -n -e '/regexp/{=;x;1!p;g;$!N;p;D;}' -e h
8) print section of file based on line numbers
sed -n '18,121p' # way 1
sed '18,121!d' # way 2
9) awk condition Check
YESTERDAY=$(date --date "1 days ago" +%d%m%y)
DATAFILE=visitor_log_$YESTERDAY.txt
#!/usr/bin/expect --
#cat $DATAFILE > VISITOR_LOG_BIG
cat DILIP_log.bad >> ALL_DILIP_log.bad
awk -F '|' '{if((length($1)==12)&&($1>910000000000)&&($1<=919999999999)) print $1,"|",$2,"|",$9}' $DATAFILE | sort | uniq > VISITOR_LOG_BIG
AWK (Aho, Weinberger, and Kernighan): is a programming language designed for text processing and typically used as a data extraction and reporting tool. It is a standard feature of most Unix-like operating system
Perl (Practical Extraction and Report Language):
Perl is a powerful text processing facility provider that makes report processing easier.
Below few one liner we can use in day to day Linux shell programming and some string management:
1) find and Replace in file
perl -pi -e 's/first/second/g' file1 file2 file3
2) find and replace only on line matching word "single"
perl -pi -e 's/first/second/g if /single/' file
perl -ne 'print if $a{$_}++' file
3) Print the N records after some pattern:
awk 'c&&c--;/pattern/{c=N}' file
4) Kill all process matching pattern
kill -9 $(ps -ef|grep search_string|grep -v grep|awk -F " " '{ print $2 }')
kill -9 `ps -ef |awk ' /search_string/ {print $2} '`
5) delete ALL blank lines from a file (same as "grep '.' ")
awk NF
awk '/./'
6) remove duplicate, nonconsecutive lines
awk '!a[$0]++' # most concise script
awk '!($0 in a){a[$0];print}' # fast
7) print the line immediately after a regexp, but not the line
# containing the regexp
sed -n '/regexp/{n;p;}'
7) print 1 line of context before and after regexp, with line number
# indicating where the regexp occurred (similar to "grep -A1 -B1")
sed -n -e '/regexp/{=;x;1!p;g;$!N;p;D;}' -e h
8) print section of file based on line numbers
sed -n '18,121p' # way 1
sed '18,121!d' # way 2
9) awk condition Check
YESTERDAY=$(date --date "1 days ago" +%d%m%y)
DATAFILE=visitor_log_$YESTERDAY.txt
#!/usr/bin/expect --
#cat $DATAFILE > VISITOR_LOG_BIG
cat DILIP_log.bad >> ALL_DILIP_log.bad
awk -F '|' '{if((length($1)==12)&&($1>910000000000)&&($1<=919999999999)) print $1,"|",$2,"|",$9}' $DATAFILE | sort | uniq > VISITOR_LOG_BIG
