Don’t forget Perl one liners

18 May, 2007 – 11:42 am

If you have the requirement to manipulate multitudes of input messages / log files / data templates with dynamic variables, you may be left wondering what is the best approach. One could argue in a friendly way for hours about which language is the best to achieve this outcome (almost like pc vs. mac), but since I often find myself working on mixed environments, you can’t go past perl one liners…

To rehash what many others have already stated, here are my own favourites:

Print HEX value of a string good for encoding those MQMD headers. If you want to reverse use pack instead of unpack… You can also use B* for binary.
perl -e 'print unpack "H*", "AMQNUS725PQ"'


Replace 8 byte value which is offset at position 128 with the ‘NEWVALUE’:
perl -pi -e 'substr($_, 128, 8)="NEWVALUE"' /root/path/to/your/files/*


To copy posn 55 len 8 to posn 128 len 8:
perl -pi -e 'substr($_, 128, 8)=substr($_, 55, 8)' /root/path/to/your/files/*


To replace posn 128 len 8 in each file with first 8 bytes of the filename:
perl -pi -e 'substr($_,128,8)=substr($ARGV,0,8)' /root/path/to/your/files/*


To access your shell environment variables:
ACCOUNTF10=OUTSIDE
perl -e 'print "I AM INSIDE MY EXPRESSION AND NOW I AM '$ACCOUNTF10' MUHAHAHAHA"'


To allow for variable interpolation, use double quoted strings which substitutes the value of a variable into a string:
perl -pi -e 'substr($_,6162,10)="'$PRODUCTEND'"' $OUTPUTFILEFULL


Global search and replace using regex for search string (could also use fixed values instead of regex):
search="2007[0-9]{4}"
replace="20070321"
perl -pi -e 's/'$search'/'$replace'/g' /root/path/to/your/files/*


Find alternate search values and replace them with a single value:
search="KENANR|SIEBEL"
replace="SIEBEL"
perl -pi -e 's/'$search'/'$replace'/g' /root/path/to/your/files/*


Find anything that starts with a search value and replace them with a single value (Notice I pad strings out as the application in use depends on fixed format):
search="^Christina"
replace="Timothy   "
perl -pi -e 's/'$search'/'$replace'/g' /root/path/to/your/files/*


Find anything that ends with a search value and replace them with a single value:
search="Bank Deposit$"
replace="Bank Credit  "
perl -pi -e 's/'$search'/'$replace'/g' /root/path/to/your/files/*


Transliteration (swaps characters around) of specific characters:
search="cd"
replace="dc"
perl -pi -e 'tr/'$search'/'$replace'/' /root/path/to/your/files/*


And here are some metacharaters/character classes you can use in your search strings if you don’t want to use literals:
[a-z]    any charater between a to z
[A-Z]    any charater between A to Z
[0-9]    any number between 0 to 9
\d       same as [0-9]
\D       inverse of [0-9] ie. any non digit
\w       same as [a-zA-Z0-9_]
\W       inverse of \w
\s       same as [ \r\t\n\f] ie. any white space
\S       inverse of \s