Search And Replace shell script helpful for Upgrades

Search And Replace shell script helpful for Upgrades

« Undetectable Sniffing On EthernetMod_Security .htaccess tricks »

This bash shell script searches in a directory you specify for any files containing a string and replaces that string with another string. It’s nice because it asks you if you would like to be prompted for each operation, or to run without user-input. I use this shell script to change static IP addresses or domains, file locations, database info, etc.

Note: I’m going to be updating this soon..

For now here is a one-liner for you that works very fast and is also low-resource-intensive!

nice -n19 sh -c 'S=askapache && R=htaccess; find . -type f|xargs -P5 -iFF grep -l -m1 "$S" FF|xargs -P5 -iFF sed -i -e s%${S}%${R}% FF'

ScreenShot

Shell Script Code

You can also download it: FIND AND REPLACE SCRIPT Version 1.4

#!/bin/bash
# Find and Replace Version 2, 2009-04-19
# Version 1.4
# GNU Free Documentation License 1.2
# 2009-04-19 - AskApache (www.askapache.com)
umask 0022
 
# set -o xtrace && set -o verbose #uncomment for verbose debugging
 
# nicify to avoid getting killed
renice 19 -p $$ &>/dev/null
 
### SHELL OPTIONS
set +o noclobber  # allowed to clobber files
set +o noglob     # globbing on
set -e            # abort on first error
shopt -s extglob  # more globbing
 
PATH=/bin:/sbin:/usr/bin:/usr/local/bin:/usr/libexec:/usr/sbin
VER="1.4"
COLORON=yes
 
SEARCH=${1:-QQQQQQQQ};
REPLACE=${2:-AskApache};
SDIR=${3:-/tmp};
 
# script_title
#==-==-==-==-==-==-==-==-==-==-==#
function script_title(){
  clear;echo -e "${C[1]} __________________________________________________________________________ "
  echo -e "| ${C[2]}             ___       __    ___                 __             ${C[1]}         |"
  echo -e "| ${C[2]}            / _ | ___ / /__ / _ | ___  ___ _____/ /  ___        ${C[1]}         |"
  echo -e "| ${C[2]}           / __ |(_-</  '_// __ |/ _ \/ _ \`/ __/ _ \/ -_)       ${C[1]}         |"
  echo -e "| ${C[3]}          /_/ |_/___/_/\_\/_/ |_/ .__/\_,_/\__/_//_/\__/        ${C[1]}         |"
  echo -e "| ${C[3]}                               /_/                              ${C[1]}         |"
  echo -e "|                                                                          |"
  echo -e "|                 ${C[4]} FIND AND REPLACE SCRIPT Version $VER ${C[1]}                    |"
  echo -e "${C[1]} __________________________________________________________________________ ${C[0]} \n\n"
}
 
# pm
#==-==-==-==-==-==-==-==-==-==-==#
function pm()
{ case ${2:-d} in d) echo -en "\n\n${C[2]}>>> ${C[4]}$1 ${C[0]}\n\n"; ;; i*) echo -e "\n${C[6]}=> ${C[4]}${1} ${C[0]}\n"; ;; esac; return 0; }
 
# ok_continue
#==-==-==-==-==-==-==-==-==-==-==#
function ok_continue()
{ local ans; echo -en "${C[4]} \n [ Press any key to continue ] ${C[0]} \n"; read -n 1 ans; return 0; }
 
# yes_no
#==-==-==-==-==-==-==-==-==-==-==#
function yes_no()
{ local a YN=65; echo -en "${C[2]}>>> ${C[4]}${1:-Q} [y/n] ${C[0]}"; read -n 1 a; case $a in [yY]) YN=0; ;; esac; return $YN; }
 
# p_done
#==-==-==-==-==-==-==-==-==-==-==#
function p_done()
{ echo -e "\n${C[8]} DONE ${C[0]} \n"; return 0; }
 
# get_settings
#==-==-==-==-==-==-==-==-==-==-==#
function get_settings(){
  local a cha
  script_title
  while true; do
  for a in "SDIR" "SEARCH" "REPLACE"; do
   cha=g; echo -en "\n ${C[4]}(Enter for Default: ${!a} )${C[0]}\n ${a}${C[6]}=> ${C[0]} ";
   read -e cha; echo; [[ ${#cha} -gt 2 ]] && eval "$a"=$cha;
  done
  yes_no "ARE THESE SETTINGS CORRECT" && echo -e "\n\n" && break
  done
}
 
# search_and_replace
#==-==-==-==-==-==-==-==-==-==-==#
function search_and_replace(){
  for i in $(seq 0 $((${#FOUNDFILES[@]} - 1))); do
  thefile=${FOUNDFILES[$i]}

  if [[ $1 -eq 0 ]]; then
   echo -e "\n\n\n___________________________________________________________________\n"
   echo -e "${C[4]}FILE:  ${C[3]}${thefile} ${C[2]}($(command du -hs ${thefile}|awk '{ print $1}'))"
   echo -e "${C[4]}Type:  ${C[6]}$(command file -b ${thefile})\n${C[7]}Matching Lines:${C[0]}"
   command grep --color=always $SEARCH $thefile;echo

   while true; do
    yes_no "Replace occurances of ${SEARCH} with ${REPLACE}?" && echo -e "\nREPLACING...\n" && sed -i -e "s%${SEARCH}%${REPLACE}%g" $thefile && p_done && break
    echo -e "\nSKIPPING..\n" && p_done && break
   done
 
   else
    sed -i -e "s%${SEARCH}%${REPLACE}%g" $thefile
   fi

  done
  return 0;
}
 
############################################################################################################
### MAIN CODE
############################################################################################################
 
if [[ $SEARCH == "QQQQQQQQ" ]]; then
  E='\033[';
  [[ -z "$TERM" ]]&&[[ -z "$1" ]]||declare -a C=('${E}0m' '${E}1\;30m' '${E}0\;32m' '${E}1\;32m' '${E}1\;37m' '${E}1\;36m' '${E}1\;35m' '${E}1\;37m' '${E}30\;42m' '${E}1\;36m' )
  script_title            # DISPLAY SCRIPT TITLE
 
  get_settings            # GET SCRIPT SETTINGS
  cd $SDIR
  declare -a FOUNDFILES=( `command grep -R -l $SEARCH . 2>/dev/null` );
  declare -a FOUNDMATCHES=(  );
  for i in $(seq 0 $((${#FOUNDFILES[@]} - 1))); do F=${FOUNDFILES[$i]}; FOUNDMATCHES[$i]=$(command grep -c . $F); done
  pm "FOUND ${SEARCH} IN ${#FOUNDFILES[@]} FILES" i
  for i in $(seq 0 $((${#FOUNDFILES[@]} - 1))); do echo -e "${C[4]}${FOUNDFILES[$i]} ${C[6]}=>${C[4]} ${FOUNDMATCHES[$i]} matches"; done
  echo -e "\n"

 while true; do
  yes_no "Replace all occurances of ${SEARCH} with ${REPLACE} without prompting?" && search_and_replace 1 && p_done && break
  search_and_replace 0 && p_done && break
 done

else
 cd $SDIR
 command grep -H -R -l $SEARCH . 2>/dev/null | xargs -iFFF sed -i -e "s%${SEARCH}%${REPLACE}%g" FFF
fi
 
exit 0

Simplified for crontab

*Mike*
To run from a crontab you wouldn’t want a full-fledged script like this, I would just create a small bash script like so:

#!/bin/sh
PATH=/usr/bin:$PATH:/usr/local/bin
command grep -H -R -l $1 $3 2>/dev/null | xargs -iFFF sed -i -e "s%${1}%${$2}%g" FFF

This is like 100x faster, to call from crontab every 35 minutes and search the directory /home/user/tmp for ’search’ and replace it with ‘replace’ every 35 minutes.

*/35 * * * * /bin/sh /home/user/search-replace.sh 'search' 'replace' '/home/user/tmp' 1>/dev/null

« Undetectable Sniffing On Ethernet
Mod_Security .htaccess tricks »

MP3 Caper