FREE THOUGHT · FREE SOFTWARE · FREE WORLD

Home  »  Shell Scripting  »  Search And Replace shell script helpful for Upgrades

by 8 comments

This bash shell script searches in a directory you specify for any files containing a string and replaces that string with another string. It's nice because it asks you if you would like to be prompted for each operation, or to run without user-input. I use this shell script to change static IP addresses or domains, file locations, database info, etc.

Note: I'm going to be updating this soon..

For now here is a one-liner for you that works very fast and is also low-resource-intensive!

nice -n19 sh -c 'S=askapache && R=htaccess; find . -type f|xargs -P5 -iFF grep -l -m1 "$S" FF|xargs -P5 -iFF sed -i -e s%${S}%${R}% FF'

ScreenShot

Search And Replace shell script helpful for Upgrades

Shell Script Code

You can also download it: FIND AND REPLACE SCRIPT Version 1.4

#!/bin/bash
# Find and Replace Version 2, 2009-04-19
# Version 1.4
# GNU Free Documentation License 1.2
# 2009-04-19 - AskApache (www.askapache.com)
umask 0022
 
# set -o xtrace && set -o verbose #uncomment for verbose debugging
 
# nicify to avoid getting killed
renice 19 -p $$ &>/dev/null
 
### SHELL OPTIONS
set +o noclobber  # allowed to clobber files
set +o noglob     # globbing on
set -e            # abort on first error
shopt -s extglob  # more globbing
 
PATH=/bin:/sbin:/usr/bin:/usr/local/bin:/usr/libexec:/usr/sbin
VER="1.4"
COLORON=yes
 
SEARCH=${1:-QQQQQQQQ};
REPLACE=${2:-AskApache};
SDIR=${3:-/tmp};
 
# script_title
#==-==-==-==-==-==-==-==-==-==-==#
function script_title(){
  clear;echo -e "${C[1]} __________________________________________________________________________ "
  echo -e "| ${C[2]}             ___       __    ___                 __             ${C[1]}         |"
  echo -e "| ${C[2]}            / _ | ___ / /__ / _ | ___  ___ _____/ /  ___        ${C[1]}         |"
  echo -e "| ${C[2]}           / __ |(_-</  '_// __ |/ _ / _ `/ __/ _ / -_)       ${C[1]}         |"
  echo -e "| ${C[3]}          /_/ |_/___/_/_/_/ |_/ .__/_,_/__/_//_/__/        ${C[1]}         |"
  echo -e "| ${C[3]}                               /_/                              ${C[1]}         |"
  echo -e "|                                                                          |"
  echo -e "|                 ${C[4]} FIND AND REPLACE SCRIPT Version $VER ${C[1]}                    |"
  echo -e "${C[1]} __________________________________________________________________________ ${C[0]} nn"
}
 
# pm
#==-==-==-==-==-==-==-==-==-==-==#
function pm()
{ case ${2:-d} in d) echo -en "nn${C[2]}>>> ${C[4]}$1 ${C[0]}nn"; ;; i*) echo -e "n${C[6]}=> ${C[4]}${1} ${C[0]}n"; ;; esac; return 0; }
 
# ok_continue
#==-==-==-==-==-==-==-==-==-==-==#
function ok_continue()
{ local ans; echo -en "${C[4]} n [ Press any key to continue ] ${C[0]} n"; read -n 1 ans; return 0; }
 
# yes_no
#==-==-==-==-==-==-==-==-==-==-==#
function yes_no()
{ local a YN=65; echo -en "${C[2]}>>> ${C[4]}${1:-Q} [y/n] ${C[0]}"; read -n 1 a; case $a in [yY]) YN=0; ;; esac; return $YN; }
 
# p_done
#==-==-==-==-==-==-==-==-==-==-==#
function p_done()
{ echo -e "n${C[8]} DONE ${C[0]} n"; return 0; }
 
# get_settings
#==-==-==-==-==-==-==-==-==-==-==#
function get_settings(){
  local a cha
  script_title
  while true; do
  for a in "SDIR" "SEARCH" "REPLACE"; do
   cha=g; echo -en "n ${C[4]}(Enter for Default: ${!a} )${C[0]}n ${a}${C[6]}=> ${C[0]} ";
   read -e cha; echo; [[ ${#cha} -gt 2 ]] && eval "$a"=$cha;
  done
  yes_no "ARE THESE SETTINGS CORRECT" && echo -e "nn" && break
  done
}
 
# search_and_replace
#==-==-==-==-==-==-==-==-==-==-==#
function search_and_replace(){
  for i in $(seq 0 $((${#FOUNDFILES[@]} - 1))); do
  thefile=${FOUNDFILES[$i]}
 
  if [[ $1 -eq 0 ]]; then
   echo -e "nnn___________________________________________________________________n"
   echo -e "${C[4]}FILE:  ${C[3]}${thefile} ${C[2]}($(command du -hs ${thefile}|awk '{ print $1}'))"
   echo -e "${C[4]}Type:  ${C[6]}$(command file -b ${thefile})n${C[7]}Matching Lines:${C[0]}"
   command grep --color=always $SEARCH $thefile;echo
 
   while true; do
    yes_no "Replace occurances of ${SEARCH} with ${REPLACE}?" && echo -e "nREPLACING...n" && sed -i -e "s%${SEARCH}%${REPLACE}%g" $thefile && p_done && break
    echo -e "nSKIPPING..n" && p_done && break
   done
 
   else
    sed -i -e "s%${SEARCH}%${REPLACE}%g" $thefile
   fi
 
  done
  return 0;
}
 
############################################################################################################
### MAIN CODE
############################################################################################################
 
if [[ $SEARCH == "QQQQQQQQ" ]]; then
  E='33[';
  [[ -z "$TERM" ]]&&[[ -z "$1" ]]||declare -a C=('${E}0m' '${E}1;30m' '${E}0;32m' '${E}1;32m' '${E}1;37m' '${E}1;36m' '${E}1;35m' '${E}1;37m' '${E}30;42m' '${E}1;36m' )
  script_title            # DISPLAY SCRIPT TITLE
 
  get_settings            # GET SCRIPT SETTINGS
  cd $SDIR
  declare -a FOUNDFILES=( `command grep -R -l $SEARCH . 2>/dev/null` );
  declare -a FOUNDMATCHES=(  );
  for i in $(seq 0 $((${#FOUNDFILES[@]} - 1))); do F=${FOUNDFILES[$i]}; FOUNDMATCHES[$i]=$(command grep -c . $F); done
  pm "FOUND ${SEARCH} IN ${#FOUNDFILES[@]} FILES" i
  for i in $(seq 0 $((${#FOUNDFILES[@]} - 1))); do echo -e "${C[4]}${FOUNDFILES[$i]} ${C[6]}=>${C[4]} ${FOUNDMATCHES[$i]} matches"; done
  echo -e "n"
 
 while true; do
  yes_no "Replace all occurances of ${SEARCH} with ${REPLACE} without prompting?" && search_and_replace 1 && p_done && break
  search_and_replace 0 && p_done && break
 done
 
else
 cd $SDIR
 command grep -H -R -l $SEARCH . 2>/dev/null | xargs -iFFF sed -i -e "s%${SEARCH}%${REPLACE}%g" FFF
fi
 
exit 0

Search And Replace shell script helpful for Upgrades

Simplified for crontab

*Mike*
To run from a crontab you wouldn't want a full-fledged script like this, I would just create a small bash script like so:

#!/bin/sh
PATH=/usr/bin:$PATH:/usr/local/bin
command grep -H -R -l $1 $3 2>/dev/null | xargs -iFFF sed -i -e "s%${1}%${$2}%g" FFF

This is like 100x faster, to call from crontab every 35 minutes and search the directory /web/user/tmp for 'search' and replace it with 'replace' every 35 minutes.

*/35 * * * * /bin/sh /web/user/search-replace.sh 'search' 'replace' '/web/user/tmp' 1>/dev/null

Tags

April 19th, 2008

Comments Welcome

  • http://www.spencerart.ku.edu Bill

    Wow! This is exactly what I was looking for in preparation of launching our new site. Thank you.

  • http://www.askapache.com/ AskApache

    @ Bill

    Great! I love finding what I'm looking for on the web, and I know it isn't an everyday occurrance.. especially with an off-the-beaten-path topic like a search-and-replace bash shell script.. and this script is really effective, Enjoy!

  • Bill

    Is there a way to search and replace for strings with, what I am assuming are, illegal characters? What I am trying to do is to replace an old string

    <!--#set var="variableName" value="variableValue" -->

    with nothing.

    Are there any tweaks or escape characters I should be throwing in there to make this happen?

  • http://www.askapache.com/ AskApache

    @ Bill

    Each shell/OS/sed version are unique.. To replace with nothing just leave the replacement blank. That tells sed to delete the text.

    As to special characters, unless you know your shell well I would recommend just typing the search and replace commands manually so you can see errors and what works.

  • http://justsoicansay.com Mike the Elder

    Being a very old guy...i used to program when it was unix 3. I am trying to get back and learn some new stuff. Your site is very interesting even if i dont understand some of what you are doing :) Although i want to. This particular script would be very helpful to me if i could run it from a cron job. The web host i am using does not allow a shell account...but i can run some stuff from scripts via a cron job. Do you have a version that i can send arguments to via a cron. As i am just getting back into programming i am learning php but i dont remember much about shell scripting. Thanks for all you share...it make it very nice for us old newbies :)

  • http://www.askapache.com/ AskApache

    @ Mike the elder

    Hey Mike welcome back! You've missed an enormous amount of progress over the last few years, but I envy your experience.. thanks for stopping by and check out the update I created for you.

  • Bill

    We are launching a new design soon and plan to use your tool as it was very effective the last time. I would like to use this opportunity to clean things up a bit in my code. For instance, I have a fair amount of inline CSS styles that I would like to change to class or id names.

    The problem I am having is that the < and > are returning syntax errors for me. If I try to replace style="margin: 1.5em;" with class="new_style" the script hangs after I return "y" for "Are these settings correct [y/n]"

    Is there a better way to do this or is this beyond the scope of this program?

    Thanks for your help.

  • Raj

    Hi,

    I downloaded the script from the link and when I run it in my machine it gives me following errors-

    : octal number out of range6: umask: 0022
    : command not foundh: line 7:
    : command not foundh: line 9:
    : Permission deniedh: line 11: /dev/null
    : command not foundh: line 12:
    : command not foundh: line 13:
    : command not foundh: line 19:

    Do anyone have any idea?

Popular Articles
My Online Tools

Related Articles
Newest Posts
Twitter



Hacking and Hackers

The use of "hacker" to mean "security breaker" is a confusion on the part of the mass media. We hackers refuse to recognize that meaning, and continue using the word to mean someone who loves to program, someone who enjoys playful cleverness, or the combination of the two. See my article, On Hacking.
-- Richard M. Stallman






[hide]

It's very simple - you read the protocol and write the code. -Bill Joy

Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution 3.0 License, just credit with a link.
This site is not supported or endorsed by The Apache Software Foundation (ASF). All software and documentation produced by The ASF is licensed. "Apache" is a trademark of The ASF. NCSA HTTPd.
UNIX ® is a registered Trademark of The Open Group. POSIX ® is a registered Trademark of The IEEE.

| Google+ | askapache

Site Map | Contact Webmaster | License and Disclaimer | Terms of Service

↑ TOPMain