Skip to main content

Lucene Solr Backup Script

#!/bin/bash
solr_home="/opt/jboss/solr/";
solr_backup_folder="/solr_backup/";
today="$(date +%d%m%Y)";
todays_backup="backup.$(date +%Y%m%d)*"
index_name_tag1="corename=\"";
index_name_tag2="\"instanceDir";
instance_dir_tag1="instanceDir=\"";
instance_dir_tag2="\"/>";
logfile="/tmp/solr/solr_backup.log";
overall_result="ok";

rm -f ${logfile};

echo ${today}" solr backup logs \n" >> ${logfile};
echo "solr_home = "${solr_home}" \n" >> ${logfile};
echo "solr_backup_folder = "${solr_backup_folder}" \n" >> ${logfile};

cd ${solr_home};

cat ${solr_home}"solr.xml" | while read line
do
 line=$(echo $line|sed 's/ //g');
 index_name_prefix="${line%${index_name_tag1}*}";
 index_name_suffix="${line#*${index_name_tag2}}";
 index_name_noprefix="${line#${index_name_prefix}${index_name_tag1}}";
 instance_dir_prefix="${line%${instance_dir_tag1}*}";
 instance_dir_suffix="${line#*${instance_dir_tag2}}";
 instance_dir_noprefix="${line#${instance_dir_prefix}${instance_dir_tag1}}";

 if [ ! "$index_name_noprefix" == "$line" ]
   then
   index_name="${index_name_noprefix%${index_name_tag2}${index_name_suffix}}"
   instance_dir="${instance_dir_noprefix%${instance_dir_tag2}${instance_dir_suffix}}"
   echo "----- index name "${index_name}" \n"  >> ${logfile}
   echo "----- instance directory "${instance_dir}" \n" >> ${logfile}

   echo "removing old snapshots \n" >> ${logfile}
   cd ${instance_dir}"data"
   old_snapshots=`ls -d backup.2*`
   echo $old_snapshots" \n" >> ${logfile}
   rm -rf ${instance_dir}data/backup.2*

   echo "taking snapshot \n" >> ${logfile}
   cd ${solr_home}
   sh bin/backup -d ${index_name}/data
   cd ${instance_dir}"data/"
   solr_snapshot=`ls -td ${todays_backup}  | awk 'NR<2'`
   echo $solr_snapshot" \n" >> ${logfile}
   echo "copying snaphot to backup folder \n" >> ${logfile}
   cp -fr ${instance_dir}data/$solr_snapshot ${solr_backup_folder}${index_name}/

   cd ${solr_backup_folder}$index_name
   echo "controlling is backup copied \n" >> ${logfile}
   if [ ! -d "${solr_backup_folder}$index_name/$solr_snapshot" ] || [ "$solr_snapshot" == "" ]
     then
     echo $solr_snapshot" does not exist in "${solr_backup_folder}$index_name" \n" >> ${logfile}
     overall_result="no";
     else
     echo $solr_snapshot" exists in "${solr_backup_folder}$index_name" \n" >> ${logfile}
     echo "removing old backups \n" >> ${logfile}
     old_backups=`ls -td backup.* | awk 'NR>2'`
     echo $old_backups" \n" >>${logfile}
     ls -td backup.* | awk 'NR>2' | xargs rm -rf
   fi
 fi
done

mailtext=$(cat ${logfile})
if [ "$overall_result" == "ok" ]
  then
  echo -e $mailtext | mail -s "solr backups overall successful" solr_report@report.test.com
  else
  echo -e $mailtext | mail -s "solr backups overall failed !!!" solr_report@report.test.com
fi

Comments

  1. Hi,

    I'm sysadmin and new in Solr stuff..
    I've to backup solr collections that I've installed, and for that I want to inspire me of your script to make mine.

    But I don't have any file in solr home bin folder, so I don't have bin/backup command.
    I would like to know if this an another script of you that you place there, or if this is a solr command that I don't have ? Or something somewhere to copy at this place?

    I'm using Solr 4.3.

    I'm french so I hope my english can be understood :)

    In any case, just to say thank you for sharing your script !

    Best Regards

    ReplyDelete
    Replies
    1. Hi,
      When i wrote this script solrcloud was not introduced. It is normal not to see backup script in your bin folder. These Scripts were superseded by the ReplicationHandler Java implementation of index replication that works over HTTP and was introduced in Solr4, and are no longer actively maintained. So you should use replication option for backup purposes. Please look at this link: http://wiki.apache.org/solr/SolrReplication

      Delete

Post a Comment

Popular posts from this blog

Find and replace with sed command in Linux

Find and replace feature is always handy. It can turn into a torture when it comes to change or delete a simple constant string in a text file. There is a handy tool in linux for doing these kind of tihngs. Actually sed is not a text editor but it is used outside of the text file to make changes.

Sending Jboss Server Logs to Logstash Using Filebeat with Multiline Support

In addition to sending system logs to logstash, it is possible to add a prospector section to the filebeat.yml for jboss server logs. Sometimes jboss server.log has single events made up from several lines of messages. In such cases Filebeat should be configured for a multiline prospector.
Filebeat takes lines do not start with a date pattern (look at pattern in the multiline section "^[[:digit:]]{4}-[[:digit:]]{2}-[[:digit:]]{2}" and negate section is set to true) and combines them with the previous line that starts with a date pattern.

server.log file excerpt where DatePattern: yyyy-MM-dd-HH and ConversionPattern: %d %-5p [%c] %m%n
Logstash filter: