Friday, July 11, 2014

Installing and Configuring RANCID on CentOS 6.x

Requirements

  • Latest version of Rancid – As of 12/23/2013: Version 2.3.8
    Download from Shrubbery.net… NOT from EPEL (old version)!
  • cvs
  • expect >= 5.40
  • gcc
  • wget

Install and Configure

  1. Install Dependencies
    1. To make sure you have what you need, type: yum install cvs expect gcc wget
      When prompted to allow the download along with dependencies, type: y
      (the letter y)
  2. Download Rancid
    1. Type: cd /root
    2. Using wget to download rancid…
    3. [root@myserver ~]# wget ftp://ftp.shrubbery.net/pub/rancid/rancid-2.3.8.tar.gz
      --2013-12-23 14:59:43--  ftp://ftp.shrubbery.net/pub/rancid/rancid-2.3.8.tar.gz
                 => “rancid-2.3.8.tar.gz”
      Resolving ftp.shrubbery.net... 129.250.47.99
      Connecting to ftp.shrubbery.net|129.250.47.99|:21... connected.
      Logging in as anonymous ... Logged in!
      ==> SYST ... done.    ==> PWD ... done.
      ==> TYPE I ... done.  ==> CWD (1) /pub/rancid ... done.
      ==> SIZE rancid-2.3.8.tar.gz ... 379908
      ==> PASV ... done.    ==> RETR rancid-2.3.8.tar.gz ... done.
      Length: 379908 (371K) (unauthoritative)
      
      100%[==================================================================================================================>] 379,908      542K/s   in 0.7s    
      
      2013-12-23 14:59:49 (542 KB/s) - “rancid-2.3.8.tar.gz” saved [379908]
C. Unpack Rancid
    1. Type: cd /root
    2. Type: tar -xvzf /root/rancid-2.3.8.tar.gz
      (Creates a new folder /root/rancid-2.3.8 with installation files)
D. Create Group and User Account for Rancid
    1. Type: groupadd netadm
    2. Type: useradd -g netadm -d /home/rancid rancid
    3. Set the rancid password by doing the following:
      [root@myserver ~]# passwd rancid
      Changing password for user rancid.
      New password: 
      Retype new password: 
      passwd: all authentication tokens updated successfully.
E. Compile and Install Rancid
    1. Type: cd /root/rancid-2.3.8
    2. Configure the installation directory:
      [root@myserver rancid-2.3.8]# ./configure --prefix=/home/rancid
    3. Type: make install
F. Copy Sample Password File To /home/rancid
    1. Type: cp /root/rancid-2.3.8/cloginrc.sample /home/rancid/.cloginrc
G. Set Permissions on /home/rancid/*
    1. Type: chmod 0640 /home/rancid/.cloginrc
    2. Type: chown -R rancid:netadm /home/rancid/
    3. Type: chmod 770 /home/rancid/
    4. You should now see something very similar to the following:
      [root@myserver rancid-2.3.8]# ls -la /home/rancid
      total 44
      drwxrwx---  6 rancid netadm 4096 Dec 23 15:58 .
      drwxr-xr-x. 3 root   root   4096 Dec 23 15:39 ..
      -rw-------  1 rancid netadm   13 Dec 23 15:54 .bash_history
      -rw-r--r--  1 rancid netadm   18 Jul 18 09:19 .bash_logout
      -rw-r--r--  1 rancid netadm  176 Jul 18 09:19 .bash_profile
      -rw-r--r--  1 rancid netadm  124 Jul 18 09:19 .bashrc
      drwxr-xr-x  2 rancid netadm 4096 Dec 23 15:54 bin
      -rw-r-----  1 rancid netadm 3772 Dec 23 15:58 .cloginrc
      drwxr-xr-x  2 rancid netadm 4096 Dec 23 15:54 etc
      drwxr-xr-x  4 rancid netadm 4096 Dec 23 15:54 share
      drwxr-xr-x  2 rancid netadm 4096 Dec 23 15:54 var
* NOTES on Using vi:
Remember to press the letter i for Insert mode
Press [ESC] to go back to command mode
To Write and Quit type: :wq
(colon, the letters w and q)
To quit without making changes, type: :q!
(colon, the letter q, the exclamation point)
H. Suggested Changes To /home/rancid/etc/rancid.conf
    1. Change to the rancid account by typing: su rancid
    2. Type: vi /home/rancid/etc/rancid.conf
    3. Search and modify lines to match the following
      (remove # from beginning of line if necessary):
      FILTER_PWDS=NO; export FILTER_PWDS
      NOCOMMSTR=NO; export NOCOMMSTR
      LIST_OF_GROUPS="SITE1 SITE2"
      MAILHEADERS=""; export MAILHEADERS

      (Replace SITE1 SITE2 with YOUR groups… separated by spaces)
    4. Write and quit VI
    5. Change back to root by typing: exit
I. Create CVS Group Directories
    1. Change to the rancid account by typing: su rancid
    2. Type: /home/rancid/bin/rancid-cvs
    3. Change back to root by typing: exit
J. Create Email Aliases For Each Of Your Groups
    1. As roottype: vi /etc/aliases
    2. Insert your aliases immediately following the warning at the top… it should look something like:
      #
      #  Aliases in this file will NOT be expanded in the header  from
      #  Mail, but WILL be visible over networks or from /bin/mail.
      #
      # >>>>>>>>>> The program "newaliases" must be run after
      # >> NOTE >> this file is updated for any changes to
      # >>>>>>>>>> show through to sendmail.
      #
      
      # Aliases for rancid alerting
      rancid-SITE1: site1@mycompany.com
      rancid-admin-SITE1: site1-admin@mycompany.com
      rancid-SITE2: site2@mycompany.com
      rancid-admin-SITE2: site2-admin@mycompany.com
    3. Write and quit VI
    4. Activate the new aliases by typing: newaliases
K. Configure A Cron Job For The rancid Account
    1. Change to the rancid account by typing: su rancid
    2. Type: crontab -e
    3. Add the following to your crontab:
      # #############################################
      # Crontab for rancid on myserver
      # #############################################
      # Hourly Check
      30 7,8,9,10,11,12,13,14,15,16,17 * * 1,2,3,4,5 /home/rancid/bin/rancid-run
      
      # Daily Clean Up of Diff Files at 11 50 pm
      50 23 * * * /usr/bin/find /home/rancid/var/logs -type f -mtime +2 -exec rm {} \;
      
      # Daily Clean Up of .SITE.run.lock Files at 11 50 pm
      50 23 * * * rm /tmp/.*.lock
      
    4. Write and quit VI
* NOTES on .clogingrc
The passwords for your devices will be on this server… PROTECT IT!
The file is read top down… if you have a device that requires a special login, make sure the section for that device is unique!
READ THE EXAMPLES IN THE SAMPLE FILE! (rancid-cloginrc-sample-file)

L. Configure The .cloginrc File
    1. Change to the rancid account by typing: su rancid
    2. Type: vi /home/rancid/.cloginrc
    3. Go through the file and make sure there is a hash (#) in front of EVERY example line!
      (which, at this time, should be EVERY line!!)
    4. The four, in my experience, most used line items are
      autoenable – Doesn’t need to use the "enable" command or password
      user – the username for logging into the device
      password – the password for logging into the device
      method – The protocol used to access the device (telnet, ssh, rsh)
      If you need to supply an "enable" password, append it to the end of the passwordline.
      Password line with autoenable being used:
      add password * {mypassword}
      
      Password line without autoenable...
      add password * {mypassword} {myenablepass} 
    5. There are many options supplied in the examples. Pretty much anything required to meet your login needs.
    6. A basic (catch all) section should look similar to the following:
      # ##################################################
      # If all other device logins fail... try this
      # ##################################################
      add autoenable * {1}
      add user * {username}
      add password * {password}
      add method * {telnet} {ssh}
      Replace username and password with the appropriate credentials for your device. DO NOT remove the braces { }… the are REQUIRED! This section would apply to ANY device. Since the file is read top down… this section should be LAST!!! But, if all your devices have the same username and password, this makes the file VERYeasy to configure. Thus why I listed it first!
    7. A device specific section should look similar to the following:
      # ##################################################
      # Only for 192.168.1.1
      # ##################################################
      add autoenable 192.168.1.1 {1}
      add user 192.168.1.1 {username}
      add password 192.168.1.1 {password}
      add method 192.168.1.1 {telnet} {ssh}
      Replace username and password with the appropriate credentials for your device. DO NOT remove the braces { }… the are REQUIRED! This section would apply to ONE device. Since the file is read top down… this section should be FIRST (or at least before the catch all)!!!
    8. A device specific section can also uses names instead of IPs and should look similar to the following:
      # ##################################################
      # Only for switch1
      # ##################################################
      add autoenable switch1 {1}
      add user switch1 {username}
      add password switch1 {password}
      add method switch1 {telnet} {ssh}
      Replace username and password with the appropriate credentials for your device. DO NOT remove the braces { }… the are REQUIRED! This section would apply to ONE device. Since the file is read top down… this section should be FIRST (or at least before the catch all)!!! Make sure the name is either in your DNS or the Rancid server’s host file!!!
    9. A group of devices, using IP, should look similar to the following:
      # ##################################################
      # For all IPs Starting with 192.168.1.1
      # ##################################################
      add autoenable 192.168.1.1* {1}
      add user 192.168.1.1* {username}
      add password 192.168.1.1* {password}
      add method 192.168.1.1* {telnet} {ssh}
      Replace username and password with the appropriate credentials for your device. DO NOT remove the braces { }… the are REQUIRED! This would apply to ALL devices starting with "192.168.1.1". So, 192.168.1.1, 192.168.1.10, 11.12, 100, 150, etc… (you get the point). Since the file is read top down… this section should be FIRST (or at least before the catch all)!!!
    10. A group of devices, using Names, should look similar to the following:
      # ##################################################
      # For all hostnames starting with switch1
      # ##################################################
      add autoenable switch* {1}
      add user switch* {username}
      add password switch* {password}
      add method switch* {telnet} {ssh}
      Replace username and password with the appropriate credentials for your device. DO NOT remove the braces { }… the are REQUIRED! This would apply to any devices having a hostname starting with "switch". So, switch1, switch 200, switch300, etc… Since the file is read top down… this section should be FIRST (or at least before the catch all)!!! Make sure the name is either in your DNS or the Rancid server’s host file!!!
    11. A 3 Site Example of the .cloginrc file:
      # #############################################
      # DEVICES ARE MATCHED IN A TOP-DOWN ORDER
      # USE WILDCARDS WITH CAUTION!
      # #############################################
      
      # #############################################
      # SITE1
      # #############################################
      
      # - Default Login
      add autoenable 192.168.1.*  {1}
      add user 192.168.1.*            {site1username}
      add password 192.168.1.*  {site1password}
      add method 192.168.1.*  {telnet} {ssh}
      
      # #############################################
      # SITE2
      # #############################################
      
      # - HP Switches are in a different subnet
      add autoenable 192.168.2.25* {1}
      add user 192.168.2.25* {hpsite2username}
      add password 192.168.2.25* {hpsite2password}
      add method 192.168.2.25* {telnet}
      
      # - Default Login
      add autoenable 192.168.2.*      {1}
      add user 192.168.2.*            {site2username}
      add password 192.168.2.*        {sit2password}
      add method 192.168.2.*          {telnet}        {ssh}
      
      # #############################################
      # SITE3
      # #############################################
      
      # - Router
      add autoenable 192.168.3.5 {1}
      add user 192.168.3.5  {site3routerusername}
      add password 192.168.3.5 {site3routerpassword}
      add method 192.168.3.5  {telnet} {ssh}
      
      # - Default Login
      add autoenable 192.168.3.*      {1}
      add user 192.168.3.*            {site3username}
      add password 192.168.3.*        {site3password}
      add method 192.168.3.*          {telnet}        {ssh}
    12. A Cisco ASA5500,ASA5510,ASA5520 Example of the .cloginrc file:
      # - Primary Firewall
      add autoenable 192.168.1.254 {0}
      add user 192.168.1.254  {fwusername}
      add password 192.168.1.254  {fwpassword} {fwenablepassword}
      add method 192.168.1.254 {ssh}
    13. Change back to root by typing: exit
M. Adding Devices – Group/Site Specific
    1. Change to the rancid account by typing: su rancid
    2. Type: vi /home/rancid/var/[GROUP]/router.db
    3. Each device occupies one line containing three items delimited by the colon (:)
      ([devicename or IP]:[manufacturer]:[up or down]
    4. If we had 3 switches in SITE1 (192.168.1.1,2, and 3) we would type the following:
      vi /home/rancid/var/SITE1/router.db
    5. The contents of the file would be as follows:
      192.168.1.1:cisco:up
      192.168.1.2:cisco:up
      192.168.1.3:cisco:up
    6. If we had 3 switches in SITE2 (192.168.2.1,2, and 3) we would type the following:
      vi /home/rancid/var/SITE2/router.db
    7. The contents of the file would be as follows:
      192.168.2.1:cisco:up
      192.168.2.2:cisco:up
      192.168.2.3:cisco:up
    8. Remember to Write and quit VI or you will lose your changes
    9. Type: /home/rancid/bin/rancid-run
    10. Change back to root by typing: exit

Testing The Configuration

  1. Test One (1) Device
    1. Change to the rancid account by typing: su rancid
    2. Type: /home/rancid/bin/rancid-run -r
  2. Test a Group
    1. Change to the rancid account by typing: su rancid
    2. Type: /home/rancid/bin/rancid-run
  3. Test All Devices
    1. Change to the rancid account by typing: su rancid
    2. Type: /home/rancid/bin/rancid-run
  4. Checking Results and Other Options
    1. In all cases, check /home/rancid/var/logs for results
    2. Optional: add an & to the end of any command to make it run in the background
      Examples:
      /home/rancid/bin/rancid-run -r 192.168.1.1 &
      /home/rancid/bin/rancid-run SITE1 &
      /home/rancid/bin/rancid-run &
      
      Press the [ENTER] key every few minutes to see 
      results that will look similar to:
      [1]+  Done                    rancid-run

Troubleshooting

  1. "cannot open CVS/Entries for reading"
    1. When running rancid for SITE1, you look in your logs (/home/rancid/var/logs) and you see something like the following:
      cvs status: cannot open CVS/Entries for reading: No such file or directory
      cvs status: use `cvs add' to create an entry for 192.168.1.1
      cvs add: in directory .:
      cvs [add aborted]: there is no version here; do 'cvs checkout' first
      cvs added missing router 192.168.1.1
      
      
      Trying to get all of the configs.
      All routers sucessfully completed.
      
      cvs diff: in directory .:
      cvs [diff aborted]: there is no version here; run 'cvs checkout' first
      cvs commit: in directory .:
      cvs [commit aborted]: there is no version here; run 'cvs checkout' first
      
      ending: Fri Dec 27 12:56:39 EST 2013
    2. Change to the rancid account by typing: su rancid
    3. Type: cat /home/rancid/var/SITE1/router.db
    4. Copy and paste the contents of /home/rancid/var/SITE1/router.db into another file or you WILL lose them!!!
    5. Type: cd /home/rancid/var/
    6. Type: rm -R ./SITE1
    7. Type: /home/rancid/bin/rancid-cvs
    8. Copy and paste the saved content back into /home/rancid/var/SITE1/router.db
    9. Test Rancid by typing: /home/rancid/bin/rancid-run SITE1
    10. Look in the logs to see if everything is working
  2. "cvs commit: Up-to-date check failed for configs/"
    1. When running rancid for SITE1 you get something like the following:
      cvs commit: Up-to-date check failed for configs/
    2. Make sure you are using the rancid account by typing: su rancid
    3. Type: cd /home/rancid/var/SITE1/configs
    4. Ensure you are in the correct directory by typing: pwd
    5. Then Update CVS by typing: cvs update
    6. Test Rancid by typing: /home/rancid/bin/rancid-run
    7. Look in the logs to see if everything is working
  3. "The following routers have not been successfully contacted"
    1. You get an e-mail for SITE1 similar the following:
      The following routers have not been successfully contacted 
      for more than 24 hours.
      
      -rw-r—– 1 rancid netadm 62023 Aug 1 21:02 
    2. Possible causes for this message are the device is down, the user id and password have changed, or the host key for SSH has changed
    3. View the latest log in /home/rancid/var/logs to determine the exact cause of the problem
    4. If the log is unclear… try accessing the device FROM THE RANCID SERVER using the SAME IP/Hostname, username, password, and method (from/home/rancid/.cloginrc) that Rancid would use
    NOTE: If you get a "host key has changed" warning on an Internet device or any device beyond your control, you should not attempt to authenticate as it is possible the device or the path to the device has been compromised.
  4. "The host key for has changed"
    1. You look in your logs and see something similar to:
       clogin error: Error: The host key for 
       has changed. Update the SSH known_hosts file 
      accordingly.
    2. ONLY DO THE FOLLOWING IF THE DEVICE AND PATH ARE FULLY UNDER YOUR CONTROL!
    3. Change to the rancid account by typing: su rancid
    4. Type: vi /home/rancid/.ssh/known_hosts
      Each line is a device record containing the DeviceName, Device IP Address, and SSH-RSA Key
    5. Move the cursor down to the line for your device
    6. Delete ONLY that line by pressing the [d] key on your keyboard twice (double click speed)
    7. Write and quit VI
    8. Update the /home/rancid/.ssh/known_hosts file:
      [rancid@myserver ~]# /home/rancid/bin/clogin 
      
      spawn ssh -c 3des -x -l rancid 
      The authenticity of host ‘ (x.x.x.x)’ can’t be established.
      RSA key fingerprint is 01:02:03:04:05:06:07:08:09:10:11:12:13:14:15:16.
      Are you sure you want to continue connecting (yes/no)?
      Host  added to the list of known hosts.
      yes
      Warning: Permanently added ‘,x.x.x.x’ (RSA) to the list of known hosts.
      
    9. Test Rancid by typing: /home/rancid/bin/rancid-run -r
    10. Look in the logs to see if everything is working
  5. False Change Notifications
    1. DELETE DEVICE CVS HISTORY
      *** WARNING ***
      THIS WILL REMOVE ALL HISTORY FOR THE DEVICE!
    2. Change to the rancid account by typing: su rancid
    3. Using vi or another editor, remove the device(s) from:
      /home/rancid/var/[SITE]/router.db
    4. Run rancid: /home/rancid/bin/rancid-run [SITE] &
    5. Delete the device history from:
      /home/rancid/var/CVS/[SITE]/configs/Attic
    6. Add the device(s) back to:
      /home/rancid/var/[SITE]/router.db
    7. Run rancid again: /home/rancid/bin/rancid-run [SITE] &

Thanks to Charles E. Gerow

No comments: