NCSS Wiki

NOTES: Make sure no earthquakes are being processed by the RT system. You do this in a terminal window by looking at the current eqassemble log file on one of the RT systems. Check that this file has not changed in the last few minutes. Then run “tail -f” on the eqassemble log to see what is happening. If a new event starts to be processed here, wait until it has finished processing in the RT systems.

First login as user aqms ro either aqms-bk-rt or aqms-nc-rt

then check which role, either primary or shadow, this RT host is set to:

  [aqms@aqms-bk-rt ~]$ showrole
  System is primary

The command 'switchrole' can be launched on either aqms-bk-rt or aqms-nc-rt. The real time switch should happen before the post processing one because currently a PP system is considered primary at startup if its corresponding RT system is primary (showrole script logic).

In order to do a complete switchover:

• Disable the 'rsync_archive' cron jobs on aqms-bk-pp & aqms-nc-pp.
• Perform the steps in the 'AQMS: Real Time Role Switch' section.
• Perform the steps in the 'AQMS/DB: Post Processing Role Switch' section.
• Enable the 'rsync_archive' cron jobs on aqms-bk-pp & aqms-nc-pp.
• Perform the steps in the 'AQMS/DB: Auxiliary Role Switch & Other Items' section.

• Switchover:

  [aqms@aqms-bk-rt ~]$ switchrole

Will switch aqms-bk-rt (rtdb1) from primary to shadow and aqms-nc-rt(rtdb2) from shadow to primary.

  OK (Y or N):Y

Enter: Y

  rtem@rtdb1 System was primary, changing database to shadow
  spawn psql -q -h aqms-bk-rt.ncss-ucb.cisn.org -d rtdb1 -U rtem -W
  Password: 
  rtdb1=> \i /home/aqms/utils/scripts/set_shadow.sql
  rtdb1=> \q
  rtem@rtdb2
  System was shadow, changing database to primary
  spawn psql -q -h aqms-nc-rt.ncss-mp.cisn.org -d rtdb2 -U rtem -W
  Password: 
  rtdb2=> \i /home/aqms/utils/scripts/set_primary.sql
  rtdb2=> \q
  [aqms@aqms-bk-rt ~]$