NCSS Wiki

The real-time alarm systems handles event notification actions resulting from automatic event detection and magnitude determination. There are independent alarm systems on each of the real-time AQMS systems, and these in turn are independent from the post-processing alarm system. There are no communications between these alarm systems

There are two important properties of the alarming system that must always be considered. These apply to any CISN alarm system, not just the post-processing alarm system. These properties affect how the alarm system can be used by other parts of the CISN system.

• An alarm action for an event cannot be canceled before that action has be carried out. This is a result of how sendcancel works.
• Once an alarm action for an event has been canceled, there is no way to un-cancel that action.
• Only the alarm system that first carried out an alarm action can cancel that action.

Each alarm system consists of a database alarm_actions table, several programs, and a CMS system to tie the programs together. There are four continuously running programs: alarmdec (several instances); telestifle, alarmact, and alarmdist And there are three programs that may be run once for a given event or alarm action: alarmevent, sendalarm, and sendcancel. These last three may be run by hand from a login window, but more often are invoked by other programs controlled by PCS as directed by the Duty Review web page. Finally, there are a number of action programs that are invoked by alarmdist to do the actual work of preparing and distributing a given alarm notification product. Currently the NCSS real-time alarm system includes seven instances of alarmdec to handle the seven different alarm inputs:

• Quicklook signal from event coordinator.
• Magnitude signal from trimag.
• Early amplitude signal from ampgen configured as ag_early.
• Late amplitude signal from ampgen configured as ag_late.
• Moment Magnitude (M_w) signal from tmts.
• Moment Tensor signal from tmts.
• TMTS Done signal from tmts.

The CISN wiki has more information about each of the alarm programs.

The alarmdec (alarm decision) module evaluates events against sets of alarm criteria. It's input is a CMS message. Each message is identified by a subject, and contains simply an event ID. Alarmdec can subscribe to one or two different message subjects. But once the CMS systems delivers a message of a subscribed subject, alarmdec has no way of knowing which module published that message. Nor does alarmdec make any use of the subject of a message. For that reason, any logic that must be based on either the subject of the message or the publisher of that message must be handled externally to alarmdec.

These limitations of alarmdec cause us to configure the NCSS system in the following way. The TMTS program can publish one message to indicate that a new M_w has been produced for an event, and another message indicating that a new mechanism has been produced, and a third to indicate that it has completed processing an event regardless of the outcome of that process. Each of these messages consist of the event ID. In order for the messages' subscribers to distinguish these different contexts of these messages, we configure TMTS to publish them under three different subjects. And we configure three instances of alarmdec to handle each one of the message subjects.

A similar situation occurs with the two instances of ampgen: if we want different actions to be taken after the early and late ampgens are completed, we need separate instances of alarmdec to make that happen. Currently we do not make use of this feature; the alarm actions for the early and late ampgen signals are the same for now.

The output of alarmdec is another CMS message. This message consists of an event ID and an alarm name. For each of the alarmdec modules of the real-time system, these messages are published with subject /alarms/PP.

The alarmdec configurations are summarized in this spreadsheet. Guidance for using the spreadsheet to generate alarm rule configuration files is here.

The alarmdec module configured as alarmdec-ql subscribes to messages with subject /signals/eventcoord/quicklook. These are the messages published by the event coordinator for a new event from binder_ew and eqassemble with 25 associated phases but no magnitude. This provides a way to publish event locations quickly without waiting for the coda magnitude.

The alarmdec module configured as alarmdec-tm subscribes to messages with subject /signals/trimag/magnitude. These messages are published by trimag to indicate that it has completed processing an event: There may be a coda magnitude (M_d), there may also be a local magnitude (M_l), or there may be no magnitude for the event; but at least there has been an attempt to compute these two magnitudes. Depending an the alarm rules, this may result in the event location and magnitude being published into PDL, and email and pager messages being sent to NCSS personnel about a potentially significant earthquake.

These two alarmdec modules are configured to listen to messages with subjects /signals/ampgen/amps_early and /signals/ampgen/amps_late, respectively. These messages indicate that ground motion amplitudes have been collected for stations within 310 km of the event, and from 310 to 800 km. These alarmdec modules provide the means to notify the ShakeMap system that amplitudes are available.

The alarmdec module configured as alarmdec-mw subscribes to CMS messages with subject /signals/tmts/magnitude. These messages are published by real-time TMTS when a newly computed M_w is to be published for an event.

The alarmdec module configured as alarmdec-mech subscribes to CMS messages with subject /signals/tmts/mechanism. These messages are published by real-time TMTS when a newly computed moment tensor mechanism is to be published for an event.

The alarmdec module configured as alarmdec-tmts-done subscribes to the CMS message with subject /signals/tmts/done. These messages indicate that TMTS has finished working on an event. Since TMTS is the last real-time AQMS module that can add to event parameters in the database, this signal is used to notify Duty Seismologists that it is OK to update the event on the Duty Review Page with little risk of causing database replication errors.

The idea behind telestifle is as follows. After a large teleseism happens somewhere in the world away from the region of concern of an AQMS system, one or more of the P phases of this event pass through the local region. If the arrivals of these phases are sufficiently impulsive, the phase detector such as pick_ew may produce phase picks. Phase associators such as binder_ew may declare one or more “local” events using these phase picks, even when no local earthquake has occurred. In addition, one or more magnitude calculators may compute relatively large magnitudes for these false events. This can result in the AQMS system emitting notifications for these false events, to the annoyance and embarrassment of the AQMS operator. Careful tuning of the AQMS system can reduce the chances of these false notifications but are not likely to eliminate them.

The first part of the telestifle system is a PDL listener (configured with exec.ini). It listens for origin and internal-origin products. (Internal-origin products are preliminary origins sent by the US Tsunami Warning Centers and the US NEIC. They are normally not available to the public.) This PDL instance runs telesListener.pl (in the PDl bin directory) every time it receives one of these products. TelesListener.pl parses event information from its command line, checks that the magnitude is at least 6.0 and that the source of the product is one of the three agencies listed above. If these conditions are met, then telesListener.pl writes a message to a FIFO that was created by the telestifle program. The message contains origin time, latitude, longitude, depth and magnitude.

The telestifle program is a CMS client which creates the above mentioned FIFO and listens for messages sent to the FIFO. When telestifle publishes a stifle window, it also sends email and pager messages to NCSS staff. These messages include the window start and end times, as well as the statement “Duty Seismologist should review all events in window on DRP.” Since all public notifications for events during the stifle window are stopped, it is essential that someone review the Duty Review Page to find real earthquakes which need to be published, as well as false events that can be canceled.

For the NCSS, all of the real-time alarmdec modules except alarmdec-tmts-done are configured to subscribe to the CMS message from telestifle, and to use an alternate set of alarm criteria during the telestifle window. These alternate alarm criteria cause the same email and pager messages as the normal rules, but do not publish ANY event information to PDL or to Shakemap.

Our experience has been that the solutions published by the Tsunami Warning Centers generally provide 2 - 3 minutes advance notice, sufficient time for this stifling system to be useful.

The alarmact (alarm action) program subscribes to the /alarms/RT CMS messages published by each of the alarmdec modules of the real-time systems. alarmact maps each of the named alarms for an event into alarm actions. These actions are written to the alarm_action database table. After an action entry is made in the database, alarmact publishes a CMS message with the subject /actions/alarm/RT. This message consists of the event ID, action name, and modification count for this particular action.

The alarmdist (alarm distribution) program subscribes to the /actions/alarm/RT CMS messages and performs the alarm actions. The actions are in the form of scripts or other programs which are named for the action they perform. For example, one of the action names which alarmact signal for execution is named EVTPRM2PDL. In alamdist's action directory, there is a directory named EVTPRM2PDL. Within the EVTPRM2PDL directory there is a script named EVTPRM2PDL. The EVTPRM2PDL runs other scripts that generate quakeML files describing event parameters and submit these files into PDL

In addition to the positive alarm actions, alarmdist can also execute alarm cancellations. Similar to the action programs, cancellation programs with specific names. The names are based on the actions which they cancel, with “CANCEL_” prepended to the name. This the program for canceling the EVTPRM2PDL action is CANCEL_EVTPRM2PDL. Each cancellation script will strive to undo (in whatever way is appropriate) whatever the corresponding action script would do. For the example of EVTPRM2PDL, the cancellation script will create a quakeML file to cancel the event notification and submit it to PDL.

Once an alarm action has been canceled, there is no provision in the alarming system to “un-cancel” the alarm action for that alarming system.

Alarm actions are described here.

The alarmevent program is used to send a CMS signal for a single event to an appropriate alarmdec module. It's command line specifies both a config file and an event ID. alarmevent reads the message subject (among other parameters) from the config file. It queries the configured database to see if the event is valid: it must exist in the database, have a preferred origin and preferred magnitude. If so, alarmevent publishes the message containing the event ID with the specified subject. If alarmevent does not find a valid event in the database, it takes no further action. Normally alarmevent is invoked by the pcsAlarmCont program, part of the process control system. In special circumstances, a user could run alarmevent by hand to initiate alarms for an event for which the real-time system failed to generate alarms.

The sendalarm program is used to send a CMS signal for a specific alarm name and event to the alarmact module. This program is rarely used. Like alarmevent, it reads a config file to determine the subject under which its message should be published.

The sendcancel program is used to cancel alarms. This is the ONLY program that can initiate alarm cancellations. The sendcancel command line includes a config file, event ID, and optionally an alarm action to be canceled. If the action is not specified, then sendcancel assumes the action is “*”, the wildcard. sendcancel queries the database alarm_action table for the specified action for the event. If the action to be canceled is “*”, the query checks that any alarm exists for that event. If the query is successful, sendcancel prepares a message containing the event ID and action name (“all” if the action is “*”) and publishes the message under the subject parsed from the config file. The configured subject should be the one to which the alarmdist module has subscribed. Because of this database check that sendcancel performs before sending its message, it is not possible to “cancel” an alarm before the alarm has been initiated.

Normally sendcancel is invoked to cancel all alarms for an event, by the pcsCancelAlarmCont program, part of the process control system. In special circumstances, a user could run sendcancel by hand to cancel specific alarms.

After the event coordinator (ec) has issued a quicklook signal, it may be necessary to cancel the event as binder continues to process picks. This can be done in ec any time until the final event message has been received. When ec decides to cancel an event, it publishes a CMS message with subject /signals/eventcoord/cancel. The program that subscribes to this message is an instance of runner configured as sendcancel_runner.cfg. When runner receives this signal, it executes the script /home/ncss/run/bin/run_sendcancel, which runs sendcancel to signal alarmdist to cancel the actions it previously took for the event.