NCSS operates two systems for generating ShakeMaps: rodgers.geo.berkeley.edu and mnlodd1.wr.usgs.gov. This page describes how ShakeMap production is handled in the NCSS. See the ShakeMap Manual for general information.
The NCSS ShakeMap systems are currently running version 3.5 and should be kept up to the most recent SVN release. When ShakeMap version 4 is released perhaps within the next year, most of the below information will need to changed!
In December 2020, MP and UCB switched production of ShakeMaps from version 3.5 to version 4.0. In this subsection we will begin to document information regarding running and maintaining ShakeMaps in version 4.0 on both the MP and UCB systems. The information on this wiki for running ShakeMap 3.5 is maintained below.
info on accessing systems here Connect to VPN and use SSH to login to MP or UCB systems to access ShakeMap
ShakeMap 4.1.1 on Virtual Machine (currently in testing mode)
MP - login to mnlosm1
ssh -l shake mnlosm1
ShakeMap 4.0
MP - login to mnlodd1new
ssh -l shake mnlodd1new
UCB - login to seiche
ssh -l ncss seiche.geo.berkeley.edu
ShakeMap 3.5
MP- login to mnlodd1
ssh -l shake mnlodd1
UCB - login in to rodgers (may have been migrated due to server change?)
ssh -l ncss rodgers.geo.berkeley.edu
Auto Run All Modules Example
example to re-run a map
shake -a 73584926
The above will run the autorun modules set in shake.conf. The current modules are (updated 12/6/2022 to implement select):
autorun_modules = aqms_eq2xml aqms_db2xml associate dyfi select assemble model mapping contour coverage gridxml info kml shape plotregr raster uncertaintymaps rupture stations transfer_pdl transfer_scp transfer_email
Here is an example on how to re-run one of the aftershocks of the Antelope Valley earthquake that occurred July 2021:
conda activate shakemap
shake -a 73585021
The terminal will output showing it is processing like below and ask for a comment. Just click enter to continue processing. All the modules above will run, including the transfer to NEIC and update the map.
(shakemap) [shake@mnlodd1new data]$ shake -a 73585021 In /home/shake/.conda/envs/shakemap/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: The text.latex.preview rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later. In /home/shake/.conda/envs/shakemap/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: The mathtext.fallback_to_cm rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later. In /home/shake/.conda/envs/shakemap/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: Support for setting the 'mathtext.fallback_to_cm' rcParam is deprecated since 3.3 and will be removed two minor releases later; use 'mathtext.fallback : 'cm' instead. In /home/shake/.conda/envs/shakemap/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: The validate_bool_maybe_none function was deprecated in Matplotlib 3.3 and will be removed two minor releases later. In /home/shake/.conda/envs/shakemap/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: The savefig.jpeg_quality rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later. In /home/shake/.conda/envs/shakemap/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: The keymap.all_axes rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later. In /home/shake/.conda/envs/shakemap/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: The animation.avconv_path rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later. In /home/shake/.conda/envs/shakemap/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: The animation.avconv_args rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later. WARNING – 2021-07-14 17:37:11 – shake.main – The -a (–autorun) option is deprecated. It is now the default behavior to use the autorun_modules in shake.conf when no modules are specified. INFO – 2021-07-14 17:37:11 – shake.main – No modules specified, checking for autoun_modules… INFO – 2021-07-14 17:37:11 – shake.main – Autorun modules found: ['aqms_eq2xml', 'aqms_db2xml', 'associate', 'dyfi', 'assemble', 'model', 'mapping', 'contour', 'coverage', 'gridxml', 'info', 'kml', 'shape', 'plotregr', 'raster', 'uncertaintymaps', 'rupture', 'stations', 'transfer_pdl', 'transfer_scp', 'transfer_email'] INFO – 2021-07-14 17:37:12 – shake.main – Running command aqms_eq2xml INFO – 2021-07-14 17:37:12 – shake.main – Finished running command aqms_eq2xml: Elapsed 0.69 secs INFO – 2021-07-14 17:37:12 – shake.main – Running command aqms_db2xml INFO – 2021-07-14 17:37:31 – shake.main – Finished running command aqms_db2xml: Elapsed 18.55 secs INFO – 2021-07-14 17:37:31 – shake.main – Running command associate INFO – 2021-07-14 17:37:31 – shake.main – Finished running command associate: Elapsed 0.00 secs INFO – 2021-07-14 17:37:31 – shake.main – Running command dyfi INFO – 2021-07-14 17:37:43 – dyfi.execute – Wrote 34 DYFI records to /home/shake/shakemap_profiles/default/data/73585021/current/dyfi_dat.xml INFO – 2021-07-14 17:37:43 – shake.main – Finished running command dyfi: Elapsed 12.33 secs INFO – 2021-07-14 17:37:43 – shake.main – Running command assemble Please enter a comment for this version. comment: INFO – 2021-07-14 17:37:55 – shake.main – Finished running command assemble: Elapsed 11.75 secs INFO – 2021-07-14 17:37:55 – shake.main – Running command model INFO – 2021-07-14 17:38:59 – shake.main – Finished running command model: Elapsed 64.01 secs INFO – 2021-07-14 17:38:59 – shake.main – Running command mapping INFO – 2021-07-14 17:39:49 – shake.main – Finished running command mapping: Elapsed 50.16 secs INFO – 2021-07-14 17:39:49 – shake.main – Running command contour INFO – 2021-07-14 17:39:53 – shake.main – Finished running command contour: Elapsed 3.78 secs INFO – 2021-07-14 17:39:53 – shake.main – Running command coverage INFO – 2021-07-14 17:39:55 – shake.main – Finished running command coverage: Elapsed 2.10 secs INFO – 2021-07-14 17:39:55 – shake.main – Running command gridxml INFO – 2021-07-14 17:39:58 – shake.main – Finished running command gridxml: Elapsed 2.44 secs INFO – 2021-07-14 17:39:58 – shake.main – Running command info INFO – 2021-07-14 17:39:58 – shake.main – Finished running command info: Elapsed 0.00 secs INFO – 2021-07-14 17:39:58 – shake.main – Running command kml INFO – 2021-07-14 17:40:08 – shake.main – Finished running command kml: Elapsed 9.98 secs INFO – 2021-07-14 17:40:08 – shake.main – Running command shape INFO – 2021-07-14 17:40:13 – shake.main – Finished running command shape: Elapsed 4.77 secs INFO – 2021-07-14 17:40:13 – shake.main – Running command plotregr INFO – 2021-07-14 17:40:21 – shake.main – Finished running command plotregr: Elapsed 7.70 secs INFO – 2021-07-14 17:40:21 – shake.main – Running command raster WARNING – 2021-07-14 17:40:21 – warnings._showwarnmsg – /home/shake/.conda/envs/shakemap/lib/python3.7/site-packages/mapio/gdal.py:490: DataSetWarning: 'Down-casting double precision floating point to single precision'
warnings.warn(DataSetWarning('Down-casting double precision '
INFO – 2021-07-14 17:40:22 – shake.main – Finished running command raster: Elapsed 0.87 secs INFO – 2021-07-14 17:40:22 – shake.main – Running command uncertaintymaps INFO – 2021-07-14 17:40:42 – shake.main – Finished running command uncertaintymaps: Elapsed 20.81 secs INFO – 2021-07-14 17:40:43 – shake.main – Running command rupture INFO – 2021-07-14 17:40:43 – rupture.execute – Writing rupture.json file… INFO – 2021-07-14 17:40:43 – shake.main – Finished running command rupture: Elapsed 0.00 secs INFO – 2021-07-14 17:40:43 – shake.main – Running command stations INFO – 2021-07-14 17:40:43 – shake.main – Finished running command stations: Elapsed 0.50 secs INFO – 2021-07-14 17:40:43 – shake.main – Running command transfer_pdl INFO – 2021-07-14 17:40:43 – transfer_base.execute – Making backup directory… INFO – 2021-07-14 17:41:31 – transfer_pdl.execute – 70 files sent. Message from sender: “70 files sent successfully: resulting in output: “urn:usgs-product:nc:shakemap:73585021:1626284444255 ”” INFO – 2021-07-14 17:41:31 – shake.main – Finished running command transfer_pdl: Elapsed 47.46 secs INFO – 2021-07-14 17:41:31 – shake.main – Running command transfer_scp INFO – 2021-07-14 17:41:32 – transport._log – Connected (version 2.0, client OpenSSH_7.4) INFO – 2021-07-14 17:41:32 – transport._log – Authentication (publickey) successful! Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/shake_result.hdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/mmi_legend.png… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/intensity.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/intensity.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/pin-thumbnail.png… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/intensity_overlay.png… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/intensity_overlay.pngw… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/pga.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/pga.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/pgv.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/pgv.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa0p3.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa0p3.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa1p0.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa1p0.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa3p0.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa3p0.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/cont_mmi.json… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/cont_mi.json… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/cont_pga.json… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/cont_pgv.json… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/cont_psa0p3.json… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/cont_psa1p0.json… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/cont_psa3p0.json… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_mmi_high_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_mmi_medium_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_mmi_low_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_pga_high_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_pga_medium_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_pga_low_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_pgv_high_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_pgv_medium_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_pgv_low_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_psa0p3_high_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_psa0p3_medium_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_psa0p3_low_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_psa1p0_high_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_psa1p0_medium_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_psa1p0_low_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_psa3p0_high_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_psa3p0_medium_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_psa3p0_low_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/grid.xml… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/uncertainty.xml… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/info.json… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/shakemap.kmz… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/shape.zip… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/mmi_regr.png… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/pga_regr.png… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/pgv_regr.png… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa0p3_regr.png… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa1p0_regr.png… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa3p0_regr.png… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/attenuation_curves.json… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/raster.zip… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/intensity_uncertainty.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/intensity_uncertainty.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/pga_uncertainty.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/pga_uncertainty.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/pgv_uncertainty.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/pgv_uncertainty.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa0p3_uncertainty.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa0p3_uncertainty.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa1p0_uncertainty.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa1p0_uncertainty.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa3p0_uncertainty.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa3p0_uncertainty.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/rupture.json… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/stationlist.json… INFO – 2021-07-14 17:42:15 – transfer_scp.execute – 69 files sent. Message from sender: “69 files sent to remote host eoc1.cisn.org” INFO – 2021-07-14 17:42:16 – transport._log – Connected (version 2.0, client OpenSSH_7.4) INFO – 2021-07-14 17:42:16 – transport._log – Authentication (publickey) successful! Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/shake_result.hdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/mmi_legend.png… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/intensity.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/intensity.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/pin-thumbnail.png… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/intensity_overlay.png… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/intensity_overlay.pngw… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/pga.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/pga.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/pgv.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/pgv.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa0p3.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa0p3.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa1p0.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa1p0.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa3p0.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa3p0.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/cont_mmi.json… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/cont_mi.json… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/cont_pga.json… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/cont_pgv.json… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/cont_psa0p3.json… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/cont_psa1p0.json… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/cont_psa3p0.json… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_mmi_high_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_mmi_medium_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_mmi_low_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_pga_high_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_pga_medium_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_pga_low_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_pgv_high_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_pgv_medium_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_pgv_low_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_psa0p3_high_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_psa0p3_medium_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_psa0p3_low_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_psa1p0_high_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_psa1p0_medium_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_psa1p0_low_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_psa3p0_high_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_psa3p0_medium_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/coverage_psa3p0_low_res.covjson… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/grid.xml… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/uncertainty.xml… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/info.json… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/shakemap.kmz… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/shape.zip… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/mmi_regr.png… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/pga_regr.png… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/pgv_regr.png… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa0p3_regr.png… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa1p0_regr.png… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa3p0_regr.png… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/attenuation_curves.json… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/raster.zip… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/intensity_uncertainty.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/intensity_uncertainty.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/pga_uncertainty.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/pga_uncertainty.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/pgv_uncertainty.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/pgv_uncertainty.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa0p3_uncertainty.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa0p3_uncertainty.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa1p0_uncertainty.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa1p0_uncertainty.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa3p0_uncertainty.pdf… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/psa3p0_uncertainty.jpg… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/rupture.json… Copying /home/shake/shakemap_profiles/default/data/73585021/current/products/stationlist.json… INFO – 2021-07-14 17:43:00 – transfer_scp.execute – 69 files sent. Message from sender: “69 files sent to remote host www2.cisn.org” INFO – 2021-07-14 17:43:00 – shake.main – Finished running command transfer_scp: Elapsed 88.16 secs INFO – 2021-07-14 17:43:00 – shake.main – Running command transfer_email INFO – 2021-07-14 17:43:00 – transfer_email.execute – Mail has already been generated for this event. Returning. INFO – 2021-07-14 17:43:00 – shake.main – Finished running command transfer_email: Elapsed 0.00 secs INFO – 2021-07-14 17:43:00 – shake.main – shake finished with event 73585021 (shakemap) [shake@mnlodd1new data]$
In some cases you may want to run individual modules for testing, etc.
Steps To Run Each Module:
First activate shakemap
conda activate shakemap
create using us evid (or use aqms_eq2xml aqms_db2xml as above to acquire from aqms drop in module)
sm_create –force 73584926
manually run each module (can do one at a time and selectively, for example to avoid auto transfer when testing)
shake –force 73584926 associate
shake –force 73584926 dyfi
shake –force 73584926 assemble
shake –force 73584926 model
shake –force 73584926 mapping
shake –force 73584926 contour
shake –force 73584926 coverage
shake –force 73584926 gridxml
shake –force 73584926 info
shake –force 73584926 kml
shake –force 73584926 shape
shake –force 73584926 plotregr
shake –force 73584926 raster
shake –force 73584926 uncertaintymaps
shake –force 73584926 rupture
shake –force 73584926 stations
To push to staging vs. production:
edit the pdl config /home/shake/run/params/config.ini There are two lines near the beginning defining the receiving hubs - confusingly named senders senders = prod01, prod02 #senders =pdldevel All three prod01, prod02, pdldevel are defined later in the configuration. I commented the first 'senders' line and uncommented the second line so products would get sent through to the staging earthquake website. I then run shake manually with the eventid. After the run, I reverse the comment changes in config.ini.
staging website links
https://jmfee-usgs.github.io/comcat-timeline/timeline.html#nc73584926
https://jmfee-usgs.github.io/comcat-timeline/timeline.html#nc73498145|staging-earthquake.usgs.gov
Acquire finite fault in json format. For example, we got this rupture.json file from Dave Wald used in the US (ShakeMap Central) ShakeMap and included it in the NC shakemap https://earthquake.usgs.gov/realtime/product/shakemap/nc73584926/us/1625968974704/download/rupture.json
(note: Checking on details for how rupture.json was resolved, for Antelope Valley we had InSAR data - see email from Bill Barnhardt)
4.4.6. Rupture Specification describes the format of the rupture.json file https://usgs.github.io/shakemap/manual4_0/sg_input_formats.html
Place rupture.json file in the event directory. For nc73584926, that's something like /home/shake/shakemap_profile/default/data/73584926/current After the rupture module runs, a similar file to the input is written to the subdir products.
Navigate to the model.conf configuration file and search for “bad_stations”
cd /home/shake/shakemap_profiles/default/install/config
Use vi or other editor to add station to exclude under the bad_station section. For example, to exclude NN.WAK following the July 2021 Antelope Valley earthquake:
vi model.conf
(do escape i to insert/edit in vi and escape :x! to save or escape :q! to quit without editing)
#---------------------------------------------------------------------------
# Bad stations: Stations may be manually flagged as "bad", and will be
# excluded from processing. Each line in the bad_stations section
# takes the form of "net.sta = ondate:offdate. E.g:
#
# CI.XYZ = 2017-04-29:2018-06-01
#
# The on date and off date take the form of YYYY-MM-DD and are compared
# with the earthquake date to determine if the station should be
# excluded or not. If there is no off date (i.e., the station is still
# bad), then the off date should be excluded but the ':' should be
# included:
#
# CI.WXY = 2017-04-29:
#
# If the start date is arbitrarily early, use 1970-01-01 and the station
# will be excluded for all dates prior to the stop date (if supplied).
#---------------------------------------------------------------------------
[[bad_stations]]
#excluded starting at 07-08-2021 Antelope Valley CA M6.0 earthquake (according to NN operator, bad station)
NN.WAK = 2021-07-07:2021-07-11
Note: as of 2021-07-12 NN had replaced the sensor at WAK so we adjusted the timeframe to include this station in future ShakeMap runs.
Some info on manual flagging is on the SM v4.0 wiki here: https://usgs.github.io/shakemap/manual4_0/sg_input_formats.html
(note: in SM3.5 grind.conf was used to exclude stations. This function is no longer included in SM4, so the steps above are the new approach to manually exclude stations)
The ShakeMap can be re-run to exclude the recently excluded station with the auto run command, i.e.:
conda activate shakemap
shake -a 73584926
You can check if station was manually flagged and excluded by looking at the station.json product file, i.e.:
{“type”: “Feature”, “id”: “NN.WAK”, “properties”: {“code”: “WAK”, “name”: “Walker, California w84”, “instrumentType”: “UNK”, “source ”: “NN”, “network”: “NN”, “commType”: “UNK”, “location”: “”, “intensity”: “null”, “intensity_flag”: “”, “intensity_stddev”: “null”, “pga”: “nul l”, “pgv”: “null”, “distance”: 9.556, “elev”: null, “vs30”: 648.35, “channels”: [{“name”: “HNE”, “amplitudes”: [{“name”: “pga”, “value”: 0.005, “units”: “%g”, “flag”: “ManuallyFlagged,Outlier”, “ln_sigma”: 0.0}, {“name”: “pgv”, “value”: 0.0012, “units”: “cm/s”, “flag”: “ManuallyFlagged ,Outlier”, “ln_sigma”: 0.0}, …
info on fault source, Vs30, etc here link to central github for CISN here (USGS gitlab?)
shake –cancel <evid>
Example. With the Dec 2020 M6.2 Petrolia Event, later in Jan 2022 the event was separated into a foreshock and mainshock. We wanted to remove the M5.7 foreshock shakemap that was auto generated
We had to modify the config shake.conf to remove commented out cancel # cancel_modules: If the -c (–cancel) option is used, this # parameter specifies teh modules that will be run and if the # –cancel option will be passed onto these other modules. # # cancel_modules = module1 –cancel module2 –cancel # #cancel_modules = transfer_pdl –cancel transfer_email –cancel
to cancel_modules = transfer_pdl –cancel
command is
shake –cancel 71127029
The cancellation was successful and example output for this run was (Note: shakemap may need to be removed manually on CISN and backup pages, contact Jennifer Taggart). Right now this event appeared on the new and old CISN shakemap pages, respectively - https://www.cisn.org/shakemap/ and https://www.cisn.org/shakemap/nc/shake/archive/
(shakemap) [shake@mnlodd1new config]$ shake –cancel 71127029 In /home/shake/.conda/envs/shakemap/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: The text.latex.preview rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later. In /home/shake/.conda/envs/shakemap/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: The mathtext.fallback_to_cm rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later. In /home/shake/.conda/envs/shakemap/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: Support for setting the 'mathtext.fallback_to_cm' rcParam is deprecated since 3.3 and will be removed two minor releases later; use 'mathtext.fallback : 'cm' instead. In /home/shake/.conda/envs/shakemap/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: The validate_bool_maybe_none function was deprecated in Matplotlib 3.3 and will be removed two minor releases later. In /home/shake/.conda/envs/shakemap/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: The savefig.jpeg_quality rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later. In /home/shake/.conda/envs/shakemap/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: The keymap.all_axes rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later. In /home/shake/.conda/envs/shakemap/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: The animation.avconv_path rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later. In /home/shake/.conda/envs/shakemap/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: The animation.avconv_args rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later. INFO – 2022-01-14 22:35:44 – shake.main – Cancelling shakemap, using cancel_modules… INFO – 2022-01-14 22:35:44 – shake.main – Cancel modules found: ['transfer_pdl', '–cancel'] INFO – 2022-01-14 22:35:44 – shake.main – Running command transfer_pdl INFO – 2022-01-14 22:36:24 – shake.main – Finished running command transfer_pdl: Elapsed 38.82 secs INFO – 2022-01-14 22:36:24 – shake.main – shake finished with event 71127029
To switch the ShakeMap instance from MP and UCB and vice-versa, simply remove/add the transfer module to the list of modules to be run for an incoming event in the configuration file shake.conf.
shake.conf can be found and edited under the directory /home/shake/shakemap_profiles/default/install/config
When the shakemap instance is in production mode (not backup) it will include the transfer_pdl, transfer_scp, and transfer_email modules like below (select implemented in CA 12/6/2022):
autorun_modules = aqms_eq2xml aqms_db2xml associate dyfi select assemble model mapping contour coverage gridxml info kml shape plotregr raster uncertaintymaps rupture stations transfer_pdl transfer_scp transfer_email
(double checking on this step)
One of the purposes of ShakeMap is to show as good a representation of ground shaking as possible for an event. The earthquake parameters (time, location, magnitude) are the starting point. But ShakeMap is not much good without measurements of ground shaking. For NCSS, that means ground motion amplitudes, which are stored in the amp table of NCSS AQMS databases. The NCSS RT systems determine ground motion amplitudes for all the broadband and accelerometer waveforms that are processed by those RT systems. There are many other stations whose waveforms do not go through the NCSS RT systems. Various systems (Caltech's AQMS system, CGS, NSMP, etc.) process those non-NCSS waveforms and submit ground motion amps to the amp import system running on the post-processing systems. The result is that ground motion amps can trickle into the NCSS AQMS databases over hours, days, months after significant earthquakes.
There is no easy way for ShakeMap or the ShakeMap operator to determine whether new amps are in the database since the last time a ShakeMap was produced. If in doubt, run retrieve by hand and see if more amps are collected than the previous time.
In both systems are the following directories under the ShakeMap base directory:
The main ShakeMap components that are of interest are:
queue is the program for controlling the automatic running of the ShakeMap programs. It runs all the time, listening on TCP port 2345 (hard-coded).
ACTION EVENT_ID REPEAT where ACTION is shake_alarm or shake_cancel. Apparently REPEAT is ignored by queue.Metro box in queue.conf.queue is configured with a repeat schedule based on event magnitude; see queue.conf. Larger events have longer repeat schedules, which causes queue to run shake in case more ground motion amplitude data becomes available in the AQMS database. The repeat schedule is based on the event origin time, not the time at which queue was notified of the event.
Note that queue will get signaled by an event several times after the event
is detected by the RT system. These signals would result from both instances
of ampgen (early and late), from TMTS, from the Duty Review Page when the
event is accepted, from TMTS web interface for a manual moment tensor, and
when the event is finalized in jiggle. Because of all these signals from AQMS,
we have reduced the number of scheduled repeats in queue.conf. We have had
complaints from USGS EHP when we generate too many versions of a ShakeMap,
so now we try to limit the number. Manual signaling of queue may be needed
when new ground motion amps get added to the database.
CISN maintains and ad hoc list of station parameters useful for some ShakeMap related programs. On the ShakeMap systems this file is lib/grind/adhoc.lis. On the post-processing systems, the file is /home/ncss/ncpp/ampexc/import/conf/adhoc.lis, where it is used by the amp import system.
The fields are Station code, Network code, SEED Channel and location codes, COSMOS Table 6 Site code, latitude, longitude, elevation, and the SNCL's descriptive name.
This file is built from smaller files maintained by UC Berkeley, Caltech, and CGS (California Geological Survey). The script getalladhocs (in the ShakeMap bin directory and in /home/ncss/ncpp/bin/) to query these three sources for their parts of the ad hoc list and then merges this into the adhoc.lis file in the current directory. The ShakeMap operator must run periodically to keep the adhoc.lis files up to date, say every 2 weeks.
The UC Berkeley portion of the ad hoc list is maintained on rodgers.geo.berkeley.edu in /home/ncss/run/StationLists. In this directory, the script genadhoc.pl (no options needed) writes to stdout, which can be redirected to adhoc_NCSS.lis-new. Compare this with adhoc_NCSS.lis; if you are satisfied with the changes, move adhoc_NCSS.lis-new to adhoc_NCSS.lis. At the same time, you should use the script genStaList.pl to update ncss_stalist. This second list is used by some process at Caltech to generate a KML map of CISN stations. These two files are served to the web by www.ncedc.org. There are symbolic links to the files on rodgers in /data/dc6/ftp/pub/doc/CISN.info/. The Berkeley portion of the ad hoc list contains data for these networks: BG,BK,GM,GS,LB,NC,NN,NP,PG,TA,UO,US,UW,WR. Thus there is no separate list for Menlo Park stations.
For earthquakes up to about magnitude 6, the fault source can be considered a
point. For larger earthquakes, ShakeMap makes better maps when provided with a
description of the fault rupture surface. This is done by creating a file
named ANYTHING_fault.txt in the event input directory, where ANYTHING
can be whatever you want. To remind you how to do this, in the ShakeMap config
directory there is a file named std_fault.txt containing comments about
this. Here's a sample file used for the South Napa M6.0 event of 2014/08/24:
# Jack Boatwright's interpolation of Doug Dreger's finite fault # For SM 3.5 and later, each line must consist of lines containing # latitude, longitude, depth points to form one or more closed polygons # Disconnected polygons should be separated by lines containing only ">". 38.220 -122.313 2 38.220 -122.313 11 38.310 -122.333 11 38.310 -122.333 2 38.220 -122.313 2
ShakeMap operators need to seek out the fault coordinates from seismologists for any events magnitude 6 and larger. This should be done within a few hours of the earthquake, not later than 24 hours later!
Whichever is the production ShakeMap system for NCSS sends ShakeMaps to the following locations. This is controlled by transfer.conf on the production ShakeMap system.
As mentioned above, ShakeMap (genex) generates html pages containing the fill
catalog of ShakeMaps. The information about the catalog is maintained in the
ShakeMap MySQL database. The html for the catalog is generated using a
template file: lib/genex/web/config/archivepages.xml. This file needs to
manually updated at the start of each year to close out the section for the
previous year and start a new section for the new year. Scroll down to the
line <archivepages template=“archive”>. You should be able to figure out
what to do!
ShakeMap needs a MySQL database to store information about earthquakes it has processed and about the command flags used for each run of ShakeMap. This last feature allows you to manually run a ShakeMap program like grind with special options to adjust the map size and location. These options will be remembered in the database so that future runs of shake (manually or drive by queue) will use the same options for that event. There is a detailed discussion of this in the ShakeMap manual.
For the NCSS, MySQL is started by system init scripts at boot time. The two databases are NOT replicated between each other.
The MySQL database on rodgers.geo.berkeley.edu is backed up nightly using a cron job /home/ncss/mysql.backup/bin/dump.csh. The backup data (text files) are stored in /home/ncss/mysql.backup/data/ where they are backed up to tape by the BSL backup system.