Operational geospace model validation
Groundbased magnetic perturbations dB/dt study
Presentations from GEM 2012 Summer Workshop (Snowmass, CO)
About the study
To support the transition of space weather research models to operations, the CCMC is leading the effort to assess the ability of geospace models to forecast geomagnetic disturbances. This geospace model validation project is built upon the ground magnetic perturbations (delta-B) metrics study which is a part of the GEM Modeling Challenge (Pulkkinen et al., 2010, 2011; Rastaetter et al., 2011).
The community-wide GEM Modeling Challenge was initiated in the summer of 2008 by the GGCM Metrics and Validation Focus Group. The CCMC is supporting the Challenge.
NOAA SWPC is planning to utilize the results of this geospace model validation project in their selection of operational geospace model. The time derivative of the horizontal magnetic field vector dB/dt and regional K index were identified as physical parameters of primary interest to NOAA SWPC customers.
Three global MHD magnetosphere models (SWMF v20110131, OpenGGCM v4.0, and CMIT-LFM-MIX vLTR-2_1_1) and two empirical delta-B models (Weimer and Weigel) participate in this activity.
Global MHD models were delivered to CCMC in January 2011. There was extensive communication with the model developers to ensure the correct installation and appropriate settings of the models. Model installation, robustness tests, necessary revisions performed by the code developers in response to CCMC feedback, and the final selection of model settings were accomplished by mid August 2011. Test simulations for all models were performed on 64 processors (or less) of the CCMC Beowulf Cluster. Simulation settings for all models were selected so that the simulations take no longer than twice the duration of the simulated event.
In support of the project the CCMC developed a tool that extracts the ground magnetic field perturbations from the global MHD model outputs (through integration over magnetospheric and ionospheric current systems).
The CCMC performed a series of sensitivity studies: sensitivity to methods of dB/dt calculations (e.g.,the CCMC post-processing tool vs. run-time calculations implemented in SWMF); sensitivity to ground station locations sensitivity to model output frequency. The results of the studies were presented to code developers and discussed at special sessions of the GEM and mini-GEM Workshops.
One of the lessons learned from the GEM Modeling Challenge is that the results of metrics studies strongly depend on the selected metrics type (how the skill score characterizing the model performance is calculated). The CCMC has performed a series of test metrics studies on the outputs of the GEM Challenge simulations. Different approaches to skill score calculations were demonstrated and discussed. Per discussion with the model developers and NOAA SWPC it was agreed to implement the threshold-based model-data comparisons and calculate the skills such as probability of detection (POD), probability of false detection (POFD), and Heidke skill score [Lopez et al, 2007, Pulkkinen et al, 2010].
Six events that include the original four GEM Challenge events and two "surprise" events were selected for the final evaluation. All simulations were conducted at the CCMC. Full 3D results are available for on-line visualization. Calculated ground magnetic perturbations dB/dt for all models participating in the study together with observational data for each event and ground stations can be viewed using the online time-series plotting tool.
The results of the operational geospace model validation based on time derivatives of magnetic field perturbations (dB/dt) will be presented and discussed at the upcoming GEM 2012 Summer Workshop.
- Selected events
- Groundbased stations list
- >> STUDY RESULTS in the time plotting tool
- Download delta B and dB/dt timeseries files
- Model runs used in dB/dt studies which were performed at the CCMC
- A. Pulkkinen, M.Kuznetsova, A.Ridley, J.Raeder, A.Vapirev, D.Weimer, R.S.Weigel, M.Wiltberger, G.Milward, L.Rastaetter, M.Hesse, H.J.Singer, A.Chulaki, Geospace Environment Modeling 2008-2009 Challenge: Ground magnetic field perturbations, Space Weather Journal, vol.9, SO2004, doi:10.1029/2010SW000600, 2011.
- A.Pulkkinen, L.Rastaetter, M.Kuznetsova, M.Hesse, A.Ridley, J.Raeder, H.J.Singer, A.Chulaki, Systematic evaluation of ground and geostationary magnetic field predictions generated by global magnetohydrodynamic models, JGR, vol.115, A03206, doi:10.1029/2009JA014537, 2010.
- R.Lopez, S.Hernandez, M.Wiltberger, C.-L.Huang., E.L.Kepko, H.Spence, C.C.Goodrich, J.G.Lyon, Predicting magnetopause crossings at geosynchronous orbit during the Halloween storms, Space Weather, vol.5, S01005, doi:10.1029/2006SW00222, 2007
- Sensitivity analyses for evaluation of ground magnetic field predictions, A.Pulkkinen, M.Kuznetsova, L.Rastaetter, 2011
- GEM 2008-2009 Challenge: ground magnetic field perturbations, A.Pulkkinen, M.Kuznetsova, A.Ridley, J. Raeder, D.Weimer, R.Weige, M.Wiltberger, G.Millward, L.Rastaetter, M.Hesse, H.J.Singer, A.Chulaki, AGU GEM mini-workshop, 2009
- From GEM metrics studies to operational geospace model selection, A.Pulkkinen, M.Kuznetsova, M.Hesse, L.Rastaetter, A.Chulaki, J.S.Shim, GEM-CEDAR Workshop, 2011
- Geospace Model Project Goals, Howard Singer, Geospace Model Workshop, 2011
- Model Delivery to CCMC, Masha Kuznetsova, Geospace Model Workshop, 2011
- Assessing Geospace Models for Use in Space Weather Operations, Howard Singer, Space Weather Workshop, 2012
