NASA/NOAA MOU Annex

Fall AGU 2018 Abstract

NOAA SWPC / CCMC Partnership to Validate Heliospheric Models
CCMC: M. Leila Mays, Peter Macneice, Aleksandre Taktakishvili,
NOAA/SWPC: Eric Adamson, Vic Pizzo, Doug Biesecker

In 2017 NOAA SWPC and CCMC started a new project under an annex to a memorandum of understanding between NASA and NOAA. The purpose of this project is to assess improvements in CME arrival time forecasts at Earth using the Air Force Data Assimilative Photospheric Flux Transport (ADAPT) model driven by data from the Global Oscillation Network Group (GONG) ground observatories. These outputs are then fed into the coupled Wang-Sheeley-Arge (WSA) - ENLIL model and then compared to the current operational version of WSA-ENLIL (without ADAPT). The project is performed in close collaboration with model developers.

SWPC has selected a set of 33 historical events over the period of three years from 2012-2014. The overall two-year project consists of multiple simulation validation studies for the entire event set: (a) benchmark (current operational version) (b) time-dependent sequence of GONG maps driving WSA-ENLIL (c) single test simulation of a time-dependent sequence of GONG maps driving ADAPT-WSA-ENLIL (d) single GONG map driving ADAPT-WSA-ENLIL (e) time-dependent sequence of GONG maps driving ADAPT-WSA-ENLIL. After each stage, the performance of the new simulation results will be compared to the benchmark (stage a) and to other previous stages. Here we present preliminary validation results for stages (a)-(c) that test the performance when using time-dependent backgrounds.
Download this presentation [PPTX]

Final Report

Final Report [PDF] (delivered 2020-05-01)

Final Report Simulation Downloads

Full download=evo *dat, evo graphics, bnd graphics, and contour plot graphics for all runs. Does not include full data cube netCDF files but these available upon request.

Downloads for Annex Tasks (a) and (b)

⊳ Download plasma parameters time-series plots comparing simulation results for (a-b), 1(a-b), 2(a-b), 3(a-b), (d-e) at Earth:
⊳ Full plots | Full plots - speed only
⊳ Zoomed in plots near the CME arrival time | Zoomed plots - speed only
⊳ (a-b) Full download (9.3 GB) GONGb WSA 2.2 ENLIL 2.6.2 a3b2 (operational prior to May 2019)
⊳ 1(a-b) Full download (1.5 GB) GONGb WSA 2.2 ENLIL 2.9e a8b1 (operational after May 2019)
⊳ 2(a-b) Full download (1.5 GB) GONGz WSA 4.5 ENLIL 2.6.2 a3b2
⊳ 3(a-b) Full download (1.5 GB) GONGz WSA 4.5 ENLIL 2.9e a8b1

Downloads for Annex Tasks (c), (d) and (e)

GONGz ADAPT WSA 4.5 ENLIL 2.9e a8b1:
⊳ (d-e): Download time-series plots comparing simulation results for (d-e) at Earth:
⊳ Full plots | Full plots - speed only
⊳ Zoomed in plots near the CME arrival time | Zoomed plots - speed only
⊳ (d-e): Full download (24 GB)
⊳ Input file listing for (d) and (e)
⊳ (c): Full download of single event resolution test (2014-08-16), and bnd graphics download.

General Downloads

⊳ Download CME arrival time error statistics for all simulations.
⊳ Bonus plots (not included in report) of CME arrival time error vs Ambient Solar Wind Speed Error: unfiltered, filtered by CME input speed: 500 km/s, 800 km/s, 1100 km/s, 1400 km/s
The ambient solar wind speed error was calculated as the average observed (1 hour OMNI) solar wind speed subtracted from the simulated ambient solar wind speed for the first day of each simulation.
⊳ Logs of input map counts: (a) (b) 1(a) 1(b) 2(a) 2(b) 3(a) 3(b) (d) (e)

General project information:

Results: Year 1 interim report [PDF]
Latest project timeline (201902) [PDF] (old project timelines: v201807 [PDF]; v20170720 [PDF])
Year 1 simulation variations planning spreadsheet
Initial project storage and CPU estimate (to be revised for 2019-2020)
Initial Status of Plans of CCMC Activities for the Annex (working document updated on 20170703) [PDF]
- CME Arrival Time working team latest report [PDF] and homepage.

Discussion Notes

Events

• Initially NOAA/SWPC selected a set of 33 historical events and 36 CME input parameters have been provided (3 events contain 2 CMEs) for the period of 2012-2014. Later, NOAA/SWPC added 5 more events in 2017-2019 to include events that can use more recent zero point corrected synoptic maps. All 38 events are available to view from CCMC's public DONKI database via an API in text or JSON format (if your are prompted for a password, please click the link again and it will bypass the login screen).
• Replication event subset for which input/output is available [TXT]

Previous Results for Annex Tasks a-c (Single and Time-dependent maps) - updated February 2019

⊳ Results: Year 1 interim report [PDF]
⊳ Download time-series plots comparing the benchmark single map simulations with time-dependent map simulations for all events (8 MB).
⊳ Same plots as above but with fixed y-axis scales for better visibility of OMNI data.
⊳ Full download of evo *dat, evo graphics, bnd graphics, and contour plot graphics for all runs (multi-block simulations for all cases) (7.3 GB).
⊳ Download pptx comparing simulation movies and timelines for events [PPTX]
⊳ Run replication outliers: See explanation google sheet and pptx
⊳ Logs of input map counts: (a) (b) 1(a) 1(b) 2(a) 2(b) 3(a) 3(b) d e
See page 1 of the year 1 report_ for a description of tasks a-e; see this spreadsheet for a description of tasks a-b and variations._

Test Simulations

Resolution Tests:

⊳ Repeat of one CME replication test swpc_test_070617_SH_2 with different grids.
Resolution: low (256x30x90), medium (512x60x180), high (1024x120x360)
Plot comparing temporal profiles at Earth for low, medium, and high resolution for three runs above.
⊳ Also see resolution comparison plots below.

Replication of event subset and resolution tests:

Comparison of GONG "mrbqs" and "mrzqs" 2007-2017 (with metrics)

From Dusan:
⊳ Time-depdendent 24hr cadence gongb (mrbqs) time series plot at Earth (low resolution) [GIF]
⊳ Time-depdendent 24hr cadence gongz (mrzqs) time series plot at Earth (low resolution) [GIF] and (medium resolution [GIF])
_Also see other plots in Dusan's calibration folder on _helioweather.net

Example of single map vs. time dependent inner boundary simulations:

All using gongb mrbqs: ftp://gong2.nso.edu/QR/bqs/.To see graphics for each run linked below, click on "list of quicklook graphics links" on the run page

Bain et al. ApJ 2016:

Single Map: August 2010
Time-dependent (24hr cadence): July 2012
Time-dependent (2hr cadence): July 2012

Luhmann et al. SWJ 2017:

Single Map: August 2010
Time-dependent (24hr cadence): March 2013
Time-dependent (24hr cadence): March 2015
(also see July 2012 above)

Luhmann et al. SWJ 2018; Lee et al. GRL 2018 (submitted):

Time-dependent (2hr cadence): July 2017
Time-dependent (2hr cadence): Sep 2017

Other (mostly prelim):

Time-dependent (4hr cadence): June 2011
Time-dependent (24hr cadence): Nov 2011
Time-dependent (4hr cadence): Jan 2012
Time-dependent (24hr cadence): March 2012
Time-dependent (24hr cadence): May 2012
Time-dependent (24hr cadence): Sep 2014
Time-dependent (24hr cadence): Oct 2014
Time-dependent (24hr cadence): Dec 2014
Time-dependent (5hr cadence): May 2016

Replication tests:

• CCMC v2.9d (version Dec 2017/Jan 2018) replication tests:
(repeat of tests below, added "b" label):
⊳ swpc_wsaenlil_bkg_20170720_0000 (18100) wcone_rep1b (tgz archive of run) and v2.6 and v2.9d temporal profile comparison gifs.
⊳ wsa_enlil_cme_09262017 (1446) wcone_rep2b (tgz archive of run) v2.6 and v2.9d temporal profile comparison gifs.
⊳ wsa_enlil_amb_09242017 wcone_rep3b (tgz archive of run) v2.6 and v2.9d temporal profile comparison gifs.

• From Dusan (September 2017):
⊳ (1) slides [PDF] and notes [TXT] on v2.9 and v2.6 replication tests.
⊳ (2) slides [PDF] and notes [TXT] on v2.9 and v2.6 replication tests.
• CCMC v2.9d (version Sep 2017) replication tests:
⊳ swpc_wsaenlil_bkg_20170720_0000 (18100) wcone_rep1 (tgz archive of run) and v2.6 and v2.9d temporal profile comparison gifs.
⊳ wsa_enlil_cme_09262017 (1446) wcone_rep2 (tgz archive of run) v2.6 and v2.9d temporal profile comparison gifs.
⊳ wsa_enlil_amb_09242017 wcone_rep3 (tgz archive of run) v2.6 and v2.9d temporal profile comparison gifs.

temporal profile comparison gifs:
swpc_test_070617_SH_2_gifs
swpc_test_081517_SH_1_gifs
swpc_test_081517_SH_2_gifs

Magnetogram input time cadence tests:

CR 2070 high resolution ambient simulations - 3 hr output (ENLIL v2.8f):

Full rotation magnetograms:

Daily-update magnetograms:

24 hr cadence input:
• WSAdt-GONGb
• WSAdt-GONGz

6 hr cadence input: (Note: 1 hr cadence GONG maps are not available for this CR)
• WSAdt-GONGb