Next Steps
Current Status (2021-10-21)
A method to calculate observational quantities for validation purposes has been created (https://github.com/ktindiana/operational-sep).
A draft code has been written which validates a variety of predicted values - see the Validation Framework page and talks describing the progress on the Conferences page.
Next Steps
Following the discussions and feedback at ISWAT 2021, we have identified these next steps in the validation effort.
The campaign organizers will:
- Create a balanced list of SEP events and non-events that will allow a statistically significant validation
- Focus on events in cycles 24 and possibly 23 to allow as many models as possible to participate
- Provide a standard set of flare and CME inputs, including uncertainties on the CME parameters
- Provide suprathermal seed spectra
- Create a pre-event definition for a forecast - i.e. specify a time stamp for each event after which no data may be used to make the forecast
- Models that make predictions using data prior to that time will be validated as operational-type forecasts
- Models that make predictions using data after that time may be validated from a scientific perspective, but the results would not inform operational useability
- Investigate the feasibility to provide multi-point observations for the SEP event list (e.g. STEREO A & B) for validation of 2D and 3D physics-based models
- Encourage all modelers with models that have the ability to operate in real-time to work with CCMC to be integrated into the SEP Scoreboard (contact Leila Mays, m.leila.mays@nasa.gov) - this would allow a true validation of forecasting capabilities using events going forward
- Work with the Moon to Mars Space Weather Analysis Office to develop an automated method to validate models running and sending forecasts in real-time to the SEP Scoreboard
- Add validation of the event-integrated differential fluence spectrum
- Improve formatting of the reports produced by the validation code
- Revisit the "onset peak" algorithm in the OpSEP code