International Forum for Space Weather Capabilities Assessment
Note: The International Forum for Space Weather Capabilities Assessment effort has been transition to the ISWAT. Please see the ISWAT website for latest updates related to any active forum activities.
- To define metrics to assess the current state of space weather modeling capabilities from the perspective of:○ end-users
○ science for space weather
- Develop a process to capture science progress in first principles models that feed into operations.
This international community-wide forum is brings together space environment experts, model and application developers, data providers, forecasters and end-users of space weather products and services to
- establish internationally recognized metrics and benchmarks that are meaningful and informative to end-users, developers, and decision makers;
- evaluate the current state of space environment models, applications and forecasting techniques;
- facilitate communications, understanding and partnerships between forecasters and researchers;
- address challenges in data-model comparison, including data quality and availability, uncertainties and sensitivities to external drivers, internal parameters and assumptions;
- identify a path for evaluation and accelerated incorporation of scientific ideas with potential to improve forecasting into space weather applications;
- quantify and track progress over time and communicate it to end-users and the community;
- facilitate further progress.
Expected Deliverables from the Working Team
Short-term deliverables (by the end of the April 2017 workshop):
- Set of metrics to quantify and track progress over time;
- Quantifying scientific progress team: catalog of metrics to aid in tracking progress of LWS program towards its goals.
- Space weather application working teams: catalog of metrics tailored for specific applications and end-users needs.
- Initial model evaluations;
- SPASE metadata templates for model output.
- Present results for community feedback and contributions at the upcoming meetings
e.g., Space Weather Workshop at NOAA/SPWC (May 2-5, 2017), Applied Space Environments Conference (Huntsville, Alabama, May 15-19, 2017), JpGU-AGU joint meeting (May 20-25, 2017), GEM and CEDAR Workshops (June 2017) , SHINE Conference (July 24th-28th, 2017 ), 2017 IAU Symposium “Space Weather of the Heliosphere: Processes and Forecasts” (July 17-21, 2017), ESWW14 (TBD), Fall AGU and mini-GEM (December 2017).
- Comprehensive model evaluations;
Model Evaluation Guide For Working Teams
International Forum for Space Weather Capabilities Assessment Below is a suggested guide for initial model evaluations of the first handful of events. This list is just a starting point to help working teams begin defining goals, tasks, and discussion items.
- Determine the physical parameters for model-data comparison.
- Determine the observational data sources.
○ Prepare SPASE metadata for the observations.
○ CCMC and Data Access Working Team will provide support with metadata generation.
- Determine the time intervals or event lists.
○ E.g., storms + preceding quiet time periods. Include a number of events, including long-duration events, and preceding quiet time periods.
- Identify sources for uncertainties.
○ E.g., due to external drivers, internal assumptions, spatial and temporal resolution, etc.
○ Suggest an approach to quantify those uncertainties.
- Determine a set of metrics relevant for specific applications, user needs, and science needs.
○ Different forecasting skill scores can represent different aspects of model or forecasting techniques performance (e.g., uncertainty, accuracy, reliability, temporal and spatial aspects).
○ A "message to the user" for each forecasting skill score is desirable to make it clear what useful information can be derived from current state of space environment modeling.
- Invite modelers to submit their results for the selected events.
○ At least one model is needed for the initial assessment.
- Perform model assessment for a handful of events with at least one model
After initial validation has been completed the working team may continue to discuss the metrics, techniques, and invite more modelers/data providers for a comprehensive validation that may be written up in a paper by the team. Click here for a description of expected deliverables from the working teams.
Team Member Responsibilities/Expectation
Working team member responsibilities:
- Actively contribute towards working team goals. Examples:
Provide/obtain data, information on impacts, communicate user needs, formulate skills scores/metrics, perform simulations, provide support with databases and metadata, bring expertise, ideas, share experience, etc.
- Support and provide feedback to leads in meeting deliverable/implementing a work plan
Working team lead responsibilities:
In general, the lead organizes the team’s work and ensures that the primary goals of the team are met. Some sample responsibilities could be:
- General responsibilities
- Invite additional participants (model developers, data providers, users of space weather products and services - see above), that would like to actively contribute to team goals
- Responsible for deliverables for the working team
- Lead/monitor progress of the working team in meeting its goals/deliverables (coordinate team communication and activities, assign action items, etc.)
- Coordinate communication with other working teams especially the Data Access Working Team, which is responsible for providing support on the generation of the meta-data needed for the validation effort
- In preparation to working meeting/workshop:
- Define a list of deliverables prior to the workshop
- Develop a Work plan on meeting the deliverable
- Finalize a schedule/format of the working meeting for their specific session
- During the working meeting:
- Organize and chair discussion sessions
- Present updates on progress/deliverable to the broader forum participants
- After the working meeting:
- Refine a list of deliverable and work plan for the next 6-12 months based on inputs/feedback from the participants
- Interested in following the forum/team(s) but not ready for an active role?
Subscribe to be a FOLLOWER. You will receive periodic working team updates (from all subscribed to teams) via the mailing list.
The CCMC together with the international community (International CCMC) facilitates this forum by:
- Developing interactive web interfaces, display and analysis tools, maintain archives.
- Supporting the implementation of SPASE Metadata Model XML Schema. Any format from modelers will be accepted and converted to the appropriate format by the CCMC team/partners.
- Organizing working meetings
List of Focused Evaluation Topics
- SUPERTOPIC: QUANTIFYING SCIENTIFIC PROGRESS (CCMC facilitator(s): B.Thompson)
- Assessment of Understanding and Quantifying Progress Toward Science Understanding and Operational Readiness
(Leads: A. Halford, A. Kellerman, K. Garcia-Sage, B. Thompson, S. Morley)
- SOLAR (CCMC facilitator(s): P. Macneice)
- Solar Flare Prediction (Leads: S. Murray, M. Georgoulis, S. Bloomfield, K.D. Leka; Scoreboard Leads: S. Murray, M.L Mays)
- Coronal & Solar Wind Structure: Coronal & SW Structure; Ambient SW; Coronal Hole Boundaries (Leads: P. Macneice, L. Jian)
- 3D CME kinematics and topology (Leads: B.Thompson, C.Moestl, D.Barnes)
- Solar Indices and Irradiance (Leads: J. Klenzing, C. Henney, K. Muglach)
- HELIOSPHERE (CCMC facilitator(s): M.L. Mays, A. Taktakishvili, P. Macneice)
- CME Arrival Time (Leads: C. Verbeke, M.L. Mays, A. Taktakishvili)
- IMF Bz at L1 (Leads: N. Savani, P. Riley)
- SEPs (Leads: I.G. Richardson. P. Quinn, M. Marsh, M.L. Mays
Scoreboard Leads: M. Dierckxsens, M. Marsh)
- GEOSPACE: Auroral Region (CCMC facilitator(s): M.Kuznetsova)
- Auroral precipitation and high latitude ionosphere electrodynamics (Leads: R. Robinson, Y. Zhang, B. Kosar)
- GEOSPACE: Geomagnetic Environment (CCMC facilitator(s): L.Rastaetter)
- Ground Magnetic Perturbations: dBdt, delta-B, GICs, FACs (Leads: D. Welling, H. Opgenoorth, C. Ngwira)
- Geomagnetic Indices (Leads: M. Liemohn)
- Magnetopause location and geosync. orbit crossing (Leads: Y. Collado-Vega, S. Merkin)
- IONOSPHERE (CCMC facilitator(s): K. Garcia-Sage, J. Yue, M. Kuznetsova)
- Neutral Density and Orbit Determination at LEO (Leads: S. Solomon, T. Fuller-Rowell, S. Bruinsma, E. Sutton)
- Global & Regional TEC (Leads: L. Scherliess, R. Calfas)
- Ionosphere Plasma Density: NmF2/foF2, hmF2, TEC (Leads: I. Tsagouri, M. Angling, K. Garcia-Sage, J. Yue)
- Ionosphere Scintillation (Leads: E. Yizengaw)
- RADIATION and PLASMA EFFECTS Scope of work (CCMC facilitator(s): Y. Zheng, M. Kuznetsova)
- Surface Charging few eV - keV electrons, plasma density (Leads: J. Minow, D. Pitchford, N. Ganushkina)
- Internal Charging keV–MeV electrons (Leads: P. O'Brien, Y. Shprits)
- Single Event Effects MeV–GeV-TeV protons, ions (Leads: M. Xapsos, J. Mazur, P. Jiggens)
- Total Ionizing Dose keV–MeV electrons, keV–GeV protons,ions (Leads: I. Jun, T. Guild, M. Xapsos)
- Radiation effects for aviation (Leads: K. Tobiska, M. Meier)
- INFORMATION ARCHITECTURE (CCMC facilitator(s): C. Wiegand)
- Information Architecture for Interactive Archives (IAIA) (Leads: C. Wiegand, D. Heynderickx, D. De Zeeuw, T. King)