Last Updated: 09/09/2022

International Forum for Space Weather Capabilities Assessment

Note: The International Forum for Space Weather Capabilities Assessment effort has been transition to the ISWAT. Please see the ISWAT website for latest updates related to any active forum activities.

Goals

  1. To define metrics to assess the current state of space weather modeling capabilities from the perspective of:○ end-users
    ○ science for space weather
  2. Develop a process to capture science progress in first principles models that feed into operations.

This international community-wide forum is brings together space environment experts, model and application developers, data providers, forecasters and end-users of space weather products and services to

  • establish internationally recognized metrics and benchmarks that are meaningful and informative to end-users, developers, and decision makers;
  • evaluate the current state of space environment models, applications and forecasting techniques;
  • facilitate communications, understanding and partnerships between forecasters and researchers;
  • address challenges in data-model comparison, including data quality and availability, uncertainties and sensitivities to external drivers, internal parameters and assumptions;
  • identify a path for evaluation and accelerated incorporation of scientific ideas with potential to improve forecasting into space weather applications;
  • quantify and track progress over time and communicate it to end-users and the community;
  • facilitate further progress.

Expected Deliverables from the Working Team

Short-term deliverables (by the end of the April 2017 workshop):

  • Set of metrics to quantify and track progress over time;
    • Quantifying scientific progress team: catalog of metrics to aid in tracking progress of LWS program towards its goals.
    • Space weather application working teams: catalog of metrics tailored for specific applications and end-users needs.
  • Initial model evaluations;
  • SPASE metadata templates for model output.
  • Present results for community feedback and contributions at the upcoming meetings
    e.g., Space Weather Workshop at NOAA/SPWC (May 2-5, 2017), Applied Space Environments Conference (Huntsville, Alabama, May 15-19, 2017), JpGU-AGU joint meeting (May 20-25, 2017), GEM and CEDAR Workshops (June 2017) , SHINE Conference (July 24th-28th, 2017 ), 2017 IAU Symposium “Space Weather of the Heliosphere: Processes and Forecasts” (July 17-21, 2017), ESWW14 (TBD), Fall AGU and mini-GEM (December 2017).

Long-term deliverables:

Model Evaluation Guide For Working Teams

International Forum for Space Weather Capabilities Assessment Below is a suggested guide for initial model evaluations of the first handful of events. This list is just a starting point to help working teams begin defining goals, tasks, and discussion items.

  • Determine the physical parameters for model-data comparison.
  • Determine the observational data sources.
    ○ Prepare SPASE metadata for the observations.
    ○ CCMC and Data Access Working Team will provide support with metadata generation.
  • Determine the time intervals or event lists.
    ○ E.g., storms + preceding quiet time periods. Include a number of events, including long-duration events, and preceding quiet time periods.
  • Identify sources for uncertainties.
    ○ E.g., due to external drivers, internal assumptions, spatial and temporal resolution, etc.
    ○ Suggest an approach to quantify those uncertainties.
  • Determine a set of metrics relevant for specific applications, user needs, and science needs.
    ○ Different forecasting skill scores can represent different aspects of model or forecasting techniques performance (e.g., uncertainty, accuracy, reliability, temporal and spatial aspects).
    ○ A "message to the user" for each forecasting skill score is desirable to make it clear what useful information can be derived from current state of space environment modeling.
  • Invite modelers to submit their results for the selected events.
    ○ At least one model is needed for the initial assessment.
  • Perform model assessment for a handful of events with at least one model

After initial validation has been completed the working team may continue to discuss the metrics, techniques, and invite more modelers/data providers for a comprehensive validation that may be written up in a paper by the team. Click here for a description of expected deliverables from the working teams.

Team Member Responsibilities/Expectation

Working team member responsibilities:

  • Actively contribute towards working team goals. Examples:
    Provide/obtain data, information on impacts, communicate user needs, formulate skills scores/metrics, perform simulations, provide support with databases and metadata, bring expertise, ideas, share experience, etc.
  • Support and provide feedback to leads in meeting deliverable/implementing a work plan

Working team lead responsibilities:

In general, the lead organizes the team’s work and ensures that the primary goals of the team are met. Some sample responsibilities could be:

  • General responsibilities
    • Invite additional participants (model developers, data providers, users of space weather products and services - see above), that would like to actively contribute to team goals
    • Responsible for deliverables for the working team
    • Lead/monitor progress of the working team in meeting its goals/deliverables (coordinate team communication and activities, assign action items, etc.)
    • Coordinate communication with other working teams especially the Data Access Working Team, which is responsible for providing support on the generation of the meta-data needed for the validation effort
  • In preparation to working meeting/workshop:
    • Define a list of deliverables prior to the workshop
    • Develop a Work plan on meeting the deliverable
    • Finalize a schedule/format of the working meeting for their specific session
  • During the working meeting:
    • Organize and chair discussion sessions
    • Present updates on progress/deliverable to the broader forum participants
  • After the working meeting:
    • Refine a list of deliverable and work plan for the next 6-12 months based on inputs/feedback from the participants

Forum followers:

  • Interested in following the forum/team(s) but not ready for an active role?
    Subscribe to be a FOLLOWER. You will receive periodic working team updates (from all subscribed to teams) via the mailing list.

CCMC Role:

The CCMC together with the international community (International CCMC) facilitates this forum by:

  • Developing interactive web interfaces, display and analysis tools, maintain archives.
  • Supporting the implementation of SPASE Metadata Model XML Schema. Any format from modelers will be accepted and converted to the appropriate format by the CCMC team/partners.
  • Organizing working meetings

List of Focused Evaluation Topics

  • SUPERTOPIC: QUANTIFYING SCIENTIFIC PROGRESS (CCMC facilitator(s): B.Thompson)
    • Assessment of Understanding and Quantifying Progress Toward Science Understanding and Operational Readiness
      (Leads: A. Halford, A. Kellerman, K. Garcia-Sage, B. Thompson, S. Morley)
  • SOLAR (CCMC facilitator(s): P. Macneice)
    • Solar Flare Prediction (Leads: S. Murray, M. Georgoulis, S. Bloomfield, K.D. Leka; Scoreboard Leads: S. Murray, M.L Mays)
    • Coronal & Solar Wind Structure: Coronal & SW Structure; Ambient SW; Coronal Hole Boundaries (Leads: P. Macneice, L. Jian)
    • 3D CME kinematics and topology (Leads: B.Thompson, C.Moestl, D.Barnes)
    • Solar Indices and Irradiance (Leads: J. Klenzing, C. Henney, K. Muglach)
  • HELIOSPHERE (CCMC facilitator(s): M.L. Mays, A. Taktakishvili, P. Macneice)
    • CME Arrival Time (Leads: C. Verbeke, M.L. Mays, A. Taktakishvili)
    • IMF Bz at L1 (Leads: N. Savani, P. Riley)
    • SEPs (Leads: I.G. Richardson. P. Quinn, M. Marsh, M.L. Mays
      Scoreboard Leads: M. Dierckxsens, M. Marsh)
  • GEOSPACE: Auroral Region (CCMC facilitator(s): M.Kuznetsova)
    • Auroral precipitation and high latitude ionosphere electrodynamics (Leads: R. Robinson, Y. Zhang, B. Kosar)
  • GEOSPACE: Geomagnetic Environment (CCMC facilitator(s): L.Rastaetter)
    • Ground Magnetic Perturbations: dBdt, delta-B, GICs, FACs (Leads: D. Welling, H. Opgenoorth, C. Ngwira)
    • Geomagnetic Indices (Leads: M. Liemohn)
    • Magnetopause location and geosync. orbit crossing (Leads: Y. Collado-Vega, S. Merkin)
  • IONOSPHERE (CCMC facilitator(s): K. Garcia-Sage, J. Yue, M. Kuznetsova)
    • Neutral Density and Orbit Determination at LEO (Leads: S. Solomon, T. Fuller-Rowell, S. Bruinsma, E. Sutton)
    • Global & Regional TEC (Leads: L. Scherliess, R. Calfas)
    • Ionosphere Plasma Density: NmF2/foF2, hmF2, TEC (Leads: I. Tsagouri, M. Angling, K. Garcia-Sage, J. Yue)
    • Ionosphere Scintillation (Leads: E. Yizengaw)
  • RADIATION and PLASMA EFFECTS Scope of work (CCMC facilitator(s): Y. Zheng, M. Kuznetsova)
    • Surface Charging few eV - keV electrons, plasma density (Leads: J. Minow, D. Pitchford, N. Ganushkina)
    • Internal Charging keV–MeV electrons (Leads: P. O'Brien, Y. Shprits)
    • Single Event Effects MeV–GeV-TeV protons, ions (Leads: M. Xapsos, J. Mazur, P. Jiggens)
    • Total Ionizing Dose keV–MeV electrons, keV–GeV protons,ions (Leads: I. Jun, T. Guild, M. Xapsos)
    • Radiation effects for aviation (Leads: K. Tobiska, M. Meier)
  • INFORMATION ARCHITECTURE (CCMC facilitator(s): C. Wiegand)
    • Information Architecture for Interactive Archives (IAIA) (Leads: C. Wiegand, D. Heynderickx, D. De Zeeuw, T. King)

Sections in this page