International Forum for Space Weather Capabilities Assessment

Forum Home | Working Teams & Topics |  Overall Goals and Deliverables  | Sign Up | FAQ |  ICCMC-LWS Working Meeting & Presentations 

Related Links | Frequently Asked Questions | Community Feedback | Downloads | Sitemap
CCMC Home | CCMC Stakeholders | Our Team | Publications | Meetings and Workshops | Concept of Operations
Models At A Glance | ModelWeb Catalog and Archive
Request Procedures | Generate Input Data Files & Parameters | Movies on Request | About the Run Process | Publications Policy
Search run database | Request run output | Special events | Kameleon Software | Space Weather Explorer | Publications policy
Instant Model Run
Forecasting Support tools | iSWA | DONKI | Mission Support | Experimental Real Time Simulations | Operational Geospace Model Validations
Intl Forum | GEM Challenge | CEDAR ETI Challenge | GEM-CEDAR Challenge | SHINE Challenge | CME Arrival Time Scoreboard | Flare Scoreboard | SEP Scoreboard | IMF Bz Scoreboard
Educational materials &activities | Space Weather REDI Initiative | SW REDI Bootcamp | Student Research Contest | Tutorial at CEDAR | Forecaster Tools
Missions near Earth/in Earth-orbit | MMS | Van Allen Probes | THEMIS | MESSENGER | STEREO | Spitzer | MAVEN | MSL | Dawn | Kepler | EPOXI | Juno | CASSINI | Voyager | New Horizons | Sounding Rockets | International
Research Community Support | CCMC Workshops | NASA Robotic Mission Operator Workshops | LWS Support | Exo-CCMC | DREAM2 Support | HELCATS Support
iSWA | DONKI | Kameleon | StereoCat | EEGGL | CME Scoreboard | SEP Scoreboard | FLR Scoreboard | SEA5

Assessment of Understanding and Quantifying Progress

We will be working towards quantifying and developing cross domain metrics and frameworks for tracking progress in our research and understanding.

Leads: A. Halford, A. Kellerman, K. Garcia-Sage, B. Thompson, S. Morley
Communications: (mailing list)

Website for ongoing forum activities

➤  Interactive team website for the 2017 CCMC Forum Workshop: agenda, feedback, meeting minutes, and more (constantly updated)
    ➤  schedule
    ➤  planning and input page
    ➤  meeting minutes
    ➤  ARLs and metrics

Working team activities during the April 2017 meeting:
This working team evaluated existing metrics and developed new metrics for measuring advancements, and current capabilities within the field of space weather focusing on three broad topics. Discussions, panels, and working groups focused on metrics for:

1- Progress in our research understanding.
2- The functionality of current models, both physics based and data driven. (including new physics e.g.)
3- Ability to include current research/physical understanding into models.

With each of these areas comes distinct challenges for developing metrics which we will explore in this working team. Topic 1 poses unique challenges for quantifying progress. Metrics such as number of papers or citation counts fail to accurately capture improvements in our understanding of the field. Metrics that accurately capture our research progress will allow us as a community to direct our research in an unbiased way towards addressing unanswered questions. This topic will consider questions such as “How critical are we?”, “Are we addressing end user concerns?”, and perhaps most importantly “How are we assessing our progress?”. Other fields have encountered similar problems, and we will consider how they have tracked progress as well as how our own field has attempted to show a continue advancement of understanding. Focus topics 2 and 3 will address questions such as “How should we compare our models against theory and observations?”, “Are our current metrics unbiased, and how can we develop ways to improve them?”, and “Are we consciously or unconsciously ignoring specific issues or topics?”. For topic 2 we will discuss and make progress towards understanding which current metrics are appropriate for the modeler and end user, as well as to identify new metrics that are required to understand model performance and appropriateness for a given problem. For topic 3 we will quantify our ability to model parts of the heliosphere with physics based models, including the time it takes to test and implement new ideas/models/observations into our larger models. We hope to find metrics that will quantify the advancements achieved by including new physics into models, as well as determining which physics is most important for different regions of the heliosphere and under what conditions.

This will be a working meeting and hope that all come prepared to engage in discussion and subsequent write ups of the results. We may also have online telecons prior to the workshop in order to make the most of the in person meeting.

National Aeronautics and Space Administration Air Force Materiel Command Air Force Office of Scientific Research Air Force Research Laboratory Air Force Weather Agency NOAA Space Environment Center National Science Foundation Office of Naval Research

| | Privacy, Security Notices | CCMC DATA Collection Consent Agreement

CCMC logo designed by artist Nana Bagdavadze