Welcome to the new CCMC website!

Please note that some pages may have moved during the migration. If you experience any issues with the new website, please reach out to gsfc-ccmc-support@lists.hq.nasa.gov.

Last Updated: 05/05/2022

SEP Scoreboard

Visit the Probability web app of the SEP Scoreboard
Visit the Intensity web app of the SEP Scoreboard
Visit the All Clear web app of the SEP Scoreboard
View all models providing output to the SEP Scoreboard

About

CCMC has implemented a beta version of the "SEP scoreboard". Scoreboard planning was initiated together with Mark Dierckxsens (BIRA-IASB), Mike Marsh (UK Met Office) and the international research community. Recently in 2018, Johnson Space Center's Space Radiation Analysis Group has become involved in the SEP scoreboard as part of a multi-year project called ISEP.

The SEP Scoreboard builds upon the flare scoreboard and CME scoreboard. It will be an automated system such that model/method developers can have their predictions automatically uploaded to an anonymous ftp in a pre-defined JSON format, which will be parsed and databased by the system.

SEP forecasts can be roughly divided into three categories: (1) Continuous/Probabilistic (2) Solar event triggered (3) Physics-based/complex. The SEP scoreboard will focus on real-time forecasts but will also coordinate with the SEP validation team to evaluate different models for a set of historical events. This is particularly useful for some physics-based models in the third category that are not yet relevant for real-time modeling.
Click here to go to the SEP Validation Team page.

Please email Mark Dierckxsens, Mike Marsh, Masha Kuznetsova, and Leila Mays with your feedback which will be shared with the SEP scoreboard planning group.

Latest News

➡ 2021-08: Combined probability and intensity all clear web application went live.

➡ 2021-03: SEPSTER2D data has been added to the SEP Scoreboard intensity web application.

➡ 2020-12: The SEP Scoreboard web applications are now live!
➡ 2020-12: ESWS 2020 presentation and screenshots [PDF]
➡ 2019-02: Final SEP Scoreboard JSON schema and helper script released
➡ 2020-10: Latest prototypes developed
See the agenda/materials of the SHINE and ESWW15 2018 community campaign.
➡ The SEP scoreboard is part of the the SEP Validation Team in the Community-wide International Forum for Space Weather Modeling Capabilities Assessment.
See the agenda of ESWW13 working meeting: Community-wide space weather Scoreboards: Research assessment of real-time forecasting models and techniques.

SEP scoreboard planning group

Please contact us to join
Leads: Mark Dierckxsens (BIRA-IASB), Mike Marsh (UK Met Office)

JSC SRAG, Ian Richardson (UMD/GSFC), Jesse Adries, Veronique Delouille (SIDC), Nathan Schwadron (UNH), Marlon Nunez (U Malaga), Anastasios Anastasiadis, Olga Malandraki (National Observatory of Athens), A. Posner (NASA HQ), B. Heber (Univ. of Kiel), J. Labrenz (Univ. of Kiel), Masha Kuznetsova (CCMC), M. Leila Mays (CCMC)

Participating partners

SEP Scoreboard and Data Access

SEP Scoreboard displays are now available publicly:

SEP Scoreboard forecast JSON file access:

CCMC Rules of the Road [PDF] apply: CCMC requests that users notify the CCMC, SEP model/technique developers and submitting forecasters before performing validation studies with the SEP Scoreboard database. It is recommended that such validation studies be performed with the knowledge and collaboration of developers and submitting forecasters/researchers.

Submission File Format Information

SEP scoreboard JSON:

A few tips on writing your data to to a JSON file.
Helper python script to create and validate JSON files:

Further JSON file examples (.json): Contact CCMC software developer Joycelyn Jones with any JSON and script questions

View the SEP Scoreboard Change Log.

Models that have registered their interest

Reports & Presentations

How to get started with the SEP Scoreboard

  • Participating modelers will respond fill in the registration questionnaire below and CCMC model on-boarding questionnaire and email both to Leila Mays. We will then create a folder for the model on our anonymous ftp site and register your model metadata on the CCMC model registry.

  • Participants will produce SEP forecasts conforming to the file format (JSON; see above).

  • SEP forecast .json files will be accepted by anonymous ftp, each model has an assigned folder.

  • All information will be stored in a database and the JSONs are made accessible to users via an API

  • SEP forecasts will be displayed side by side on the "SEP Scoreboard" website - currently in development (beta version is live now).

CCMC Rules of the Road [PDF] apply: CCMC requests that users notify the CCMC, SEP model/technique developers and submitting forecasters before performing validation studies with the SEP Scoreboard database. It is recommended that such validation studies be performed with the knowledge and collaboration of developers and submitting forecasters/researchers.

Model/Technique Registration

(*=required)

Information about your group/project:

  • Name(s) and e-mail (s) (please list primary contact first)*

  • Associated Institution/Project name/Group name*

  • Website URL(s)

  • Logo(s)

Information about your method:

  • Forecasting method name*

  • Shorthand unique identifier for your method (methodname_version, e.g. ModelName_1 or ModelName_201201)

  • Short description*

  • Model Inputs*

  • Model Outputs*

  • References

Further Model Details:

  1. Is the forecast made continuously (e.g. probability for the next 24 hours, or a time series), or event-triggered (e.g. by a flare or CME).

  2. Is the forecast human generated, human generated but model-based, model-based, or other.

  3. If model-based: is the model empirical, physics-based or both.

SEP Models in the Community and Literature

Compiled by Mike Marsh

Model TypeModel NamePrincipal Developer(s)Observational InputsOutputs
EmpiricalAER SEP modelLisa Winter (LANL)Type II, Type III, and Langmuir wave properties measured from Wind/WAVESprobability of a > 10 MeV proton event (> 10 pfu)
EmpiricalAFRL PPSStephen Kahler (AFRL)GOES x-ray peak flux & locationE>5 MeV intensities
Empirical/Machine learningBoubrahimi modelSoukaina Boubrahimi (GSU)GOES x-ray and proton fluxE>100 MeV intensities
PhysicsSEP modeling with EUHFORIANicolas Wijsen, Angels AranCoupled with the EUHFORIA MHD model
PhysicsEPREMNathan Schwadron (UNH)Can be driven by in-situ proton observations, can be coupled with MHDUser defined flux range, also dose calculations within EMMREM framework
PhysicsFLAMPA (SWMF)University of MichiganSWMF module coupled with MHD
EmpiricalFORSPEFAnastasios Anastasiadis (NOA)Magnetograms, x-ray flares, or CMEsE > 30,60,100 MeV integral proton energy flux and fluence
PhysicsKota SEP (SWMF)University of MichiganSWMF module coupled with MHD
PhysicsLuhmann ModelJanet Luhmann (UCB SSL)Coupled with WSA-ENLIL+Cone (magnetograms, coronagraphs)User defined flux range
EmpiricalMAG4David Falconer (NASA/MSFC, UAH)Magnetograms, x-ray flares24 hour event probabilistic forecast
PhysicsiPATHGang Li, Gary Zank (UAH)
Physics & EmpiricalPREDICCSNathan Schwadron (UNH)(coupled version of EMMREM and REleASE)
EmpiricalREleASEArik PosnerSOHO/COSTEP-EPHIN high energy electron flux. ACE/EPAM in new versionE=4-9, 9-16, 16-40, 28-50 MeV proton flux
EmpiricalSEPForecast (COMESEP)Mark Dierckxsens (BIRA IASB)GOES x-ray peak flux & location, CME width & velocity, GLE observationsE>10 MeV and >60 MeV integral proton energy peak flux and probability
PhysicsSOLPENCOAngels Aran (Univ. Barcelona)CME/Flare location & shock velocity estimateUser defined flux range
PhysicsSPARXSilvia Dalla (UCLan) Mike Marsh (UK Met Office)Flare location, peak x-ray fluxUser defined flux range
Empirical/Machine learningSPRINTSAlec Engel (NextGen Federal Systems)x-ray flares10 MeV onset and peak flux
EmpiricalSWPC PPMChristopher Balch (NOAA/SWPC)GOES x-ray, SEON radio burst, H-alpha/EUV imagingE>10 MeV integral peak proton flux, peak time, and probability
EmpiricalSWPCNOAA/SWPCDay 1-3 event probabilistic forecast
EmpiricalUMASEPMarlon Nuñez (Univ. Malaga)Goes x-ray & proton fluxesE>10 MeV integral proton flux. E>100 MeV proton flux in new version.
EmpiricalUK Met OfficeUK Met OfficeDay 1-4 event probabilistic forecast
PhysicsZhang modelMing Zhang (FIT)