Related Links | Frequently Asked Questions | Community Feedback | Downloads | Sitemap
Home | Stakeholders| Steering Committee | GSFC Staff | Publications | Meetings and Workshops | Concept of Operations
Home | Models At A Glance | ModelWeb Catalog and Archive | Model Submission | Metrics Studies
Home | Request Procedures | Generate Input Data Files & Parameters | About the Runs-on-Request Process
Home | STEREO Support | Search Simulation Results Database | Special Sun-Earth Connection Events | 3D VRML Output for Selected Events
Home | Instant Model Run
Home | Solar & Heliospheric RT Run | Global Magnetosphere RT Run | Ionospheric Electrodynamics RT Run | Ionospheric RT Run: HF Vertical Signal Loss | Movies of Storms

Metrics Studies

Metrics studies are standardized, repeatable comparisons of model output to a measurable parameter of interest. They allow us to objectively compare the performance of models with comparable output, to measure the improvement of model capabilities over time, or to measure model usefulness for operations in comparison with some simple standard model.

The result is usually expressed as a skill score, which measures the performance of a model against the standard model based on modeling of one particular parameter. This allows us to construct a simple, “one number” measure of model capabilities. When the model's skill score(M) is less than 0, the model is worse than standard model; M = 0 means the model is as good as standard one; 0 < M < 1: the model is better than the standard model; M = 1 signifies the perfect score.

Metrics Studies, Verification and Validation - overview of CCMC metrics activites (presented at the 2005 CCMC Workshop, October 2005, Clearwater Beach, FL)

Examples of CCMC Metrics Studies:
Model Name
Model Output Parameters
Measurements
Standard Models
Status/
Report/
Presentation
Velocity and density at ACE location

ACE velocity and density

mean,
persistence

Completed
Report

Connection to active regions
Energetic particles precipitation (event/no event)
persistence
Started
Started
Magnetopause position as boundary surface between open and closed filed lines. .

Magnetopause crossings by satellites

persistence
Started
Magnetic perturbations at geosynch. orbit GOES magnetic field data
no perturbations
Started
Boundary between open and closed field lines mapped to ionosphere
Polar cap boundary observed by POLAR and IMAGE
mean
Completed
Presentation
Polar cap from field-aligned current pattern
Particle fluxes at geosynchronous orbit. Particle fluxes observed by LANL geosynch. sat.
mean

persistence,
average,
27-day recurrence

Completed
Report
Ground magnetic perturbations calculated from ionospheric potential pattern Ground magnetometer data (H component of magnetic perturbations)
no perturbations
Completed
Report
In progress
In progress

Electron density parameters (vertical profiles, TEC, NmF2)

Incoherent scatter radars, GPS stations, ionosonds
mean
Started

First community metrics challenge evaluation

E x B drift in ionosphere Velocity measurements from DMSP F13
zero velocity

 

Presentations and Publications on Metrics

National Aeronautics and Space Administration Air Force Materiel Command Air Force Office of Scientific Research Air Force Research Laboratory Air Force Weather Agency NOAA Space Environment Center National Science Foundation Office of Naval Research

| | | Privacy, Security Notices

CCMC logo designed by artist Nana Bagdavadze