Kameleon

Overview

Kameleon is a software suite that is being developed at the CCMC to address the difficulty in analyzing and disseminating the varying output formats of space weather model data. Through the employment of a comprehensive data format standardization methodology, Kameleon allows heterogeneous model output to be stored uniformly in a common science data format. The converted files contain both the original model output as well as additional metadata elements to create platform independent and self-descriptive data files. To facilitate model data dissemination, data reuse, and code reuse – the Kameleon access and interpolation library provides direct access to both the model data as well as the embedded metadata.

In a nutshell, the Kameleon converter reads model output data and re-writes the data into a standard format. The Kameleon access/interpolation library reads data already converted by Kameleon providing a high level interface to the data.

To get started right away, go to the Quick Start page.

Kameleon Software Suite

The Kameleon Software Suite began life as two packages written in C. The Kameleon Converter Package and the Kameleon Access/Interpolation Library. Both converter and interpolator packages evolved, and were eventually re-written with successor distributions now in JAVA and C++ respectively.

The suite now consists of:

• Original Kameleon Converter ( C )
• Generation 2 KameleonJ/Kameleon6 Converter ( JAVA ) [tgz]
• Original Kameleon Interpolator ( C )
• Generation 2 Kameleon+ Interpolator ( C++ ) [tgz]

For Developers

If you are a developer and want to add new model conversion functionality to the Kameleon Converter, you can use either the C or JAVA version depending on what language you are most comfortable with. The CCMC is investing primarily in KameleonJ/Kameleon6 development, but we are not opposed to new external development activities/branches that extend the utility, features, functionality of the original version. If you have model(s) and/or data and are interested in writing directly to the Kameleon supported format, please make sure all standard global and variable meta-data is created and compatible data structures are used. Please note that a formal specification document is currently under development for this... If you want to add new access/interpolation support to Kameleon, we recommend development using the Kameleon+ (C++) version of the library as a baseline. The Kameleon+ version is expected to receive the bulk of both CCMC and community support.

For Users

If you already have Kameleon compatible data and want to read it, you should first verify that the KameleonJ/Kameleon6 version supports the model output you have. If it does, we recommend you use the KameleonJ/Kameleon6 version. However, if the original Kameleon access/interpolation library is the only version that supports the data you have and/or you are already comfortable with the original version, please feel free to use the original distribution. * Please note that the CCMC is investing primarily in the new generations of KameleonJ/Kameleon6 and Kameleon+

Detailed Kameleon Information Printable 2-page summary: Kameleon Software [PDF]

Primary Features

• Model Readers and Interpolators for models hosted at the CCMC. For a list of supported models, see Supported Models.
• Model-specific field line integration
• Wrappers for C, Fortran, Python, and Java

News

• Model plug-ins: Python-embedded readers and interpolators
• Python Visualization tools
• Coordinate Transformation

Installation

See Quick Start for cross-platform installation.

Tutorials

The new Coordinate Transformations tutorial illustrates coordinate-aware interpolation. Users may choose from a set of standard Heilospheric or Magnetospheric coordinate systems. Kameleon will automatically transform interpolation positions to the coordinate system of the underlying model.

The new Grid Tutorial illustrates usage of the command-line tool grid.py, which interpolates variables onto a user-specified grid.

Source code

The Kameleon-plus source code is hosted on github: https://github.com/ccmc/ccmc-software

To contribute, check out the CCMC source code:

git clone https://github.com/ccmc/ccmc-software

The Kameleon-plus source is in the subdirectory /kameleon-plus/trunk/kameleon-plus-working

Contact

If you are having issues, please use the contact form.

Quick Start

Kameleon Installers

These platform-dependent installers include pre-built binaries and executables.

Run the installer with bash: $ bash kameleon-6.1.1-MacOSX-x86_64.txt

Then follow the prompts:

Welcome

kameleon will now be installed into this location:
<your/home/directory>/kameleon

  - Press ENTER to confirm the location
  - Press CTRL-C to abort the installation
  - Or specify an different location below

[<your/home/directory>/kameleon] >>>

After choosing your install location preference, you’ll see the following output:

PREFIX=/Users/apembrok/kameleon
installing: python-2.7.11-0 ...
Python 2.7.11 :: Continuum Analytics, Inc.
installing: boost-1.59.0-py27_0 ...
installing: bzip2-1.0.6-0 ...
installing: cdf36-3.6.1-0 ...
installing: cmake-3.3.1-0 ...
installing: hdf5-1.8.15.1-2 ...
installing: kameleon-6.1.0-py27_0 ...
installing: openssl-1.0.2g-0 ...
installing: pcre-8.31-0 ...
installing: pip-8.1.1-py27_0 ...
installing: readline-6.2-2 ...
installing: setuptools-20.3-py27_0 ...
installing: sqlite-3.9.2-0 ...
installing: swig-3.0.8-1 ...
installing: tk-8.5.18-0 ...
installing: wheel-0.29.0-py27_0 ...
installing: zlib-1.2.8-0 ...
installation finished.

Finally, choose whether to prepend the kameleon install location to your system path.:

Do you wish the installer to prepend the kameleon install location
to PATH in your /Users/apembrok/.bash_profile ? [yes|no]
[yes] >>> no

You may wish to edit your .bashrc or prepend the kameleon install location:

$ export PATH=/Users/apembrok/kameleon/bin:$PATH

Thank you for installing kameleon!

Note that because the kameleon installer includes its own version of python, choosing to prepend kameleon’s bin directory will cause kameleon’s python to take precedence over your system python.

Executables

The list below shows all executables packaged with the installer.

Note: The executable paths will be path/to/kameleon/bin/ccmc/examples/<language>.

• C++/
  • kameleon_prog: opens and samples kameleon supported Model
  • time_interp: tests interpolation between time steps
  • tracer_prog: tests the field line tracer on supported models
  • integrator_prog: tests field line integrator on supported models
  • coordinate_transformation_test: tests cxform’s coordinate transformations
  • lfm_test: tests the reader and interpolator for LFM
  • enlil_test: tests the reader/interpolator for Enlil
  • adapt3d_test: tests the reader/interpolator for Adapt3d
  • open_ggcm_test: tests the reader/interpolator for open-GGCM
  • mas_test: tests the reader/interpolator for MAS
  • swmf_iono_test: tests the reader/interpolator for SWMF ionosphere model
  • CDFReader: tests the reader CDF file reader
• C/
  • filereader_compat_test: tests c wrapper for filereader
  • kameleon_compat_test: tests c wrapper for time interpolator
  • time_interp_c: tests c wrapper for kameleon
  • tracer_c: tests c wrapper for tracer
• fortran/
  • generalfilereader_compat_f: tests FORTRAN wrapper for general file reader
  • kameleon_compat_f: tests FORTRAN access to models through Kameleon
  • timeinterp_compat_f: tests FORTRAN compatibility for time interpolator
  • tracer_compat_f: tests FORTRAN compatibility for field line tracer
  • time_series_test: saves time series of multiple variables to an output file
• python/
  • kameleon_test.py: tests python access to kameleon objects
  • tracer.py: tests python access to field line tracer
  • ARMS_test.py: tests python access to ARMS reader (python extension)
  • grid.py: basic grid interpolation/cut plane visualization
  • pyModel_test.py: test of embedded python readers/interpolators

Libraries

The installer includes the following platform-dependent libraries (names corresponding to a Mac build shown here). The example programs automatically link to these. The library paths will begin with /path/to/kameleon/lib/ccmc. Headers are in /path/to/kameleon/include.

Test Data

Download sample kameleon test data here.

Download, untar and unzip any of the above test data, e.g.:

wget http://ccmc.gsfc.nasa.gov/downloads/sample_data/ENLIL.tar.bz2
tar -vxjf ENLIL.tar.bz2

Supported Models

As of Version 6.1, the following models are supported.

Accessing the Models

The Kameleon Class provides access to supported models through a common interface.

For example, the following python code is model-independent (e.g. it may be used to access the BATS-R-US model or the LFM):

import CCMC
kameleon = CCMC.Kameleon()
kameleon.open("/path/to/ccmc-converted/file.cdf")
kameleon.loadVariable("bz")
interpolator = kameleon.createNewInterpolator()
bz = interpolator.interpolate("bz",25,0,0)
print "bz:", bz
kameleon.close()

This encourages code re-use when comparing results from different models. See Kameleon Class Demo for a complete example.

Alternatively, you may invoke a model’s reader directly:

import CCMC
lfm = CCMC.LFM()
lfm.open("/path/to/converted/lfm_file.cdf") #could also be "/path/to/converted/lfm_file.H5"
lfm.loadVariable('bz')
interpolator = lfm.createNewInterpolator()
bz, dx, dy, dz = interpolator.interpolate_dc('bz',-30,0,0) # dx,dy,dz will store the local resolution
print bz
lfm.close()

This is useful for accessing model-specific variables and routines.

• Model Overview - detail descriptions of accessible models.
• Full Examples - more examples in C++, Python, Fortran.
• Model-Independent Classes - API and wrappers.

Model Overview

Heliosphere

MAS

MHD Model of Solar Corona. Model domain: 1 - 30 solar radii.

Model Authors/Developers: J. Linker, Z. Mikic, R. Lionello, P. Riley Science Applications International Corporation (SAIC) San Diego, California

Adapt3D

Adapt3D is a Solar model on an unstructured tetrahedral mesh.

From Spicer et al. “A New 3D, Fully Parallel, Unstructured AMR MHD High Order Godunov Code for Modeling Sun-Earth Connection Phenomena” (preprint)

ENLIL

ENLIL is a time-dependent 3D MHD model of the heliosphere. It solves for plasma mass, momentum and energy density, and magnetic field, using a Flux-Corrected-Transport (FCT) algorithm.

Model Authors/Developers: D. Odstrcil - University of Boulder, Colorado

Magnetosphere

BATS-R-US

BATS-R-US, the Block-Adaptive-Tree-Solarwind-Roe- Upwind-Scheme, was developed by the Computational Magnetohydrodynamics (MHD) Group at the University of Michigan, now Center for Space Environment Modeling (CSEM).

The BATS-R-US code solves 3D MHD equations in finite volume form using numerical methods related to Roe’s Approximate Riemann Solver. BATSRUS uses an adaptive grid composed of rectangular blocks arranged in varying degrees of spatial refinement levels. The magnetospheric MHD part is attached to an ionospheric potential solver that provides electric potentials and conductances in the ionosphere from magnetospheric field-aligned currents.

More details can be found on the ccmc website here.

Open-GGCM

The UCLA/NOAA Geospace General Circulation Model (GGCM) open_ggcm: was originally developed as a magnetohydrodynamic ( MHD ) model of Earth’s magnetosphere at UCLA in the early 1990’s by J.Raeder.

LFM

The Lyon-Fedder-Mobarry global MHD model solves the 3D MHD equations for the Eath’s magnetosphere using a stretched logically spherical grid.

Ionosphere

SWMF-Ionosphere (not supported)

We are in the process of folding the SWMF Ionospheric interpolator into the main development branch. In the meantime, you may access the beta version of the SWMF Ionosphere here:

ftp://hanna.ccmc.gsfc.nasa.gov/pub/GEM-CEDAR/out/high-latitude-drivers/

• Download the following files:
  • fortran_test.tar.gz
  • A few timesteps from SWMF_CDF/E.2001.243/ for testing
• Unzip fortran_test
• cd into cdf35_0-dist and complile the CDF library (see Help.all for information on setting OS and ENV variables within the Makefile): make all
• Follow these instructions contained in the fortran_test/README:

Included in this directory are CDF files containing the ionospheric component
of SWMF for the 2006 December event and the example fortran test code to
access and interpolate the data. The CDF files each contain one timestep.

The example fortran code can be compiled by extracting fortran_test.tar.gz and
modifying the Makefile to select the appropriate configure options for your
operating system. The compilation has been tested on Linux and OSX.

1. Edit Makefile
   CDF_OPTIONS and KAMELEON_PLUS_CONFIGURE must be defined to the correct
   variables for the operating system
   F77, CC, and CXX must point to the system fortran, c, and c++ compilers
   NJOBS should be set to the number of cores on the system. If this is left
   blank, an infinite number of parallel jobs are used during compilation.
2. make all

If your compiler does not support the -Ofast option, replace it with -O3 -ffast
-math or -O3 or your preferred compiler optimizations (replace all instances)

The example fortran program, fortran_test, opens a CDF file and interpolates
the selected variable (such as 'ep' [electric potential]) onto a 181x181 grid
to standard output. The default missing value is -1.09951163E+12

./fortran_test CCMC_CCMC_052312_1.swmf.i_e20061213-000500-000.cdf ep
Model name: 'swmf'
 done opening file
loading ep
 done loading variable ep
 created interpolator
  lat:    90.0000000      mlt:    12.0000000      interpolated value:   -1.09951163E+12
  lat:    89.0000000      mlt:    12.0000000      interpolated value:   -6.34560013
  lat:    88.0000000      mlt:    12.0000000      interpolated value:   -6.66540003
  lat:    87.0000000      mlt:    12.0000000      interpolated value:   -6.97889996
...
  lat:   -87.0000000      mlt:    12.0000000      interpolated value:  -0.790960014
  lat:   -88.0000000      mlt:    12.0000000      interpolated value:  -0.625360012
  lat:   -89.0000000      mlt:    12.0000000      interpolated value:  -0.470510006
  lat:   -90.0000000      mlt:    12.0000000      interpolated value:   -1.09951163E+12

NOTE: the SWMF ionospheric electrodynamics files do not have values at the poles.

main.f and kameleon_f.f show how to use the time interpolator using a
directory of timestep files.
In this example, the directory path and prefix of all files is hardcoded in
kameleon_f.f, so this
must be updated to reflect the actual location of the CDF files.

 file_path='/path/to/your/test_files_directory'

You must also choose an interpolation time to be within the range of time
steps to be loaded; set the following variables in main.f:

         itime(1)= 2001      ! year
         itime(2)= 08        ! month
         itime(3)= 29        ! day
         itime(4)= 04         ! hour
         itime(5)= 0        ! min
         itime(6)= 0        ! sec

If you have questions, please contact MARSHALL SWINDELL (marshall.j.swindell@nasa.gov).

Model Classes

See Model Classes Page.

Interpolator Classes

See Interpolator Classes Page.

Coordinate Transformations

See Coordinate Transformations Page.

Grid Tutorial

See Grid Tutorial Page.

Full Examples

The Kameleon Class can be used to access any of Kameleon’s Supported Models.

Kameleon Class Demo

This demo should work on any of Kameleon’s Supported Models.

import sys
@CCMC_MODULE_PATH_STR@
import _CCMC as ccmc

def main(argv):
	if (len(argv) == 5):
	    filename = argv[0]
	    variable = argv[1]

	    c0 = float(argv[2])
	    c1 = float(argv[3])
	    c2 = float(argv[4])

	    kameleon = ccmc.Kameleon()
	    kameleon.open(filename)
	    kameleon.loadVariable(variable)

	    interpolator = kameleon.createNewInterpolator()
	    var = interpolator.interpolate(variable,c0, c1, c2)

	    print variable, var

	    kameleon.close()
	else:
		print 'Usage: <filename> <variable> x, y, z
 python kameleon_test rho -40 0 0'

if __name__ == '__main__':
    main(sys.argv[1:])

Tracer Class Demo

Python

#1/usr/bin/python
"""
This program takes as input a kameleon-compatible cdf or hdf5 file, a variable, and a seed position.
It then computes a fieldline and interpolates the variable onto it, plotting the resulting fieldline in 3d
and a graph of the variable as a function of arc length.

todo:
"""
import numpy as np
import sys, getopt
import time
@CCMC_MODULE_PATH_STR@
import _CCMC as ccmc


def main(argv):
    if (len(argv) == 5):
        filename = argv[0]
        variable = argv[1]
        c0 = float(argv[2])
        c1 = float(argv[3])
        c2 = float(argv[4])

        print 'Input file is "', filename
        print 'seed position:', c0, c1, c2

        kameleon = ccmc.Kameleon()

        kameleon.open(filename)
        kameleon.loadVariable("b")
        kameleon.loadVariable(variable)
        tracer = ccmc.Tracer(kameleon)
        tracer.setMaxIterations(20000)
        tracer.setDn(.2)

        print 'Testing bidirectional trace at seed position:', c0, c1, c2
        start = time.clock()
        fieldline1 = tracer.bidirectionalTrace('b',c0,c1,c2)
        finish = time.clock()
        elapsed = finish - start
        print 'Bidirectional trace completed. Elapsed time: ', elapsed, 'seconds.'



        interpolator = kameleon.createNewInterpolator()

        if module_exists("matplotlib"):
            import matplotlib.pyplot as plt
            from mpl_toolkits.mplot3d import axes3d
            fig = plt.figure()
            ax = fig.add_subplot(111, projection='3d')

            npts = fieldline1.size()
            x = np.zeros(npts); y = np.zeros(npts); z = np.zeros(npts)
            data = np.zeros(npts); arcLength = np.zeros(npts)

            for i in range(npts):
                p = fieldline1.getPosition(i)
                (x[i], y[i], z[i]) = (p.component1, p.component2, p.component3)
                data[i] = interpolator.interpolate(variable, x[i], y[i], z[i])
                arcLength[i] = fieldline1.getLength(i)

            def plotExtrema(extrema, color = 'r', marker = 'o'):
            #plot min and max values along fieldline
                for i in range(extrema.size()):
                    p  = fieldline1.getPosition(extrema[i])
                    ax.scatter(p.component1, p.component2, p.component3, c = color, marker  = marker)

            fieldline1.minmax()
            plotExtrema(fieldline1.minima)
            plotExtrema(fieldline1.maxima, 'b', '^')

            ax.plot(x, y, z, label='Magnetic field line')
            ax.legend()

            fig = plt.figure()
            plt.plot(arcLength, data, label='parametric data')
            plt.xlabel('arc length [' + kameleon.getVisUnit('x') + ']')
            plt.ylabel(variable + ' [' + kameleon.getVisUnit(variable) + ']')



            plt.show()

        else:
            print 'matplotlib not installed.. no 3d view supported.'

    else: print 'Usage: <filename> <variable> x, y, z
 python kameleon_test rho -40 0 0'




def module_exists(module_name):
    try:
        __import__(module_name)
    except ImportError:
        return False
    else:
        return True

if __name__ =="__main__":
    main(sys.argv[1:])

Fortran

      program tracer_f

c     Five functions to create a Kameleon Object, open the cdf file, load the variable, and close the file

      external f_kameleon_create, f_kameleon_open, f_kameleon_delete
      external f_kameleon_load_vector_variable, f_kameleon_close
      external f_kameleon_create_c_string

c     Three Tracer functions
      external f_tracer_create, f_tracer_bidirectionaltrace, f_tracer_delete

c     Variables to be used for interpolation and data extraction

      character*300 cdf_file_path
      character*301 cdf_file_path_c
      character*50 variable
      character*50 variable_c
c      real*8 x,y,z
c      real*8 interpolated_value
      integer status,kid,tid,s_length,actual_steps
      real x_array(1000)
      real y_array(1000)
      real z_array(1000)
      character*100 c0,c1,c2
      real rc0, rc1, rc2
      integer argc

c      character*50 var_to_read

c     --- set your actual path name here ---
      argc = iargc()
      if (argc.ne.5) then
        print *,"tracer_compat_f <filename> variable c0 c1 c2"
        print *,"    Adapt3D, OpenGGCM, BATSRUS, LFM: x y z"
        print *,"    ENLIL, MAS: r theta(latitude) phi(longitude)"
        print *,"    SWMF Iono: 0 theta(latitude) phi(MLT)"
       call exit(1)
      endif

      call getarg(1, cdf_file_path)
      call getarg(2, variable)
      call getarg(3, c0)
      call getarg(4, c1)
      call getarg(5, c2)

      read(c0,*) rc0
      read(c1,*) rc1
      read(c2,*) rc2

      write(*,*) 'Processing file: ', cdf_file_path

c     Open the cdf file


      kid=0
      tid=0
      status=0
      actual_steps=0
      call f_kameleon_create_c_string(cdf_file_path, cdf_file_path_c)
      call f_kameleon_create_c_string(variable, variable_c)
      call f_kameleon_create(kid)
      call f_kameleon_open(kid,cdf_file_path_c,status)
      call f_kameleon_load_vector_variable(kid,variable_c)
      call f_tracer_create(tid,kid)

      print *, 'c0: ', rc0, ' c1: ', rc1, ' c2: ', rc2
      print *, 'passing variables:', 'tid: ', tid, 'variable ', variable

c     make sure the step size used is appropriate and the target arrays have 2 times
c     the step_max so the arrays have enough for two directions.
      call f_Tracer_bidirectionaltrace(tid,variable,rc0,rc1,rc2,10000,
     &       .2, actual_steps,x_array,y_array,z_array)
      write(*,*) 'finished tracing'
      write(*,*) 'actual_steps: ',actual_steps
      call f_tracer_delete(tid);
      call f_kameleon_close(kid)
      call f_kameleon_delete(kid, status)
      end

Available Wrappers

Kameleon offers several interfaces into its core c++ library. The most complete is the Python interface, but there is basic functionality for C, Fortran, and Java.

• Python
• C
• FORTRAN
• Java

Developer Guide

See the Developer Guide.