NASA/NOAA ACE

Dom will try to access the ACE resource to make sure GLOTEC is working correctly.

• Discussed and provided feedback to NOAA/SWPC on the R2O2R model readiness level checklist
• Discussed resources needed for any upcoming milestones. Planning for UMASEP in the shared ACE environment

Continued providing feedback on SWPT R2O process checklist

Continued providing feedback on SWPT R2O process checklist

• CCMC is still experiencing download issue from services.swpc.noaa.gov. AI: Steve will follow up with Bill.
• Bringing WSA Dashboard to ACE? Postpone the decision until after CCMC Workshop.
• Provided feedback on SWPT R2O process checklist
• GloTEC
• AI: Steve will figure out if there are any issue sharing COSMIC2 data with ACE
• Dom provided GloTEC output as part of validation effort
• AI: Tina creates account for Dom
• *.SKL files are not in gitlab yet. AI: Ratina will check and upload those files, if they are not there. She will also do the same for BNC files.

• Dom is working on generating GloTEC output for Min-Yang. Dom mentioned the GloTEC historical run version is not containerized.
• AI: Tina creates account for Dom
• Ratina thought Marcus uploaded the *SKL files. AI: Chinwe will check.

• Both SWPC and CCMC's version of GloTEC uses the public f10.7 json files. However, Dom made a change to the code to handle when the f10.7 json is not available. Dom will get access to ACE to change and rebuild the docker image himself. AI: CCMC creates an account for Dom.
• Min-Yang is evaluating the performance of ionosphere models during solar max and solar min storm times. Dom will help generating some historical data for Min-Yang.

• CCMC has been having issues when trying to get data from services.swpc.noaa.gov. Initially, we were getting timeouts when sending requests, but more recently we are seeing requests hanging during the SSL handshake portion of the request. Corey, from CCMC’s SA team has been interfacing with SWPC directly and gave more technical details of the issue. Corey expressed that AWS likely needs to be involved in the resolution. AI: Corey to forward previous correspondence with SWPC to Steve and Tamara. Steve will escalate.
• GloTEC: Dom was supposed to fix when f10.7 data is not available and the code reads from a file but the data in the file is bigger than the array in the code, which causing a segmentation fault. The code also needed to read from a public JSON file rather than from their internal (therefore unreachable from ACE) URL. AI: Using public JSON file will eliminate the need to read from f10.7 file. This is preferable. 40 day json is publicly available (https://services.swpc.noaa.gov/json/f107_cm_flux.json) and SWPC operations is using the public file. Ratina will check what CCMC’s version is using. If already using this URL, then nothing to do. If not, then Dom will follow up with updating the code to use the public URL.
• Steve introduced new developers at SWPC.
• NCAR is under agreement to provide the data to only SWPC. It is not allowed to be distributed publicly. AI: Steve to check with NCAR if they can share it privately. If so, Ratina will setup an rsync service to ACE.
• AI Steve will email Mihail to check if Mihail has any issue accessing ACE
• Masha agreed to implement the secure/restricted web site for modelers/researchers to check their model output. AI: Steve will dig up the requirement and send to us.
• WSA model team is interested giving WSA dashboard to SWPC. Leila explained the steps that CCMC will take to facilitate this process. Steve is in agreement. AI: He will bring it up with Michelle to discuss the next steps.

NOAA/SWPC shared the R2O2R model readiness level checklist with CCMC

CCMC demonstrated iSWA to NOAA/SWPC

• CTIPe: Will give Mihail a space to work, let him make the code change then check it back to ACE gitlab.
• Guidance from the team will be needed. It will also be beneficial for CCMC to learn and transfer the knowledge.
• Build docker image in ACE environment will need Visual Studio Code, docker, docker compose. AI: Tina will install Visual Studio Code.
• Masha suggested to come up a list of essential space environment quantities. Will start with CTIPe and understand this quantity first.
• GloTEC:
• Ratina hasn’t show Dom GloTEC output image yet, but has shown him the logs, which looks okay. She still needs to show him the images.
• Steve will follow up on how to get COSMIC2 data into ACE
• Need more requirements for the password protected website and data – define what need to be accessed controlled and what should be available

NOAA/SWPC (Rob Steenburgh) gave CCMC a briefing on SWPC Aviation services and ICAO

• GloTEC:
• Model output page is on hold until further detail on the password protected regulation.
• For validation: Chinwe will check if the Python plot script is running and generating outputs. If so, we will push the plot out for Dom to access.
• The version of GloTEC in ACE can ingests COSMIC2 and CWDP data stream. Dom will help us reach out to NCAR to get the COSMIC2 data.
• Will have some background briefs on:
• Aviation services from the Aviation of weather center (SWPC)
• NOAA test experiments look like in general (SWPC)
• CCMC demo ISWA and its replay capabilities in the context of experiments and customers.

• CTIPe: Prompt penetration electric fields (PPEF) data is being downloaded; however, the code is not being used. Mihail recommends enabling the code for the ACE setup.
• Will need to rebuild the image and wrapper service and maybe update/add the volume mount to point to the PPEF data
• No expected change to the model output after enabling PPEF.
• The real time version uses PPEF data, but the forecast version can’t use the PPEF code since the data is not available. The forecast version will need to be commented out in the meantime. The combined wrapper service might not work anymore. It would be nice to have 1 version of CTIPe to handle both.
• Consideration to enable password protected web site/pages for data that’s not ready for public
• Not sure if S3 is capable to handle authentication. If not, we might have to move away from s3 and stand up a web server for this.
• AI: Tina checks in with SMCE ISO regarding if there is any regulation and/or policy that we need to comply and/or alternative options.

• GloTEC
• Brazil BNC file is ready. SWPC will add to the docker-compose file to set up the ingest.
• New PNG files are being generated. Will need a way to display those output.
• Possible ingest COSMIC satellite data from NCAR as another data source.
• No plan to move GloTEC to CCMC, yet. An agreement needs to be in place beforehand.
• Future milestone(s) for this group will more or less align with SWPC FY 22 plans.

• GloTEC
• Ingest is working correctly; however, pre-processor part is not. Will meet with Dom to troubleshoot.
• Will check in the BNC files after getting pre-processor working.
• Steve submitted the AGU abstract.
• Planning FY22 milestones.

• GloTEC
• Established accounts for all data sources. Still need to clean up the BNC files for IBGE.
• Model ingest is working now.
• Maybe reuse the GloTEC output page that’s integrated with ICAO or will create a page similar to CTIPe.
• Steve will draft an AGU abstract and circulate for review.
• Min Yang will lead the science part of GloTEC for CCMC.

• GloTEC
• Will push the 1.7 version of GloTEC to Gitlab and install it in ACE.
• Updated the BNC files to reflect the recent changes.
• Submitted access request to Natural Resources Canada and credential is being worked on.
• Unable to reach the IBGE (Brazil) data resource contact. Dom will find another POC for us. If not, we might be able to use the Brazil stations from CDDIS.
• Uses dual frequency GNSS stations, version 2.11 rinex format.
• Will create a web page with animation for visualization.
• Steve and Masha met to discuss the milestones, but internal conversation still need to take place at SWPC. Team will continue as planned.
• SWPC will take the lead on the AGU abstract
• Masha suggests a joint website where both SWPC and CCMC can edit the content. AI: Tina will move and get feedback on the current ACE status page on the public s3 website, including other collaboration works/tasks with SWPC.

• Kubernetes training was on 6/1 & 6/3.
• GloTEC:
• Was able to pull the CDDIS data stream via NTRIP client; however, don’t know which streams we need.
• SWPC will install 1.6 next week.
• Masha and Steve will meet later of this month to define future milestone(s).

• Kubernetes training is in the works. Tentatively scheduled for week of June 1st.
• GloTEC status:
• CCMC registered CDDIS GNSS already; however, they were experiencing issues and couldn’t create our accounts until next week.
• Ratina provided information on other data feeds that we need to register.
• SWPC pushed 1.6 image to SMCE Gitlab registry
• Ratina, with Marcus’ and Kelvin’ help, will install 1.6 containers first on ACE. Then share what they did with NASA later.
• The model will continue running even when the input data is not available. The more input data it has, the more accurate its map will be.
• Assessment doc:
• We don’t need to include the transition section in our assessment.
• Consider it done and will update the document as we continue our work
• As of now, the target audience of this document is us. It might become the supporting material for ACE, if we decided to expand the role of ACE.
• Masha and Steve will tag up to brainstorm about future milestones.
• Model self-recovery, health monitoring and notification features is desirable for all operation/production tool and model.

• Will set up a meeting with Ramon to discuss AWS Elastic Kubernetes Service usage potentially in ACE.
• GloTEC
• Increased the instance type to C5.2xlarge, which comes with 8 vCPU and 16 GB memory, as a starting point. We can increase it as needed.
• Source code and images haven’t been uploaded to SMCE Gitlab or in ACE. AI: Ratina will create a story to keep track.
• AI: Tina will need to talk to Masha about how to register as a group for CDDIS.
• There have been some internal discussions at SWPC regarding beneficial uses of ACE, how to partner with CCMC, transition and validations. Steve will meet with others to discuss more before inviting others.
• Masha also suggests performing some model benchmark testing for both cost and performance evaluations.

• GloTEC v.1.6 is fully functional and tested.
• SWPC will provide GloTEC (ingest and assimilator) and a list of ingest data site. There is no separate product generation yet.
• Potentially needs agreements to retrieve some data via NTRIP.
• Model flow: Retrieve the US TEC data and pre-process by the ingest, then modeling code integrated with the ground base (the non-TEC base).
• Will also share the base image that’s included in the assimilator.
• Is computational heavy – 16GB of memory for assimilation code and 4 core CPU (separately) so combine shouldn’t add more than 4-6 processors and little bit more memory.
• Replicating all input data source.
• Real time run is not crucial, if there is a sufficient sample of it. It will be less accurate, but it will still run.
• Validation – will only be done in ACE environment and maybe just grab from SWPC.
• Dom recommends using CDDIS, repo for GNSS data, as a starting point for data. (https://cddis.nasa.gov/Data_and_Derived_Products/Data_caster_description.html). CDDIS might have similar data usage agreement like SWPC in place. He also recommends that we figure out how CDDIS permits other NASA organization to use or distribute that data.
• AI: Dom will come up the list of data source and forward it to us.
• The plan is to install GloTEC on the same instance as CTIPe, check the source code and docker images into SMCE Gitlab and use CDDIS as the initial data source.
• Governance – a separate meeting is needed to discuss and define the guidelines for ACE.
• Assessment report
• Need to figure out how to wrap up this report.
• It didn’t have the best practice or rules of the road, maybe that should be part of the governance documents and not here.
• Should use the same consistent language.

• NOAA/SWPC will invite CCMC to their slack channel.
• SWPC is looking to deploy GloTEC v.1.6 to NCS next Thursday. AI: Steve will check with Terry and/or Dom regarding providing CCMC the list of TEC input for GloTEC.
• Plan to install both model ingest and processing on the same host instead of different hosts.
• SWPC has explicit agreements with some data providers so there might be a challenge to get those data in ACE. AI: SWPC will take a look at that.
• CARI-7, a model candidate to be installed in ACE and for validation.
• Masha suggests to installing CARI-7 in another collaboration environment for research community to avoid opening ACE to the community.
• FAA requests not to distribute the source code, only the executable, docker image(s) of the model will be shared.
• Hazel will get Kyle’s approval for installing CARI-7 in ACE. Upon agreement, Masha will formally invite Sam to CCMC, under Bob Robinson and Yihua’s supervision.
• CARI-7 are being run in real-time in SWPC. Hazel is working on Run on Request version and validation still need to be done with it.
• CCMC is interested in adding this model to iSWA. Hazel needs to check the IKO agreement. Alternative option is CCMC fetches CARI-7 output from SWPC for iSWA.
• In addition to assessment report, we also need to start looking into defining guideline for intellectual property rights, criteria for installation, accessibility by other parties other than SWPC and CCMC, communication with public, etc. for ACE.

• There are a lot of cloud initiatives and NOAA & NASA collaboration activities for earth science data simulation (space weather is included) happening right now.
• Masha recommends taking this opportunity to show case what ACE is doing and coming up activities for the team.
• Steve also wants to demonstrate ACE as a usable resource for model validation, which enables scientist and developer collaborations.
• Terry suggests defining how we should use ACE to demonstrate the value of it.
• GloTEC will match what’s in SWPC, to the best extent, in AWS.
• Marcus would like to set up GloTEC via AWS EKS due to more external dependencies and directories setup (volume).
• Plan to share GloTEC 1.7 version, which include the ingest and assimilator pieces, in few weeks.
• Masha recommends mirroring GloTEC (operational version) as close as possible, ported back to CCMC without any change at first. Benchmark performed for performance and stability before introducing changes. In addition, with the changes, it will demonstrate the model transition (funnel) process.
• Is data agreement needed for ACE to get/use the same input data as SWPC?
• Majority of the data is available via internet. Certain number of ground systems will require you to register with them.
• **AI:**SPWC will provide us a list of sites (& contacts) and CCMC will setup with those sites, with our IP, so we can get the same thing.
• NOAA is working with NCAR to containerize Model Evaluation Tools (METplus) and METviewer. Terry is willing to share in ACE.
• NOAA/SWPC uses docker host that’s running multiple docker containers on and running VMs. They are moving toward Kubernetes. They simply expand the docker host, if needed.
• A dedicated service account user for GloTEC (linux). AI: Tina creates the accounts for GloTEC. It needs UID and GID 1001, and part of docker group.
• Data will be stored in /data volume initially and will determine the plan for them in the future. Maybe pushed to S3 for display (Currently, there is not web content to display the NetCDF and/or plots.) Marcus and Ratina will think about how to organize the data for the static website.

• Dominic presented high level GloTEC to the team.
• Suggestion for the model validation and transition best practices:
• Define how we have validated the model for transitions and how we can make those processes more robust
• Link to Space Weather R2O2R Framework document for the definitions of readiness levels
• Would like to see real practical processes for: what do you actually do [to transition]; what does the Pi need to do; what does CCMC (before and after ACE) need to do; what does SWPC need to do; when does model come into ACE and move on?
• Use AGU poster schematics to clarify the model flow (relevant to the readiness level) and illustrate our end goal
• AI: Tina add editable schematics to the assessment paper and Masha will work on the data flow arrows.
• Dominic recommends starting with CDDIS as the initial data input.

• Steve added the model transition section for the Assessment of AWS Environment. He would like to add a checklist, but a framework is there to explain the different stages of R2O2R process. The team will review that section and discuss what we should do both with that section and the report.
• SWPC is very close with GloTEC so we are still on schedule to receive by Feb. There are 2 pieces of the model: ingest/assimilation and processing. The plan is to have CCMC install GloTEC on their production servers, so both pieces will need to be transitioned to ACE.
• GloTEC uses Networked Transport of RTCM data via Internet (NTRIP) protocol. We should be able to subscribe to those data feeds.
• Masha suggests creating a model procedure to gather model requirements such as identify data source, directory structures, etc. for efficiency and possibly part of best practice.
• Marcus is interested in using AWS Kubernetes service. AI: Tina will look into and get back to the team.
• Steve will share some high level GloTEC documentation/presentation with the team so CCMC can get a better understanding of it

• Steve will pull together some SWPC material for the model transition section of the Assessment of AWS Environment.
• Delivery of GloTEC is still on schedule for January to early February 2021.
• Steve hoped once the governance is in place then the live commercial data that was part of the GloTEC evaluation and validation could be available in ACE.
• Cray Linux Environment (CLE)
• SUSE Linux would certainly work. However, just a question of how divergent those two models, the operational one and the one in the ACE, will be.
• NCO has no plan to use container technology with the new generation of HPC hardware.
• Eric noted that some models exhibit variations depending on how it was compiled, how many cores and type of cores were used. If ACE can’t reproduce the exact SWPC operational environment for validation purpose, then it makes the validation process not useful.
• Steve reminded us this cloud environment’s goal is to be as similar to the SWPC operational environment as is physically and logically possible so model adaptations to the operational environment can be made earlier in the validation process. Part of this effort is also to determine how ACE can mirror the SWPC operational environment; and how close ACE needs to and can get to.
• Chinwe found a publication https://cfwebprod.sandia.gov/cfdocs/CompResearch/docs/PID5099047.pdf on a group that compared an HPC application running in a Singularity container on a Cray system to the same application running in Docker on AWS. The group concluded that running an HPC application is more suited for Cray systems, but docker on AWS offers a good testbed option.

• Need SWPC to contribute to the model transition section of the Assessment of AWS Environment.
• SWPC is refactoring the GloTEC code making it more robust and modular. Tentatively schedule to receive the code sometime in Jan. 2021. Will determine the due date after the kickoff/opening brief.
• AWS doesn’t offer Cray Linux Environment (CLE). Since CLE builds on top of SUSE Linux, the closest OS alternatives in AWS is SUSE Linux. However, given that CLE is proprietary, we won’t be able to install and/or use any CLE libraries/packages in this environment. Need to follow up with Eric to get his feedback.
• Maybe instead of mimicking the operational environment, we should just approximate it, but this will require some analysis.
• Marcus suggested this might be a good opportunity to try out AWS Elastic Kubernetes Service (EKS) for GloTEC. Tina will investigate in EKS once we start working on GloTEC.
• In addition having GloTEC completely functional in the environment, another possible milestone could be validating GloTEC with commercial weather data for model evaluation.

• GloTEC is the next serial model this group will bring into ACE. It is not as ready as CTIPe and more complex. In general, GloTEC takes different type of input data (TEC measurement, computation data and ground GPS data) and inverts them into ionosphere model. It consists of pre-processor and data simulation/simulator.
• Probably need to get the docker architecture up to standard and address any issues along the way to make this model more operational ready.
• Leverage S3 bucket storage, if possible
• We also need to decide how much data we should keep
• SWPC will push the Docker images to SMCE GitLab and need to find out how to get input data (NTRIP GPS)
• Masha recommends establishing an ionosphere team to figure out benchmark for the ionosphere models. Will work independently to establish the team.
• AWS parallel cluster is set up. CCMC needs to install a model and compare its performance with in-house cluster. CCMC’s idea is to use WSA and ADAPT or ENLIL for this collaboration effort.
• SWPC will discuss offline their plan for geospace parallel models and whether it makes sense to bring to this environment.
• Masha suggests we need to think about milestones for the upcoming year.
• Operationally, SWPC will be using Cray Linux Environment and Cray parallel tools. It might be problematic for us to replicate/mirror in AWS.
• AI: Tina will look into AWS AMI to see if AWS offers CLE or not.
• Masha suggests mirroring as closely as possible, so the modelers have more confidence that indeed their model will work in NOAA/SWPC. It will also streamline the model upgrade from research to operation.
• SWPC parallel code runs with a utility call ecFlow, instead of cronjobs, to schedule execution. Recommends installing ecFlow in the AWS environment.
• CCMC won’t proceed, yet, as more information is needed before then.
• SWPC is in the process of defining the validation process for themselves. SWPC has some documentation for the framework and the different readiness stages that can be included in the Assessment report.
• Masha suggests coming up with a benchmark (validation parameters) based on historical and real time events or time interval. Model should also be running for a while to evaluate the performance.
• The initial AWS Assessment due date was end of Oct. Not sure what we have met the milestone or not. SWPC will evaluate the assessment and determine a new due date, if needed.
• CTIPe docker image has been checked in to SMCE GitLab image repository. It reduces the amount of transfer time between the environments
• CTIPe has a web server docker container and is running. We can stop it since we are not using it.

• Our AGU poster session is scheduled for Monday, 7 December 2020; 04:00 - 20:59 PST.
• Chinwe submitted an abstract to the ESWS (European Space Weather Symposium). It was selected for presentation and is scheduled for Nov 6, 14:30-14:45 UT https://www.iopconferences.org/iop/frontend/reg/titem.csp?pageID=992798&eventID=1528. Please review and provide feedback on her slides. It needs to be submitted before then.
• Please test the SMCE docker image registry and provide any feedback. AI: Mike H. will check with SWPC team.
• ctipe’s web server is running in private subnet, which is not accessible via the internet publicly. The validation website is also hosted via AWS S3. Do we need to keep the web server helper container running? AI: Mike H. will check with SWPC team.
• Assessment of AWS environment
• Almost done. Please review and provide feedback
• Need help with "Best Practices for Model Transition" section. It needs to include the entrance and exit criteria for the model transition process, steps to promote model from development to production/operation, validation criteria for model operation.
• AI: Jia will connect us to CTIPe model developers to see what they went through

• AGU abstract was accepted for poster session. It is scheduled for Session Date and Time: Monday, 7 December 2020; 04:00 - 20:59 PST. Need to plan for the content of the poster.
• SMCE System Admin configured SMCE gitlab repo to host docker images. SWPC will test if it meets their requirement
• Assessment of AWS environment
• Need help writing up the content for "what we did"
• AWS: Tina
• Moving/installing container to CCMC: Chinwe
• Installing in AWS: SWPC
• Add a short summary at the top and future plan.
• No update for 2nd model yet. Team will focus on the assessment paper until a 2nd model has been identified.

• Chris demoed how SWPC uses Nagios XI.
• CCMC removed the validation page links and removed the IP restrictions. The website is publicly accessible.
• During today's meeting, the team discussed the list of software/model best practice questions that CCMC gathered from https://12factor.net
• No update for the 2nd SWPC model

• Introduced Adam, CCMC AWS System Admin
• Collaboration status website is available at ccmc.gsfc.nasa.gov/ace - it includes the abstract, high level project timeline, and the bi-weekly meeting notes (this page)
• Input plot is now updated
• Validation page is still not working
• Not part of end Aug milestone, but it will be nice to have in the future. Additional planning is needed for this ionosphere validation page to bring it alive.
• AI: For the time being, we will remove this validation page links
• S3 bucket is restricted to NASA & CCMC IP. AI: CCMC will remove those restrictions and make it publicly accessible.
• Masha suggests to have a health monitor in place for the CTIPe model to provide some type of output warning.
• SWPC uses health check docker container and checks the age of the output images. They also use nagios for centralized monitoring. Steve recommends SWPC to give CCMC a nagios demo. AI: SWPC will demo at the next tag-up.
• It would also be nice to include monitoring for input stream and docker execution
• Steve suggests to have this global monitoring solution included next time
• Clarification on what's expected for the Oct. 31 milestone (initial assessment of AWS environment for collaboration) The assessment should include:
• Elements of process - entrance and exit criteria for the model
• What's the validation criteria for model operation
• Best practice recommendations, such as:
• Status monitoring
• Version control and version control log
• Formalize steps to promote model from development to production/operation.
• Refer to https://12factor.net/
• Recommended to involve others to brainstorm/identify process.
• Identify 2nd SWPC model for AWS environment
• One possibility: GloTEC. It is another containerized model and ingests data from NASA resource.
• It would be nice if it is a model heavily used at SWPC and not available at CCMC, yet. Ideally, ACE should have all the models and using the same monitoring tool at SWPC.
• AI: Steve will work with others to identify the next model.
• Thanks to the SWPC team for addressing the input plot errors and Chinwe for identifying the errors. Resulted completing the end of Aug milestone (CTIPe model) before the deadline.

• Submitted AGU Fall 2020 abstract
• CTIPe is able to call IDL to generate plot and images are being pushed to s3 bucket for display.
• Website contains the latest images, except "Inputs" (plot of the inputs for the real time run of CTIPe) is still missing (http://ccmc-swpc.s3-website-us-east-1.amazonaws.com/inputs.html).
  AI: Chris/SWPC will look into this.
• Chinwe is trying to reinstall the CTIPe container in another AWS development instance before reinstall on in-house servers. Making good progress with it.

• Created a container for CTIPe pushes data to S3
• S3 bucket is configured for public static website and available at http://ccmc-swpc.s3-website-us-east-1.amazonaws.com/index.html
• AI: Look into directory listing to list the images
• AI: CCMC will resolve the IDL incompatibility version - CTIPe uses IDL v 8.7.2; however, CCMC installed v.8.5.
• If unable to get 8.7.2, maybe SWPC will consider python workaround instead of IDL or downgrade to v.8.5.
• SMCE Gitlab is incomplete. It is missing Dockerfile, source code and other files for PPE, SWPC_IDL. AI: SWPC will upload the missing items to Gitlab.
• CTIPe model doesn't seem to have a good version control tagging strategy. AI: SWPC will confirm with CTIPe model lead.
• AI: Masha suggested to create a web page to keep track of collaboration status

• Introduced Chinwe
• AWS access key is available through the EC2 IAM_Role
• https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html#instance-metadata-security-credentials
• AI: SWPC will look into how to obtain the credentials via the environment variable for the docker-compose.yml file.
• IDL - CCMC is in the process of getting the floating license.
• Went over the 8/30/2020 milestone (CTIPe). The team agreed on the term for each sub tasks.
• Completely functional
• Being able to write to the s3 bucket
• Being able to plot with IDL
• Output available outside AWS environment
• Making data available by the s3 website
• Exchange of tagged versions between SWPC and CCMC
• Will tag both the source code (in SMCE Gitlab) and docker image (Docker Hub)
• Making the image public - AI: Chris will look into whether this is allow or not
• AWS Elastic Container Registry (ECR) will not be part of this milestone as not sure how long it will take to make data available by the s3 website (2nd requirement)
• For CCMC, we need to be able to reinstall and execute this CTIPe container on in-house server
• AI: Once CTIPe is functional, CCMC will try to re-install while Tina is looking into how to make the data available.
• Will use NOAA slack channel to communicate for this collaboration effort

• AWS access key - Tina is still having trouble authenticating herself to read/write to the bucket.
• Masha encourages purchasing a dedicated floating IDL license
• Masha & Steve agree that we need to identify success criterial for this collaboration effort
• Meeting on following Monday (6/22/2020) to identify progress, issues and plans

• S3 is mounted as a storage volume; however, it is not stable.
• Maybe a script to monitor the mount and remount
• Replace the s3fs library with another one
• Make the data publicly accessible via leveraging s3 solution. Refer to CCMC-SWPC/swpc-ctipe-wrapper/ctipeweb for website content
• Temporary installed IDL license for plots generation; however, not ideal.
• SWPC would like to directly write to the bucket; however, it requires AWS access keys. AI: CCMC will figure where are the authentication keys

Priority: Keys & IDL, web server

• Action Item (AI): CCMC installs miniconda
• AI: Configure sudo rights to run docker-compose as m_ctipe user
• CTIPe generates 30 gb/month
• current local storage is not enough - additional storage is needed
• would like to display the data via web
• s3 bucket sounds like a good solution for both requirements
• AI: Tina figures out s3 permission for website
• AI: CCMC will install IDL