Commit 18da3fa1 authored by Tami Fairless's avatar Tami Fairless
Browse files

fixed git clone statement and added DDT link

parent e11dad2d
# Running LASSO-O Shallow Convection via Container
This project helps users to run the LASSO-O workflow software \<insert repository link>
in a local container environment - either Docker,Singularity, or Shifter depending upon the host policy.
LASSO stands for the Large-Eddy Simulation
(LES) Atmospheric Radiation Measurement (ARM) Symbiotic Simulation and Observation
(LASSO) activity. More information about LASSO can be found
[here](https://arm.gov/capabilities/modeling/lasso).
Questions about LASSO and this software can be directed to <lasso@arm.gov>.
## Getting Started
Follow these instructions to run LASSO-O at your local institution:
#### 1) Check out the LASSO-O Shallow Convection runtime helper
```bash
$ git clone https://gitlab.com/gov-doe-arm/docker/run-lasso-o_shcu.git
$ cd run-lasso-o_shcu
```
#### 2) Set up your container runtime environment
See the appropriate README file for setup instructions for your respective
environment:
* [README-SHIFTER.md](./README-SHIFTER.md). Use these instructions if you are running at NERSC or have Shifter installed on your HPC cluster.
* [README-DOCKER.md](./README-DOCKER.md). Use these instructions if you are running from your local desktop. **NOTE: this should be used for small test runs only**
* [README-SINGULARITY.md](./README-SINGULARITY.md). Use these instructions if you have Singularity installed on your HPC cluster.
#### 3) Prepare Simulation Data
Place or symbolically link your WRF LES simulation wrfstat and wrfout files into the `data/inputs` directory.
You may add 1 - 10 different simulation outputs to the inputs folder. Wrfout
and wrfstat files should each be placed in a separate directory named with the correct simulation number,
as shown in the below example:
```bash
$ ls data/inputs/*
data/inputs/sgpwrfout3C1.00:
wrfout_d01_2018-07-10_12:00:00.nc wrfout_d01_2018-07-10_20:00:00.nc
wrfout_d01_2018-07-10_13:00:00.nc wrfout_d01_2018-07-10_21:00:00.nc
wrfout_d01_2018-07-10_14:00:00.nc wrfout_d01_2018-07-10_22:00:00.nc
wrfout_d01_2018-07-10_15:00:00.nc wrfout_d01_2018-07-10_23:00:00.nc
wrfout_d01_2018-07-10_16:00:00.nc wrfout_d01_2018-07-11_00:00:00.nc
wrfout_d01_2018-07-10_17:00:00.nc wrfout_d01_2018-07-11_01:00:00.nc
wrfout_d01_2018-07-10_18:00:00.nc wrfout_d01_2018-07-11_02:00:00.nc
wrfout_d01_2018-07-10_19:00:00.nc wrfout_d01_2018-07-11_03:00:00.nc
data/inputs/sgpwrfout4C1.00:
wrfout_d01_2018-07-10_12:00:00.nc wrfout_d01_2018-07-10_20:00:00.nc
wrfout_d01_2018-07-10_13:00:00.nc wrfout_d01_2018-07-10_21:00:00.nc
wrfout_d01_2018-07-10_14:00:00.nc wrfout_d01_2018-07-10_22:00:00.nc
wrfout_d01_2018-07-10_15:00:00.nc wrfout_d01_2018-07-10_23:00:00.nc
wrfout_d01_2018-07-10_16:00:00.nc wrfout_d01_2018-07-11_00:00:00.nc
wrfout_d01_2018-07-10_17:00:00.nc wrfout_d01_2018-07-11_01:00:00.nc
wrfout_d01_2018-07-10_18:00:00.nc wrfout_d01_2018-07-11_02:00:00.nc
wrfout_d01_2018-07-10_19:00:00.nc wrfout_d01_2018-07-11_03:00:00.nc
data/inputs/sgpwrfstat3C1.00:
wrfstat_d01_2018-07-10_12:00:00.nc
data/inputs/sgpwrfstat4C1.00:
wrfstat_d01_2018-07-10_12:00:00.nc
```
<div style="background-color: #F9F5D2; border: 1px solid grey; margin: 10px; padding: 10px;">
<strong>NOTE: </strong>
You do not have to copy any observational data (e.g., sgpcldfracset15mC1.c1, sgplassodiagobsC1.c1),
as this data is embedded in the container image.
</div>
### 4) Edit config.yml file
Edit the config.yml file to provide parameters about your run. The config.yml file contains
detailed descriptions for each parameter. If you run following these instructions, you
should be able to use the default parameters with the exception of `begin_datetime` and
`container_runtime`, which should be modified for your specific environment, as shown below:
``` yaml
#-----------------------------------------------------------------
# Begin Datetime
#
# Enter the UTC start date and time for your simulation data
# in the format YYYYMMDD.HHMMSS
#
# The entered date MUST be one listed in the adjacent lasso_dates.txt
# file.
#-----------------------------------------------------------------
begin_datetime: 20180710.115900
#-----------------------------------------------------------------
# Container Runtime
#
# Identify whether to run via Docker or Singularity. Use 'docker'
# for Docker and 'singularity' for singularity
#
# (Note, Docker is intended for small-scale runs only.)
#-----------------------------------------------------------------
#container_runtime: docker
#container_runtime: shifter
container_runtime: singularity
```
### 5) Run LASSO-O
Start LASSO-O via the `run.sh` script from the run-lasso-o_shcu folder.
See the appropriate README file for setup instructions for your respective
environment:
* [README-SHIFTER.md](./README-SHIFTER.md). Use these instructions if you are running at NERSC or have Shifter installed on your HPC cluster.
* [README-DOCKER.md](./README-DOCKER.md). Use these instructions if you are running from your local desktop. **NOTE: this should be used for small test runs only**
* [README-SINGULARITY.md](./README-SINGULARITY.md). Use these instructions if you have Singularity installed on your HPC cluster.
<div style="background-color: #F9F5D2; border: 1px solid grey; margin: 10px; padding: 10px;">
<strong>NOTE: </strong>
The specific run command will vary depending upon the container runtime and scheduler
used at your HPC cluster.
</div>
### 6) Plotting output and skill scores via Jupyter Notebook
See the [notebooks/README.md](notebooks/README.md) file for instructions on how to start
up a Jupter Notebook for visualizing the output of your LASSO-O run.
# Running LASSO-O Shallow Convection via Container
This project helps users to run the LASSO-O workflow software \<insert repository link>
in a local container environment - either Docker,Singularity, or Shifter depending upon the host policy.
LASSO stands for the Large-Eddy Simulation
(LES) Atmospheric Radiation Measurement (ARM) Symbiotic Simulation and Observation
(LASSO) activity. More information about LASSO can be found
[here](https://arm.gov/capabilities/modeling/lasso).
Questions about LASSO and this software can be directed to <lasso@arm.gov>.
## Getting Started
Follow these instructions to run LASSO-O at your local institution:
#### 1) Check out the LASSO-O Shallow Convection runtime helper
```bash
$ git clone git@code.arm.gov:lasso/containers/run-lasso-o_shcu.git
$ cd run-lasso-o_shcu
```
#### 2) Set up your container runtime environment
See the appropriate README file for setup instructions for your respective
environment:
* [README-SHIFTER.md](./README-SHIFTER.md). Use these instructions if you are running at NERSC or have Shifter installed on your HPC cluster.
* [README-DOCKER.md](./README-DOCKER.md). Use these instructions if you are running from your local desktop. **NOTE: this should be used for small test runs only**
* [README-SINGULARITY.md](./README-SINGULARITY.md). Use these instructions if you have Singularity installed on your HPC cluster.
#### 3) Prepare Simulation Data
Place or symbolically link your WRF LES simulation wrfstat and wrfout files into the `data/inputs` directory.
You may add 1 - 10 different simulation outputs to the inputs folder. Wrfout
and wrfstat files should each be placed in a separate directory named with the correct simulation number,
as shown in the below example:
```bash
$ ls data/inputs/*
data/inputs/sgpwrfout3C1.00:
wrfout_d01_2018-07-10_12:00:00.nc wrfout_d01_2018-07-10_20:00:00.nc
wrfout_d01_2018-07-10_13:00:00.nc wrfout_d01_2018-07-10_21:00:00.nc
wrfout_d01_2018-07-10_14:00:00.nc wrfout_d01_2018-07-10_22:00:00.nc
wrfout_d01_2018-07-10_15:00:00.nc wrfout_d01_2018-07-10_23:00:00.nc
wrfout_d01_2018-07-10_16:00:00.nc wrfout_d01_2018-07-11_00:00:00.nc
wrfout_d01_2018-07-10_17:00:00.nc wrfout_d01_2018-07-11_01:00:00.nc
wrfout_d01_2018-07-10_18:00:00.nc wrfout_d01_2018-07-11_02:00:00.nc
wrfout_d01_2018-07-10_19:00:00.nc wrfout_d01_2018-07-11_03:00:00.nc
data/inputs/sgpwrfout4C1.00:
wrfout_d01_2018-07-10_12:00:00.nc wrfout_d01_2018-07-10_20:00:00.nc
wrfout_d01_2018-07-10_13:00:00.nc wrfout_d01_2018-07-10_21:00:00.nc
wrfout_d01_2018-07-10_14:00:00.nc wrfout_d01_2018-07-10_22:00:00.nc
wrfout_d01_2018-07-10_15:00:00.nc wrfout_d01_2018-07-10_23:00:00.nc
wrfout_d01_2018-07-10_16:00:00.nc wrfout_d01_2018-07-11_00:00:00.nc
wrfout_d01_2018-07-10_17:00:00.nc wrfout_d01_2018-07-11_01:00:00.nc
wrfout_d01_2018-07-10_18:00:00.nc wrfout_d01_2018-07-11_02:00:00.nc
wrfout_d01_2018-07-10_19:00:00.nc wrfout_d01_2018-07-11_03:00:00.nc
data/inputs/sgpwrfstat3C1.00:
wrfstat_d01_2018-07-10_12:00:00.nc
data/inputs/sgpwrfstat4C1.00:
wrfstat_d01_2018-07-10_12:00:00.nc
```
You can order WRF Simulation data to test with from the [ARM Data Discovery Tool](https://adc.arm.gov/discovery/#/results/datastream::sgplassodiagraw3C1.m1/start_date::2018-07-10)
<div style="background-color: #F9F5D2; border: 1px solid grey; margin: 10px; padding: 10px;">
<strong>NOTE: </strong>
You do not have to copy any observational data (e.g., sgpcldfracset15mC1.c1, sgplassodiagobsC1.c1),
as this data is embedded in the container image.
</div>
### 4) Edit config.yml file
Edit the config.yml file to provide parameters about your run. The config.yml file contains
detailed descriptions for each parameter. If you run following these instructions, you
should be able to use the default parameters with the exception of `begin_datetime` and
`container_runtime`, which should be modified for your specific environment, as shown below:
``` yaml
#-----------------------------------------------------------------
# Begin Datetime
#
# Enter the UTC start date and time for your simulation data
# in the format YYYYMMDD.HHMMSS
#
# The entered date MUST be one listed in the adjacent lasso_dates.txt
# file.
#-----------------------------------------------------------------
begin_datetime: 20180710.115900
#-----------------------------------------------------------------
# Container Runtime
#
# Identify whether to run via Docker or Singularity. Use 'docker'
# for Docker and 'singularity' for singularity
#
# (Note, Docker is intended for small-scale runs only.)
#-----------------------------------------------------------------
#container_runtime: docker
#container_runtime: shifter
container_runtime: singularity
```
### 5) Run LASSO-O
Start LASSO-O via the `run.sh` script from the run-lasso-o_shcu folder.
See the appropriate README file for setup instructions for your respective
environment:
* [README-SHIFTER.md](./README-SHIFTER.md). Use these instructions if you are running at NERSC or have Shifter installed on your HPC cluster.
* [README-DOCKER.md](./README-DOCKER.md). Use these instructions if you are running from your local desktop. **NOTE: this should be used for small test runs only**
* [README-SINGULARITY.md](./README-SINGULARITY.md). Use these instructions if you have Singularity installed on your HPC cluster.
<div style="background-color: #F9F5D2; border: 1px solid grey; margin: 10px; padding: 10px;">
<strong>NOTE: </strong>
The specific run command will vary depending upon the container runtime and scheduler
used at your HPC cluster.
</div>
### 6) Plotting output and skill scores via Jupyter Notebook
See the [notebooks/README.md](notebooks/README.md) file for instructions on how to start
up a Jupter Notebook for visualizing the output of your LASSO-O run.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment