Inputs/Outputs
From OpenGGCM
Contents |
[edit]
Inputs
- Time series of solar wind (SW) parameters and interplanetary magnetic field (IMF). These usually come from solar wind monitors such as ACE or WIND. Alternatively one can also use made up SW/IMF data to test model responses or specific processes.
- Solar F10.7 cm radio flux (as proxy for solar UV/EUV radiation). This is a slowly varying parameter.
[edit]
Outputs
- Magnetosphere: 3d fields of magnetic field, electric field, plasma density, flow velocity vector, and temperature or pressure. 2D fileds of the same quantities at prescribed planes (this allows for much higher cadence).
- Ionosphere: Filed aligned currents, potential, Hall and Pedersen conductane, e- precipitation energy flux and mean energy.
- Ground: magnetic peerturbations at mid- and high latitudes.
- CTIM: 3D fields of various e-, ion, and neutral parameters: Ne, NfM2, hFm2, winds, ion/neutral temperature, N/O ratio, etc.
[edit]
Configure the user environment
On Zaphod, and many other High Performance Clustered computers, a system for switching between different versions of compilers and libraries exists, it is called 'modules'. Here is an example of how to find what compilers are possible on Zaphod:
user@h101:~> module available ------------------------- /usr/local/Modules3.1.6/modulefiles -------------------------- Chombo-2.0/mpich-1.2.7_gcc-3.3.3_p4 null CommandTailContainer/gcc-3.3.3 openmpi/1.2.5 cmake-2.6.0 pgi/5.2 dot pgi/7.0 gcc/4.1.1 pgi/7.0-2 gdbClusterUtils/mpich_1.2.7-gcc_3.3.3 pgi/7.1 gm/2.0.22 pgi/7.1-1 hdf5/1.6.3/gcc-3.3.3/mpich-1.2.7/p4 pgi/7.1-5 hdf5/1.6.3/gcc-3.3.3/mpich-1.2.7/serial pgi/7.2-2 hdf5/1.6.3/gcc-4.1.1/gm pgi32/5.2 hdf5/1.6.3/gcc-4.1.1/p4 pgi32/7.0 hdf5/1.6.3/gcc-4.1.1/serial pgi32/7.0-2 hdf5/1.8.1 pgi32/7.1 module-info pgi32/7.1-1 modules pgi32/7.1-5 mpich/gm/1.2.6..14a(default) pgi32/7.2-2 mpich/p4/1.2.6 pgi64/5.2 mpich/p4/1.2.7(default) pgi64/7.0 mpich/p4/1.2.7-gcc-3.3.3 pgi64/7.0-2 mpich/p4/1.2.7-gcc3.3.3 pgi64/7.1 mpich/p4/1.2.7-pgi7.0 pgi64/7.1-1 mpich-gcc/gm/1.2.6..14a-gcc pgi64/7.1-5 mpich-gcc4/gm/1.2.7..15-gcc4 pgi64/7.2-2 mpich-gcc4/p4/1.2.7-gcc4 pgi_32/5.2 mpich-pgi7/gm/1.2.7..15-pgi7 use.own mpich-pgi7/p4/1.2.7-pgi7
The following dialog shows loading a compiler and mpich library, verifying that it was loaded, purging the selection, and verifying that the purge was successful.
user@h101:~> module load pgi64/7.2-2 mpich-pgi7/gm/1.2.7..15-pgi7 user@h101:~> module list Currently Loaded Modulefiles: 1) pgi64/7.2-2 2) mpich-pgi7/gm/1.2.7..15-pgi7 user@h101:~> module purge user@h101:~> module list No Modulefiles Currently Loaded.
[edit]
Check out a version of OpenGGCM from the Subversion repository
$ svn checkout svn+ssh://<user>@artemis.sr.unh.edu/svn/raid/ggcm_legacy/branches/OpenGGCM_HEAD
where "user" is a Artemis user.
The repository is organized into a "trunk" and a number of "branches." the ggcm_legacy/branches directory contains the latest development version, OpenGGCM, as well as all of the stable versions. To get a listing of the available versions, do the following:
$ svn ls svn+ssh://<usr>@artemis.sr.unh.edu/svn/raid/ggcm_legacy/branches
[edit]
Create a RUN directory
Runs are identified by the RUN environment variable, which is set during the build process. The user is required to create a RUN directory and populate it with a "runme" file. For example, if RUN=ggcmrun0001:
$ mkdir $HOME/ggcmrun0001
$ cd $HOME/ggcmrun0001
$ cp /directory/where/runme/exists/runme .
One often also includes the following "clean" script, which is useful if one needs to start the build process over from scratch. We usually create the file as $HOME/bin/clean since most users add $HOME/bin to their PATH.
#!/bin/sh
HERE=`pwd`
RUN=`basename ${HERE}`
/bin/rm -r -f BASETIME DIPOLTIME MONITOR PDATA SWMONITOR \
*~ core geo.h* grid* in.* map.* maxmaps* script.* \
*.grid *.grid2 *.smf *.ps run *.o a.out *.exe *.f tmp* \
lall* postsc* grid_include.* in.* gridinfo.* map.dat.* \
minvar run.tar run.tar.gz *.tar throwerror \
*.sh *.tar.gz ${RUN}*
ls -l
exit 0
[edit]
Model or Event Study ?
[edit]
Model
If you are planning a study of MHD basic physics, you will need to create an "swdata" input file. This file will describe the time-dependent solar wind boundary conditions at the sunward boundary of the simulation domain. For example:
-999.00 0.000 3.535 3.535 -400.000 0.000 0.000 7.000 4.000 1.0 0.0 0.0 120.000 0.000 3.535 3.535 -400.000 0.000 0.000 7.000 4.000 1.0 0.0 0.0 239.000 0.000 3.535 3.535 -400.000 0.000 0.000 7.000 4.000 1.0 0.0 0.0
Each line of this file corresponds to a different time (data is linearly interpolated to the simulation time steps at the boundary). The columns describe various solar wind parameters:
- Column 1: Time (min)
- Column 2: IMF <math>B_x</math> (nT)
- Column 3: IMF <math>B_y</math> (nT)
- Column 4: IMF <math>B_z</math> (nT)
- Column 5: Solar wind <math>U_x</math> (km/sec)
- Column 6: Solar wind <math>U_y</math> (km/sec)
- Column 7: Solar wind <math>U_z</math> (km/sec)
- Column 8: Solar wind density (particles/cc)
- Column 9: Solar wind pressure (pPa)
- Column 10: X component of sunward boundary plane
- Column 11: Y component of sunward boundary plane
- Column 12: Z component of sunward boundary plane
The "runme" will refer to swdata on the SWDATA variable line.
The "runme" file is both a configuration file and a script (which calls script.runme in OpenGGCM/bin):
#!/usr/bin/env csh
# NOTE:You will most likely have to adjust the path, # or create a symlink.
if ( ! $?OPENGGCMDIR ) then
setenv OPENGGCMDIR `pwd | sed 's:OpenGGCM-tests/head/.*$:branches/OpenGGCM_HEAD:g'`
endif
exec $OPENGGCMDIR/bin/script.runme `pwd` `basename $0` 0
# tst00001
#
#--------------------------------------------------------------
# This is the magic file that makes global MHD runs happen.
# Don't change the first two lines -- they do the magic.
# Everything below here is input to the model; obviously,
# lines that start with a '#' are comments and blank lines
# are ignored.
# Unified input -- JR, IGPP/UCLA 12/28/2000
#--------------------------------------------------------------
#runstring case-tst00001-swdata
#------------- some top level options ----------------------------
RUN default # run id, default sets to current dir
DOSMF 1 # switch to create parameter database
DOINP 1 # switch to create input file
DOGRD 1 # switch to create the grid
RUNLIN 1 # switch to run on local Linux cluster
KEEPTMP 0 # switch to keep tmp files for debugging
TARGET test
MACH lin # which machine type to run on (lin/sp2/ix1)
TUSER {{ $ENV{'USER'} }}
RUNTIME 18:00:00 # CPU limit, only for SDSC,NCSA
CLASS normal # batch class
PRECISION single # REAL precision
PROTOCOL default # message passing protocol, default should be fine
MHDCOORD MHD
#------------- compiling, profiling and debugging ----------------
LOCALMPI mpif77 # the local mpi compiler script
#MPIF77OPT { -O2 -fast -Msave -Bstatic -v } # options for compilaton
MPIF77OPT { -O2 -fast -Msave -v } # options for compilaton
LOCALF77 {{ $ENV{'FC'} }}
F77PROF false # switch to merge in profiling code
f77debug none
check_mhdvar false # check mhd variables for sane values every step
CHECKSUMS false # write checksums to compare code changes
DUMPFIELDS false # This is used for debugging, copious output
#------------- directories ---------------------------------------
GGCMDIR default # base directory for the code
OUTDIR default # where the output goes
#------------- the code input files ------------------------------
GLOBU default # code input (utilities), default: latest
GRIDA default # ditto, grid/x-grid generation
GRIDB default # ditto, grid/partitioning
GRIDC default # ditto, grid/mapping coefficients
MHDSC default # ditto, constants and definitions
MHDCA default # ditto, MHD core code
#MHDC true # Turn on the r.h.s. calculation in C.
#USESSE2 true # true will activate the sse2 compile options
MHDC false # Turn on the r.h.s. calculation in C.
USESSE2 false # true will activate the sse2 compile options
MHDMI default # ditto, MI coupling code/sigmas/CTIM interf.
IOSOL default # ditto, ionosphere potential solver
IOPOS default # ditto, ionosphere variables post-prpcessing
MHDSO default # Satellite output routines.
COTR1 default # ditto, coord. transf. and time routines
MACHS default # ditto, machine dependent stuff
MINVA default # ditto, minimum variance program
CTIM none # ctim version or "none" for sigaa2 sigma computation
#-------------- define the grid ------------------------------------
#(MCNG) medium resolution, no specific region emphasis, 3,150,000 grid points
NPX 5 # number of processors in x
NPY 1 # number of processors in y
NPZ 2 # number of processors in z
NX 315 # number of cells in x
NY 100 # number of cells in y
NZ 100 # number of cells in z
DY0 0.30 # minimum cell size in y
DZ0 0.30 # minimum cell size in z
XX1 -24.01 # sunward extension of box
XX2 350.01 # anti-sunward extension of box
YY2 48.01 # box size in +-y
ZZ2 48.01 # box size in +-z
#.... how the x-grid is defined
# f = fak*(hm + (h0-hm)*tanhm(b1*(x-x1)) + (hn-hm)*tanhp(b2*(x-x5))
# * - (hm-hmm)*(#one-tanhm(b3*(x-(x3-dmm)))-tanhp(b3*(x-(x3+dmm)))) )
GGX1 $v="-n $NX -x0 $XX1 -xn $XX2"
GGX2 $v="-x1 -26 -x3 10 -dmm 8 -x5 80 "
GGX3 $v="-h0 2.0 -hn 43 -hm 1.7 -hmm 0.70"
GGX4 $v="-b1 0.15 -b2 0.025 -b3 0.3"
#.... if we run locally on a linux cluster we need to construct the machine file
#-------------- resistivity ----------------------------------------
MAGDIFFU nl1 # resistivity model res1/nl1
diffthreshold 0.65 # resistivity threshold
diffconstant 0.05 # resistivity constant
#-------------- output control -------------------------------------
# ':' seperated lists, unit is seconds, +val is only that step, -val is modulus
outputmode combined # single/combined (use single only for jobs with many procs on SP2)
outtime2d -60
outtimeio -60
outtime3d -300
outtimeex 7200:16210:1 # A:B:C.. increase output cadence by C between A nd B
outplanex 0 # output plane location in RE
outplaney 0
outplanez 0
ORBITFILE orbits.txt
orbit_dt 1
modtty 1 # output line to log file every modtty
etajout true # E on edge centers, B on face centers
modrestart 2000
#-------------- time step control ----------------------------------
RRMIN 0.10 # minimum permitted number density in cm**-3
SPEEDLIMIT 1500.0 # maximum permitted Alfven speed in km/s
thx 0.40 # cfl time step factor
modnewstep 1 # recalculate cfl limit every 'modnewstep' step
#-------------- Times, Solar Wind / IMF ----------------------------
#.. UT times, use DIPOLTIME "1967:01:01:00:00:0.0" for generic runs with no dipole tilt
STARTTIME 1999:12:31:22:00:0.0 # UT when the run starts
# Just do 20 minutes
ENDTIME 1999:12:31:22:20:0.0
#ENDTIME 2000:01:01:02:30:0.10
DIPOLTIME 1967:01:01:00:00:0.0 # time for dipole orientation
alphafak 1.00 # density factor to account for SW alphas
#
#...... SW/IMF input by hand (comment section out if not used)
SWMON fi # solar wind monitor sc (wi/ac/i8/ge/fi), use 'fi' if none
#... T(min) bx by bz vx vy vz N(cc) P(pP) flag Nx Ny Nz
SWFILE ./swdata # file with SW/IMF, generated here
#
f107 150. # F10.7 solar flux
MOX default # solar wind monitor position (GSE)
MOY default
MOZ default
swturb1 false # solar wind turbulence (true/false)
swturb_rr 0.0 # solar wind trbulence random perturbation for density in 1/cc
swturb_vv 40.0 # solar wind trbulence random perturbation for velocity in km/sec
#-------------- MI coupling and ionosphere -------------------------
ISPHERE 3.0 # inner boundary radius (RE)
io_shield_lat 50.0 # ionosphere shielding latitude
BND_IO_RR 1.5 # density at inner boundary in cc
BND_IO_TT 500 # temperature at inner bndry in eV
NTHETA 361 # ionosphere grid latitude points
NPHI 121 # ionosphere grid longitude points
NIOX 40 # no of panels in theta for one hemisphere
NIOY 8 # no of modes in phi
NIOGO 4 # order of gauss legendre integration
njslide 2 # fac sliding time averages
modiono 20 # recalc ionosphere every soso step
sigmatrans 5000000 # after which timestep begin transition to s-c sigma
sig_const 5 # Pedersen conductance
sigh_const 5 # constant Hall conductance
sigmafak 1.0 # factor to multiply all ionosphere conductances
mape1 1.0 # MI coupling factor for mapped E
fac_filt 6 # how often to spatially filter FAC
sig_floor 5 # minimum sigmas
fak_fac_dyn 1.0 # factor for dynamo currents (only w/ctim)
sig_smooth 2 # spatially smoothing sigmas
sig_h_back 0.3
sig_p_back 0.5 # nightside H/P background conductances
sigfac_fak 1.00 # fudge factor for discrete e-flux
sigdif_fak 1.00 # fudge factor for diffuse e-flux
BIGDIM 8000000 # global memory
#-------------- CTIM inputs ----------------------------------------
ctim_potfak 0.6 # factor for driving ctim potential
CTIM_IN ./
ctim_f_prsxxxx {"$CTIM_IN/inp_ctim.fields.20000921000.asc"}
ctim_f_mmmxxxx {"$CTIM_IN/inp_ctim.mmm20000921000"}
ctim_f_velxxxx {"$CTIM_IN/inp_ctim.vel20000921000"}
ctim_f_bmagfld {"$CTIM_IN/inp_ctim.bmagfld"}
ctim_f_substorm {"$CTIM_IN/inp_ctim.substorm"}
ctim_f_cormag {"$CTIM_IN/inp_ctim.cormag"}
ctim_f_datam {"$CTIM_IN/inp_ctim.datam"}
ctim_f_levpi {"$CTIM_IN/inp_ctim.levpi"}
ctim_f_gwatts {"$CTIM_IN/inp_ctim.gwatts"}
ctim_f_cpcpd {"$CTIM_IN/inp_ctim.cpcpd"}
ctim_f_ionprof {"$CTIM_IN/inp_ctim.ionprof"}
ctim_f_prof2 {"$CTIM_IN/inp_ctim.prof2"}
ctim_f_dmsp {"$CTIM_IN/inp_ctim.dmsp"}
ctim_f_holt {"$CTIM_IN/inp_ctim.holt"}
ctim_f_dmspmod {"$CTIM_IN/inp_ctim.dmspmod"}
ctim_f_rice91 {"$CTIM_IN/inp_ctim.rice91"}
ctim_f_riceprof {"$CTIM_IN/inp_ctim.riceprof"}
ctim_f_hprof {"$CTIM_IN/inp_ctim.hprof"}
ctim_f_hough {"$CTIM_IN/inp_ctim.hough"}
ctim_f_inshaun {"$CTIM_IN/inp_ctim.d1d2"}
ctim_i10 5 # I10 is the smothing frequency.
ctim_i5 1 # i5 is mwt calling frequency.
ctim_i8 6 # i8 is shaun frequency.
ctim_windmx 1000.0 # maximum wind
ctim_tmpmin 130.0 # minimum temperature
ctim_ampl22 608.0 # amplitudes of the tidal model
ctim_ampl23 500.0
ctim_ampl24 138.0
ctim_ampl25 138.0
ctim_ampl11 200.0
ctim_phase22 9.6 # phases of tidal model.
ctim_phase23 4.6
ctim_phase24 1.2
ctim_phase25 2.2
ctim_phase11 12.0
ctim_kp 3 # kp index
ctim_xx 91 # global setting???
ctim_amp 1.3 # ratio of E foster field to real E field.
ctim_switch_1 default # ctim boolean switches
ctim_switch_2 default
ctim_switch_3 default
ctim_switch_4 default
ctim_switch_5 default
ctim_switch_6 default
ctim_switch_7 default
ctim_switch_8 default
ctim_switch_9 default
ctim_switch_10 default
ctim_switch_11 default
ctim_switch_12 default
ctim_switch_13 default
ctim_switch_14 default
ctim_switch_15 default
ctim_switch_16 default
ctim_switch_17 default
ctim_switch_18 default
ctim_switch_19 default
ctim_switch_20 default
ctim_switch_21 default
ctim_switch_22 default
ctim_switch_23 default
ctim_switch_24 default
ctim_switch_25 default
ctim_switch_26 default
ctim_switch_27 default
ctim_switch_28 default
ctim_switch_30 default
ctim_switch_31 default
ctim_switch_32 default
ctim_switch_33 default
ctim_switch_34 default
ctim_switch_35 default
ctim_switch_37 default
#-------------------------- The End
The user must set the OPENGGCMDIR variable to be the location of the working directory checked out of the subversion repository. In this case, OPENGGCMDIR=$HOME/OpenGGCM_HEAD. Note also that the user must set TARGET=target.local-zaphod-myri to build the code locally on zaphod (previous versions of the target script "precompile" the code on apollo and then build the code on a remote machine via ssh.)
[edit]
Event Study
The alternative to the swdata file is to use actual satellite measurements as the input. This is called an event study.
At this time, there is no automated way of generating the runme, so try to copy an event runme to the run directory. This should remind you of the different manual edits that you must make. You will end up manually setting; "runstring" to some relevent name, TARGET, TUSER, RUNTIME, PROTOCOL (usually mpi), the usual grid definitions, STARTTIME, ENDTIME, DIPOLTIME, alphafak, SWFILE will be set to minvar, SWMON to your designated satellite, INPAVE to the averaging interval, IDATA to where your timecoded satellite data resides, f107 to the F10.7cm solar flux during the event, and MOX, MOY, MOZ, the satellite position at the time of observation. There are other edits that can be made, but those are advanced.
here is an example of an event runme:
#!/usr/bin/env csh
if ( ! $?OPENGGCMDIR ) then
setenv OPENGGCMDIR `pwd | sed 's:OpenGGCM-tests/head/.*$:branches/OpenGGCM_HEAD:g'`
endif
exec $OPENGGCMDIR/bin/script.runme `pwd` `basename $0` 0
# tst0005
#runstring tst00005-event-2002.11.18-orbits-no-ctim
#------------- some top level options ----------------------------
RUN default # run id, default sets to current dir
DOSMF 1 # switch to create parameter database
DOINP 1 # switch to create input file
DOGRD 1 # switch to create the grid
RUNLIN 1 # switch to run on local Linux cluster
KEEPTMP 1 # switch to keep tmp files for debugging
TARGET test
##------------- some top level options ----------------------------
MACH lin # which machine type to run on (ling77/lin/sp2/ix1)
TUSER {{ $ENV{'USER'} }}
RUNTIME 02:00 # CPU limit, only for SDSC,NCSA
CLASS normal # batch class (plat:standarf|capability)
PRECISION single # REAL precision
PROTOCOL mpi # message passing protocol, default should be fine
MHDCOORD MHD
#------------- compiling, profiling and debugging ----------------
LOCALMPI mpif77 # the local mpi compiler script
MPIF77OPT { -O2 } # options for compilaton
LOCALF77 {{ $ENV{'FC'} }}
F77PROF false # switch to merge in profiling code
f77debug none
check_mhdvar false # check mhd variables for sane values every step
CHECKSUMS false # write checksums to compare code changes
#------------- directories ---------------------------------------
GGCMDIR default # base directory for the code
OUTDIR default # where the output goes
#------------- the code input files ------------------------------
GLOBU default # code input (utilities), default: latest
GRIDA default # ditto, grid/x-grid generation
GRIDB default # ditto, grid/partitioning
GRIDC default # ditto, grid/mapping coefficients
MHDSC default # ditto, constants and definitions
MHDCA default # ditto, MHD core code
MHDC default # Turn on the r.h.s. calculation in C.
USESSE2 default # true will activate the sse2 compile options
MHDMI default # ditto, MI coupling code/sigmas/CTIM interf.
IOSOL default # ditto, ionosphere potential solver
IOPOS default # ditto, ionosphere variables post-prpcessing
COTR1 default # ditto, coord. transf. and time routines
MACHS default # ditto, machine dependent stuff
MINVA default # ditto, minimum variance program
CTIM none
#-------------- define the grid ------------------------------------
#---------------------------------------------------------------
#------ grid definition for 'tail1'
#---------------------------------------------------------------
npx 1
npy 1
npz 1
nx 128
ny 64
nz 64
dy0 0.4
dz0 0.4
xx1 -21.01
xx2 1000.01
yy2 40.01
zz2 40.01
#.... how the x-grid is defined
# f = fak*(hm + (h0-hm)*tanhm(b1*(x-x1)) + (hn-hm)*tanhp(b2*(x-x5))
# * - (hm-hmm)*(#one-tanhm(b3*(x-(x3-dmm)))-tanhp(b3*(x-(x3+dmm)))) )
#
ggx1 $v="-n $NX -x0 $XX1 -xn $XX2 "
ggx2 $v="-x1 -26 -x3 10 -dmm 8 -x5 80"
ggx3 $v="-h0 2.0 -hn 43 -hm 1.7 -hmm 0.7"
ggx4 $v="-b1 0.15 -b2 0.025 -b3 0.3 "
#-------------- resistivity ----------------------------------------
MAGDIFFU nl1 # resistivity model res1/nl1
diffthreshold 0.75 # resistivity threshold
diffconstant 0.03 # resistivity constant
#-------------- output control -------------------------------------
# ':' seperated lists, unit is seconds, +val is only that step, -val is modulus
outputmode combined # single/combined (use single only for jobs with many procs on SP2)
outtimerun -1200000 # frequency for runtime graphics, 0 if not wanted
outtime2d -300
outtimeio -300
outtime3d -600000
outtimeex 280000:1080000:1
outplanex 10000 # none
outplaney 0
outplanez 10000 # none
modtty 1 # output line to log file every modtty
etajout true # E on edge centers, B on face centers
#-------------- time step control ----------------------------------
RRMIN 0.10 # minimum permitted number density in cm**-3
SPEEDLIMIT 1500.0 # maximum permitted Alfven speed in km/s
thx 0.40 # cfl time step factor
modnewstep 1 # recalculate cfl limit every 'modnewstep' step
modrestart 10000000 # write restart every soso steps
#-------------- Times, Solar Wind / IMF ----------------------------
# event.2002.11.18
STARTTIME 2002:11:18:00:00:0.000 # UT when the run starts
ENDTIME 2002:11:18:16:00:0.000
DIPOLTIME 2002:11:18:08:00:0.000 # time for dipole orientation
alphafak 1.10 # density factor to account for SW alphas
##
###...... automatic SW file generation (comment section out if not used)
SWMON ge # solar wind monitor sc (wi/ac/i8/ge/fi), use 'fi' if none
SWFILE minvar # file with SW/IMF, can be auto/minvar
SETIMFBX none # set Bx to fixed value, or "none"
INPAVE 30 # averaging interval for input files, seconds
IDATA ../events/event.2002.11.18
f107 174.8 # F10.7 solar flux
MOX 19.60686 # solar wind monitor position (GSE)
MOY -22.69474
MOZ 5.11076
swturb1 false # solar wind turbulence (true/false)
swturb_rr 0.0 # solar wind trbulence random perturbation for density in 1/cc
swturb_vv 0.0 # solar wind trbulence random perturbation for velocity in km/sec
#-------------- MI coupling and ionosphere -------------------------
ISPHERE 3.5 # inner boundary radius (RE)
io_shield_lat 50.0 # ionosphere shielding latitude
BND_IO_RR 1.0 # density at inner boundary in cc
BND_IO_TT 400 # temperature at inner bndry in eV
NTHETA 181 # ionosphere grid latitude points
NPHI 61 # ionosphere grid longitude points
NIOX 24 # no of panels in theta for one hemisphere
NIOY 6 # no of modes in phi
NIOGO 4 # order of gauss legendre integration
njslide 2 # fac sliding time averages
modiono 20 # recalc ionosphere every soso step
sigmatrans 10000 # after which timestep begin transition to s-c sigma
sig_const 5.00
sigmafak 1.0 # factor to multiply all ionosphere conductances
mape1 1.0 # MI coupling factor for mapped E
fac_filt 2 # how often to spatially filter FAC
sig_floor 0.5 # minimum sigmas
fak_fac_dyn 1.0 # factor for dynamo currents (only w/ctim)
sig_smooth 2 # spatially smoothing sigmas
sig_h_back 0.3
sig_p_back 0.5 # nightside H/P background conductances
sigfac_fak 2.00 # fudge factor for discrete e-flux
sigdif_fak 2.00 # fudge factor for diffuse e-flux
BIGDIM 20000000 # global memory
# <SATOUT>
orbitfile orbits.txt
orbit_dt 5
# </SATOUT>
#-------------------------- The End ------------------------------------
Currently, there is no automated way to acquire the event data. One typically goes to cdaweb.nasa.gov and grabs data from the NSSDC. The raw data must be converted into ASCII files in a specific format (*** THIS NEEDS TO BE FLESHED OUT, since amak.get_cdaweb is hopelessly broken due to changes in the cdaweb.nasa.gov web site ***).
After you have acquired the event data, it must be processed into a form that OpenGGCM can use. The following script accomplishes this; it must be run from within the IDATA directory.
#!/bin/csh
set AC = $#argv
#echo "AC=$AC"
if ( -d /usr/local/public/bin ) then
set TOOLPATH=/usr/local/public/bin
else
set TOOLPATH=$HOME/ggcm_bin
endif
echo "Tool path is $TOOLPATH"
set PU = 1
if ($AC == 0) then
while (1)
echo "Spacecraft prefix (ac, cl, g0, g8, ge, l0, l4, l7, l9, ma, po, sa, wi) ?"
set SC = $<
echo $SC | egrep --regexp="ac|cl|g0|g8|ge|l0|l4|l7|l9|ma|po|sa|wi"
set TT = $status
#echo "status = $TT"
if ( $TT == 0 ) break
end
echo "WARNING: There will be no checks on your Timecode entries."
echo "T1 (YEAR:MO:DD:HR:MM:0.0) = ?"
set T1 = $<
echo "T2 (YEAR:MO:DD:HR:MM:0.0) = ?"
set T2 = $<
echo "Fillgap, in seconds ?"
set SECONDS = $<
else
# We process the command line arguments in this case.
if ($AC == 4) then
set SC = $1; shift
set T1 = $1; shift
set T2 = $1; shift
set SECONDS = $1; shift
else
# There must be exactly 4 arguments; SC, T1, T2
set PU = 0
endif
if ( $PU == 1 ) then
foreach AA ( $SC $T1 $T2 )
echo $AA | egrep --regexp="-h|-help"
set TT = $status
if ( $TT == 0 ) then
set PU = 0
break
endif
end
endif
# Despite having 4 arguments, the SC could still be bogus.
if ( $PU == 1 ) then
echo $SC | egrep --regexp="ac|cl|g0|g8|ge|l0|l4|l7|l9|ma|po|sa|wi"
set TT = $status
if ( $TT == 1 ) then
set PU = 0
endif
endif
# There should be checks for T1 and T2, but csh is a pain.
endif
if ( $PU == 0 ) then
echo " "
echo "Usage: $0 Spacecraft tstart tend fillgap"
echo " "
echo "Spacecraft choices {ac, cl, g0, g8, ge, l0, l4, l7, l9, ma, po, sa, wi}"
echo " "
echo "Example:"
echo "$0 ac 1999:02:18:00:00:0.0 1999:02:19:00:00:0.0 30"
echo " SAT YEAR:MO:DD:HR:MM:0.0 YEAR:MO:DD:HR:MM:0.0 FILLGAP"
echo " Note that FILLGAP is in seconds."
echo " "
exit 1
else
echo "Spacecraft= $SC T1=$T1 T2=$T2 Fillgap=$SECONDS"
endif
#exit 1
foreach AA ( `ls $SC.*` )
if ( -e $AA && -z $AA ) then
echo "Removing $AA because it is zero size."
rm $AA
else
ls -l $AA
endif
end
if ( -e $SC.np ) then
if ( ! -e $SC.rr ) then
cp $SC.np $SC.rr
else
echo "$SC.np and $SC.rr both exist."
ls -l $SC.np $SC.rr
endif
else
echo "Missing $SC.np"
endif
# Rationalize all the times to one base.
foreach AA ( np rr pp temp bxgse bygse bzgse btot vxgse vygse vzgse vtot vth xgse ygse zgse )
if ( -e $SC.$AA ) then
if( ! -e save.$SC.$AA ) then
echo "cp $SC.$AA save.$SC.$AA"
cp $SC.$AA save.$SC.$AA
else
echo "$SC.$AA and save.$SC.$AA exist."
ls -l $SC.$AA save.$SC.$AA
endif
set echo
cat save.$SC.$AA | tser_ave -t1 $T1 -t2 $T2 -tav $SECONDS -fillgaps > $SC.$AA
unset echo
head --verbose --lines=2 $SC.$AA
else
echo "Missing $AA"
endif
end
#exit
csh $TOOLPATH/amak.derived $SC -v
ls -lrt $SC.*
The subversion repository on Artemis has a script that may be useful to you when working on a project. The idea is that a project will have multiple runs associated with it. So the cycle script will allow you to dryrun the compile and submit process. Experienced users who are modifying the OpenGGCM source will find the dryrun feature useful.
svn list svn+ssh://YourUserName@artemis.sr.unh.edu/usr/svn/cluster_scripts
Grab the one called cycle.sh
svn co svn+ssh://YourUserName@artemis.sr.unh.edu/usr/svn/cluster_scripts/cycle.sh
The command
sh cycle.sh -h
will execute the script with a help message.
more to come etc.
[edit]
Final steps.
Finally, the user should create a $HOME/ggcm_data directory, which is where data files will be written to. It's best to use one of the Zaphod storage nodes (s101-s106) for this purpose. For example, on the head node h101, storage node s105 is NFS mounted as /mnt/data07.
$ cd $HOME
$ ln -s /mnt/data07 ggcm_data
[edit]
Execute the runme script
Theoretically, the user should now be ready to build the executable on zaphod:
$ cd $HOME/$RUN
$ ./runme
After a minute or so, a new $HOME/ggcm_data/$RUN directory is created. This directory contains some input files as well as the executable. For example:
$ ls $HOME/ggcm_data/$RUN
clock45_0006.exe clock45_0006.smf script.runtimegraphics
clock45_0006.f in.clock45_0006
[edit]
Submit a job to the Zaphod batch system
Currently, Zaphod makes use of the TORQUE batch system, which is essentially just a new version of the unsupported OpenPBS batch system (see ZaphodAdmin:SysAdmin:Batch system for details). You should therefore create a batch script in $HOME/ggcm_data/$RUN to execute $RUN.exe (in this example, clock45_0006.exe). For various reasons, it's best if you use mpiexec rather than mpirun (see the Zaphod:Run an MPI job page for general instructions).
Here is an example batch script /home/jdorelli/ggcm_data/test0001/test0001.batch:
#!/bin/csh #PBS -l nodes=2:ppn=2 #PBS -l walltime=00:10:00 #PBS -q debug cd $HOME/ggcm_data/test0001 /usr/local/bin/mpiexec ./test0001.exe </dev/null >& test0001.log
One submits the job as follows:
$ qsub test0001.batch
The code output is redirected to test0001.log
[edit]
Checking on your job
There are some nice commands to tell you about the status of your batch jobs. One such command is the showstate command. It provides copious detail about all jobs and information on which nodes are in use.
you@h101:~> showstate
cluster state summary for Tue Feb 7 13:47:20
JobName S User Group Procs Remaining StartTime
------------------ - --------- -------- ----- ----------- -------------------
(A) 8141 R nbessho nbessho 16 19:34:04 Tue Feb 7 09:21:24
(B) 8142 R nbessho nbessho 16 19:34:04 Tue Feb 7 09:21:24
(C) 8147 R dpontin users 32 2:01:41:15 Tue Feb 7 13:28:35
(D) 8099 R pzhu pzhu 64 2:21:56:07 Fri Feb 3 11:43:27
(E) 8110 R kai users 16 3:02:58:49 Fri Feb 3 16:46:09
(F) 8130 R djl djl 34 5:00:29:52 Mon Feb 6 14:17:12
(G) 8119 R wenhuil wenhuil 66 5:11:45:49 Mon Feb 6 01:33:09
usage summary: 7 active jobs 122 active nodes
[0][0][0][0][0][0][0][0][0][1][1][1][1][1][1][1][1][1][1][2]
[1][2][3][4][5][6][7][8][9][0][1][2][3][4][5][6][7][8][9][0]
Frame 01: [ ][!][ ][!][ ][ ]XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Frame 10: [ ][ ][ ][ ][ ][C][C][C][C][C][D][D][D][D][D][D][D][D][!][D]
Frame 11: [D][D][D][D][D][!][D][#][D][#][D][D][D][D][D][D][D][D][C][C]
Frame 12: [C][C][C][C][!][C][C][C][C][C][B][B][B][B][B][B][D][D][B][B]
Frame 13: [G][G][G][G][G][G][!][!][G][G][G][D][D][D][D][D][D][!][G][G]
Frame 14: [G][G][G][G][G][G][G][G][G][G][G][G][G][G][G][G][G][G][G][G]
Frame 15: [G][G][A][A][A][!][A][A][A][A][A][E][E][E][!][E][E][E][!][E]
Frame 16: [!][E]XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Frame 17: [ ][ ][ ][ ][ ][ ][ ][ ][#][ ]XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Frame 18: [ ][ ][ ][ ][ ][ ][#][ ][ ][F]XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Frame 19: [F][#][F][#][F][F][F][F][F][F]XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Frame 20: [F][F][F][F][F][F][F][F]XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Key: [?]:Unknown [*]:Down w/Job [#]:Down [ ]:Idle [@] Busy w/No Job [!] Drained
node m128 is down
node m130 is down
node n109 is down
node n117 is down
node n122 is down
node n124 is down
A less detailed command in the 'show' family is:
you@h101:~> showq
ACTIVE JOBS--------------------
JOBNAME USERNAME STATE PROC REMAINING STARTTIME
8141 nbessho Running 16 19:30:27 Tue Feb 7 09:21:24
8142 nbessho Running 16 19:30:27 Tue Feb 7 09:21:24
8147 dpontin Running 32 2:01:37:38 Tue Feb 7 13:28:35
8099 pzhu Running 64 2:21:52:30 Fri Feb 3 11:43:27
8110 kai Running 16 3:02:55:12 Fri Feb 3 16:46:09
8130 djl Running 34 5:00:26:15 Mon Feb 6 14:17:12
8119 wenhuil Running 66 5:11:42:12 Mon Feb 6 01:33:09
7 Active Jobs 244 of 299 Processors Active (81.61%)
122 of 148 Nodes Active (82.43%)
IDLE JOBS----------------------
JOBNAME USERNAME STATE PROC WCLIMIT QUEUETIME
8131 wollheim Idle 36 2:00:00 Mon Feb 6 15:14:06
1 Idle Job
BLOCKED JOBS----------------
JOBNAME USERNAME STATE PROC WCLIMIT QUEUETIME
Total Jobs: 8 Active Jobs: 7 Idle Jobs: 1 Blocked Jobs: 0
