Set environment variables for parallelization¶
GEOS-Chem Classic uses OpenMP parallelization, which is an implementation of shared-memory (aka serial) parallelization.
OpenMP-parallelized programs (such as GEOS-Chem Classic) cannot execute on more than 1 computational node. Most modern computational nodes typically contain between 16 and 64 cores. Therefore, GEOS-Chem Classic simulations will not be able to take advantage of more cores than these.
We recommend that you consider using GCHP for more computationally-intensive simulations.
In the the sample environment files for GNU and Intel, we define the following environment varaiables for OpenMP parallelization:
OMP_NUM_THREADSenvironment variable sets the number of computational cores (aka threads) that you would like GEOS-Chem Classic to use.
For example, the command below will tell GEOS-Chem Classic to use 8 cores within parallel sections of code:
$ export OMP_NUM_THREADS=8
We recommend that you define
OMP_NUM_THREADSnot only in your environment file, but also in your GEOS-Chem run script.
In order to use GEOS-Chem Classic with OpenMP parallelization, you must request the maximum amount of stack memory in your Unix environment. (The stack memory is where local automatic variables and temporary
$OMP PRIVATEvariables will be created.)
Add the following lines to your system startup file (e.g.
.bashrc) and to your GEOS-Chem run scripts:
ulimit -s unlimited export OMP_STACKSIZE=500m
The ulimit -s unlimited will tell the bash shell to use the maximum amount of stack memory that is available.
The environment variable
OMP_STACKSIZEmust also be set to a very large number. In this example, we are nominally requesting 500 MB of memory. But in practice, this will tell the GNU Fortran compiler to use the maximum amount of stack memory available on your system. The value 500m is a good round number that is larger than the amount of stack memory on most computer clusters, but you can increase this if you wish.
Errors caused by incorrect environment variable settings¶
Be on the lookout for these errors:
OMP_NUM_THREADSis set to 1, then your simulation will execute using only one computational core. This will make your simulation take much longer than necessary.
OMP_STACKSIZEenvironment variable is not included in your environment file (or if it is set to a very low value), you might encounter a segmentation fault error after the TPCORE transport module is initialized. In this case, GEOS-Chem Classic “thinks” that it does not have enough memory to perform the simulation, even though sufficient memory may be present.