In order to be able to use different versions of Python flexibly, the Anaconda software has been installed on the masternode, which allows users to create, according to their needs, different environments with different versions of Python.

To load the Anaconda environment, the user must load the anaconda3 module

module load anaconda3

by loading this module you have access to the various anaconda executables including conda , which is the utility that allows the creation and activation of the various Python environments.

The first command to use, ONLY THE FIRST TIME is:

conda init bash

after running this command, the user must disconnect and reconnect from the masternode because his profile is changed so that every time the user opens a session on the masternode he already has access to Anaconda WITHOUT having to load the module.
So the commands:

module load anaconda3
conda init bash

they have to be used only the first time.

The version of Python that is installed with Anaconda is 3.8.3, if you want to use a different version you can see the list of available versions with the command:

conda search python

which returns the list of all usable Python versions.

Once you have chosen the version of Python you want to use, you need to create the environment, with the command in the example you create a Python environment 3.7.9 with the name py37 :

conda create -n py37 python=3.7.9 anaconda

this environment will then be activated in the job file with the command:

conda activate py37

===== PBS jobfile =====

<code bash>
#!/bin/bash
#PBS -S /bin/bash

# Set Jobname
#PBS -N S-ALE

 # gandalf, merlino, morgana, legolas
#PBS -q covenant

#PBS -o job.out

#PBS -e job.err

# Set request for N nodes,C cores per node, M memory per node
#PBS -l select=1:ncpus=8:mpiprocs=8:mem=20gb

# Pass environment to job
#PBS -V

mkdir /scratch_local/$(echo $PBS_O_LOGNAME)_$(basename $PBS_O_WORKDIR)_jobid_$(echo $PBS_JOBID)
cd /scratch_local/$(echo $PBS_O_LOGNAME)_$(basename $PBS_O_WORKDIR)_jobid_$(echo $PBS_JOBID)
cp -R $PBS_O_WORKDIR/* /scratch_local/$(echo $PBS_O_LOGNAME)_$(basename $PBS_O_WORKDIR)_jobid_$(echo $PBS_JOBID)
REMOTE_MOUNT_CMD="ssh masternode mount_remote_scratch $(head -n 1  $PBS_NODEFILE) $(echo $PBS_O_LOGNAME)_$(basename $PBS_O_WORKDIR)_jobid_$(echo $PBS_JOBID)"
REMOTE_UMOUNT_CMD="ssh masternode umount_remote_scratch $(echo $PBS_O_LOGNAME)_$(basename $PBS_O_WORKDIR)_jobid_$(echo $PBS_JOBID)"
eval $REMOTE_MOUNT_CMD

module purge
module load intel impi lsdyna
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/opt/ohpc/pub/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/opt/ohpc/pub/anaconda3/etc/profile.d/conda.sh" ]; then
        . "/opt/ohpc/pub/anaconda3/etc/profile.d/conda.sh"
    else
        export PATH="/opt/ohpc/pub/anaconda3/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<

conda activate py<nome modulo conda>

echo "---------------------------------------------------------------------"
echo  "---Starting job at: `date`"
echo
echo "------Current working directory is `pwd`"
np=$(wc -l < $PBS_NODEFILE)
echo "------Running on ${np} processes (cores) on the following nodes:"
cat $PBS_NODEFILE
echo "----  xxxxxxxx"

python prova.py

#mpirun -np 20 -f $PBS_NODEFILE $SOFT_PATH/R12DEV i=S-ALE.k



echo "-----job finished at `date`"
echo "---------------------------------------------------------------------"

eval $REMOTE_UMOUNT_CMD
mv /scratch_local/$(echo $PBS_O_LOGNAME)_$(basename $PBS_O_WORKDIR)_jobid_$(echo $PBS_JOBID)/* $PBS_O_WORKDIR
rm -Rf /scratch_local/$(echo $PBS_O_LOGNAME)_$(basename $PBS_O_WORKDIR)_jobid_$(echo $PBS_JOBID)

Please report new interesting or broken links to clusterhpc-dcmc@polimi.it

Quick guides and papers: