Python is preinstalled on all clusters. You must install your own packages. Not every package is installed (see below). You must run a batch or interactive job to run python jobs.
Installing Python Packages
You must install your own packages. Not every package is preinstalled on the system. You can install specific python packages in your account. You must install from an interactive job using pip.
For example:
First start an interactive job using at least one core:
qsub -I -l nodes=1:ppn=1
List the available base python versions.
module avail python
Next, load the python module matching the version you require.
module load python/3.9.5
Use pip to install the setup tools first. The tool will allow you to install a package. You only have to install the tool once.
pip install -U --user setuptools
Lastly, install your package. You may want to use the ‘–no-deps’ option.
pip install -U --user <package_name>
Exit the interactive job when finished to halt charges.
Your package is now installed in your account (in .local/lib/python/site-packages/). You only have to install the package once. You can now use it for any job from any compute node.
Interactive Python job
First start an interactive job. In this example, we are using 1 copper node and 24 cores.
qsub -I -l nodes=1:copper:ppn=24
Next, load the python module matching the version you require.
module load python/3.9.5
Run your script
python3 run.py
Exit the interactive job when finished to halt charges.
exit
Python job in batch mode example
Below is a simple python pbs script. This script will run python in batch using a ‘copper’ node and 16 cores. Note: “python3” must be used. There is a bug where the alias “python” -> “python3” will not work with batch jobs. Be sure to install any packages you may need prior to running the batch job.
#PBS -l nodes=1:copper:ppn=16 cd $PBS_O_WORKDIR module load python/3.9.3 python3 run.py