I’m running some python scripts on some Linux clusters using SGE or SLURM. I already have my conda environment set up properly using the login node. I have been writing something like
source ~/.bashrc module purge #Kill all active modules conda init bash conda deactivate conda deactivate conda activate my_env python my_script.py
to activate the environment properly. (I have done a lot of work to figure this out) However, I just found some example codes like
/anaconda3/envs/my_env/bin/python my_script.py
seems to do the same thing without the need for tedious deactivation and activation. Are they actually doing the same thing? If so, which would be the better practice?
Advertisement
Answer
Programmatic execution with an environment is usually better done through the conda run
subcommand. E.g.,
my_slurm_script.sh
#!/bin/bash -l conda run -n my_env python my_script.py
Read the conda run --help
for details.