biac:cluster:modules
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
biac:cluster:modules [2021/05/12 12:22] – cmp12 | biac:cluster:modules [2023/03/15 15:54] – cmp12 | ||
---|---|---|---|
Line 1: | Line 1: | ||
====== Accessing Available Software ====== | ====== Accessing Available Software ====== | ||
- | The cluster uses [[https:// | + | The cluster uses [[https:// |
+ | ==== What's Loaded? ==== | ||
- " | - " | ||
< | < | ||
Line 12: | Line 12: | ||
</ | </ | ||
+ | ==== What's Available? ==== | ||
- " | - " | ||
< | < | ||
Line 31: | Line 31: | ||
</ | </ | ||
+ | ==== Get more info about a module ==== | ||
+ | To get more info about a specific module use the " | ||
- | - to load additional software use " | + | < |
+ | [cmp12@blade01 ~]$ module display fsl | ||
+ | ------------------------------------------------------------------- | ||
+ | / | ||
+ | |||
+ | module-whatis loads the fsl v6.0.3 environment | ||
+ | setenv FSLDIR / | ||
+ | setenv FSLSUBMIT 0 | ||
+ | setenv FSL_DIR / | ||
+ | setenv FSL_BIN / | ||
+ | setenv FSLTCLSH / | ||
+ | setenv FSLWISH / | ||
+ | setenv FSLOUTPUTTYPE NIFTI_GZ | ||
+ | prepend-path PATH / | ||
+ | ------------------------------------------------------------------- | ||
+ | |||
+ | |||
+ | [cmp12@blade01 ~]$ module whatis fsl | ||
+ | fsl : loads the fsl v6.0.3 environment | ||
+ | |||
+ | </ | ||
+ | |||
+ | ==== How to modify the environment ==== | ||
+ | - to load additional software use " | ||
+ | - just calling the modulename will load the highest version number of an available software, unless there is a " | ||
< | < | ||
Line 43: | Line 69: | ||
</ | </ | ||
- | - to switch to an alternative version of a loaded | + | If there is a package already |
+ | - " | ||
< | < | ||
Line 78: | Line 105: | ||
</ | </ | ||
+ | |||
+ | |||
+ | ==== Make it consistent ==== | ||
You can add module calls to your submission scripts to keep the environment consistent for your jobs or you can add module calls to your ~/.bashrc initialization file to have a specific set of modules across all of your cluster jobs or interactive sessions. | You can add module calls to your submission scripts to keep the environment consistent for your jobs or you can add module calls to your ~/.bashrc initialization file to have a specific set of modules across all of your cluster jobs or interactive sessions. | ||
Line 93: | Line 123: | ||
</ | </ | ||
+ | |||
+ | |||
+ | If there are certain packages that are not available you can always email biac-help@duke.edu to request them. Additionally, | ||
+ | |||
+ | < | ||
+ | [cmp12@blade01 ~]$ which python3 | ||
+ | / | ||
+ | |||
+ | [cmp12@blade01 ~]$ which cmake3 | ||
+ | ~/ | ||
+ | </ | ||
+ | |||
+ | |||
+ | ====== Setting Up Experiment Specific Conda Environments ====== | ||
+ | |||
+ | [[https:// | ||
+ | |||
+ | First, you want to unload the default python3 and load miniconda via modules to begin: | ||
+ | < | ||
+ | [cmp12@blade29 ~]$ module unload python3 | ||
+ | [cmp12@blade29 ~]$ module load miniconda | ||
+ | |||
+ | [cmp12@blade29 ~]$ conda --version | ||
+ | conda 22.11.1 | ||
+ | |||
+ | #set the default location to download packages to TMPDIR so you do not fill up your home directory | ||
+ | [cmp12@blade29 ~]$ export CONDA_PKGS_DIRS=${TMPDIR}/ | ||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | Now, create your environment in a mounted location. | ||
+ | |||
+ | < | ||
+ | conda create -p / | ||
+ | |||
+ | #activate it | ||
+ | conda activate / | ||
+ | |||
+ | #install some of the python modules you may need | ||
+ | (/ | ||
+ | / | ||
+ | |||
+ | (/ | ||
+ | |||
+ | #verify | ||
+ | (/ | ||
+ | / | ||
+ | |||
+ | |||
+ | (/ | ||
+ | ipython==8.11.0 | ||
+ | jedi==0.18.2 | ||
+ | matplotlib-inline==0.1.6 | ||
+ | nibabel==5.0.1 | ||
+ | numpy==1.24.2 | ||
+ | ....... | ||
+ | </ | ||
+ | |||
+ | Now you have a consistent environment that you can use/ | ||
+ | |||
+ | To deactivate just run "conda deactivate" | ||
+ | |||
+ | To make it consistent, you can modify your ~/ | ||
+ | |||
+ | < | ||
+ | module remove python3 | ||
+ | module add miniconda | ||
+ | |||
+ | conda activate / | ||
+ | </ | ||
+ | |||
+ | |||
+ | Alternatively, | ||
+ | |||
+ | < | ||
+ | | ||
+ | Python 3.10.4 (main, Mar 31 2022, 08:41:55) [GCC 7.5.0] on linux | ||
+ | Type " | ||
+ | >>> | ||
+ | >>> | ||
+ | ' | ||
+ | >>> | ||
+ | '// | ||
+ | </ | ||
+ | |||
+ | Add something else to your environment ( like r ): | ||
+ | < | ||
+ | (/ | ||
+ | |||
+ | (/ | ||
+ | / | ||
+ | |||
+ | (/ | ||
+ | R version 3.6.1 (2019-07-05) -- " | ||
+ | Copyright (C) 2019 The R Foundation for Statistical Computing | ||
+ | Platform: x86_64-conda_cos6-linux-gnu (64-bit) | ||
+ | |||
+ | |||
+ | </ | ||
+ | |||
+ | |||
biac/cluster/modules.txt · Last modified: 2024/06/21 15:44 by 127.0.0.1