Brain Imaging & Analysis Center

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Accessing Available Software ======

The cluster uses [[https://modules.readthedocs.io/en/latest/module.html|environment modules]] to control what software is loaded in your environment.  Certain software will be loaded by default, but there are other versions available to keep analyses consistent across a study or additional software that you may wish to load.  Modules work by adding specific environmental variables needed for each package and/or adding them to your default search path. 

==== What's Loaded? ====
- "**module list**" will show you what is currently loaded 
<code>
[cmp12@blade08 ~]$ module list
Currently Loaded Modulefiles:
  1) fsl/6.0.3           2) freesurfer/7.1.1    3) afni/21.1.04        4) ants/2.3.4          
  5) mrtrix3/3.0.2       6) singularity/3.6.3   7) matlab/R2019b
</code>

==== What's Available? ====
- "**module avail**" will show you what additional software or alternative are available 
<code>
[cmp12@blade08 ~]$ module avail

------------------------------------------ /etc/modulefiles -----------------------------------------------
afni/18.2.15                dtk/0.6.4.1                 fsl/5.0.10                  laynii/2.0.0                miniconda                   singularity/2.5.2
afni/21.1.04                ffmpeg/4.1                  fsl/5.0.6                   mango/4.0.1                 mpi/mpich-3.0-x86_64        singularity/3.6.3
ansysEM/19.0                fidl/2.66                   fsl/5.0.9                   mango/4.1                   mpi/mpich-x86_64            slicer/4.6.2
ansysEM/19.5                freesurfer/0_dev            fsl/6.0.0                   matlab/R2014a               mpi/openmpi-x86_64(default) trackvis/0.6.1
ants/2.3.4                  freesurfer/5.3.0            fsl/6.0.3                   matlab/R2017a               mricron                     weka/3.8.4
c3d/1.1.0                   freesurfer/6.0.0            hcp_pipelines/3.27.0        matlab/R2018a               mrtrix3/3.0.2(default)
camino/2.1                  freesurfer/6.0.1            hcp_workbench/1.4.0         matlab/R2019b(default)      mrtrix3/mact
dcm2niix                    freesurfer/7.1.1            hcp_workbench/1.4.2         matlab/R2021a               mrtrix3/tissue
dramms/1.5.1                fscan                       hcp_workbench/1.5.0         medinria/2.2.3(default)     rar/5.70
dtiprep/1.2.4               fsl/5.0.1                   laynii/1.5.6                medinria/4.0.0              simnibs/2.0.1

</code>

==== Get more info about a module ====
To get more info about a specific module use the "module display" or "module whatis" command.

<code>
[cmp12@blade01 ~]$ module display fsl
-------------------------------------------------------------------
/etc/modulefiles/fsl/6.0.3:

module-whatis	 loads the fsl v6.0.3 environment 
setenv		 FSLDIR /usr/local/packages/fsl-6.0.3 
setenv		 FSLSUBMIT 0 
setenv		 FSL_DIR /usr/local/packages/fsl-6.0.3 
setenv		 FSL_BIN /usr/local/packages/fsl-6.0.3/bin 
setenv		 FSLTCLSH /usr/local/packages/fsl-6.0.3/bin/fsltclsh 
setenv		 FSLWISH /usr/local/packages/fsl-6.0.3/bin/fslwish 
setenv		 FSLOUTPUTTYPE NIFTI_GZ 
prepend-path	 PATH /usr/local/packages/fsl-6.0.3/bin 
-------------------------------------------------------------------


[cmp12@blade01 ~]$ module whatis fsl
fsl                  : loads the fsl v6.0.3 environment

</code>

==== How to modify the environment ====
 - to load additional software use "**module load modulename(s)**"\\
 - just calling the modulename will load the highest version number of an available software, unless there is a "default" designation in the listing

<code>
[cmp12@blade08 ~]$ module load laynii dcm2niix fscan

[cmp12@blade08 ~]$ module list
Currently Loaded Modulefiles:
  1) fsl/6.0.3           3) afni/21.1.04        5) mrtrix3/3.0.2       7) matlab/R2019b       9) dcm2niix
  2) freesurfer/7.1.1    4) ants/2.3.4          6) singularity/3.6.3   8) laynii/2.0.0       10) fscan
</code>

If there is a package already loaded you should use the **switch** command to change versions\\
- "**module switch modulename/version**"

<code>
[cmp12@blade08 ~]$ which feat
/usr/local/packages/fsl-6.0.3/bin/feat

[cmp12@blade08 ~]$ module switch fsl/5.0.10

[cmp12@blade08 ~]$ module list
Currently Loaded Modulefiles:
  1) **fsl/5.0.10**      3) afni/21.1.04        5) mrtrix3/3.0.2       7) matlab/R2019b       9) dcm2niix
  2) freesurfer/7.1.1    4) ants/2.3.4          6) singularity/3.6.3   8) laynii/2.0.0       10) fscan

[cmp12@blade08 ~]$ which feat
/usr/local/packages/fsl-5.0.10/bin/feat

</code>

- to remove a loaded package "**module rm modulename**"
- to unload everything "**module purge**"

<code>
[cmp12@blade08 ~]$ module rm fsl

[cmp12@blade08 ~]$ module list
Currently Loaded Modulefiles:
  1) freesurfer/7.1.1    3) ants/2.3.4          5) singularity/3.6.3   7) laynii/2.0.0        9) fscan
  2) afni/21.1.04        4) mrtrix3/3.0.2       6) matlab/R2019b       8) dcm2niix

[cmp12@blade08 ~]$ module purge

[cmp12@blade08 ~]$ module list
No Modulefiles Currently Loaded.

</code>


==== Make it consistent ====

You can add module calls to your submission scripts to keep the environment consistent for your jobs or you can add module calls to your ~/.bashrc initialization file to have a specific set of modules across all of your cluster jobs or interactive sessions.

<code>
[cmp12@blade01 ~]$ tail -n 10 ~/.bashrc 

module purge
module load fsl/6.0.0 freesurfer/6.0.1 matlab/R2018a


[cmp12@blade01 ~]$ module list
Currently Loaded Modulefiles:
  1) fsl/6.0.0          2) freesurfer/6.0.1   3) matlab/R2018a

</code>


If there are certain packages that are not available you can always email biac-help@duke.edu to request them.  Additionally, standard linux software will not show up in the module list ( ie: cmake3, gcc ).  Usually the which or locate command is an easy way to see of they are available.

<code>
[cmp12@blade01 ~]$ which python3
/usr/bin/python3

[cmp12@blade01 ~]$ which cmake3
~/bin/cmake3
</code>


====== Setting Up Experiment Specific Conda Environments ======

[[https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html|Conda]] is available via modules to setup and maintain your own specific environments if you want to use specific versions of python ( or R ) modules that are not tied to the default cluster installation.

First, you want to unload the default python3 and load miniconda via modules to begin:
<code>
[cmp12@blade29 ~]$ module unload python3
[cmp12@blade29 ~]$ module load miniconda

[cmp12@blade29 ~]$ conda --version
conda 22.11.1

#set the default location to download packages to TMPDIR so you do not fill up your home directory
[cmp12@blade29 ~]$ export CONDA_PKGS_DIRS=${TMPDIR}/pkgs

</code>


Now, create your environment in a mounted location.  For example and Lab directory for general access from your other lab members, or in an experiment so it can be used throughout all the analysis of an entire experiment.

<code>
conda create -p /mnt/munin/Song/Lab/Chris/myenv python

#activate it
conda activate /mnt/munin/Song/Lab/Chris/myenv

#install some of the python modules you may need
(/mnt/munin/Song/Lab/Chris/myenv) [cmp12@blade29 ~]$ which pip3
/mnt/munin/Song/Lab/Chris/myenv/bin/pip3

(/mnt/munin/Song/Lab/Chris/myenv) [cmp12@blade29 ~]$ pip3 install numpy scipy nibabel ipython

#verify
(/mnt/munin/Song/Lab/Chris/myenv) [cmp12@blade29 ~]$ which python
/mnt/munin/Song/Lab/Chris/myenv/bin/python


(/mnt/munin/Song/Lab/Chris/myenv) [cmp12@blade29 ~]$ pip3 freeze
ipython==8.11.0
jedi==0.18.2
matplotlib-inline==0.1.6
nibabel==5.0.1
numpy==1.24.2
.......
</code>

Now you have a consistent environment that you can use/maintain throughout the experiment.

To deactivate just run "conda deactivate" which will remove the environment from your path.

To make it consistent, you can modify your ~/.bash_profile ( or ~/.bashrc ) to automatically add miniconda and activate the environment:

<code>
module remove python3
module add miniconda

conda activate /mnt/munin/Song/Lab/Chris/myenv
</code>


Alternatively, you can activate within submission scripts and/or make sure you are explicitly calling your own version of python within scripts:

<code>
 [cmp12@blade29 ~]$ //mnt/munin/Song/Lab/Chris/myenv/bin/python
Python 3.10.4 (main, Mar 31 2022, 08:41:55) [GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy.__version__
'1.24.2'
>>> numpy.__file__
'//mnt/munin/Song/Lab/Chris/myenv/lib/python3.10/site-packages/numpy/__init__.py'
</code> 

Add something else to your environment ( like R ):
<code>
(/mnt/munin/Song/Lab/Chris/myenv) [cmp12@blade29 ~]$ conda search r-base

#if you don't see the version you want you can add additional channels
conda config --add channels conda-forge
conda config --set channel_priority strict

#then search again
conda search r-base


#install the version you would like to use
(/mnt/munin/Song/Lab/Chris/myenv) [cmp12@blade29 ~]$ conda install r-base=4.2.2

(/mnt/munin/Song/Lab/Chris/myenv) [cmp12@blade29 ~]$ which R
/mnt/munin/Song/Lab/Chris/myenv/bin/R

(/mnt/munin/Song/Lab/Chris/myenv) [cmp12@blade29 ~]$ R --version
R version 4.2.2 (2022-10-31) -- "Innocent and Trusting"
Copyright (C) 2022 The R Foundation for Statistical Computing
Platform: x86_64-conda-linux-gnu (64-bit)


</code>


====== Other Software ======

Matlab Libraries : /usr/local/packages/MATLAB

Premade Singularity Images : /usr/local/packages/singularity/images/
Brain Imaging & Analysis Center

User Tools

Site Tools

Page Tools
