HPC
Run neurodesktop in a high-performance computing environment
Ways of using Neurocommand in Linux:
- You can use Neurocontainers (i.e., download Singularity containers) directly via CVMFS: https://www.neurodesk.org/docs/getting-started/neurocontainers/cvmfs/
- or you can install Neurocommand as described here:
Requirements:
Required for installation
- python 3.6+ https://docs.conda.io/en/latest/miniconda.html#linux-installers
- singularity https://docs.sylabs.io/guides/3.5/user-guide/quick_start.html
- git
Required for use
Optional, but recommended
NB: Your HPC will likely have lmod and Singularity already installed - check with your sysadmin
Command line mode (e.g. running on an HPC or CVL)
- Load singularity and for best performance, it should be 3.x e.g.
module load singularity/3.5.0
# Note: Some HPCs install singularity/apptainer on the compute nodes directly (e.g. Bunya at UQ), so you don't need to do this step then.
- Load or install aria2 to optimize the download performance of our containers (THIS IS OPTIONAL)
module load aria2c
- To install the repository, run the following (make sure to clone this to a directory with enough storage, write permissions and NOT a symbolic link (to be sure run cd `pwd -P`)!). It is recommended to perform this setup in a Python venv or conda environment:
git clone https://github.com/NeuroDesk/neurocommand.git
cd neurocommand
pip3 install -r neurodesk/requirements.txt --user
bash build.sh --cli
bash containers.sh
export SINGULARITY_BINDPATH=$PWD
# OR
export APPTAINER_BINDPATH=$PWD
Install Containers
- If these steps are successful, the help will be displayed
- Install all or only specific containers by following the instructions, e.g.:
To search for containers that have “itksnap” in the name:
bash containers.sh itksnap
Then you can copy and paste the specific install command or you can install all containers with that name:
bash containers.sh --itksnap
To download all containers (be careful - there are a lot of containers!):
bash containers.sh --all
Adding your containers to lmod
- To add each container to the module search path, run the following:
module use $PWD/local/containers/modules/
- It may be a good idea to add this to your .bashrc if it works. When adding to your .bashrc you will need to replace $PWD to point to the correct path, i.e.
module use ~/neurocommand/local/containers/modules/
- It is very important to also set the SINGULARITY_BINDPATH or the APPTAINER_BINDPATH variable in your .bashrc. This variable must contain a comma-separated list of directories you want to access with the Neurodesk tools.
e.g.:
export SINGULARITY_BINDPATH=/scratch/,/data/
export APPTAINER_BINDPATH=/scratch/,/data/
#Note: User the correct line depending on your installation. Do not add a directory that does not exist, otherwise the containers will not start!
- to see the installed containers at the top of the list (neurodesk containers will take preference over system modules with the same name), run:
module --ignore_cache avail
Starting a new shell
- Every time you start a new shell you will need to run
conda activate neurocommand
. Unless you added it to your .bashrc, you will also need to runmodule use PathToYourContainers
.
To update
Run
git pull
bash build.sh --cli
- this updates the neurocommand but not the modules
- install.sh does not need to be run again to update containers go into the neurodesktop directory and run
bash containers.sh
Choose the module you want to update for example you want to update mrtrix3/3.0.2 module with the eddy_cuda fix:
~/neurocommand/local/fetch_containers.sh mrtrix3 3.0.2 20221108 mrview $@
or run this to update all containers
bash containers.sh --all