Linux and HPC

Install neurocommand on Linux, HPC

Ways of using Neurocommand in Linux and HPC:

  1. You can use the module files for Neurocontainers directly via CVMFS: https://www.neurodesk.org/docs/getting-started/neurocontainers/cvmfs/
  2. or you can install Neurocommand as described here:

Requirements:

Setup Instructions:

Install command line (e.g. running on Linux or HPC)

  1. Load singularity For optimal performance, ensure you are using Singularity version 3.x:
module load singularity/3.5.0
#or
module load singularity
#or 
module load apptainer

If this doens’t work it might already be loaded. Just run singularity and check.

  1. Load aria2 (Optional) To speed up container downloads, you can optionally install or load aria2c:
module load aria2c
  1. Clone and Set Up the Repository Clone the repository into a directory with enough storage and ensure you are not using a symbolic link (to be sure run cd `pwd -P`). It’s recommended to perform this setup within a Python virtual environment (venv) or a Conda environment:
git clone https://github.com/NeuroDesk/neurocommand.git 
cd neurocommand 
pip3 install -r neurodesk/requirements.txt --user 
bash build.sh --cli
bash containers.sh
export SINGULARITY_BINDPATH=`pwd -P`
# OR, depending on your installation:
export APPTAINER_BINDPATH=`pwd -P`

Install Containers

  • If these steps are successful, the help will be displayed
  • Install all or only specific containers by following the instructions, e.g.:
  1. Search and Install Specific Containers To search for containers that have “itksnap” in the name:
bash containers.sh itksnap
  1. Install a Specific Version To install a specific version, (e.g., itksnap version 4.0.2 from 20240117):
./local/fetch_containers.sh itksnap 4.0.2 20240117

To install all containers with that name:

bash containers.sh --itksnap

To download all containers (be careful - there are a lot of containers!):

bash containers.sh --all

Add your containers to lmod

  • To add each container to the module search path, run the following:
module use $PWD/local/containers/modules/
  • It may be a good idea to add this to your .bashrc if it works. When adding to your .bashrc you will need to replace $PWD to point to the correct path, i.e. the output of this
echo "module use $PWD/local/containers/modules/"
  • It is very important to also set the SINGULARITY_BINDPATH or the APPTAINER_BINDPATH variable in your .bashrc. This variable must contain a comma-separated list of directories you want to access with the Neurodesk tools.

e.g.:

export SINGULARITY_BINDPATH=/scratch/,/data/
# OR, depending on your installation:
export APPTAINER_BINDPATH=/scratch/,/data/
#Note: User the correct line depending on your installation. Do not add a directory that does not exist, otherwise the containers will not start!
  • Run ml avail to see the installed containers at the top of the list (neurodesk containers will take preference over system modules with the same name), run:
module --ignore_cache avail
  • Every time you start a new shell you need to run module use PathToYourContainers or add this command to you .bashrc file.

To update

1. Update the Neurocommand Repository

First, ensure your local repository is up to date by pulling the latest changes:

git pull

Next, rebuild Neurocommand by running:

bash build.sh

2. Update Containers

To update the containers, navigate to the neurodesktop directory and run:

bash containers.sh

3. Update Specific Modules

Choose the module you want to update for example you want to update mrtrix3/3.0.2 module with the eddy_cuda fix:

~/neurocommand/local/fetch_containers.sh mrtrix3 3.0.2 20221108 mrview $@

Examples for specific HPCs:

Greatlakes - University of Michigan

Setup on a desktop

module load singularity
# now change to a directory with enough storage, e.g. /nfs/turbo/username
git clone https://github.com/NeuroDesk/neurocommand.git 
cd neurocommand 
pip3 install -r neurodesk/requirements.txt --user 
bash build.sh --cli
bash containers.sh
export SINGULARITY_BINDPATH=`pwd -P`
bash containers.sh itksnap
# now select a version of itksnap to install. For this copy and paste the installation
echo "module load singularity" >> ~/.bashrc
echo "module use $PWD/local/containers/modules/" >> ~/.bashrc
echo "export SINGULARITY_BINDPATH=/nfs/,/scratch/" >> ~/.bashrc

Setup on with a jupyter notebook

Start new Jupyter notebook by entering “load singularity” in the Module Commands field:

image

Then run these commands:

!pip install jupyterlmod

# Restart the kernel by clicking Kernel -> Restart Kernel

import module
await module.load('niimath')