SHPC Condo Software Configuration

In this section, we discuss the SHPC Condos software configuration. Our software environment uses Linux environment modules to perform this configuration. The software modules available to users also contain preconfigured compiler toolchains, or programming environments which include parallel compiler wrappers and associated MPI stacks. There are also workflow tools that may help with your applications as well.

Modules

SHPC has more than one hundred software packages installed. Our software environment uses Linux (CentOS 7.x) environment modules to manage versions and dependencies of software packages. When you load a module, it sets the environment variables necessary for running your program.

A list of available software modules can be viewed by typing module avail.

Modules: Local repository
By default the local repository is used as a source of software installations. To list available modules, type module avail. To load a module, use module load module_name. Similarly, unload modules by typing module unload module_name.

Modules: CVMFS-based repository
A CVMFS (Cern Virtual File System)-based repository is available that has several software packages. To use the CVMFS-based repository run the following commands from your login node:

source /software/dev_tools/swtree/cs400/modulefiles/switch-modules.sh

switch_modules oasis

After entering the above commands the new repository should be active and the command below will list the software available for use:

module avail

Similarly switch_modules local will bring back the local modules to use.

Additional information on SHPC modules may be found here.

Compiling Your Own Software

What system should I compile on?
When compiling your own software on the condos, do not use the login nodes for large parallel compilations. Small compilation tasks are fine, but the login nodes have strict limits placed on the resources available to each user, and are not meant to be used for large or time consuming compilation tasks. Instead, we encourage you to start an interactive session on a compute node and run the parallel compilation from there. Information on starting an interactive session in Moab is available here, and information on starting an interactive session in Slurm is available here.

What directory should I compile my program in?
We discourage compiling software in your home directory, as that filesystem is not meant for high-performance parallel file-io. Instead, we encourage you to compile in your Lustre filesystem directory, or in the /localscratch directory on a compute node, then copy or run make install with the proper prefix to place the program binaries into your home directory.

Compiler ToolChains

Depending on the application/code you are working on, you might choose a specific compiler to achieve the best performance of your programs. Compiler toolchains such as GNU, Intel, PGI and NAG are already installed to work with other libraries. See here for more information on SHPC compilers

Workflow Tools

Workflow tools orchestrate multi-stage computations. Several workflow tools are available on SHPC.

Notes on Specific Software Usage

Visualizing Remote Data over SSH using Visit

Visit is a well-known visualization software package that is available on SHPC. Visit may be configured by creating the profile of SHPC Condo to visualize data from your local Visit client. In order to do so, follow the steps as shown below on your local Visit:

Start Visit and go to Options → Host profiles
Press New Host and populate the fields as described below
- Host nickname: cades
- Remote host name: or-slurm-login.ornl.gov
- Host name aliases: or-slurm-login#g
- Path to Visit installation: /software/dev_tools/swtree/cs400_centos7.2_pe2016-08/visit/2.13.3/centos7.5_binary
- Username: <your_ucams_id>
Check the Tunnel data ... checkbox
Click on Apply
Now in the file → open menu, click on the dropdown (▾) and select cades
Enter your password in the dialog box that opens
You should be able to see your data on the SHPC file system which can be opened just as if it were local data

Visualizing by Paraview

ParaView is an open-source, multi-platform data analysis and visualization application. ParaView users can quickly build visualizations to analyze their data using qualitative and quantitative techniques.

Launch ParaView on laptop (must be the same version as remote):

   In ParaView: File →Connect →Add Server:
   Name: Pick a name
   Server Type: Client/Server
   Host: localhost Port: 11111
    - press Configure button
    - Startup Type: Manual
    - click Save

Open a local terminal 1: ssh username@or-slurm-login07.ornl.gov

    - Start a job and get a compute node (say or-condo-c57):

       [uid@or-slurm-login07 ~]$ srun -A staff -p batch -N 1 -n 1 -c 1 – mem=8G -t 1:00:00 --pty /bin/bash
       srun: job 2575867 queued and waiting for resources
       srun: job 2575867 has been allocated resources

    - Launch the server:

      [uid@or-condo-c57 ~]$ module load paraview/5.8.0
      [uid@or-condo-c57 ~]$ pvserver --force-offscreen-rendering
      Waiting for client...
      Connection URL: cs://or-condo-c57.ornl.gov:11111
      Accepting connection(s): or-condo-c57.ornl.gov:11111

Open a 2nd local TERMINAL 2:

    - ssh -L 11111:localhost:11111 username@or-slurm-login07.ornl.gov
    - ssh -L 11111:localhost:11111 or-condo-c57.ornl.gov

Launch ParaView on Laptop:

   Connect to server you made previously in the ParaView GUI.
   If successful, you’ll see in “Terminal 1” a message that says “Client connected”

      [uid@or-condo-c57 ~]$ pvserver --force-offscreen-rendering
      Waiting for client...
      Connection URL: cs://or-condo-c57.ornl.gov:11111
      Accepting connection(s): or-condo-c57.ornl.gov:11111
      Client connected.