Usage guide¶
Synopsis¶
neos options [-s –scenario scenario]
Description¶
NEOS is a software which provides a generic framework to manage job sessions on HPC clusters.
The framework is based on scenarios. A scenario defines the sequential steps to properly setup and initialize the job session. Users select the scenario for their jobs and can even define their own scenarios. The frawework can manage a wide variety of workloads from classic MPI parallel computing to advanced visualization sessions with 3D remote rendering.
Global options¶
-h, --help Show help message and exit. --version Print NEOS version and exit. -d, --debug Enable debug output. --dry-run Dry-run mode: print commands to run without actually running them. -l, --log Log commands output instead of printing on stdout/stderr. -k, --keep Do not remove temporary files generated by NEOS. -L, --list List available scenarios with their optional parameters. -s, --scenario=SCENARIO Name of scenario to run. -S, --scenarios-dir=DIR Path to directory with additional scenarios to load. -m, --module=MODULE Name of environment module to load? -M, --modules-dir=DIR Path to directory with additional modules. -o, --opts=OPTS Optional parameters for scenarios.
Environment modules¶
NEOS can load system-wide and user defined environment modules files. To load system-wide module, just set their name with -m, –module parameter.
You can also define specific environment modules files and load them with NEOS. For example, if you define this environment module in ~/modules/mymodule:
#%Module
module-whatis {Description: My own module for my own scenario}
if { ![is-loaded OtherModule/0.0.1] } {
module load OtherModule/0.0.1
}
prepend-path PATH ~/bin
setenv ENVFORSCENARIO "testenv"
The module file first contains a description and a dependency to another module named OtherModule/0.0.1. Then, it prepends the $PATH environment variable with ~/bin. Finally, it sets the $ENVFORSCENARIO with value testenv. Please refer to modules documentation for more details.
This specific module file can be loaded by NEOS with the following parameters:
neos --modules-dir=~/modules --module=mymodule [OPTIONS]
Specific scenarios¶
With NEOS, users can run scenarios available on the system but they can also define their own scenarios. Please refer to the scenario writing guide for reference.
As an example, if a scenario is defined in file ~/scenarios/simple.py and it is called foo, it can be loaded and run by NEOS using the following parameters:
neos --scenarios-dir=~/scenarios --scenario=foo
Integration with Slurm¶
NEOS can run scenarios in Slurm job step context only. In other words, it is impossible to run scenario outside of jobs, except for listing scenarios and options, getting command line helper and printing version.
To run NEOS scenarios, Slurm commands must be used.
srun command¶
The most straighforward way to launch NEOS scenario is to use srun command from the frontend node. For example:
$ ssh login@cluster
$ srun -n 4 neos
In this example, srun will create an job allocation for 4 tasks (or CPUs) and launch NEOS with default scenario on those resources. Please note that you may need to specify other parameters (such as the partition or the QOS) depending on the cluster configuration.
Please refer to Slurm documentation and srun manpage for reference.
salloc command¶
Slurm salloc command create a resources allocation and runs a command from the frontend node. By default, without any command given in parameter, it spawns a new shell on the frontend node within the job allocation environment. To run a scenario on the allocated resources, you must use srun command as well. For example:
$ ssh login@cluster
$ salloc -n 4 # allocate 4 CPUs
$ srun neos # runs NEOS on all allocated resources
The main advantage of this mode is that you can run multiple scenarios in one job allocation.
Please refer to Slurm documentation and salloc manpage for reference.
sbatch command¶
Slurm sbatch command can submit job for later execution. It is recommanded to give a submission script in parameter of this command. This script specifies the allocation constraints. It is run on the first allocated node once the job is started. Here is an example of sbatch submission script submit.sh to run a NEOS scenario:
#!/bin/sh
#SBATCH -n 4
neos
This script can then be submited to Slurm with sbatch command:
$ sbatch submit.sh
Please refer to Slurm documentation and sbatch manpage for reference.
Controlling output¶
NEOS produces 2 types of outputs:
- NEOS internal output, including information, warning, errors reported by the software itself, as well as debug messages when -d, –debug flag is enabled on the command line.
- outputs of sub-commands launched by scenarios (except when explicitely controlled by the scenarios).
By default, all these outputs are mixed within stdout and stderr of NEOS process. The sub-commands outputs can be redirected in a logfile using the -l, –log flag. The destination log file can be set using the –opts parameter like this:
$ neos --log --opts logfile:~/neos.log
The default value of this optional parameter can be obtained with -L, –list.
As stated before, NEOS must be launched by Slurm workload manager. Slurm is able to manage job processes outputs in many advanced ways.
By default, when run interactively, srun redirects all the programs outputs launched on the allocated nodes to the TTY of the user on the frontend node. This behaviour can be controlled using –output parameter. For example, outputs can be completely discarded with:
srun --output=none neos [OPTIONS]
All outputs can also be written in a logfile:
srun --output=srun.log neos [OPTIONS]
This way, srun.log will contain all outputs (including errors) produced by NEOS and all the sub-commands launched by the scenarios. The filenames can also contain various job related variables such as the job ID and the step ID using placeholders (ex: –output=srun_%j-%s_out.log). Please refer to srun manpage for the full list of available placeholders.
It is also possible to separate stdout and stderr using the –error parameter:
srun --output=srun_%j-%s_out.log --error=srun_%j-%s_err.log neos [OPTIONS]
The outputs of batch jobs submitted with the sbatch command are redirected by default to the file ./slurm-%j.out. Just as well as srun command, the sbatch command has the –output and –error parameters to control the destination of the outputs.
Please refer to sbatch manpage for all details.
Slurm and NEOS parameters can be combined to separate all types of outputs:
srun --output=srun-out.log --error=srun-err.log neos --log --opts=logfile:~/neos.log
With the previous commands, the content of the various files are:
- srun-out.log contains the information (and eventually debug if enabled) produced internally by NEOS,
- srun-err.log contains the warning and errors produced internally by NEOS,
- neos.log contains the outputs the sub-commands launched by NEOS scenarios.
Examples¶
Run default scenario:
neos
List available scenarios with their optional parameters:
neos --list
Run gnome scenario with specific resolution:
srun neos --scenario=gnome --opts=resolution:1280x1024
Run paraview scenario within openmpi-1.10 module environment:
srun neos --scenario=paraview --module=openmpi-1.10
Run user-specific foo scenario within bar specific module in dry-run mode:
srun neos --dry-run --scenario-dirs=~/scenarios --scenario=foo --modules-dir=~/modules --module=bar