Command line interface

See the related page in the Getting Started guide.

More documentation is accessible by executing:

smttask COMMAND --help

smttask run

Execute the Task(s) defined by TASKDESC. If multiple TASKDESC files are passed, these are executed in parallel, with the number of parallel processes determined by CORE.

A taskdesc can be obtained by calling .save() on an instantiated task.

This commands creates the environment variable SMTTASK_PROCESS_NUM, which stores an integer >= 0. This number is unique between concurrent calls to smttask run, and can be used to assign unique path names to each concurrent process. This is useful if for example tasks require a compilation directory, and one wants to avoid simultaneous tasks attempting to use the same directory.

Usage

smttask run [OPTIONS] [TASKDESC]...

Options

-n, --cores <cores>

The number of parallel processes to use, if more than one TASKDESC is given. This parameter is ignored if there is only one TASKDESC. Zero or negative values indicate to use all available cores, minus the specified value. (Example: on a 4 core machine, the value -1 indicates to use 3 cores.) Default value value is -1.

--record, --no-record

Use –no-record to disable recording (and thereby also the check that the version control repository is clean).

--keep, --clean

By default, after successfully running a task, the taskdesc file is removed from disk. This can be disabled with the ‘–keep’ option.Cleaning is done on a best-effort basis: if the file cannot be found (e.g. because it was moved), no error is raised.

--recompute, --no-recompute

Add the flag ‘–recompute’ to force computation of a task, even if a previous run is found.

--reason <reason>

Override the reason provided in the TASKDESC.

-v, --verbose

Specify up to 2 times (-vv) to increase the logging level which is printed. Default is to print info, warning and error messages. -v: debug and up -vv: everything.

-q, --quiet

Turn off info messages. Specifying multiple times will also turn off warning (-qq), error (-qqq) and critical (-qqqq) messages.

-r, --reverse <reverse>

If multiple tasks are passed, iterate over them in reversed order. Possible use case: smttask is already running throw a list of tasks, and while we have some extra compute time, we want to knock off some tasks from the end of the list.

--start-spacing <start_spacing>

When running with more than one core, the amount of time (in seconds) to wait before starting the next job. This can help avoid lock conflicts (all recorded tasks need to access the .smt record store when they start and when they end). Default: 1.5 s

--progress-interval <progress_interval>

Set the interval, in minutes, at which the progress bar will be updated. Especially useful when the output will be sent to a log file, to avoid logging thousands of progress bar updates. The default is to let tqdm dynamically adjust the update rate, for a smooth but CPU-friendly progress bar appropriate for console output.

--record-store <record_store>

Specify the Sumatra record store to which execution records should be saved; if no file exists at the specified location, a new record store is created. This option is only required to use a different store that the one specified in project file. The imagined use case is when launching multiple simultaneous runs – the default SQLite backend is not very robust against concurrent access, and can suffer corruption in such cases. Using separate record store avoids this problem, at the cost of having to merge record stores afterwards. Warning: this feature relies on possibly unsafe manipulations of Sumatra objects. If you need this feature, consider switching to a PostgreSQL backend instead. See also the smttask store merge command for combining record stores.

--pdb, --no-pdb

Launch the pdb post-mortem debugger if an exception is raised while running the task. If there is more than one task, this option is mostly ignored since only errors in the root process will trigger a debugging session.

--wait <wait>

Specify an amount of time to wait before starting the task(s). Formats: ‘1h30m’, ‘1hour 30min’.

--import <pkg_imports>

Import these packages before running the task. Also adds them to the list of safe packages. Intended for importing the base project package.

Arguments

TASKDESC

Optional argument(s)

smttask project

Manage SumatraTask projects.

Usage

smttask project [OPTIONS] COMMAND [ARGS]...

Commands

clone

Clone a project from SRC to DEST.

init

Run the initialization wizard for…

smttask store

Inspect or manipulate the records and data stores.

Usage

smttask store [OPTIONS] COMMAND [ARGS]...

Commands

create-surrogates

Create surrogate records for outputs…

delete

Delete one or more records, including the…

find-output

Find output files for a previously run…

merge

Merge entries from multiple record stores.

rebuild

Rebuild the input datastore.

recreate

Recreate the task file used to create…

smttask smt

Execute Sumatra commands

A version of Sumatra is packaged with SumatraTask; it provides the recordstore and provenance tracking functionality. Some of its commands are exposed in this scope.

Note that these are provided as a fallback; ideally all operations would be performed with a proper smttask commands.

Usage

smttask smt [OPTIONS] COMMAND [ARGS]...

Commands

configure

export

info

list

sync

version

view

smttask debug

Debugging utilities.

Usage

smttask debug [OPTIONS] COMMAND [ARGS]...

Commands

compare

Compare two task files that should be the…

humanize

Reformat a task file to make it more human…

key-parents

Recurse into a task’s fields and print the…