Release 20.09

Ability to override worker description per batch

When starting a gspc::scoped_runtime_system one traditionally specifies a topology_description for the workers, which is reused for any add_worker() call. An overload has been added that allows giving a specific description per call, in order to allow asymmetric topologies.

RIF strategy ‘local’

Developers testing on machines that support neither ssh nor pbsdsh may now at least validate their application on the local node by specifying --rif-strategy=local. The strategy expects the only node given to be hostname().

Modify worker environment variables

It is now possible to specify the environment variables to be set when starting a runtime system’s workers, rather than having a predefined, empty, environment.

By default the environment still is completely empty, and no variables other than the ones referenced in the options will be set in the remote process. The following options have been added:

--worker-env-set-variable to explicitly set the value of a variable on the command line
--worker-env-copy-file path to set all variables in the given files, where every line is a key-value pair in env style (key=value)
--worker-env-copy-variable name and --worker-env-copy-current to copy variables set in the process starting up by allow-list or blanket copy respectively

Ability to disable function-does-not-unload check

It is now possible to disable the check for a module call function not leaking any dynamic libraries (and thus tainting the worker process). Workflow developers may tag their module with require_function_unloads_without_rest="false" (default: true) to mark the taint as known and okay.

Orchestrator was inlined into Agent

The orchestrator process has been removed and the functionality was moved into agent. This has also removed the --orchestrator-port command line option.

Due to removing this level, log messages may slightly differ. Besides that post-job cleanup scripts should be adopted to no longer try to kill orchestrator.

$install/lib and $install/libexec/gspc is no longer added to LD_LIBRARY_PATH of worker automatically. Users may use --worker-env-set-variable to reconstruct this behavior, but proper rpaths are suggested instead.
The stencil_cache library was removed as it was too specific for generic use.

Fixes

When raising a missing_binding exception, the full path into a struct is given, instead of just the root.
It is now possible to start 0 workers per node in a topology description, to free clients from having to filter the description in degenerated cases.
Error messages have been improved when using the RIF ssh strategy but neither having USER or HOME exported or having an id_rsa file, nor specifying the respective option on the command line.
All RIF strategies now support --help, if they support command line arguments.
Improved the performance of parallel applications that are generating large numbers of independent tasks, but have way fewer workers than tasks.
pnetc no longer throws an exception when synthesizing virtual places with tokens on them.
The virtual memory client no longer fails to transfer when a large amount of ranges is given.
The expression language parser now correctly handles integral 'f' for floats.
Properly clean up processes when using rif strategy pbsdsh