Command Tool

A class that wraps a CommandLineTool with named argument

Declaration

class janis.CommandTool(**connections)[source]

A CommandTool is an interface between Janis and a program to be executed. Simply put, a CommandTool has a name, a command, inputs, outputs and a container to run in.

This class can be inherited to created a CommandTool, else a CommandToolBuilder may be used.

class janis.CommandToolBuilder(tool: str, base_command: Union[str, List[str], None], inputs: List[janis_core.tool.commandtool.ToolInput], outputs: List[janis_core.tool.commandtool.ToolOutput], container: str, version: str, friendly_name: Optional[str] = None, arguments: List[janis_core.tool.commandtool.ToolArgument] = None, env_vars: Dict[KT, VT] = None, tool_module: str = None, tool_provider: str = None, metadata: janis_core.utils.metadata.ToolMetadata = None, cpus: Union[int, Callable[[Dict[str, Any]], int]] = None, memory: Union[int, Callable[[Dict[str, Any]], int]] = None, time: Union[int, Callable[[Dict[str, Any]], int]] = None, disk: Union[int, Callable[[Dict[str, Any]], int]] = None, directories_to_create: Union[janis_core.operators.selectors.Selector, str, List[Union[janis_core.operators.selectors.Selector, str]]] = None, files_to_create: Union[Dict[str, Union[str, janis_core.operators.selectors.Selector]], List[Tuple[Union[janis_core.operators.selectors.Selector, str], Union[janis_core.operators.selectors.Selector, str]]]] = None, doc: str = None)[source]
__init__(tool: str, base_command: Union[str, List[str], None], inputs: List[janis_core.tool.commandtool.ToolInput], outputs: List[janis_core.tool.commandtool.ToolOutput], container: str, version: str, friendly_name: Optional[str] = None, arguments: List[janis_core.tool.commandtool.ToolArgument] = None, env_vars: Dict[KT, VT] = None, tool_module: str = None, tool_provider: str = None, metadata: janis_core.utils.metadata.ToolMetadata = None, cpus: Union[int, Callable[[Dict[str, Any]], int]] = None, memory: Union[int, Callable[[Dict[str, Any]], int]] = None, time: Union[int, Callable[[Dict[str, Any]], int]] = None, disk: Union[int, Callable[[Dict[str, Any]], int]] = None, directories_to_create: Union[janis_core.operators.selectors.Selector, str, List[Union[janis_core.operators.selectors.Selector, str]]] = None, files_to_create: Union[Dict[str, Union[str, janis_core.operators.selectors.Selector]], List[Tuple[Union[janis_core.operators.selectors.Selector, str], Union[janis_core.operators.selectors.Selector, str]]]] = None, doc: str = None)[source]

Builder for a CommandTool.

Parameters:
  • tool – Unique identifier of the tool
  • friendly_name – A user friendly name of your tool (must be implemented for generated docs)
  • base_command – The command of the tool to execute, usually the tool name or path and not related to any inputs.
  • inputs – A list of named tool inputs that will be used to create the command line.
  • outputs – A list of named outputs of the tool; a ToolOutput declares how the output is captured.
  • arguments – A list of arguments that will be used to create the command line.
  • container – A link to an OCI compliant container accessible by the engine.
  • version – Version of the tool.
  • env_vars – A dictionary of environment variables that should be defined within the container.
  • tool_module – Unix, bioinformatics, etc.
  • tool_provider – The manafacturer of the tool, eg: Illumina, Samtools
  • metadata – Metadata object describing the Janis tool interface
  • cpu – An integer, or function that takes a dictionary of hints and returns an integer in ‘number of CPUs’
  • memory – An integer, or function that takes a dictionary of hints and returns an integer in ‘GBs’
  • time – An integer, or function that takes a dictionary of hints and returns an integer in ‘seconds’
  • disk – An integer, or function that takes a dictionary of hints and returns an integer in ‘GBs’
  • directories_to_create – A list of directories to create, accepts an expression (selector / operator)
  • files_to_create – Either a List of tuples [path: Selector, contents: Selector],

or a dictionary {“path”: contents}. The list of tuples allows you to use an operator for the pathname :param doc: Documentation string

Overview

A janis.CommandTool is the programmatic interface between a set of inputs, how these inputs bind on the Command Line to call the tool, and how to collect the outputs. It would be rare that you should directly instantiate CommandTool, instead you should subclass and and override the methods you need as declared below.

Like a workflow, there are two methods to declare a command tool:

  • Use the CommandToolBuilder class
  • Inherit from the CommandTool class,

Template

CommandToolBuilder:

import janis_core as j

ToolName = j.CommandToolBuilder(
    tool: str="toolname",
    base_command=["base", "command"],
    inputs: List[j.ToolInput]=[],
    outputs: List[j.ToolOutput]=[],
    container="container/name:version",
    version="version",
    friendly_name=None,
    arguments=None,
    env_vars=None,
    tool_module=None,
    tool_provider=None,
    metadata: ToolMetadata=j.ToolMetadata(),
    cpu: Union[int, Callable[[Dict[str, Any]], int]]=None,
    memory: Union[int, Callable[[Dict[str, Any]], int]]=None,
)

This is equivalent to the inherited template:

from typing import List, Optional, Union
import janis_core as j

class ToolName(j.CommandTool):
    def tool(self) -> str:
        return "toolname"

    def base_command(self) -> Optional[Union[str, List[str]]]:
        pass

    def inputs(self) -> List[j.ToolInput]:
        return []

    def outputs(self) -> List[j.ToolOutput]:
        return []

    def container(self) -> str:
        return ""

    def version(self) -> str:
        pass

     # optional

     def arguments(self) -> List[j.ToolArgument]:
         return []

     def env_vars(self) -> Optional[Dict[str, Union[str, Selector]]]:
         return {}

     def friendly_name(self) -> str:
         pass

     def tool_module(self) -> str:
         pass

     def tool_provider(self) -> str:
         pass

     def cpu(self, hints: Dict) -> int:
         pass

     def memory(self, hints: Dict) -> int:
         pass

     def bind_metadata(self) -> j.ToolMetadata:
         pass

Structure

A new tool definition must subclass the janis.CommandTool class and implement the required abstract methods:

  • janis.CommandTool.tool(self) -> str: Unique identifier of the tool
  • janis.CommandTool.base_command(self) -> str: The command of the tool to execute, usually the tool name or path and not related to any inputs.
  • janis.CommandTool.inputs(self) -> List[janis.ToolInput ]: A list of named tool inputs that will be used to create the command line.
  • janis.CommandTool.arguments(self) -> List[janis.ToolArgument ]: A list of arguments that will be used to create the command line.
  • janis.CommandTool.outputs(self) -> List[janis.ToolOutput ]: A list of named outputs of the tool; a ToolOutput declares how the output is captured.
  • janis.CommandTool.container(self): A link to an OCI compliant container accessible by the engine. Previously, docker().
  • janis.CommandTool.version(self): Version of the tool.
  • janis.CommandTool.env_vars(self) -> Optional[Dict[str, Union[str, Selector]]]: A dictionary of environment variables that should be defined within the container.

To better categorise your tool, you can additionally implement the following methods:

  • janis.Tool.friendly_name(self): A user friendly name of your tool (must be implemented for generated docs)
  • janis.Tool.tool_module(self): Unix, bioinformatics, etc.
  • janis.Tool.tool_provider(self): The manafacturer of the tool, eg: Illumina, Samtools

Tool Input

class janis.ToolInput(tag: str, input_type: Union[Type[Union[str, float, int, bool]], janis_core.types.data_types.DataType, Type[janis_core.types.data_types.DataType]], position: Optional[int] = None, prefix: Optional[str] = None, separate_value_from_prefix: bool = None, prefix_applies_to_all_elements: bool = None, presents_as: str = None, secondaries_present_as: Dict[str, str] = None, separator: str = None, shell_quote: bool = None, localise_file: bool = None, default: Any = None, doc: Union[str, janis_core.tool.documentation.InputDocumentation, None] = None)[source]
__init__(tag: str, input_type: Union[Type[Union[str, float, int, bool]], janis_core.types.data_types.DataType, Type[janis_core.types.data_types.DataType]], position: Optional[int] = None, prefix: Optional[str] = None, separate_value_from_prefix: bool = None, prefix_applies_to_all_elements: bool = None, presents_as: str = None, secondaries_present_as: Dict[str, str] = None, separator: str = None, shell_quote: bool = None, localise_file: bool = None, default: Any = None, doc: Union[str, janis_core.tool.documentation.InputDocumentation, None] = None)[source]

A ToolInput represents an input to a tool, with parameters that allow it to be bound on the command line. The ToolInput must have either a position or prefix set to be bound onto the command line.

Parameters:
  • tag – The identifier of the input (unique to inputs and outputs of a tool)
  • input_type (janis.ParseableType) – The data type that this input accepts
  • position – The position of the input to be applied. (Default = 0, after the base_command).
  • prefix – The prefix to be appended before the element. (By default, a space will also be applied, see separate_value_from_prefix for more information)
  • separate_value_from_prefix – (Default: True) Add a space between the prefix and value when True.
  • prefix_applies_to_all_elements – Applies the prefix to each element of the array (Array inputs only)
  • shell_quote – Stops shell quotes from being applied in all circumstances, useful when joining multiple commands together.
  • separator – The separator between each element of an array (defaults to ‘ ‘)
  • localise_file – Ensures that the file(s) are localised into the execution directory.
  • default – The default value to be applied if the input is not defined.
  • doc – Documentation string for the ToolInput, this is used to generate the tool documentation and provide

hints to the user.

Note

A ToolInput (and ToolArgument) must have either a position AND / OR prefix to be bound onto the command line.

  • The prefix is a string that precedes the inputted value. By default the prefix is separated by a space, however this can be removed with separate_value_from_prefix=False.
  • The position represents the order of how arguments are bound onto the command line. Lower numbers get a higher priority, not providing a number will default to 0.
  • prefix_applies_to_all_elements applies the prefix to each element in an array (only applicable for array inputs).
  • The localise_file attribute places the file input within the execution directory.
  • presents_as is a mechanism for overriding the name to localise to. The localise_file parameter MUST be set to True for presents_as
  • secondaries_present_as is a mechanism for overriding the format of secondary files. localise_file does NOT need to be set for this functionality to work. In CWL, this relies on https://github.com/common-workflow-language/cwltool/pull/1233

Tool Argument

class janis.ToolArgument(value: Any, prefix: Optional[str] = None, position: Optional[int] = 0, separate_value_from_prefix=None, doc: Union[str, janis_core.tool.documentation.DocumentationMeta, None] = None, shell_quote: bool = None)[source]
__init__(value: Any, prefix: Optional[str] = None, position: Optional[int] = 0, separate_value_from_prefix=None, doc: Union[str, janis_core.tool.documentation.DocumentationMeta, None] = None, shell_quote: bool = None)[source]

A ToolArgument is a CLI parameter that cannot be override (at runtime). The value can

Parameters:
  • value (str | janis.InputSelector | janis.StringFormatter) –
  • position – The position of the input to be applied. (Default = 0, after the base_command).
  • prefix – The prefix to be appended before the element. (By default, a space will also be applied, see separate_value_from_prefix for more information)
  • separate_value_from_prefix – (Default: True) Add a space between the prefix and value when True.
  • doc – Documentation string for the argument, this is used to generate the tool documentation and provide
  • shell_quote – Stops shell quotes from being applied in all circumstances, useful when joining multiple commands together.

Tool Output

class janis.ToolOutput(tag: str, output_type: Union[Type[Union[str, float, int, bool]], janis_core.types.data_types.DataType, Type[janis_core.types.data_types.DataType]], selector: Union[janis_core.operators.selectors.Selector, str, None] = None, presents_as: str = None, secondaries_present_as: Dict[str, str] = None, doc: Union[str, janis_core.tool.documentation.OutputDocumentation, None] = None, glob: Union[janis_core.operators.selectors.Selector, str, None] = None, _skip_output_quality_check=False)[source]
__init__(tag: str, output_type: Union[Type[Union[str, float, int, bool]], janis_core.types.data_types.DataType, Type[janis_core.types.data_types.DataType]], selector: Union[janis_core.operators.selectors.Selector, str, None] = None, presents_as: str = None, secondaries_present_as: Dict[str, str] = None, doc: Union[str, janis_core.tool.documentation.OutputDocumentation, None] = None, glob: Union[janis_core.operators.selectors.Selector, str, None] = None, _skip_output_quality_check=False)[source]

A ToolOutput instructs the the engine how to collect an output and how it may be referenced in a workflow.

Parameters:
  • tag – The identifier of a output, must be unique in the inputs and outputs.
  • output_type – The type of output that is being collected.
  • selector – How to collect this output, can accept any janis.Selector.
  • glob – (DEPRECATED) An alias for selector
  • doc – Documentation on what the output is, used to generate docs.
  • _skip_output_quality_check – DO NOT USE THIS PARAMETER, it’s a scapegoat for parsing CWL ExpressionTools when an cwl.output.json is generated