pyiron_workflow.nodes.macro module

A base class for macro nodes, which are composite like workflows but have a static interface and are not intended to be internally modified after instantiation.

class pyiron_workflow.nodes.macro.Macro(*args, label: str | None = None, parent: Composite | None = None, delete_existing_savefiles: bool = False, autoload: BackendIdentifier | StorageInterface | None = None, autorun: bool = False, checkpoint: BackendIdentifier | StorageInterface | None = None, strict_naming: bool = True, **kwargs)[source]

Bases: Composite, StaticNode, ScrapesIO, ABC

A macro is a composite node that holds a graph with a fixed interface, like a pre-populated workflow that is the same every time you instantiate it.

At instantiation, the macro uses a provided callable to build and wire the graph, then builds a static IO interface for this graph. This callable must use the macro object itself as the first argument (e.g. adding nodes to it). The provided callable may optionally specify further args and kwargs; these are used to pre-populate the macro with UserInput nodes, although they may later be trimmed if the IO can be connected directly to child node IO without any loss of functionality. This can be especially helpful when more than one child node needs access to the same input value. Similarly, the callable may return any number of child nodes’ output channels (or the node itself in the case of single-output nodes) as long as a commensurate number of labels for these outputs were provided to the class constructor. These function-like definitions of the graph creator callable can be used to build only input xor output, or both together. Macro input channel labels are scraped from the signature of the graph creator; for output, output labels can be provided explicitly as a class attribute or, as a fallback, they are scraped from the graph creator code return statement (stripping off the “{first argument}.”, where {first argument} is whatever the name of the first argument is.

Macro IO is _value linked_ to the child IO, so that their values stay synchronized, but the child nodes of a macro form an isolated sub-graph.

As with function nodes, subclasses of Macro may define a method for creating the graph.

As with Workflow`, all DAG macros can determine their execution flow automatically, if you have cycles in your data flow, or otherwise want more control over the execution, all you need to do is specify the node.signals.input.run connections and starting_nodes list yourself. If only _one_ of these is specified, you’ll get an error, but if you’ve provided both then no further checks of their validity/reasonableness are performed, so be careful. Unlike Workflow, this execution flow automation is set up once at instantiation; If the macro is modified post-facto, you may need to manually re-invoke configure_graph_execution().

Examples

Let’s consider the simplest case of macros that just consecutively add 1 to their input:

>>> from pyiron_workflow import as_macro_node, macro_node
>>> def add_one(x):
...     result = x + 1
...     return result
>>>
>>> def add_three_macro(self, one__x):
...     self.one = self.create.function_node(add_one, x=one__x)
...     self.two = self.create.function_node(add_one, self.one)
...     self.three = self.create.function_node(add_one, self.two)
...     self.one >> self.two >> self.three
...     self.starting_nodes = [self.one]
...     return self.three

In this case we had _no need_ to specify the execution order and starting nodes –it’s just an extremely simple DAG after all! – but it’s done here to demonstrate the syntax.

We can make a macro by passing this graph-building function (that takes a macro as its first argument, i.e. self from the macro’s perspective) to the Macro class. Then, we can use it like a regular node! Just like a workflow, the io is constructed from unconnected owned-node IO by combining node and channel labels.

>>> macro = macro_node(add_three_macro, output_labels="three__result")
>>> out = macro(one__x=3)
>>> out.three__result
6

We can also nest macros, rename their IO, and provide access to internally-connected IO by inputs and outputs maps:

>>> def nested_macro(self, inp):
...     self.a = self.create.function_node(add_one, x=inp)
...     self.b = self.create.macro_node(
...         add_three_macro, one__x=self.a, output_labels="three__result"
...     )
...     self.c = self.create.function_node(add_one, x=self.b)
...     return self.c, self.b
>>>
>>> macro = macro_node(
...     nested_macro, output_labels=("out", "intermediate")
... )
>>> macro(inp=1)
{'out': 6, 'intermediate': 5}

Macros and workflows automatically generate execution flows when their data is acyclic. Let’s build a simple macro with two independent tracks:

>>> def modified_flow_macro(self, a__x=0, b__x=0):
...     self.a = self.create.function_node(add_one, x=a__x)
...     self.b = self.create.function_node(add_one, x=b__x)
...     self.c = self.create.function_node(add_one, x=self.b)
...     return self.a, self.c
>>>
>>> m = macro_node(modified_flow_macro, output_labels=("a", "c"))
>>> m(a__x=1, b__x=2)
{'a': 2, 'c': 4}

We can override which nodes get used to start by specifying the starting_nodes property and (if necessary) reconfiguring the execution signals. Care should be taken here, as macro nodes may be creating extra input nodes that need to be considered. It’s advisable to use draw() or to otherwise inspect the macro’s children and their connections before manually updating execution flows.

Let’s use this and then observe how the a sub-node no longer gets run:

>>> _ = m.disconnect_run()
>>> m.starting_nodes = [m.b]
>>> _ = m.b >> m.c
>>> m(a__x=1000, b__x=2000)
{'a': 2, 'c': 2002}

(The _ is just to catch and ignore output for the doctest, you don’t typically need this.)

Note how the a node is no longer getting run, so the output is not updated! Manually controlling execution flow is necessary for cyclic graphs (cf. the while loop meta-node), but best to avoid when possible as it’s easy to miss intended connections in complex graphs.

If there’s a particular macro we’re going to use again and again, we might want to consider making a new class for it using the decorator, just like we do for function nodes. If no output labels are explicitly provided as arguments to the decorator itself, these are scraped from the function return value, just like for function nodes (except the initial macro (or self or whatever the first argument is named) on any return values is ignored):

>>> from pyiron_workflow.api import Macro
>>> @Macro.wrap.as_macro_node
... def AddThreeMacro(self, x):
...     add_three_macro(self, one__x=x)
...     # We could also simply have decorated that function to begin with
...     return self.three
>>>
>>> macro = AddThreeMacro()
>>> macro(x=0).three
3

Alternatively (and not recommended) is to make a new child class of Macro that overrides the graph_creator() arg such that the same graph is always created.

>>> class AddThreeMacro(Macro):
...     _output_labels = ["three"]
...
...     def graph_creator(self, x):
...         add_three_macro(self, one__x=x)
...         return self.three
>>>
>>> macro = AddThreeMacro()
>>> macro(x=0).three
3

We can also modify an existing macro at runtime by replacing nodes within it, as long as the replacement has fully compatible IO. There are three syntacic ways to do this. Let’s explore these by going back to our add_three_macro and replacing each of its children with a node that adds 2 instead of 1.

It’s possible for the macro to hold nodes which are not publicly exposed for data and signal connections, but which will still internally execute and store data, e.g.:

>>> @Macro.wrap.as_macro_node("lout", "n_plus_2")
... def LikeAFunction(self, lin: list,  n: int = 1):
...     self.plus_two = n + 2
...     self.sliced_list = lin[n:self.plus_two]
...     self.double_fork = 2 * n
...     return self.sliced_list, self.plus_two.channel
>>>
>>> like_functions = LikeAFunction(lin=[1,2,3,4,5,6], n=3)
>>> sorted(like_functions().items())
[('lout', [4, 5]), ('n_plus_2', 5)]

>>> like_functions.double_fork.value
6

abstractmethod graph_creator(*args, **kwargs) → StaticNode | OutputData | tuple[StaticNode | OutputData, ...] | None[source]: Build the graph the node will run.

property inputs: Inputs

property outputs: OutputsWithInjection

Create and return a Macro node instance using the given node function.

Parameters:

graph_creator (callable) – Function to create the graph for the Macro.
node_args – Positional arguments for the Macro initialization – parsed as node input data.
output_labels (str | tuple[str, ...] | None) – Labels for the Macro’s outputs. Default is None, which scrapes these from the return statement in the decorated function’s source code.
validate_output_labels (bool) – Whether to validate the output labels. Defaults to True.
use_cache (bool) – Whether this node should default to caching its values. (Default is True.)
node_kwargs – Keyword arguments for the Macro initialization – parsed as node input data when the keyword matches an input channel.

Returns:

An instance of the Macro subclass.

Return type:

Macro