fusionlab.nn.components.MultiScaleLSTM

class fusionlab.nn.components.MultiScaleLSTM

Bases: Layer, NNLearner

MultiScaleLSTM layer applying multiple LSTMs at different sampling scales and concatenating their outputs [1].

Each LSTM can either return the full sequence or only the last hidden state, controlled by return_sequences. The user specifies scales to sub-sample the time dimension. For example, a scale of 2 processes every 2nd time step.

Parameters:

• lstm_units (int) – Number of units in each LSTM.

• scales (list of int or str or None, optional) – List of scale factors. If ‘auto’ or None, defaults to [1] (no sub-sampling).

• return_sequences (bool, optional) – If True, each LSTM returns the entire sequence. Otherwise, it returns only the last hidden state. Defaults to False.

• **kwargs – Additional arguments passed to the parent Keras Layer.

Notes

• If return_sequences=False, the output is concatenated along features: \((B, \text{units} \times \text{num\_scales})\).

• If return_sequences=True, a list of sequence outputs is returned. Each may have a different time dimension if scales differ.

call(`inputs`, training=False)

Forward pass, applying each LSTM at the specified scale.

get_config()

Returns the layer’s configuration dict.

from_config(`config`)

Builds the layer from the config dict.

Examples

>>> from fusionlab.nn.components import MultiScaleLSTM
>>> import tensorflow as tf
>>> x = tf.random.normal((32, 20, 16))  # (B, T, D)
>>> # Instantiating a multi-scale LSTM
>>> mslstm = MultiScaleLSTM(lstm_units=32,
...     scales=[1, 2], return_sequences=False)
>>> y = mslstm(x)  # shape => (32, 64)
>>> # because scale=1 and scale=2 each produce 32 units,
... # which are concatenated => 64

See also

DynamicTimeWindow

For slicing sequences before applying multi-scale LSTMs.

TemporalFusionTransformer

A complex model that can incorporate multi-scale modules.

References

[1]

Lim, B., & Zohren, S. (2021). “Time-series forecasting with deep learning: a survey.” Philosophical Transactions of the Royal Society A, 379(2194), 20200209.

__init__(lstm_units, scales=None, return_sequences=False, **kwargs)

Parameters:

• lstm_units (int)

• scales (str | List[int] | None)

• return_sequences (bool)

Methods

__init__(lstm_units[, scales, return_sequences])
add_loss(losses, **kwargs)	Add loss tensor(s), potentially dependent on layer inputs.
add_metric(value[, name])	Adds metric tensor to the layer.
add_update(updates)	Add update op(s), potentially dependent on layer inputs.
add_variable(*args, **kwargs)	Deprecated, do NOT use! Alias for add_weight.
add_weight([name, shape, dtype, ...])	Adds a new variable to the layer.
build(input_shape)	Creates the variables of the layer (for subclass implementers).
build_from_config(config)	Builds the layer's states with the supplied config dict.
call(inputs[, training])	Forward pass that processes the input at multiple scales.
compute_mask(inputs[, mask])	Computes an output mask tensor.
compute_output_shape(input_shape)	Computes the output shape of the layer.
compute_output_signature(input_signature)	Compute the output tensor signature of the layer based on the inputs.
count_params()	Count the total number of scalars composing the weights.
finalize_state()	Finalizes the layers state after updating layer weights.
from_config(config)	Builds MultiScaleLSTM from the given config dictionary.
get_build_config()	Returns a dictionary with the layer's input shape.
get_config()	Returns a config dictionary containing 'lstm_units', 'scales', and 'return_sequences'.
get_input_at(node_index)	Retrieves the input tensor(s) of a layer at a given node.
get_input_mask_at(node_index)	Retrieves the input mask tensor(s) of a layer at a given node.
get_input_shape_at(node_index)	Retrieves the input shape(s) of a layer at a given node.
get_output_at(node_index)	Retrieves the output tensor(s) of a layer at a given node.
get_output_mask_at(node_index)	Retrieves the output mask tensor(s) of a layer at a given node.
get_output_shape_at(node_index)	Retrieves the output shape(s) of a layer at a given node.
get_params([deep])	Get the parameters for this learner.
get_weights()	Returns the current weights of the layer, as NumPy arrays.
help(**kwargs)
load(file_path[, format])	Load the learner's state from a specified file in the desired format.
load_own_variables(store)	Loads the state of the layer.
save([file_path, format, overwrite, ...])	Save the learner's state to a specified file in the desired format.
save_own_variables(store)	Saves the state of the layer.
set_params(**params)	Set the parameters of this learner.
set_weights(weights)	Sets the weights of the layer, from NumPy arrays.
summary()	Provide a summary of the learner's parameters.
with_name_scope(method)	Decorator to automatically enter the module name scope.

Attributes

activity_regularizer	Optional regularizer function for the output of this layer.
compute_dtype	The dtype of the layer's computations.
dtype	The dtype of the layer weights.
dtype_policy	The dtype policy associated with this layer.
dynamic	Whether the layer is dynamic (eager-only); set in the constructor.
inbound_nodes	Return Functional API nodes upstream of this layer.
input	Retrieves the input tensor(s) of a layer.
input_mask	Retrieves the input mask tensor(s) of a layer.
input_shape	Retrieves the input shape(s) of a layer.
input_spec	InputSpec instance(s) describing the input format for this layer.
losses	List of losses added using the add_loss() API.
metrics	List of metrics attached to the layer.
my_params
name	Name of the layer (string), set in the constructor.
name_scope	Returns a tf.name_scope instance for this class.
non_trainable_variables	Sequence of non-trainable variables owned by this module and its submodules.
non_trainable_weights	List of all non-trainable weights tracked by this layer.
outbound_nodes	Return Functional API nodes downstream of this layer.
output	Retrieves the output tensor(s) of a layer.
output_mask	Retrieves the output mask tensor(s) of a layer.
output_shape	Retrieves the output shape(s) of a layer.
stateful
submodules	Sequence of all sub-modules.
supports_masking	Whether this layer supports computing a mask using compute_mask.
trainable
trainable_variables	Sequence of trainable variables owned by this module and its submodules.
trainable_weights	List of all trainable weights tracked by this layer.
updates
variable_dtype	Alias of Layer.dtype, the dtype of the weights.
variables	Returns the list of all layer variables/weights.
weights	Returns the list of all layer variables/weights.

__init__(lstm_units, scales=None, return_sequences=False, **kwargs)

Parameters:

• lstm_units (int)

• scales (str | List[int] | None)

• return_sequences (bool)

call(inputs, training=False)

Forward pass that processes the input at multiple scales.

Parameters:

• inputs (tf.Tensor) – Shape (B, T, D).

• training (bool, optional) – Training mode. Defaults to False.

Returns:

• If return_sequences=False, returns a single 2D tensor of shape (B, lstm_units * len(scales)).

• If return_sequences=True, returns a list of 3D tensors, each with shape (B, T’, lstm_units), where T’ depends on the scale sub-sampling.

Return type:

tf.Tensor or list of tf.Tensor

get_config()

Returns a config dictionary containing ‘lstm_units’, ‘scales’, and ‘return_sequences’.

Returns:

Configuration dictionary.

Return type:

dict

classmethod from_config(config)

Builds MultiScaleLSTM from the given config dictionary.

Parameters:

config (dict) – Must include ‘lstm_units’, ‘scales’, ‘return_sequences’.

Returns:

A new instance of this layer.

Return type:

MultiScaleLSTM

help(**kwargs)

my_params = MultiScaleLSTM(lstm_units, scales=None, return_sequences=False)