fusionlab.nn.components.MultiScaleLSTM

class fusionlab.nn.components.MultiScaleLSTM

Bases: Layer, NNLearner

MultiScaleLSTM layer applying multiple LSTMs at different sampling scales and concatenating their outputs [1].

Each LSTM can either return the full sequence or only the last hidden state, controlled by return_sequences. The user specifies scales to sub-sample the time dimension. For example, a scale of 2 processes every 2nd time step.

Parameters:

• lstm_units (int) – Number of units in each LSTM.

• scales (list of int or str or None, optional) – List of scale factors. If ‘auto’ or None, defaults to [1] (no sub-sampling).

• return_sequences (bool, optional) – If True, each LSTM returns the entire sequence. Otherwise, it returns only the last hidden state. Defaults to False.

• **kwargs – Additional arguments passed to the parent Keras Layer.

Notes

• If return_sequences=False, the output is concatenated along features: \((B, \text{units} \times \text{num\_scales})\).

• If return_sequences=True, a list of sequence outputs is returned. Each may have a different time dimension if scales differ.

call(`inputs`, training=False)

Forward pass, applying each LSTM at the specified scale.

get_config()

Returns the layer’s configuration dict.

from_config(`config`)

Builds the layer from the config dict.

Examples

>>> from fusionlab.nn.components import MultiScaleLSTM
>>> import tensorflow as tf
>>> x = tf.random.normal((32, 20, 16))  # (B, T, D)
>>> # Instantiating a multi-scale LSTM
>>> mslstm = MultiScaleLSTM(lstm_units=32,
...     scales=[1, 2], return_sequences=False)
>>> y = mslstm(x)  # shape => (32, 64)
>>> # because scale=1 and scale=2 each produce 32 units,
... # which are concatenated => 64

See also

DynamicTimeWindow

For slicing sequences before applying multi-scale LSTMs.

TemporalFusionTransformer

A complex model that can incorporate multi-scale modules.

References

[1]

Lim, B., & Zohren, S. (2021). “Time-series forecasting with deep learning: a survey.” Philosophical Transactions of the Royal Society A, 379(2194), 20200209.

__init__(lstm_units, scales=None, return_sequences=False, **kwargs)

Parameters:

• lstm_units (int)

• scales (str | List[int] | None)

• return_sequences (bool)

Methods

__init__(lstm_units[, scales, return_sequences])
add_loss(loss)	Can be called inside of the call() method to add a scalar loss.
add_metric(*args, **kwargs)
add_variable(shape, initializer[, dtype, ...])	Add a weight variable to the layer.
add_weight([shape, initializer, dtype, ...])	Add a weight variable to the layer.
build(input_shape)
build_from_config(config)	Builds the layer's states with the supplied config dict.
call(inputs[, training])	Forward pass that processes the input at multiple scales.
compute_mask(inputs, previous_mask)
compute_output_shape(*args, **kwargs)
compute_output_spec(*args, **kwargs)
count_params()	Count the total number of scalars composing the weights.
from_config(config)	Builds MultiScaleLSTM from the given config dictionary.
get_build_config()	Returns a dictionary with the layer's input shape.
get_config()	Returns a config dictionary containing 'lstm_units', 'scales', and 'return_sequences'.
get_params([deep])	Get the parameters for this learner.
get_weights()	Return the values of layer.weights as a list of NumPy arrays.
help(**kwargs)
load(file_path[, format])	Load the learner's state from a specified file in the desired format.
load_own_variables(store)	Loads the state of the layer.
quantize(mode[, type_check, config])
quantized_build(input_shape, mode)
quantized_call(*args, **kwargs)
rematerialized_call(layer_call, *args, **kwargs)	Enable rematerialization dynamically for layer's call method.
save([file_path, format, overwrite, ...])	Save the learner's state to a specified file in the desired format.
save_own_variables(store)	Saves the state of the layer.
set_params(**params)	Set the parameters of this learner.
set_weights(weights)	Sets the values of layer.weights from a list of NumPy arrays.
stateless_call(trainable_variables, ...[, ...])	Call the layer without any side effects.
summary()	Provide a summary of the learner's parameters.
symbolic_call(*args, **kwargs)

Attributes

compute_dtype	The dtype of the computations performed by the layer.
dtype	Alias of layer.variable_dtype.
dtype_policy
input	Retrieves the input tensor(s) of a symbolic operation.
input_dtype	The dtype layer inputs should be converted to.
input_spec
losses	List of scalar losses from add_loss, regularizers and sublayers.
metrics	List of all metrics.
metrics_variables	List of all metric variables.
my_params
non_trainable_variables	List of all non-trainable layer state.
non_trainable_weights	List of all non-trainable weight variables of the layer.
output	Retrieves the output tensor(s) of a layer.
path	The path of the layer.
quantization_mode	The quantization mode of this layer, None if not quantized.
supports_masking	Whether this layer supports computing a mask using compute_mask.
trainable	Settable boolean, whether this layer should be trainable or not.
trainable_variables	List of all trainable layer state.
trainable_weights	List of all trainable weight variables of the layer.
variable_dtype	The dtype of the state (weights) of the layer.
variables	List of all layer state, including random seeds.
weights	List of all weight variables of the layer.

__init__(lstm_units, scales=None, return_sequences=False, **kwargs)

Parameters:

• lstm_units (int)

• scales (str | List[int] | None)

• return_sequences (bool)

call(inputs, training=False)

Forward pass that processes the input at multiple scales.

Parameters:

• inputs (tf.Tensor) – Shape (B, T, D).

• training (bool, optional) – Training mode. Defaults to False.

Returns:

• If return_sequences=False, returns a single 2D tensor of shape (B, lstm_units * len(scales)).

• If return_sequences=True, returns a list of 3D tensors, each with shape (B, T’, lstm_units), where T’ depends on the scale sub-sampling.

Return type:

tf.Tensor or list of tf.Tensor

get_config()

Returns a config dictionary containing ‘lstm_units’, ‘scales’, and ‘return_sequences’.

Returns:

Configuration dictionary.

Return type:

dict

classmethod from_config(config)

Builds MultiScaleLSTM from the given config dictionary.

Parameters:

config (dict) – Must include ‘lstm_units’, ‘scales’, ‘return_sequences’.

Returns:

A new instance of this layer.

Return type:

MultiScaleLSTM

help(**kwargs)

my_params = MultiScaleLSTM(lstm_units, scales=None, return_sequences=False)