fusionlab.nn.transformers.TemporalFusionTransformer

class fusionlab.nn.transformers.TemporalFusionTransformer

Bases: Model, NNLearner

TemporalFusionTransformer model implementation for multi-horizon forecasting, with optional static, past, and future inputs.

This class extends Keras Model and integrates with the gofast NNLearner interface. It supports dynamic (past) inputs, optional static inputs, and newly added optional future inputs (future_input_dim). By including the future covariates, the TemporalFusionTransformer can account for known future features (e.g., events, planned discount rates, etc.) in its predictions.

Parameters:

• dynamic_input_dim (int) – Dimensionality of the dynamic (past) inputs. This is mandatory for the TFT model.

• static_input_dim (int, optional) – Dimensionality of static inputs. If not None, the call method will expect static inputs.

• future_input_dim (int, optional) – Dimensionality of future (known) inputs. If not None, the call method will expect future inputs to handle exogenous covariates known in the future (e.g., events, planned promotions, etc.).

• hidden_units (int, default 32) – Number of hidden units for the layers that do not have a distinct specification (e.g., GRNs, variable selection networks).

• num_heads (int, default 4) – Number of attention heads in the multi-head attention layer.

• dropout_rate (float, default 0.1) – Dropout rate for various layers (GRNs, attention, etc.).

• forecast_horizon (int, default 1) – Number of timesteps to forecast into the future.

• quantiles (list of float, optional) – List of quantiles for probabilistic forecasting. If None, a single deterministic output is produced.

• activation (str, default 'elu') – Activation function. Must be one of {'elu', 'relu', 'tanh', 'sigmoid', 'linear', 'gelu'}.

• use_batch_norm (bool, default False) – Whether to apply batch normalization in various sub-layers.

• num_lstm_layers (int, default 1) – Number of LSTM layers in the encoder.

• lstm_units (list of int or None, default None) – If provided, each index corresponds to the number of LSTM units for that layer. If None, uses hidden_units for each layer.

Examples

>>> from fusionlab.nn._tensor_validation import validate_tft_inputs
>>> from fusionlab.nn.tft import TemporalFusionTransformer
>>> model = TemporalFusionTransformer(
...     dynamic_input_dim=10,
...     static_input_dim=5,
...     future_input_dim=8,
...     hidden_units=32,
...     num_heads=4,
...     dropout_rate=0.1,
...     forecast_horizon=7,
...     quantiles=[0.1, 0.5, 0.9],
...     activation='elu',
...     use_batch_norm=True,
...     num_lstm_layers=2,
...     lstm_units=[64, 32]
... )

Notes

The newly added future_input_dim allows the model to incorporate future covariates known at forecast time. In the call method, if future_input_dim is not None, the model expects three inputs: (static_inputs, dynamic_inputs, future_inputs). Otherwise, it expects only (static_inputs, dynamic_inputs).

See also

VariableSelectionNetwork

For feature selection and embedding.

GatedResidualNetwork

A GRN used in various sub-layers.

LSTM

Keras LSTM layers for sequence processing.

References

[1]

Lim, B., Arık, S. Ö., Loeff, N., & Pfister, T. (2019). Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting. https://arxiv.org/abs/1912.09363

__init__(dynamic_input_dim, static_input_dim=None, future_input_dim=None, hidden_units=32, num_heads=4, dropout_rate=0.1, forecast_horizon=1, quantiles=None, activation='elu', use_batch_norm=False, num_lstm_layers=1, lstm_units=None, output_dim=1, **kw)

Methods

__init__(dynamic_input_dim[, ...])
add_loss(loss)	Can be called inside of the call() method to add a scalar loss.
add_metric(*args, **kwargs)
add_variable(shape, initializer[, dtype, ...])	Add a weight variable to the layer.
add_weight([shape, initializer, dtype, ...])	Add a weight variable to the layer.
build(input_shape)
build_from_config(config)	Builds the layer's states with the supplied config dict.
call(inputs[, training])	The main forward pass for NTemporalFusionTransformer.
compile([optimizer, loss, loss_weights, ...])	Configures the model for training.
compile_from_config(config)	Compiles the model with the information given in config.
compiled_loss(y, y_pred[, sample_weight, ...])
compute_loss([x, y, y_pred, sample_weight, ...])	Compute the total loss, validate it, and return it.
compute_mask(inputs, previous_mask)
compute_metrics(x, y, y_pred[, sample_weight])	Update metric states and collect all metrics to be returned.
compute_output_shape(*args, **kwargs)
compute_output_spec(*args, **kwargs)
count_params()	Count the total number of scalars composing the weights.
evaluate([x, y, batch_size, verbose, ...])	Returns the loss value & metrics values for the model in test mode.
export(filepath[, format, verbose, ...])	Export the model as an artifact for inference.
fit([x, y, batch_size, epochs, verbose, ...])	Trains the model for a fixed number of epochs (dataset iterations).
from_config(config)	Recreate NTemporalFusionTransformer instance from config.
get_build_config()	Returns a dictionary with the layer's input shape.
get_compile_config()	Returns a serialized config with information for compiling the model.
get_config()	Return the model configuration for serialization.
get_layer([name, index])	Retrieves a layer based on either its name (unique) or index.
get_metrics_result()	Returns the model's metrics values as a dict.
get_params([deep])	Get the parameters for this learner.
get_state_tree([value_format])	Retrieves tree-like structure of model variables.
get_weights()	Return the values of layer.weights as a list of NumPy arrays.
help(**kwargs)
load(file_path[, format])	Load the learner's state from a specified file in the desired format.
load_own_variables(store)	Loads the state of the layer.
load_weights(filepath[, skip_mismatch])	Load the weights from a single file or sharded files.
loss(y, y_pred[, sample_weight])
make_predict_function([force])
make_test_function([force])
make_train_function([force])
predict(x[, batch_size, verbose, steps, ...])	Generates output predictions for the input samples.
predict_on_batch(x)	Returns predictions for a single batch of samples.
predict_step(data)
quantize(mode[, config])	Quantize the weights of the model.
quantized_build(input_shape, mode)
quantized_call(*args, **kwargs)
rematerialized_call(layer_call, *args, **kwargs)	Enable rematerialization dynamically for layer's call method.
reset_metrics()
save(filepath[, overwrite, zipped])	Saves a model as a .keras file.
save_own_variables(store)	Saves the state of the layer.
save_weights(filepath[, overwrite, ...])	Saves all weights to a single file or sharded files.
set_params(**params)	Set the parameters of this learner.
set_state_tree(state_tree)	Assigns values to variables of the model.
set_weights(weights)	Sets the values of layer.weights from a list of NumPy arrays.
stateless_call(trainable_variables, ...[, ...])	Call the layer without any side effects.
stateless_compute_loss(trainable_variables, ...)
summary([line_length, positions, print_fn, ...])	Prints a string summary of the network.
symbolic_call(*args, **kwargs)
test_on_batch(x[, y, sample_weight, return_dict])	Test the model on a single batch of samples.
test_step(data)
to_json(**kwargs)	Returns a JSON string containing the network configuration.
train_on_batch(x[, y, sample_weight, ...])	Runs a single gradient update on a single batch of data.
train_step(data)

Attributes

compiled_metrics
compute_dtype	The dtype of the computations performed by the layer.
distribute_reduction_method
distribute_strategy
dtype	Alias of layer.variable_dtype.
dtype_policy
input	Retrieves the input tensor(s) of a symbolic operation.
input_dtype	The dtype layer inputs should be converted to.
input_spec
jit_compile
layers
losses	List of scalar losses from add_loss, regularizers and sublayers.
metrics	List of all metrics.
metrics_names
metrics_variables	List of all metric variables.
my_params
non_trainable_variables	List of all non-trainable layer state.
non_trainable_weights	List of all non-trainable weight variables of the layer.
output	Retrieves the output tensor(s) of a layer.
path	The path of the layer.
quantization_mode	The quantization mode of this layer, None if not quantized.
run_eagerly
supports_masking	Whether this layer supports computing a mask using compute_mask.
trainable	Settable boolean, whether this layer should be trainable or not.
trainable_variables	List of all trainable layer state.
trainable_weights	List of all trainable weight variables of the layer.
variable_dtype	The dtype of the state (weights) of the layer.
variables	List of all layer state, including random seeds.
weights	List of all weight variables of the layer.

__init__(dynamic_input_dim, static_input_dim=None, future_input_dim=None, hidden_units=32, num_heads=4, dropout_rate=0.1, forecast_horizon=1, quantiles=None, activation='elu', use_batch_norm=False, num_lstm_layers=1, lstm_units=None, output_dim=1, **kw)

help(**kwargs)

my_params = TemporalFusionTransformer(     dynamic_input_dim,     static_input_dim=None,     future_input_dim=None,     hidden_units=32,     num_heads=4,     dropout_rate=0.1,     forecast_horizon=1,     quantiles=None,     activation='elu',     use_batch_norm=False,     num_lstm_layers=1,     lstm_units=None,     output_dim=1 )

call(inputs, training=False, **kw)

The main forward pass for NTemporalFusionTransformer.

1. Validate and unpack inputs using validate_tft_inputs.

2. Apply variable selection to static, dynamic, and future inputs.

3. Perform positional encoding on dynamic+future sequences.

4. Compute static context vectors if static is present.

5. Pass through LSTM encoders.

6. Optionally enrich dynamic with static context.

7. Temporal attention for interpretable weighting of time steps.

8. Position-wise feedforward (GRN).

9. Final slicing (forecast horizon) and output (quantiles or single).

Parameters:

• inputs (tuple) – Should contain up to three elements: (dynamic_inputs, future_inputs, static_inputs) or fewer if not all are provided.

• training (bool, default False) – Whether in training mode (affects dropout, BN, etc.).

Returns:

Final predicted sequences of shape (batch_size, forecast_horizon, num_quantiles or 1).

Return type:

tf.Tensor

get_config()

Return the model configuration for serialization. Includes all hyperparameters that define the structure of the NTemporalFusionTransformer.

classmethod from_config(config)

Recreate NTemporalFusionTransformer instance from config. This classmethod is invoked by Keras to deserialize the model.