fusionlab.nn.components.aggregate_multiscale_on_3d

fusionlab.nn.components.aggregate_multiscale_on_3d(lstm_output, mode='auto')[source]

Aggregate multi-scale LSTM outputs using a specified strategy.

This function combines outputs from MultiScaleLSTM. It is designed to either produce a single 3D sequence tensor (for attention mechanisms) or a single 2D context vector (by collapsing the time dimension).

Parameters:

  • lstm_output (list of tf.Tensor or tf.Tensor) –

    The output from MultiScaleLSTM. - If a list: Expected to be from an LSTM with

    return_sequences=True. Each element is a 3D tensor (B, T_scale, U) where T_scale can vary.

    • If a single tensor: Assumed to be from an LSTM with return_sequences=False, shape (B, U * num_scales). In this case, it’s returned as is.

  • mode ({'auto', 'sum', 'average', 'flatten', 'concat', 'last'}, optional) –

    Aggregation strategy: - ‘concat’: (For 3D output) Pads sequences to the max

    length and concatenates along the feature axis. This is the primary mode for creating a rich sequence representation for downstream attention layers. Result shape: (B, T_max, U*S).

    • ’last’ or ‘auto’: (For 2D output) Takes the last time step from each sequence in the list and concatenates them. Result shape: (B, U*S).

    • ’average’: (For 2D output) Averages each sequence over its time dimension and concatenates the results.

    • ’sum’: (For 2D output) Sums each sequence over its time dimension and concatenates the results.

    • ’flatten’: (For 2D output) Concatenates and flattens all dimensions except the batch. Requires sequences to have the same length.

Returns:

The aggregated feature tensor, either 2D or 3D depending on the mode.

Return type:

tf.Tensor