fusionlab.nn.components.LearnedNormalization

class fusionlab.nn.components.LearnedNormalization

Bases: Layer, NNLearner

Learned Normalization layer that learns mean and standard deviation parameters for normalizing input features. This layer can be used to replace or augment standard data preprocessing steps by allowing the model to learn the optimal scaling dynamically.

Parameters:

None – This layer does not define additional initialization parameters besides standard Keras Layer.

Notes

This layer maintains two trainable weights: 1) mean: shape \((D,)\) 2) stddev: shape \((D,)\) where D is the last dimension of the input (feature dimension).

call(`inputs`, training=False)

Forward pass. Normalizes the input by subtracting the learned mean and dividing by the learned standard deviation plus a small epsilon.

get_config()

Returns the configuration dictionary for serialization.

from_config(`config`)

Instantiates the layer from a config dictionary.

Examples

>>> from fusionlab.nn.components import LearnedNormalization
>>> import tensorflow as tf
>>> # Create input of shape (batch_size, features)
>>> x = tf.random.normal((32, 10))
>>> # Instantiate the learned normalization layer
>>> norm_layer = LearnedNormalization()
>>> # Forward pass
>>> x_norm = norm_layer(x)

See also

MultiModalEmbedding

An embedding layer that can be used alongside learned normalization in a pipeline.

HierarchicalAttention

Another specialized layer for attention mechanisms.

__init__(**kws)

Methods

__init__(**kws)
add_loss(loss)	Can be called inside of the call() method to add a scalar loss.
add_metric(*args, **kwargs)
add_variable(shape, initializer[, dtype, ...])	Add a weight variable to the layer.
add_weight([shape, initializer, dtype, ...])	Add a weight variable to the layer.
build(input_shape)	Build method that creates trainable weights for mean and stddev according to the last dimension of the input.
build_from_config(config)	Builds the layer's states with the supplied config dict.
call(inputs[, training])	Forward pass of the LearnedNormalization layer.
compute_mask(inputs, previous_mask)
compute_output_shape(*args, **kwargs)
compute_output_spec(*args, **kwargs)
count_params()	Count the total number of scalars composing the weights.
from_config(config)	Instantiates the layer from a config dictionary.
get_build_config()	Returns a dictionary with the layer's input shape.
get_config()	Returns the configuration dictionary for this layer.
get_params([deep])	Get the parameters for this learner.
get_weights()	Return the values of layer.weights as a list of NumPy arrays.
help(**kwargs)
load(file_path[, format])	Load the learner's state from a specified file in the desired format.
load_own_variables(store)	Loads the state of the layer.
quantize(mode[, type_check, config])
quantized_build(input_shape, mode)
quantized_call(*args, **kwargs)
rematerialized_call(layer_call, *args, **kwargs)	Enable rematerialization dynamically for layer's call method.
save([file_path, format, overwrite, ...])	Save the learner's state to a specified file in the desired format.
save_own_variables(store)	Saves the state of the layer.
set_params(**params)	Set the parameters of this learner.
set_weights(weights)	Sets the values of layer.weights from a list of NumPy arrays.
stateless_call(trainable_variables, ...[, ...])	Call the layer without any side effects.
summary()	Provide a summary of the learner's parameters.
symbolic_call(*args, **kwargs)

Attributes

compute_dtype	The dtype of the computations performed by the layer.
dtype	Alias of layer.variable_dtype.
dtype_policy
input	Retrieves the input tensor(s) of a symbolic operation.
input_dtype	The dtype layer inputs should be converted to.
input_spec
losses	List of scalar losses from add_loss, regularizers and sublayers.
metrics	List of all metrics.
metrics_variables	List of all metric variables.
my_params
non_trainable_variables	List of all non-trainable layer state.
non_trainable_weights	List of all non-trainable weight variables of the layer.
output	Retrieves the output tensor(s) of a layer.
path	The path of the layer.
quantization_mode	The quantization mode of this layer, None if not quantized.
supports_masking	Whether this layer supports computing a mask using compute_mask.
trainable	Settable boolean, whether this layer should be trainable or not.
trainable_variables	List of all trainable layer state.
trainable_weights	List of all trainable weight variables of the layer.
variable_dtype	The dtype of the state (weights) of the layer.
variables	List of all layer state, including random seeds.
weights	List of all weight variables of the layer.

__init__(**kws)

build(input_shape)

Build method that creates trainable weights for mean and stddev according to the last dimension of the input.

Parameters:

input_shape (tuple) – Shape of the input, typically (batch_size, …, feature_dim).

call(inputs, training=False)

Forward pass of the LearnedNormalization layer.

Subtracts the learned mean from inputs and divides by stddev + 1e-6 to avoid division by zero.

Parameters:

• inputs (tf.Tensor) – Input tensor of shape \((B, ..., D)\).

• training (bool, optional) – Flag indicating if the layer is in training mode. Defaults to False.

Returns:

Normalized tensor of the same shape as inputs.

Return type:

tf.Tensor

get_config()

Returns the configuration dictionary for this layer.

Returns:

Configuration dictionary.

Return type:

dict

classmethod from_config(config)

Instantiates the layer from a config dictionary.

Parameters:

config (dict) – Configuration dictionary.

Returns:

A new instance of this layer.

Return type:

LearnedNormalization

help(**kwargs)

my_params = LearnedNormalization()