fusionlab.nn.components.LearnedNormalization

class fusionlab.nn.components.LearnedNormalization

Bases: Layer, NNLearner

Learned Normalization layer that learns mean and standard deviation parameters for normalizing input features. This layer can be used to replace or augment standard data preprocessing steps by allowing the model to learn the optimal scaling dynamically.

Parameters:

None – This layer does not define additional initialization parameters besides standard Keras Layer.

Notes

This layer maintains two trainable weights: 1) mean: shape \((D,)\) 2) stddev: shape \((D,)\) where D is the last dimension of the input (feature dimension).

call(`inputs`, training=False)

Forward pass. Normalizes the input by subtracting the learned mean and dividing by the learned standard deviation plus a small epsilon.

get_config()

Returns the configuration dictionary for serialization.

from_config(`config`)

Instantiates the layer from a config dictionary.

Examples

>>> from fusionlab.nn.components import LearnedNormalization
>>> import tensorflow as tf
>>> # Create input of shape (batch_size, features)
>>> x = tf.random.normal((32, 10))
>>> # Instantiate the learned normalization layer
>>> norm_layer = LearnedNormalization()
>>> # Forward pass
>>> x_norm = norm_layer(x)

See also

MultiModalEmbedding

An embedding layer that can be used alongside learned normalization in a pipeline.

HierarchicalAttention

Another specialized layer for attention mechanisms.

__init__(**kws)

Methods

__init__(**kws)
add_loss(losses, **kwargs)	Add loss tensor(s), potentially dependent on layer inputs.
add_metric(value[, name])	Adds metric tensor to the layer.
add_update(updates)	Add update op(s), potentially dependent on layer inputs.
add_variable(*args, **kwargs)	Deprecated, do NOT use! Alias for add_weight.
add_weight([name, shape, dtype, ...])	Adds a new variable to the layer.
build(input_shape)	Build method that creates trainable weights for mean and stddev according to the last dimension of the input.
build_from_config(config)	Builds the layer's states with the supplied config dict.
call(inputs[, training])	Forward pass of the LearnedNormalization layer.
compute_mask(inputs[, mask])	Computes an output mask tensor.
compute_output_shape(input_shape)	Computes the output shape of the layer.
compute_output_signature(input_signature)	Compute the output tensor signature of the layer based on the inputs.
count_params()	Count the total number of scalars composing the weights.
finalize_state()	Finalizes the layers state after updating layer weights.
from_config(config)	Instantiates the layer from a config dictionary.
get_build_config()	Returns a dictionary with the layer's input shape.
get_config()	Returns the configuration dictionary for this layer.
get_input_at(node_index)	Retrieves the input tensor(s) of a layer at a given node.
get_input_mask_at(node_index)	Retrieves the input mask tensor(s) of a layer at a given node.
get_input_shape_at(node_index)	Retrieves the input shape(s) of a layer at a given node.
get_output_at(node_index)	Retrieves the output tensor(s) of a layer at a given node.
get_output_mask_at(node_index)	Retrieves the output mask tensor(s) of a layer at a given node.
get_output_shape_at(node_index)	Retrieves the output shape(s) of a layer at a given node.
get_params([deep])	Get the parameters for this learner.
get_weights()	Returns the current weights of the layer, as NumPy arrays.
help(**kwargs)
load(file_path[, format])	Load the learner's state from a specified file in the desired format.
load_own_variables(store)	Loads the state of the layer.
save([file_path, format, overwrite, ...])	Save the learner's state to a specified file in the desired format.
save_own_variables(store)	Saves the state of the layer.
set_params(**params)	Set the parameters of this learner.
set_weights(weights)	Sets the weights of the layer, from NumPy arrays.
summary()	Provide a summary of the learner's parameters.
with_name_scope(method)	Decorator to automatically enter the module name scope.

Attributes

activity_regularizer	Optional regularizer function for the output of this layer.
compute_dtype	The dtype of the layer's computations.
dtype	The dtype of the layer weights.
dtype_policy	The dtype policy associated with this layer.
dynamic	Whether the layer is dynamic (eager-only); set in the constructor.
inbound_nodes	Return Functional API nodes upstream of this layer.
input	Retrieves the input tensor(s) of a layer.
input_mask	Retrieves the input mask tensor(s) of a layer.
input_shape	Retrieves the input shape(s) of a layer.
input_spec	InputSpec instance(s) describing the input format for this layer.
losses	List of losses added using the add_loss() API.
metrics	List of metrics attached to the layer.
my_params
name	Name of the layer (string), set in the constructor.
name_scope	Returns a tf.name_scope instance for this class.
non_trainable_variables	Sequence of non-trainable variables owned by this module and its submodules.
non_trainable_weights	List of all non-trainable weights tracked by this layer.
outbound_nodes	Return Functional API nodes downstream of this layer.
output	Retrieves the output tensor(s) of a layer.
output_mask	Retrieves the output mask tensor(s) of a layer.
output_shape	Retrieves the output shape(s) of a layer.
stateful
submodules	Sequence of all sub-modules.
supports_masking	Whether this layer supports computing a mask using compute_mask.
trainable
trainable_variables	Sequence of trainable variables owned by this module and its submodules.
trainable_weights	List of all trainable weights tracked by this layer.
updates
variable_dtype	Alias of Layer.dtype, the dtype of the weights.
variables	Returns the list of all layer variables/weights.
weights	Returns the list of all layer variables/weights.

__init__(**kws)

build(input_shape)

Build method that creates trainable weights for mean and stddev according to the last dimension of the input.

Parameters:

input_shape (tuple) – Shape of the input, typically (batch_size, …, feature_dim).

call(inputs, training=False)

Forward pass of the LearnedNormalization layer.

Subtracts the learned mean from inputs and divides by stddev + 1e-6 to avoid division by zero.

Parameters:

• inputs (tf.Tensor) – Input tensor of shape \((B, ..., D)\).

• training (bool, optional) – Flag indicating if the layer is in training mode. Defaults to False.

Returns:

Normalized tensor of the same shape as inputs.

Return type:

tf.Tensor

get_config()

Returns the configuration dictionary for this layer.

Returns:

Configuration dictionary.

Return type:

dict

classmethod from_config(config)

Instantiates the layer from a config dictionary.

Parameters:

config (dict) – Configuration dictionary.

Returns:

A new instance of this layer.

Return type:

LearnedNormalization

help(**kwargs)

my_params = LearnedNormalization()