fusionlab.nn.anomaly_detection.SequenceAnomalyScoreLayer

class fusionlab.nn.anomaly_detection.SequenceAnomalyScoreLayer

Bases: Layer, NNLearner

Computes an anomaly score from input features using a Multi-Layer

Perceptron (MLP).

This layer processes input features, typically representing learned embeddings or aggregated sequence information from upstream layers, through a configurable MLP to produce a scalar anomaly score for each input sample.

It provides flexibility in defining the depth and width of the MLP, activation functions, normalization, and dropout for regularization. The output score reflects the model’s learned assessment of how anomalous the input features are.

Parameters:

• hidden_units (int or list of int) –

  Specifies the structure of the hidden layers in the MLP. * If int: A single hidden layer with that many units is used. * If list[int]: Creates multiple hidden layers, where each

  integer in the list defines the number of units for the corresponding layer.

• activation (str, default 'relu') – Activation function applied after each hidden dense layer (but before normalization or dropout). Common choices include ‘relu’, ‘elu’, ‘gelu’, ‘tanh’.

• dropout_rate (float, default 0.1) – Dropout rate applied after activation (and normalization, if used) in each hidden layer. Value between 0 and 1.

• use_norm (bool or str, default False) –

  Specifies whether to apply normalization after the activation in hidden layers. * False: No normalization. * True or ‘layer’: Use Layer Normalization. * ‘batch’: Use Batch Normalization. Note that Batch Normalization

  behaves differently during training and inference.

• final_activation (str, default 'linear') –

  Activation function applied to the final output neuron that produces the scalar anomaly score. * ‘linear’: Produces an unbounded score. * ‘sigmoid’: Produces a score between 0 and 1, interpretable

  as a probability or normalized score.

  • Other activations like ‘softplus’ can also be used to ensure non-negative scores.

• **kwargs – Additional keyword arguments passed to the parent Keras Layer.

Notes

This layer typically expects input features with shape (Batch, Features). If your input is sequential (Batch, TimeSteps, Features), you might need to flatten or pool it before feeding it to this layer.

Use Case and Importance

This layer is designed to be a part of a larger model, acting as a dedicated “scoring head” that learns to map complex internal features to an anomaly score. It’s useful when you want the model to learn what constitutes an anomaly based on learned representations, rather than relying solely on reconstruction error or predefined rules. This approach aligns well with the concept of feature-based anomaly detection within models like XTFT. Training this layer effectively requires integrating it into a larger network and defining a suitable loss function that utilizes its output score, potentially combining it with the primary task’s loss (e.g., forecasting loss) or using anomaly labels if available (supervised training).

Mathematical Formulation

The layer implements a standard Multi-Layer Perceptron (MLP). For an input feature vector \(\mathbf{h}\) and $L$ hidden layers:

Let \(\mathbf{h}^{(0)} = \mathbf{h}\). For each hidden layer $i = 1 dots L$:

\[\begin{split}\mathbf{a}^{(i)} = \text{Dense}_i(\mathbf{h}^{(i-1)}) \\ \mathbf{n}^{(i)} = \text{Activation}(\mathbf{a}^{(i)}) \\ \mathbf{o}^{(i)} = \text{Normalization}(\mathbf{n}^{(i)}) \quad (\text{if use_norm=True}) \\ \mathbf{h}^{(i)} = \text{Dropout}(\mathbf{o}^{(i)} \text{ or } \mathbf{n}^{(i)})\end{split}\]

The final score is computed from the last hidden layer’s output \(\mathbf{h}^{(L)}\):

\[\text{Score} = \text{FinalActivation}(\text{Dense}_{out}(\mathbf{h}^{(L)}))\]

where Dense includes weights, biases, and the specified activation or normalization steps.

call(inputs, training=False)

Performs the forward pass to compute anomaly scores.

Examples

>>> from fusionlab.nn.anomaly_detection import SequenceAnomalyScoreLayer
>>> import tensorflow as tf
>>> B, F = 32, 64 # Batch, Features
>>> # Assume 'features' are output from another layer
>>> features = tf.random.normal((B, F))
>>> # Instantiate with multiple hidden layers and LayerNorm
>>> anomaly_scorer = SequenceAnomalyScoreLayer(
...     hidden_units=[64, 32], # Two hidden layers
...     activation='relu',
...     dropout_rate=0.2,
...     use_norm='layer', # Use Layer Normalization
...     final_activation='sigmoid' # Output score between 0 and 1
... )
>>> # Get anomaly scores
>>> scores = anomaly_scorer(features, training=True) # Pass training flag
>>> scores.shape
TensorShape([32, 1])

See also

tensorflow.keras.layers.Layer

Base class for Keras layers.

tensorflow.keras.layers.Dense

Fully-connected layer used internally.

tensorflow.keras.layers.Dropout

Dropout regularization layer.

tensorflow.keras.layers.LayerNormalization

Normalization layer option.

tensorflow.keras.layers.BatchNormalization

Normalization layer option.

fusionlab.nn.transformers.XTFT

Can incorporate feature-based anomaly detection potentially using layers like this.

fusionlab.nn.losses.AnomalyLoss

Loss component for anomaly scores.

LSTMAutoencoderAnomaly

Alternative reconstruction-based component.

References

[1]

Chalapathy, R., & Chawla, S. (2019). Deep learning for anomaly detection: A survey. arXiv preprint arXiv:1901.03407.

__init__(hidden_units, activation='relu', dropout_rate=0.1, use_norm=False, final_activation='linear', kernel_regularizer=None, bias_regularizer=None, **kwargs)

Parameters:

• hidden_units (int | List[int])

• activation (str)

• dropout_rate (float)

• use_norm (bool | str)

• final_activation (str)

Methods

__init__(hidden_units[, activation, ...])
add_loss(loss)	Can be called inside of the call() method to add a scalar loss.
add_metric(*args, **kwargs)
add_variable(shape, initializer[, dtype, ...])	Add a weight variable to the layer.
add_weight([shape, initializer, dtype, ...])	Add a weight variable to the layer.
build(input_shape)
build_from_config(config)	Builds the layer's states with the supplied config dict.
call(inputs[, training])	Forward pass: (Dense -> [Norm] -> Dropout) * N -> Dense Output.
compute_mask(inputs, previous_mask)
compute_output_shape(*args, **kwargs)
compute_output_spec(*args, **kwargs)
count_params()	Count the total number of scalars composing the weights.
from_config(config)	Creates layer from its config.
get_build_config()	Returns a dictionary with the layer's input shape.
get_config()	Returns the layer configuration.
get_params([deep])	Get the parameters for this learner.
get_weights()	Return the values of layer.weights as a list of NumPy arrays.
help(**kwargs)
load(file_path[, format])	Load the learner's state from a specified file in the desired format.
load_own_variables(store)	Loads the state of the layer.
quantize(mode[, type_check, config])
quantized_build(input_shape, mode)
quantized_call(*args, **kwargs)
rematerialized_call(layer_call, *args, **kwargs)	Enable rematerialization dynamically for layer's call method.
save([file_path, format, overwrite, ...])	Save the learner's state to a specified file in the desired format.
save_own_variables(store)	Saves the state of the layer.
set_params(**params)	Set the parameters of this learner.
set_weights(weights)	Sets the values of layer.weights from a list of NumPy arrays.
stateless_call(trainable_variables, ...[, ...])	Call the layer without any side effects.
summary()	Provide a summary of the learner's parameters.
symbolic_call(*args, **kwargs)

Attributes

compute_dtype	The dtype of the computations performed by the layer.
dtype	Alias of layer.variable_dtype.
dtype_policy
input	Retrieves the input tensor(s) of a symbolic operation.
input_dtype	The dtype layer inputs should be converted to.
input_spec
losses	List of scalar losses from add_loss, regularizers and sublayers.
metrics	List of all metrics.
metrics_variables	List of all metric variables.
my_params
non_trainable_variables	List of all non-trainable layer state.
non_trainable_weights	List of all non-trainable weight variables of the layer.
output	Retrieves the output tensor(s) of a layer.
path	The path of the layer.
quantization_mode	The quantization mode of this layer, None if not quantized.
supports_masking	Whether this layer supports computing a mask using compute_mask.
trainable	Settable boolean, whether this layer should be trainable or not.
trainable_variables	List of all trainable layer state.
trainable_weights	List of all trainable weight variables of the layer.
variable_dtype	The dtype of the state (weights) of the layer.
variables	List of all layer state, including random seeds.
weights	List of all weight variables of the layer.

__init__(hidden_units, activation='relu', dropout_rate=0.1, use_norm=False, final_activation='linear', kernel_regularizer=None, bias_regularizer=None, **kwargs)

Parameters:

• hidden_units (int | List[int])

• activation (str)

• dropout_rate (float)

• use_norm (bool | str)

• final_activation (str)

call(inputs, training=False)

Forward pass: (Dense -> [Norm] -> Dropout) * N -> Dense Output. Expects inputs of shape (Batch, Features).

get_config()

Returns the layer configuration.

classmethod from_config(config)

Creates layer from its config.

help(**kwargs)

my_params = SequenceAnomalyScoreLayer(     hidden_units,     activation='relu',     dropout_rate=0.1,     use_norm=False,     final_activation='linear',     kernel_regularizer=None,     bias_regularizer=None )