fusionlab.metrics.time_weighted_accuracy_score¶

fusionlab.metrics.time_weighted_accuracy_score(y_true, y_pred, time_weights='inverse_time', sample_weight=None, nan_policy='propagate', multioutput='uniform_average', eps=1e-08, verbose=0)[source]¶

Compute the Time-Weighted Accuracy (TWA) score.

This metric evaluates classification accuracy over sequences, applying weights to different time steps. It is suitable for scenarios where the importance of correct predictions varies across the time horizon. The last dimension of inputs is treated as the time dimension.

For a single sample \(i\) and output \(o\), the TWA is: .. math:

\mathrm{TWA}_{i,o} = \sum_{t=1}^{T} w_t \cdot
\mathbf{1}\{y_{i,o,t} = \hat y_{i,o,t}\},

where \(T\) is the number of time steps, \(w_t\) are the time weights (normalized to sum to 1), \(y_{i,o,t}\) is the true class label, \(\hat y_{i,o,t}\) is the predicted class label, and \(\mathbf{1}\{\cdot\}\) is the indicator function (1 if true, 0 if false). The final score is an average of these \(\mathrm{TWA}_{i,o}\) values over samples and potentially outputs.

Parameters:

y_true (array-like) – True class labels. Expected shapes: - (n_timesteps,) for a single sequence. - (n_samples, n_timesteps) for single output, multiple samples. - (n_samples, n_outputs, n_timesteps) for multi-output. Labels can be of any type that supports equality comparison.
y_pred (array-like) – Predicted class labels, matching y_true in shape and type.
time_weights (array-like of shape (n_timesteps,), str, or None, :class:``) –
default=’inverse_time’ Weights to apply to each time step’s accuracy. - If ‘inverse_time’, weights are \(w_t = 1/t\)

(1-indexed t), normalized to sum to 1.
- If an array-like is provided, it’s used directly and normalized to sum to 1. Its length must match n_timesteps.
- If None, uniform weights (1/T) are applied.
sample_weight (array-like of shape (n_samples,), optional) – Sample weights. If None, samples are equally weighted. Sum of weights must be > eps.
nan_policy ({'omit', 'propagate', 'raise'}, default 'propagate') –
How to handle NaNs in y_true or y_pred:
- 'raise': Raise an error on any NaN.
- 'omit': Drop samples (rows) containing NaNs.
- 'propagate': The TWA for samples/outputs with NaNs will be NaN.
multioutput ({'raw_values', 'uniform_average'}, default 'uniform_average') –
Defines aggregation if inputs are multi-output.
- 'raw_values': Returns a TWA score for each output.
- 'uniform_average': Scores of all outputs are averaged.
eps (float, default 1e-8) – Small epsilon value to prevent division by zero when sum of sample weights is very close to or is zero.
verbose (int, default 0) – Verbosity level: 0 (silent), 1 (summary), >=2 (debug details).

Returns:

score – Mean TWA score. Values range from 0 to 1, with higher values indicating better time-weighted accuracy. Scalar if multioutput=’uniform_average’ or if inputs represent a single output. Array of shape (n_outputs,) if multioutput=’raw_values’ and inputs are multi-output.

Return type:

float or ndarray of floats

Examples

>>> import numpy as np
>>> # from fusionlab.metrics import time_weighted_accuracy_score

>>> # Single output (2 samples, 3 timesteps)
>>> y_t = np.array([[1, 0, 1], [0, 1, 1]])
>>> y_p = np.array([[1, 1, 1], [0, 1, 0]])
>>> # Correctness: S0: [1,0,1], S1: [1,1,0]
>>> # Default time_weights (T=3): approx [0.545, 0.273, 0.182]
>>> # TWA_S0 = 1*0.545 + 0*0.273 + 1*0.182 = 0.727
>>> # TWA_S1 = 1*0.545 + 1*0.273 + 0*0.182 = 0.818
>>> # Avg TWA = (0.727 + 0.818) / 2 = 0.7725
>>> score = time_weighted_accuracy_score(y_t, y_p)
>>> print(f"TWA score (default weights): {score:.4f}")
TWA score (default weights): 0.7727

>>> # With NaN
>>> y_t_nan = np.array([[1, np.nan, 1], [0,1,1]])
>>> y_p_nan = np.array([[1, 1, 1], [0,1,0]])
>>> score_prop = time_weighted_accuracy_score(y_t_nan, y_p_nan, nan_policy='propagate')
>>> print(f"TWA score (propagate NaN): {score_prop:.4f}")
TWA score (propagate NaN): nan