fusionlab.metrics.prediction_stability_score

fusionlab.metrics.prediction_stability_score(y_pred, sample_weight=None, nan_policy='propagate', multioutput='uniform_average', verbose=0)[source]

Compute the Prediction Stability Score (PSS).

Measures the temporal smoothness of consecutive forecasts. Lower values indicate smoother, more coherent trajectories. Assumes predictions are ordered in time along the last axis.

Formally, for B samples, O outputs (optional), and horizon T: .. math:

\mathrm{PSS}_{i,o} = \frac{1}{T-1} \sum_{t=2}^{T}
|\hat y_{i,o,t} - \hat y_{i,o,t-1}|

The final score is an average over samples and possibly outputs.

Parameters:

  • y_pred (array-like) – Forecast trajectories. Expected shapes: - (n_samples, n_timesteps) for single output. - (n_samples, n_outputs, n_timesteps) for multi-output. The last dimension is always treated as the time dimension.

  • sample_weight (array-like of shape (n_samples,), optional) – Sample weights. If None, samples are equally weighted.

  • nan_policy ({'omit', 'propagate', 'raise'}, default 'propagate') –

    How to handle NaNs in y_pred:

    • 'raise': Raise an error on any NaN.

    • 'omit': Drop samples (rows) containing NaNs.

    • 'propagate': Score for samples/outputs with NaNs will be NaN.

  • multioutput ({'raw_values', 'uniform_average'}, default 'uniform_average') –

    Defines aggregation if y_pred has an n_outputs dimension.

    • 'raw_values': Returns a score for each output.

    • 'uniform_average': Scores of all outputs are averaged.

  • verbose (int, default 0) – Verbosity level: 0 (silent), 1 (summary), >=2 (debug details).

Returns:

score – Mean PSS. Scalar if multioutput=’uniform_average’ or if y_pred is 2D. Array if multioutput=’raw_values’ and y_pred is 3D.

Return type:

float or ndarray of floats

Examples

>>> import numpy as np
>>> # Single output (3 samples, 5 timesteps)
>>> y_p1 = np.array([[1,1,2,2,3], [2,3,2,3,2], [0,1,0,1,0]])
>>> prediction_stability_score(y_p1)
0.5833333333333334
>>> # Multi-output (2 samples, 2 outputs, 3 timesteps)
>>> y_p2 = np.array([[[1,2,1], [5,5,5]], [[3,2,3], [0,1,0]]])
>>> prediction_stability_score(y_p2, multioutput='raw_values')
array([1.  , 0.25])