Version 0.3.0

(Release Date: June 17, 2025)

Focus: Advanced PINNs and Flexible Attentive Architectures

This is a release that significantly refactors the core attentive model architecture and introduces a powerful, flexible new generation of Physics-Informed Neural Networks (PINNs). The focus has been on modularity, robustness, and providing users with greater control over the model architecture to tackle a wider range of complex forecasting problems. This new foundation enables a state-of-the-art generation of PINNs and is complemented by a unified hyperparameter tuner, HydroTuner, designed for all hydrogeological models.

New Features

• New Introduced BaseAttentive, a powerful and flexible base class that encapsulates the entire encoder-decoder and attention logic. This provides a consistent, modular foundation for all advanced forecasting models in the library.

• New The base architecture includes a mode parameter ('tft_like' or 'pihal_like') to control how known future features are handled, allowing for both TFT-style input enrichment and standard encoder-decoder data flows.

• New TransFlowSubsNet: A state-of-the-art, fully-coupled PINN that simultaneously models both transient groundwater flow and aquifer-system consolidation. This is the new flagship model for complex hydrogeological forecasting.

• New HALNet: The powerful, data-driven Hybrid Attentive LSTM Network is now available as a standalone model, serving as a robust, general-purpose forecasting tool without physics components.

• New PiTGWFlow: A self-contained, pure-physics PINN for solving the 2D transient groundwater flow equation, ideal for forward and inverse problems.

• New Introduced HydroTuner: A flexible, model-agnostic tuner for all hydrogeological PINNs. By simply passing the model class (PIHALNet or TransFlowSubsNet), it dynamically adapts its search space.

• New Added HALTuner, a dedicated tuner for the new standalone HALNet model.

• Code Example (New Flexible HALNet):

  import numpy as np
  import tensorflow as tf
  from fusionlab.nn.models import HALNet
  
  # 1. Define model config for "tft_like" mode
  TIME_STEPS, HORIZON = 8, 4
  halnet_tft = HALNet(
      mode='tft_like',
      static_input_dim=3, dynamic_input_dim=5, future_input_dim=2,
      output_dim=1, forecast_horizon=HORIZON, max_window_size=TIME_STEPS
  )
  
  # Data must span both lookback and forecast periods
  future_span_tft = TIME_STEPS + HORIZON
  inputs_tft = [
      tf.random.normal((2, 3)), # Static
      tf.random.normal((2, TIME_STEPS, 5)), # Dynamic
      tf.random.normal((2, future_span_tft, 2)) # Future
  ]
  output_tft = halnet_tft(inputs_tft)
  print(f"TFT-like mode output shape: {output_tft.shape}")
  
  # 2. Define model config for "pihal_like" mode
  halnet_pihal = HALNet(
      mode='pihal_like',
      static_input_dim=3, dynamic_input_dim=5, future_input_dim=2,
      output_dim=1, forecast_horizon=HORIZON, max_window_size=TIME_STEPS
  )
  # Data only needs to cover the forecast horizon for future features
  future_span_pihal = HORIZON
  inputs_pihal = [
      tf.random.normal((2, 3)), # Static
      tf.random.normal((2, TIME_STEPS, 5)), # Dynamic
      tf.random.normal((2, future_span_pihal, 2)) # Future
  ]
  output_pihal = halnet_pihal(inputs_pihal)
  print(f"PIHAL-like mode output shape: {output_pihal.shape}")
  

• New A suite of PINN-specific utilities have been added, including the critical prepare_pinn_data_sequences() function for handling complex input requirements.

• New New spatial utilities in spatial_utils, including create_spatial_clusters and batch_spatial_sampling, have been introduced to aid in geospatial feature engineering.

• New Feature Engineering Utility: Added a new utility function, create_time_features(), which can generate a rich set of features (e.g., day of week, month, week of year) directly from a datetime column.

• New Tuning Summary Visualization: The HydroTuner and other tuners now automatically generate and save a plot of the tuning trials vs. objective scores upon completion, providing immediate visual insight into the search process.

Improvements

• Enhancement The legacy PIHALNet has also been re-architected to inherit from BaseAttentive, benefiting from the new configuration system and modularity.

• Enhancement Visualization functions like plot_forecast_by_step() and forecast_view() have been added and improved for more insightful analysis of forecast results.

• Enhancement This new base class is highly configurable via an architecture_config dictionary, supporting:

  • Two distinct encoder architectures: 'hybrid' (default), which uses MultiScaleLSTM, and 'transformer', which uses a pure self-attention stack.

  • A fully modular decoder_attention_stack, giving users fine-grained control over the attention mechanisms.

• Enhancement The new tuners’ .create() factory method can automatically infer data-dependent parameters (like input/output dimensions) directly from NumPy arrays, significantly simplifying the setup process.

• Code Example (New HydroTuner Workflow):

  from fusionlab.nn.forecast_tuner import HydroTuner
  from fusionlab.nn.pinn.models import TransFlowSubsNet
  # Assume 'inputs' and 'targets' are pre-prepared NumPy dicts
  
  # 1. Define a search space for the model
  search_space = {
      "embed_dim": [32, 64],
      "dropout_rate": {"type": "float", "min_value": 0.1, "max_value": 0.3},
      "K": ["learnable", 1e-4],  # Physics HP for TransFlowSubsNet
      "learning_rate": [1e-3, 5e-4]
  }
  
  # 2. Instantiate the Tuner using the .create() factory method
  tuner = HydroTuner.create(
      model_name_or_cls=TransFlowSubsNet, # Specify the model to tune
      inputs_data=inputs,
      targets_data=targets,
      search_space=search_space,
      max_trials=10,
      project_name="New_TransFlow_Tuning"
  )
  
  # 3. Run the search
  print("Starting tuning with the new HydroTuner...")
  best_model, best_hps, _ = tuner.run(
      inputs=inputs,
      y=targets,
      epochs=20,
      callbacks=[tf.keras.callbacks.EarlyStopping('val_loss', patience=5)]
  )
  

• Enhancement Tuner Usability: The HydroTuner and other class-based tuners now have a refit_best_model argument in their .run() method. Setting this to False allows users to quickly find the best hyperparameters without waiting for the final, potentially time-consuming model retraining step. Ideal for rapid experimentation.

• Enhancement PINN Model Control: The TransFlowSubsNet and PIHALNet models now accept a correction_mlp_config parameter. This allows users to customize the architecture (number of layers and units) of the coordinate-based correction MLPs, offering finer control over the physics-informed component.

• Enhancement Performance: The prepare_pinn_data_sequences() utility has been optimized for performance, resulting in a significant speed-up (up to 30%) when generating sequences from very large DataFrames.

API Changes

• API Change The `BaseAttentive` class is now the recommended base for all hybrid and PINN models. Its __init__ signature introduces the `architecture_config` dictionary as the new standard for controlling model structure.

• API Change The legacy PiHALTuner is now shadowed by the flexible `HydroTuner`, which requires a model_name_or_cls argument to specify the model to tune.

• API Change The primary argument for defining the search space in new tuners is now `search_space`. The old param_space argument is maintained for backward compatibility but raises a FutureWarning.

• API Change The refactored PIHALNet now takes a single physics loss weight, `lambda_pde`, in its .compile() method.

• API Change The MultiObjectiveLoss component was refactored to accept anomaly_scores during __init__, making its call() signature standard and compatible with the default model.fit().

Fixes

• Fix Architectural Overhaul: Completely refactored the internal logic of PIHALNet (now BaseAttentive) to use a robust encoder-decoder architecture. This permanently fixes a series of ValueError and InvalidArgumentError exceptions related to shape mismatches that occurred when time_steps and forecast_horizon were different.

• Fix Residual Connections: Corrected the logic for residual connections (Add + LayerNormalization) to handle the use_residuals=False case correctly, preventing TypeError exceptions. All feature dimensions within the attention blocks are now consistent, resolving shape mismatches.

• Fix Positional Encoding: Replaced the naive linear positional encoding with the standard, robust sinusoidal implementation from “Attention Is All You Need”. Fixed an issue where a single instance was incorrectly used on tensors with different feature dimensions.

• Fix PINN Gradient Calculation: Refactored PiTGWFlow to decouple prediction from residual calculation. The train_step now manages a single GradientTape context, fixing a ValueError where gradients could not be computed due to a broken computational path.

• Fix Corrected the gradient flow in PINN models to improve stability, especially when training with very low physics-loss weights.

• Fix Enhanced the prepare_pinn_data_sequences() utility to better handle edge cases with single-group time series.

• Fix The search method in PINNTunerBase now correctly handles TensorFlow Datasets where the target y is already a tensor, improving compatibility with non-PINN models.

• Fix PINN Gradient Stability: Fixed a critical bug where setting a physics loss weight (e.g., lambda_cons) to exactly 0.0 could lead to None gradients during backpropagation in tf.function mode. The training step now correctly handles zero-weighted losses.

• Fix Data Preparation: Corrected an issue in prepare_pinn_data_sequences() where datasets containing only a single time series group could cause shape errors during processing.

• Fix Hyperparameter Tuner: Resolved a bug in HydroTuner.create where explicitly passing quantiles=None in fixed_params was not being correctly recognized, causing it to fall back to defaults.

• Fix Plotting Utilities: The forecast_view function no longer raises an error when view_years contains years that are not present in the data, it now correctly ignores them and prints a warning.

Tests

• Tests Added a comprehensive Pytest suite for the new HALNet and TransFlowSubsNet models, validating both 'tft_like' and 'pihal_like' modes.

• Tests Created a robust test suite for PiTGWFlow, covering instantiation, learnable parameter tracking, forward pass with multiple input formats, and the custom train_step.

• Tests Added a Pytest suite for the PositionalEncoding layer to ensure numerical stability, shape consistency, and serialization.

• Tests Added a comprehensive test suite for the new HydroTuner, covering instantiation, the .create() factory method, and end-to-end runs for both PIHALNet and TransFlowSubsNet.

• Tests Implemented unit tests for all new data utilities, including prepare_pinn_data_sequences(), widen_temporal_columns(), and the spatial utilities.

• Tests Expanded model tests to validate the new architecture_config functionality in BaseAttentive.

• Tests Added new tests for the HydroTuner to validate the new refit_best_model=False workflow.

• Tests Implemented regression tests to ensure PINN models train correctly when one or more lambda weights are set to zero.

• Tests Expanded test coverage for all data utilities (e.g., nan_ops, widen_temporal_columns) with edge-case DataFrames, including those with all-NaN columns or a single row.

Documentation

• Docs Added a new User Guide page, HALNet (Hybrid Attentive LSTM Network), to detail the flexible new HALNet model and its dual-mode architecture.

• Docs Updated the Physics-Informed Neural Networks (PINNs) page to reflect the new, more powerful TransFlowSubsNet and PITGWFlow models.

• Docs Added the Forecast Visualization Utilities page to document the new and improved visualization utilities, including forecast_view, plot_forecast_by_step, and plot_history_in.

• Docs Added an exercise page, Exercise: Forecasting with HALNet (All Inputs Required), to provide a hands-on tutorial for using the new HALNet model.

• Docs Completely reorganized the User Guide for improved clarity and navigation. The guide is now structured thematically around Models, Utilities, Tuning, and Exercises.

• Docs Added new, in-depth documentation pages for all PINN models: Hybrid Physics-Data Models: PiHALNet & PIHALNet. TransFlowSubsNet: A Physics-Informed Hybrid Forecasting Model, and Physics-Informed Transient Groundwater Flow (PiTGWFlow).

• Docs Added a new, comprehensive guide for the /user_guide/forecast_tuner/hydro_tuner.

• Docs Created new hands-on tutorials for all major models and utilities in the Hands-On Exercises.

• Docs Added a new “Tips & Tricks” page to the User Guide with practical advice on setting lambda weights for PINN models and choosing between ‘hybrid’ and ‘transformer’ architectures.

• Docs Updated the HydroTuner guide with an advanced section on defining custom hyperparameter search spaces and interpreting the newly generated tuning results plot.

• Docs All fusionlab.utils modules now have more detailed API docstrings with runnable examples for every public function.

Contributors

• Laurent Kouadio (Lead Developer)