Hyperparameter Tuning (Legacy PiHALTuner)

This section details the legacy PiHALTuner, a class designed specifically for hyperparameter optimization of the original PiHALNet model.

Warning

This tuner is specific to the legacy PiHALNet model. For tuning the modern, BaseAttentive-based models like PIHALNet and TransFlowSubsNet, please use the more flexible HydroTuner.

The PiHALTuner automates the process of searching for the best architectural and physics-informed hyperparameters using the Keras Tuner backend.

Core Concepts

The design of PiHALTuner is centered on separating the static, unchanging aspects of a model from the parameters you wish to optimize.

  • Fixed Parameters (`fixed_model_params`): This is a dictionary

    that defines the “problem.” It holds all the parameters that will not be tuned, primarily the data-dependent dimensions like static_input_dim, dynamic_input_dim, and forecast_horizon. These are constant for a given tuning job.

  • Hyperparameter Space (`param_space`): This dictionary allows you

    to override the tuner’s default search space for any given hyperparameter. For example, while the tuner might search for a dropout_rate between 0.0 and 0.3 by default, you can provide param_space={‘dropout_rate’: [0.1, 0.15]} to test only those two specific values.

End-to-End Workflow

The primary workflow involves using the PiHALTuner.create() factory method. This simplifies setup by automatically inferring data dimensions and merging them with defaults and user-provided parameters.

### Step 1: Prepare Data First, prepare your input features and target variables as dictionaries of NumPy arrays.

 1import numpy as np
 2
 3# Define data dimensions for the example
 4B, T, H = 128, 12, 5
 5S_DIM, D_DIM, F_DIM = 4, 6, 3
 6O_DIM = 1
 7
 8# Generate dummy data arrays
 9inputs = {
10    "coords": np.random.rand(B, H, 3).astype(np.float32),
11    "static_features": np.random.rand(B, S_DIM).astype(np.float32),
12    "dynamic_features": np.random.rand(B, T, D_DIM).astype(np.float32),
13    "future_features": np.random.rand(B, H, F_DIM).astype(np.float32),
14}
15targets = {
16    "subsidence": np.random.rand(B, H, O_DIM).astype(np.float32),
17    "gwl": np.random.rand(B, H, O_DIM).astype(np.float32)
18}

### Step 2: Create the Tuner with .create()

The .create() method is the recommended way to instantiate the tuner. It inspects your data to determine the necessary dimensions, combines them with sensible defaults, and prepares the tuner for the search. You can still provide a fixed_model_params dictionary to override any specific default or inferred values.

 1from fusionlab.nn.pinn.tuning import PiHALTuner
 2
 3# The .create() method infers dimensions from data
 4tuner = PiHALTuner.create(
 5    inputs_data=inputs,
 6    targets_data=targets,
 7    # You can override specific fixed params if needed
 8    fixed_model_params={'quantiles': [0.1, 0.5, 0.9]},
 9    # Define a custom search space for specific HPs
10    param_space={
11        'learning_rate': [1e-3, 5e-4],
12        'lambda_pde': {'type': 'float', 'min_value': 0.05, 'max_value': 0.5}
13    },
14    # Keras Tuner settings
15    objective='val_loss',
16    max_trials=10,
17    project_name="Legacy_PIHALNet_Tuning",
18    directory="./pihal_tuner_results",
19    overwrite=True
20)
21print("Tuner created and configured.")

### Step 3: Run the Hyperparameter Search Call the .run() method (an alias for .fit()) to start the tuning process. This method handles the creation of tf.data.Dataset objects internally and executes the Keras Tuner search loop.

 1import tensorflow as tf
 2
 3# Split data for validation
 4val_split = -20
 5train_inputs = {k: v[:val_split] for k, v in inputs.items()}
 6val_inputs = {k: v[val_split:] for k, v in inputs.items()}
 7train_targets = {k: v[:val_split] for k, v in targets.items()}
 8val_targets = {k: v[val_split:] for k, v in targets.items()}
 9
10# Run the search
11best_model, best_hps, tuner_instance = tuner.run(
12    inputs=train_inputs,
13    y=train_targets,
14    validation_data=(val_inputs, val_targets),
15    epochs=20, # Max epochs for each trial
16    batch_size=32,
17    callbacks=[tf.keras.callbacks.EarlyStopping('val_loss', patience=5)]
18)

### Step 4: Analyze Results After the search, the tuner object contains the best hyperparameters and a model instance retrained on the full dataset using those settings.

 1print("\n--- Tuning Complete: Best Hyperparameters ---")
 2if best_hps:
 3    for hp, value in best_hps.values.items():
 4        if isinstance(value, float):
 5            print(f"  - {hp}: {value:.5f}")
 6        else:
 7            print(f"  - {hp}: {value}")
 8
 9    # The best model is ready to be saved or used for prediction
10    # best_model.save("best_legacy_pihalnet_model.keras")

API Reference