fusionlab.utils.ts_utils.trend_ops¶

fusionlab.utils.ts_utils.trend_ops(df, dt_col, value_col, ops=None, check_stationarity=True, trend_type='both', error='raise', strategy='adf', verbose=0, view=False, fig_size=(10, 4), show_grid=False, **kw)[source]¶

Perform transformations on a time series (e.g., removing upward/downward trends or applying differencing) based on automatically detected trends. The function first determines if the series is stationary or non-stationary and then applies the specified operation to remove or mitigate the detected trend.

In particular, when differencing is applied (for example in the 'detrend' option), the mathematical operator is:

\[\nabla Y_t = Y_t - Y_{t-1},\]

which removes first-order trends. For a linear trend removal through ordinary least squares (OLS), the model:

\[Y_t = \beta_0 + \beta_1 \cdot t + \epsilon_t\]

is fitted and subtracted from the original series if the slope \(\beta_1\) indicates an upward or downward trend.

Parameters:

df (pandas.DataFrame) – A DataFrame containing the time series data.
dt_col (str) – The column name representing datetime in the DataFrame. Must be a valid time-like column or convertible to one.
value_col (str or array-like) – The name (or direct series) of the target variable to transform.
ops ({'remove_upward', 'remove_downward', 'remove_both',) –
‘detrend’, ‘none’}, optional The transformation operation to perform:
- 'remove_upward': Detect and remove upward trend only.
- 'remove_downward': Detect and remove downward trend only.
- 'remove_both': Remove any identified trend (upward/downward).
- 'detrend': Apply differencing if the series is non-stationary.
- 'none': No transformation is performed.
check_stationarity (bool, optional) – Whether to apply a stationarity test (ADF or KPSS) before deciding on transformations. If False, transformations rely solely on the linear trend detection.
trend_type ({'both', 'upward', 'downward'}, optional) –
Type of trend detection applied in conjunction with stationarity checks:
- 'both': Check for both upward and downward slopes.
- 'upward': Focus on detecting a positive slope only.
- 'downward': Focus on detecting a negative slope only.
error ({'raise', 'warn', 'ignore'}, optional) –
Behavior to adopt if transformation yields invalid data (e.g., all NaNs):
- 'raise': Raises a ValueError.
- 'warn': Issues a warning.
- 'ignore': Silently ignores it.
strategy ({'adf', 'kpss'}, optional) –
Stationarity test used if check_stationarity=True:
- 'adf': Augmented Dickey-Fuller.
- 'kpss': Kwiatkowski–Phillips–Schmidt–Shin test.
verbose (int, optional) –
Verbosity level controlling console output:
- 0: No printing.
- 1: Basic info.
- 2: More detailed logs.
- 3: Very detailed logs (debug mode).
view (bool, optional) – If True, displays a plot comparing original vs. transformed data after trend removal or differencing.
fig_size (tuple of (int, int), optional) – Figure size for the resulting plot in inches (width, height).
show_grid (bool, optional) – Whether to display grid lines on the generated plots.
**kw (dict, optional) – Additional keyword arguments forwarded to lower-level functions or the plotting calls (e.g., markers, line widths, alpha, etc.).

Returns:

df – The original DataFrame with its target column potentially replaced by the transformed version.

Return type:

pandas.DataFrame

Examples

>>> import pandas as pd
>>> from fusionlab.utils.ts_utils import trend_ops
>>> data = {
...     'Date': [
...         '2020-01-01', '2020-01-02', '2020-01-03',
...         '2020-01-04', '2020-01-05'
...     ],
...     'Value': [10, 12, 15, 13, 14]
... }
>>> df = pd.DataFrame(data)
>>> # Remove both upward or downward linear trends if found
>>> transformed_df = trend_ops(
...     df=df, dt_col='Date',
...     value_col='Value',
...     ops='remove_both',
...     view=True,
...     verbose=1
... )
Detected Trend: upward
Stationarity Test p-value: 0.3147
Both upward and downward trends removed.