Statistical Analysis·Pairs Trading Methods·Intermediate

Dynamic Hedge Ratio

Estimate time-varying dynamic hedge ratios between cointegrated asset pairs using rolling window OLS regression, Kalman filter state-space models, and vector error correction models that continuously adapt to the evolving long-run equilibrium relationship across changing market regimes.

pairs-tradingquant-analysis

Dynamic Hedge Ratio Estimation — Statistical Analysis

Category: Statistical Analysis | Subcategory: Pairs

What This Notebook Does

A static hedge ratio (β from OLS) is estimated once and fixed. But in crypto markets, the relationship between assets evolves — β at the start of a bull market may differ from β during a correction.

Dynamic hedge ratio methods:

Method	Description	Pros
Rolling OLS	Estimate β over rolling window	Simple, interpretable
Kalman Filter	Bayesian updating of β	Smooth, optimal tracking
DCC-GARCH	Dynamic conditional correlation	Best for volatility modeling

This notebook:

Computes rolling OLS hedge ratio with adjustable window
Implements a Kalman Filter for smooth β estimation
Compares static vs dynamic hedge ratio performance
Measures residual stationarity under each approach
Visualises β evolution and spread quality
Exports the dynamic hedge ratio series

[ ]

!pip install numpy pandas matplotlib seaborn scipy statsmodels --quiet

[ ]

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from statsmodels.tsa.stattools import adfuller
import statsmodels.api as sm
import warnings

warnings.filterwarnings('ignore')
%matplotlib inline
plt.rcParams['figure.figsize'] = (14, 5)
plt.rcParams['axes.spines.top'] = False
plt.rcParams['axes.spines.right'] = False
print('Imports ready.')

Imports ready.

Section 1 — Configuration

Configuration Parameters for Dynamic Hedge Ratio Estimation

This cell defines the core parameters that control the behavior of our dynamic hedge ratio models:

ROLLING_WINDOW: Specifies the look-back period for the Rolling OLS method. A shorter window makes the estimation more reactive to recent changes but potentially more volatile, while a longer window provides smoother but slower-adapting estimates.
KF_Q: Represents the process noise for the Kalman Filter. This parameter dictates how quickly the Kalman Filter's estimated state (the hedge ratio) is allowed to change over time. A higher value means the filter will adapt more aggressively to new observations.
KF_R: Represents the observation noise for the Kalman Filter. This parameter reflects the uncertainty or variance in the observed data. A higher KF_R indicates more noisy observations, leading the filter to rely more on its previous state estimate.
SIMULATION_DAYS: Determines the total number of days for our synthetic data generation. This allows us to create a sufficiently long time series to observe regime shifts and test the adaptability of the dynamic models.

Why it matters: These parameters are critical for tuning the responsiveness and stability of the dynamic hedge ratio estimates. Incorrectly set parameters can lead to either overfitting (too reactive) or underfitting (too slow to adapt) the true underlying relationship between assets.

Connection to topic: These configurations directly impact how well the dynamic methods (Rolling OLS and Kalman Filter) will perform in tracking a regime-shifting hedge ratio, which is the central problem addressed in this notebook.

[ ]

ROLLING_WINDOW  = 60
KF_Q            = 1e-5   # Kalman: process noise
KF_R            = 1e-3   # Kalman: observation noise
SIMULATION_DAYS = 730
print('Config ready.')

Config ready.

Section 2 — Data with Regime-Shifting β

Data Generation with Regime-Shifting Beta and Static OLS Baseline

This section performs two key tasks:

Synthetic Data Generation (generate_regime_shifting_pair function): This function creates a pair of synthetic price series, Y and X, where the true underlying hedge ratio (true_beta) deliberately shifts mid-series. Specifically, the true_beta is 0.5 for the first half of the data and 0.8 for the second half. This simulation includes a mean-reverting spread component to make the data more realistic for pairs trading scenarios.
Static OLS Hedge Ratio Calculation: After generating the data, a traditional Ordinary Least Squares (OLS) regression is performed on the entire dataset to calculate a static hedge ratio (beta_static) and an intercept (alpha_static). This provides a benchmark against which the dynamic methods will be compared. The spread_static is also calculated using this fixed beta.

Why it matters:

Regime-Shifting Data: Simulating data with a known regime shift is crucial for validating the effectiveness of dynamic hedge ratio estimation methods. If a method can accurately track the true_beta in this controlled environment, it gives us confidence in its potential application to real-world, non-stationary markets.
Static Benchmark: The beta_static serves as a baseline. By comparing the dynamic betas and their resulting spreads against this static value, we can quantitatively demonstrate the advantages of adapting the hedge ratio over time, especially in markets where relationships evolve.

Connection to topic: This section lays the foundation for the entire analysis. It creates the very problem (a changing hedge ratio) that the dynamic methods aim to solve and establishes the basic measurement tools (spread_static) needed to evaluate their performance.

[ ]

def generate_regime_shifting_pair(n_days=730, seed=42):
    """
    Generate pair where true β shifts mid-series (regime change).

    β = 0.5 for first half, β = 0.8 for second half.

    Returns
    -------
    pd.DataFrame  Columns: Y, X, true_beta.
    """
    rng = np.random.default_rng(seed)
    idx = pd.date_range('2023-01-01', periods=n_days, freq='D')
    X_log = 9.0 + np.cumsum(rng.normal(0, 0.012, n_days))

    true_beta = np.concatenate([np.linspace(0.5, 0.5, n_days//2),
                                  np.linspace(0.8, 0.8, n_days - n_days//2)])
    # OU spread
    spread = [0.0]
    for _ in range(n_days - 1):
        spread.append(spread[-1] * 0.95 + rng.normal(0, 0.015))

    Y_log = 1.0 + true_beta * X_log + np.array(spread)
    return pd.DataFrame({'Y': np.exp(Y_log), 'X': np.exp(X_log),
                          'true_beta': true_beta}, index=idx)


df = generate_regime_shifting_pair(SIMULATION_DAYS)

# Static OLS hedge ratio (full sample)
reg_static = sm.OLS(np.log(df['Y']), sm.add_constant(np.log(df['X']))).fit()
beta_static = reg_static.params.iloc[1]
alpha_static = reg_static.params.iloc[0]
df['spread_static'] = np.log(df['Y']) - alpha_static - beta_static * np.log(df['X'])
print(f'Static β = {beta_static:.4f}')

Static β = -5.2486

Section 3 — Rolling OLS Hedge Ratio

Rolling OLS Hedge Ratio Estimation

This section introduces one of the primary dynamic methods for estimating the hedge ratio: Rolling Ordinary Least Squares (OLS).

rolling_ols_hedge function: This function calculates the hedge ratio (beta) by applying OLS regression to a continuously moving window of data. For each point in time, it considers only the most recent window observations to estimate the relationship between Y and X. The function takes logarithmic returns of Y and X as input, which is standard practice in financial time series analysis.
Application to DataFrame: The rolling_ols_hedge function is then applied to our simulated data (df['Y'], df['X']) using the ROLLING_WINDOW defined in the configuration. The resulting dynamic hedge ratio is stored in df['beta_rolling']. Subsequently, a spread_rolling is calculated using this time-varying beta.

Why it matters: Rolling OLS is a straightforward and interpretable approach to dynamic hedging. It addresses the limitation of static OLS by allowing the hedge ratio to adapt to recent market conditions. While simple, it effectively demonstrates how a changing market environment necessitates a flexible hedge.

Connection to topic: This is the first concrete implementation of a dynamic hedge ratio method. It directly contrasts with the previously calculated static beta by demonstrating how the hedge ratio (and thus the spread) can evolve based on recent data, moving us closer to the goal of robust pairs trading in non-stationary markets.

[ ]

def rolling_ols_hedge(Y: pd.Series, X: pd.Series, window: int) -> pd.Series:
    """
    Compute rolling OLS hedge ratio β.

    Parameters
    ----------
    Y, X   : Price series.
    window : Rolling window size.

    Returns
    -------
    pd.Series  Rolling β.
    """
    betas = [np.nan] * window
    lX = np.log(X.values)
    lY = np.log(Y.values)
    for i in range(window, len(Y)):
        x = lX[i-window:i]
        y = lY[i-window:i]
        A = np.vstack([np.ones_like(x), x]).T
        try:
            beta = np.linalg.lstsq(A, y, rcond=None)[0][1]
        except:
            beta = np.nan
        betas.append(beta)
    return pd.Series(betas, index=Y.index)


df['beta_rolling'] = rolling_ols_hedge(df['Y'], df['X'], ROLLING_WINDOW)
df['spread_rolling'] = np.log(df['Y']) - df['beta_rolling'] * np.log(df['X'])

Section 4 — Kalman Filter Hedge Ratio

Kalman Filter for Dynamic Hedge Ratio Estimation

This section implements a more sophisticated dynamic estimation technique: the Kalman Filter.

kalman_hedge_ratio function: This function employs a Kalman Filter to recursively estimate the hedge ratio (β) and intercept (α) over time. Unlike Rolling OLS, which re-estimates from scratch for each window, the Kalman Filter updates its previous estimates based on new observations, resulting in a smoother and often more statistically optimal tracking of the underlying state.
- It models the state as [α, β], where both the intercept and slope are allowed to evolve.
- Q (process noise) and R (observation noise) parameters control the filter's responsiveness and confidence in its estimates vs. new data.
Application to DataFrame: The kalman_hedge_ratio function is applied to the log prices of Y and X from our simulated dataset, utilizing the KF_Q and KF_R parameters from the configuration. The estimated dynamic beta is stored in df['beta_kalman'], and the corresponding spread_kalman is calculated.

Why it matters: The Kalman Filter is a powerful algorithm for state estimation in dynamic systems. For hedge ratios, it provides a more robust and smooth estimate compared to simple rolling regressions, as it incorporates uncertainty and continually refines its belief about the true underlying beta. This can lead to more stable spreads and potentially better trading signals.

Connection to topic: This represents a more advanced approach to dynamic hedge ratio estimation than Rolling OLS. By comparing beta_kalman and spread_kalman against the static and rolling OLS results, we can evaluate the benefits of a Bayesian, recursive filtering approach in tracking regime shifts and improving spread quality for pairs trading.

[ ]

def kalman_hedge_ratio(Y: pd.Series, X: pd.Series, q: float, r: float) -> pd.Series:
    """
    Estimate dynamic hedge ratio β using a Kalman Filter.

    State: [α, β] — both intercept and slope are tracked.
    Observation: log(Y) = α + β * log(X) + noise.

    Parameters
    ----------
    Y, X : Price series (logs used internally).
    q    : Process noise (how fast β can change).
    r    : Observation noise.

    Returns
    -------
    pd.Series  Kalman-estimated β time series.
    """
    lX = np.log(X.values)
    lY = np.log(Y.values)
    n  = len(lY)

    # State [alpha, beta], 2x2
    x  = np.array([0.0, 0.5])    # init: alpha=0, beta=0.5
    P  = np.eye(2)
    Q  = q * np.eye(2)
    R  = r

    betas = []
    for i in range(n):
        H = np.array([[1.0, lX[i]]])  # observation matrix
        # Predict
        P_pred = P + Q
        # Kalman gain
        S = H @ P_pred @ H.T + R
        K = P_pred @ H.T / S[0, 0]
        # Update
        y_pred = H @ x
        x = x + K.flatten() * (lY[i] - y_pred[0])
        P = (np.eye(2) - K.reshape(-1, 1) @ H) @ P_pred
        betas.append(x[1])

    return pd.Series(betas, index=Y.index)


df['beta_kalman'] = kalman_hedge_ratio(df['Y'], df['X'], KF_Q, KF_R)
df['spread_kalman'] = np.log(df['Y']) - df['beta_kalman'] * np.log(df['X'])

Section 5 — Spread Quality Comparison

Spread Quality Comparison: Stationarity and Volatility

This section provides a quantitative assessment of the effectiveness of each hedge ratio estimation method by analyzing the quality of the resulting spreads.

Metrics Calculated: For each spread (static, rolling OLS, and Kalman Filter), the following metrics are computed:
- Standard Deviation (Std): A measure of the spread's volatility. Lower standard deviation generally indicates a tighter, more predictable spread.
- Augmented Dickey-Fuller (ADF) p-value: The p-value from the ADF statistical test, which is used to determine if a time series is stationary. A p-value less than 0.05 typically suggests that the spread is stationary, meaning it tends to revert to its mean.
Evaluation Loop: The code iterates through each of the calculated spreads, drops any NaN values (common at the start of rolling windows), and then performs the calculations and prints the results.

Why it matters: For effective pairs trading, a stationary spread is paramount. A stationary spread implies that the difference between the two assets (adjusted by the hedge ratio) will tend to revert to its mean over time, allowing for mean-reversion trading strategies. The standard deviation further indicates the tightness and predictability of this mean reversion. This analysis directly addresses whether the dynamic methods are successfully creating better, more tradable spreads than a static approach.

Connection to topic: This section is a crucial part of the evaluation. It moves beyond just estimating the beta to directly assessing the output quality (the spread) for each method. By quantitatively comparing stationarity and volatility, we can determine which dynamic approach offers the most robust foundation for a pairs trading strategy under regime-shifting conditions.

[ ]

for label, col in [('Static β', 'spread_static'), ('Rolling OLS β', 'spread_rolling'), ('Kalman β', 'spread_kalman')]:
    s = df[col].dropna()
    adf_p = adfuller(s)[1]
    print(f'{label:16s}: std={s.std():.4f}  ADF p={adf_p:.4f}  stationary={"YES" if adf_p < 0.05 else "NO"}')

Static β        : std=1.2185  ADF p=0.5679  stationary=NO
Rolling OLS β   : std=59.9945  ADF p=0.0020  stationary=YES
Kalman β        : std=0.5035  ADF p=0.5698  stationary=NO

Section 6 — Visualization

Visualization of Dynamic Hedge Ratios and Spread Volatility

This section provides a visual comparison of the performance of the static, Rolling OLS, and Kalman Filter methods. Visualizations are key to intuitively understanding complex time series dynamics.

Top Plot: Dynamic vs Static Hedge Ratio: This subplot displays the evolution of the true_beta (our known ground truth), the fixed beta_static, and the estimated beta_rolling and beta_kalman over the entire simulation period. This allows for a clear visual assessment of how well each dynamic method tracks the true, regime-shifting hedge ratio.
- The true_beta is shown as a dashed black line, clearly indicating the point of regime shift.
- beta_static is a flat line, highlighting its inability to adapt.
- beta_rolling and beta_kalman show their respective adaptive behaviors.
Bottom Plot: Spread Volatility by Method: This subplot shows the 5-day rolling standard deviation (volatility) of the spreads generated by each method (spread_static, spread_rolling, spread_kalman). A lower and more stable spread volatility is generally desirable for pairs trading.
- We expect dynamic methods to produce spreads with lower volatility, especially around the regime shift.

Why it matters: Visualizing these trends helps to quickly identify the strengths and weaknesses of each approach. We can see how quickly and accurately beta_rolling and beta_kalman adapt to the change in true_beta, and critically, how this adaptation translates into more stable and potentially more tradable spreads (lower volatility).

Connection to topic: This visualization provides a compelling summary of the notebook's core findings. It allows for a direct visual comparison of how effectively each dynamic method addresses the challenge of regime-shifting hedge ratios and ultimately contributes to the goal of maintaining a healthy, stationary spread for pairs trading.

[ ]

fig, axes = plt.subplots(2, 1, figsize=(14, 8), sharex=True)
fig.suptitle('Dynamic Hedge Ratio Estimation', fontsize=14, fontweight='bold')

ax1 = axes[0]
ax1.plot(df.index, df['true_beta'],    color='black',   lw=2,   ls='--', label='True β')
ax1.plot(df.index, beta_static * np.ones(len(df)), color='gray', lw=1.5, ls=':', label=f'Static β ({beta_static:.2f})')
ax1.plot(df.index, df['beta_rolling'], color='#1976d2', lw=1.5, label=f'Rolling OLS (w={ROLLING_WINDOW})')
ax1.plot(df.index, df['beta_kalman'],  color='#e53935', lw=2,   label='Kalman Filter')
ax1.set_ylabel('Hedge Ratio β')
ax1.legend(fontsize=8)
ax1.set_title('Dynamic vs Static Hedge Ratio')

ax2 = axes[1]
ax2.plot(df.index, df['spread_static'].rolling(5).std(), color='gray', lw=1, ls=':', label='Static spread volatility')
ax2.plot(df.index, df['spread_rolling'].rolling(5).std(), color='#1976d2', lw=1.5, label='Rolling OLS spread vol')
ax2.plot(df.index, df['spread_kalman'].rolling(5).std(),  color='#e53935', lw=2,   label='Kalman spread vol')
ax2.set_ylabel('Spread Volatility (5d)')
ax2.legend(fontsize=8)
ax2.set_title('Spread Volatility by Method')

plt.tight_layout()
plt.show()

Section 7 — Export

Exporting the Results

This final section is dedicated to saving the comprehensive dataset generated and analyzed throughout the notebook.

df.to_csv('dynamic_hedge_ratio.csv'): This command saves the entire pandas DataFrame (df) to a CSV file named dynamic_hedge_ratio.csv. The DataFrame includes the original synthetic price series (Y, X), the true_beta, the calculated static beta components, and all dynamically estimated hedge ratios (beta_rolling, beta_kalman) along with their corresponding spreads.

Why it matters: Exporting the data ensures that the results of our analysis are persistent and accessible. This is crucial for several reasons:

Further Analysis: The saved data can be easily loaded into other environments or scripts for more in-depth analysis, backtesting, or integration into larger trading systems.
Reproducibility: It allows others (or yourself later) to reproduce or verify the results without re-running the entire notebook.
Input for Other Models: The dynamically estimated hedge ratios are valuable outputs that could serve as inputs for other quantitative models or trading algorithms.

Connection to topic: This step completes the workflow by making the valuable output of the dynamic hedge ratio estimation process readily available. It signifies the practical application of the methods explored, providing concrete data that can be used to inform actual pairs trading strategies.

[ ]

df.to_csv('dynamic_hedge_ratio.csv')
print('Saved: dynamic_hedge_ratio.csv')

Saved: dynamic_hedge_ratio.csv

Conclusion

This notebook has explored different approaches to estimating dynamic hedge ratios, a critical component for robust pairs trading strategies in non-stationary markets. We compared a static OLS approach against two dynamic methods: Rolling OLS and the Kalman Filter, using synthetically generated data with a known regime-shifting beta.

Key Findings:

Static OLS Limitations: As expected, the static hedge ratio failed to adapt to the regime shift, leading to a higher volatility and often non-stationary spread when the underlying relationship changed.
Rolling OLS: This method demonstrated adaptability by reflecting recent market conditions. While it is simple and interpretable, its estimates can be somewhat noisy, and its responsiveness is directly tied to the chosen window size.
Kalman Filter: The Kalman Filter provided a smoother and often more accurate tracking of the true, underlying hedge ratio. Its ability to recursively update estimates based on process and observation noise resulted in a more stable and potentially more tradable spread, especially during periods of regime change.

Practical Implications:

For real-world pairs trading, particularly in volatile markets like cryptocurrency where relationships can evolve rapidly, dynamic hedge ratio estimation is crucial. The Kalman Filter, with proper tuning of its Q and R parameters, offers a powerful tool to maintain stationary spreads and generate more reliable trading signals. The ability to track a changing beta helps in better risk management and potentially higher profitability compared to relying on a fixed, static hedge.

Further extensions could involve exploring other dynamic models like DCC-GARCH for volatility co-movement, incorporating exogenous variables, or applying these methods to real market data.