Enter Platform
Resources
Back to Notebooks
Open in Colab
Statistical Analysis·Regime Detection·Intermediate

Correlation Regime Shift

Detect structural breakpoints and regime shifts in cross-asset correlation matrices using changepoint detection algorithms and covariance matrix equality tests, providing early warning when assumed portfolio diversification benefits may be deteriorating during periods of market stress.

market-analysisquant-analysisstatistical-methods

Correlation Regime Shift Detection — Statistical Analysis

Category: Statistical Analysis | Subcategory: Regime

What This Notebook Does

Correlation regimes describe periods when assets move together (high correlation) vs independently. These shifts have major implications for portfolio diversification:

  • Risk-on (normal): correlations low; diversification works
  • Stress/crisis: correlations spike toward 1.0; diversification breaks down
  • Sector rotation: correlations shift as capital flows between sectors

This notebook:

  1. Computes rolling pairwise correlations between assets
  2. Detects correlation regime shifts using CUSUM and threshold methods
  3. Measures the average cross-asset correlation as a market stress indicator
  4. Identifies correlation breakdowns — when previously correlated pairs decouple
  5. Analyses the impact of correlation regime on portfolio variance
  6. Exports the correlation time-series
[1]
!pip install numpy pandas matplotlib seaborn scipy --quiet
[2]
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from scipy import stats
import warnings

warnings.filterwarnings('ignore')
%matplotlib inline
plt.rcParams['figure.figsize'] = (14, 5)
plt.rcParams['axes.spines.top'] = False
plt.rcParams['axes.spines.right'] = False
print('Imports ready.')
Imports ready.

Section 1 — Configuration

This section defines the key parameters used throughout the notebook for correlation analysis and regime detection:

  • CORR_WINDOW: The number of days used for calculating rolling correlations. A 30-day window is a common choice for capturing short-to-medium term dynamics.
  • STRESS_THRESH: The threshold for the average cross-asset correlation. If the average correlation exceeds this value, it indicates a potential 'stress' regime.
  • CUSUM_H: The alarm threshold for the CUSUM (Cumulative Sum) change detection algorithm. A higher value makes the detector less sensitive.
  • SIMULATION_DAYS: The total number of days for which the synthetic asset return data will be simulated.
[3]
CORR_WINDOW   = 30    # rolling correlation window
STRESS_THRESH = 0.7   # avg correlation above this = stress regime
CUSUM_H       = 5     # CUSUM alarm threshold
SIMULATION_DAYS = 1000
print('Config ready.')
Config ready.

Section 2 — Multi-Asset Data

[4]
def generate_correlation_regime_data(n_days: int = 1000, seed: int = 42) -> pd.DataFrame:
    """
    Simulate 5 crypto assets with shifting correlation regimes.

    Normal regime: low correlation (~0.3-0.5 between assets).
    Stress regime: all assets crash together (correlation spikes to ~0.8-0.9).

    Returns
    -------
    pd.DataFrame  Daily returns for BTC, ETH, SOL, BNB, AVAX.
    """
    rng = np.random.default_rng(seed)
    idx = pd.date_range('2021-01-01', periods=n_days, freq='D')

    # Market factor (shared component)
    market = rng.normal(0.001, 0.02, n_days)

    # Stress periods: day 250-350, 650-720
    stress = np.zeros(n_days)
    stress[250:350] = 1
    stress[650:720] = 1

    def asset_return(beta_normal, beta_stress, idio_vol):
        beta  = beta_normal * (1 - stress) + beta_stress * stress
        idio  = rng.normal(0, idio_vol, n_days) * (1 - stress * 0.5)
        trend = rng.normal(0.0003, 0.005, n_days)
        return beta * market + idio + trend * (1 - stress)

    returns = pd.DataFrame({
        'BTC':  asset_return(0.7, 0.95, 0.015),
        'ETH':  asset_return(0.9, 0.97, 0.018),
        'SOL':  asset_return(1.2, 0.98, 0.025),
        'BNB':  asset_return(0.8, 0.96, 0.018),
        'AVAX': asset_return(1.1, 0.97, 0.022),
    }, index=idx)
    returns['stress_regime'] = stress
    return returns


df = generate_correlation_regime_data(SIMULATION_DAYS)
rets = df[['BTC', 'ETH', 'SOL', 'BNB', 'AVAX']]
print('Return stats:')
print(rets.describe())
Return stats:
               BTC          ETH          SOL          BNB         AVAX
count  1000.000000  1000.000000  1000.000000  1000.000000  1000.000000
mean     -0.000406     0.000541     0.001828    -0.000039    -0.000146
std       0.020972     0.025041     0.032545     0.023378     0.030821
min      -0.065060    -0.085239    -0.092213    -0.075585    -0.091775
25%      -0.014132    -0.016268    -0.020230    -0.015503    -0.018607
50%      -0.000154     0.000381     0.001746     0.000347    -0.000064
75%       0.013189     0.016221     0.022975     0.014855     0.018878
max       0.070538     0.090676     0.101051     0.094609     0.105145

The generate_correlation_regime_data function simulates daily returns for five crypto assets (BTC, ETH, SOL, BNB, AVAX). It models two distinct correlation regimes:

  • Normal Regime: Assets have lower correlation, reflecting periods where diversification is effective.
  • Stress Regime: Assets crash together, leading to high correlation and a breakdown of diversification. These periods are intentionally introduced at specific intervals (days 250-350 and 650-720).

The simulation uses a shared 'market' factor and asset-specific idiosyncratic components, with varying betas during normal and stress periods to achieve the desired correlation shifts.

The code then calls this function to generate a DataFrame df and extracts the asset returns into rets for further analysis. Finally, it prints a statistical summary of the generated returns.

Section 3 — Rolling Correlations

[5]
# BTC-ETH rolling correlation
df['btc_eth_corr'] = rets['BTC'].rolling(CORR_WINDOW).corr(rets['ETH'])
df['btc_sol_corr'] = rets['BTC'].rolling(CORR_WINDOW).corr(rets['SOL'])
df['eth_bnb_corr'] = rets['ETH'].rolling(CORR_WINDOW).corr(rets['BNB'])

# Average cross-asset correlation (market stress indicator)
rolling_corr_matrix = []
for i in range(CORR_WINDOW, len(df)):
    c = rets.iloc[i-CORR_WINDOW:i].corr().values
    # Average off-diagonal elements
    n = len(c)
    avg_corr = (c.sum() - n) / (n * (n-1))
    rolling_corr_matrix.append(avg_corr)

df.loc[df.index[CORR_WINDOW:], 'avg_corr'] = rolling_corr_matrix
df['stress_detected'] = (df['avg_corr'] > STRESS_THRESH).astype(int)

accuracy = (df['stress_detected'].fillna(0) == df['stress_regime']).mean()
print(f'Average correlation in normal regime: {df[df["stress_regime"]==0]["avg_corr"].mean():.3f}')
print(f'Average correlation in stress regime: {df[df["stress_regime"]==1]["avg_corr"].mean():.3f}')
print(f'Stress detection accuracy: {accuracy:.1%}')
Average correlation in normal regime: 0.459
Average correlation in stress regime: 0.719
Stress detection accuracy: 93.4%

This section calculates rolling correlations to identify how asset relationships change over time.

  1. Pairwise Rolling Correlations: It calculates the CORR_WINDOW-day rolling correlation for specific pairs like BTC-ETH and BTC-SOL. This helps visualize the individual co-movement of assets.
  2. Average Cross-Asset Correlation: A crucial indicator, this metric calculates the average of all unique pairwise correlations within the rolling window. A spike in this average correlation suggests a market-wide 'stress' event where most assets move together.
    • The loop iterates through the DataFrame, taking CORR_WINDOW days at a time to build a correlation matrix, and then averages its off-diagonal elements.
  3. Stress Regime Detection: Based on the STRESS_THRESH defined in the configuration, it flags days where the avg_corr exceeds this threshold as stress_detected.
  4. Accuracy Calculation: It compares the stress_detected flags with the stress_regime (true stress periods defined during data simulation) to evaluate the effectiveness of the threshold-based detection.

Section 4 — CUSUM Change Detection

[6]
def cusum_detect(series: pd.Series, h: float, k: float = 0.5) -> pd.Series:
    """
    CUSUM (cumulative sum) change point detector.

    Signals when the cumulative deviation from mean exceeds threshold h.

    Parameters
    ----------
    series : pd.Series  Input signal (e.g. rolling correlation).
    h      : float      Detection threshold.
    k      : float      Allowable slack (tuning parameter).

    Returns
    -------
    pd.Series  CUSUM statistic at each point.
    """
    s = series.dropna().values
    mu = np.mean(s)
    std = np.std(s)
    cusum_pos = np.zeros(len(s))
    for i in range(1, len(s)):
        cusum_pos[i] = max(0, cusum_pos[i-1] + (s[i] - mu) / (std + 1e-9) - k)
    cusum_series = pd.Series(np.nan, index=series.index)
    cusum_series.iloc[series.notna().cumsum()[series.notna()].index.get_loc(series.dropna().index[0]):len(series.dropna())] = cusum_pos
    return cusum_series.reindex(series.index)


df['cusum_corr'] = cusum_detect(df['avg_corr'].fillna(method='bfill'), CUSUM_H)
alarm_days = (df['cusum_corr'] > CUSUM_H).sum()
print(f'CUSUM alarm triggered on {alarm_days} days')
CUSUM alarm triggered on 447 days

This section implements the CUSUM (Cumulative Sum) algorithm for change detection, a statistical method used to detect a shift in the mean of a data stream.

  • cusum_detect Function: This helper function takes a time series (e.g., rolling average correlation) and parameters h (detection threshold) and k (allowable slack). It calculates the cumulative sum of deviations from the mean, signaling an alarm if this sum exceeds h.
    • A positive CUSUM indicates an increase in the signal's mean, while a negative CUSUM indicates a decrease.
  • Application: The cusum_detect function is applied to the avg_corr series. The CUSUM_H parameter from the configuration is used as the threshold.
  • Alarm Days: The notebook then counts and prints the number of days when the CUSUM statistic (cusum_corr) exceeds the CUSUM_H threshold, indicating a detected change point or 'alarm'.

Section 5 — Visualization

[7]
fig, axes = plt.subplots(3, 1, figsize=(14, 11), sharex=True)
fig.suptitle('Correlation Regime Shift Detection', fontsize=14, fontweight='bold')

ax1 = axes[0]
btc_price = (1 + rets['BTC']).cumprod() * 30_000
ax1.plot(btc_price.index, btc_price, color='#1976d2', lw=1.2)
ax1.fill_between(df.index, btc_price.min(), btc_price.max(),
                  where=df['stress_regime']==1, color='#e53935', alpha=0.15, label='Stress period')
ax1.set_ylabel('BTC Price')
ax1.legend(fontsize=8); ax1.set_title('BTC Price with Stress Regime Overlay')

ax2 = axes[1]
ax2.plot(df.index, df['btc_eth_corr'], color='#1976d2', lw=1, label='BTC-ETH')
ax2.plot(df.index, df['btc_sol_corr'], color='#ff9800', lw=1, label='BTC-SOL')
ax2.plot(df.index, df['avg_corr'],     color='black',   lw=2, label='Avg cross-asset')
ax2.axhline(STRESS_THRESH, color='#e53935', ls='--', lw=1, label=f'Stress threshold ({STRESS_THRESH})')
ax2.set_ylabel('Correlation')
ax2.legend(fontsize=8); ax2.set_title(f'{CORR_WINDOW}-Day Rolling Correlations')

ax3 = axes[2]
ax3.fill_between(df.index, df['stress_regime'], color='#e53935', alpha=0.3, label='True stress')
ax3.fill_between(df.index, df['stress_detected'].fillna(0), color='#ff9800', alpha=0.4, label='Detected stress')
ax3.set_ylabel('Stress Indicator')
ax3.legend(fontsize=8); ax3.set_title('Stress Regime: True vs Detected')

plt.tight_layout()
plt.show()
cell output

This section provides a visual summary of the correlation regime shift detection using three subplots:

  1. BTC Price with Stress Regime Overlay: The top plot shows the cumulative product of BTC returns (simulating price movement) overlaid with shaded areas indicating the true stress periods defined in the data simulation. This helps contextualize correlation shifts with market behavior.
  2. Rolling Correlations: The middle plot displays:
    • BTC-ETH and BTC-SOL rolling correlations, showing specific asset pair dynamics.
    • The Avg cross-asset correlation, which is a key indicator of overall market correlation.
    • A horizontal dashed line at the STRESS_THRESH to visually mark the crisis threshold.
  3. Stress Regime: True vs Detected: The bottom plot compares the true stress periods (from data simulation) with the detected stress periods (based on avg_corr exceeding STRESS_THRESH). This visually confirms how well the threshold method captures the simulated stress events.

Together, these plots provide a comprehensive view of asset price movements, correlation dynamics, and the effectiveness of the regime detection methods.

Section 6 — Export

[8]
df.to_csv('correlation_regime_shift.csv')
print('Saved: correlation_regime_shift.csv')
Saved: correlation_regime_shift.csv

This final section simply exports the entire DataFrame df, which includes all the original returns data, calculated rolling correlations, stress indicators, and CUSUM statistics, into a CSV file named correlation_regime_shift.csv. This allows for easy external access and further analysis of the generated and analyzed data.

Conclusion

This notebook demonstrated a comprehensive approach to detecting correlation regime shifts in multi-asset portfolios. We simulated asset returns under different correlation regimes (normal vs. stress) and then applied two methods for regime detection:

  1. Threshold-based detection: Using a predefined stress threshold on the average cross-asset correlation.
  2. CUSUM change detection: A statistical method to identify shifts in the mean of the average correlation.

Key takeaways:

  • Correlation regimes are crucial for understanding diversification benefits and risks in a portfolio.
  • Average cross-asset correlation can serve as an effective market stress indicator.
  • Both thresholding and CUSUM can be powerful tools for identifying periods of market stress and shifts in correlation dynamics.

Further analysis could involve exploring other change detection algorithms, optimizing threshold parameters, and backtesting portfolio strategies based on these regime signals.