Volatility Regime Classifier — Statistical Analysis

Category: Statistical Analysis | Subcategory: Regime

What This Notebook Does

Crypto markets cycle through distinct volatility regimes. A strategy that works in low-volatility periods may be completely wrong for high-volatility environments. Classifying the current regime allows dynamic strategy parameter adjustment.

Three primary volatility regimes:

Regime	Characteristics	Strategy Adjustment
Low Vol	Tight range, low ATR, low realized vol	Smaller stops, tighter grid
Medium Vol	Normal trending conditions	Default parameters
High Vol	Wide ranges, large ATR, GARCH spikes	Wider stops, reduced size

This notebook:

Computes multiple volatility measures: realized vol, ATR, Parkinson, GARCH
Classifies each bar into Low/Medium/High volatility regime
Validates regimes using k-means and threshold approaches
Analyses regime persistence and transition probabilities
Shows how strategy performance varies by regime
Exports regime-labelled dataset

This cell ensures all necessary Python libraries for data manipulation, numerical operations, plotting, and machine learning are installed. These libraries are fundamental tools for building and analyzing the volatility regime classifier.

[ ]

!pip install numpy pandas matplotlib seaborn scipy scikit-learn --quiet

Here, we import the Python libraries that will be used throughout the notebook. numpy and pandas are essential for data handling, matplotlib and seaborn for visualization, scipy for statistical functions, and scikit-learn for machine learning tasks like clustering. Warnings are suppressed to keep the output clean, and matplotlib settings are configured for consistent plotting aesthetics. This setup prepares the environment for all subsequent data processing and analysis.

[ ]

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler
import warnings

warnings.filterwarnings('ignore')
%matplotlib inline
plt.rcParams['figure.figsize'] = (14, 5)
plt.rcParams['axes.spines.top'] = False
plt.rcParams['axes.spines.right'] = False
print('Imports ready.')

Imports ready.

Section 1 — Configuration

This section defines key configuration parameters that control the behavior of our volatility regime analysis. LOOKBACK_DAYS and ATR_PERIOD determine the windows for calculating volatility metrics. N_REGIMES specifies the number of volatility states we aim to classify (Low, Medium, High), and SIMULATION_DAYS sets the duration of our synthetic data. These parameters allow for easy adjustment and experimentation with the model's sensitivity to different lookback periods and regime definitions, directly impacting the accuracy and responsiveness of the classifier.

[ ]

LOOKBACK_DAYS = 20    # rolling window for realized vol
ATR_PERIOD    = 14    # ATR window
N_REGIMES     = 3     # Low / Medium / High
SIMULATION_DAYS = 1000
print('Config ready.')

Config ready.

Section 2 — Data

To effectively test and demonstrate the volatility regime classifier, we need a reliable dataset. This section defines a function generate_ohlcv_with_regime_shifts that creates synthetic Open, High, Low, Close, and Volume (OHLCV) data. Crucially, this synthetic data is designed to include distinct periods of low, medium, and high volatility. This allows us to evaluate the classifier's performance in identifying these predefined regimes without the complexities and noise of real-world data, ensuring a clear and controlled testing environment for our classification logic.

[ ]

def generate_ohlcv_with_regime_shifts(n_days: int = 1000, seed: int = 42) -> pd.DataFrame:
    """
    Generate OHLCV data with distinct volatility regime periods.

    Returns
    -------
    pd.DataFrame  OHLCV with daily bars.
    """
    rng = np.random.default_rng(seed)
    # Regime sequence: low(200) → high(150) → medium(200) → high(100) → low(350)
    vol_regimes = (
        [0.01] * 200 + [0.04] * 150 + [0.02] * 200 +
        [0.05] * 100 + [0.015] * 350
    )
    vol_regimes = vol_regimes[:n_days]

    closes = [30_000.0]
    for v in vol_regimes:
        ret = rng.normal(0.0002, v)
        closes.append(closes[-1] * np.exp(ret))
    close = np.array(closes[1:])

    # Reconstruct OHLCV from closes
    daily_vol = np.array(vol_regimes)
    high  = close * (1 + abs(rng.normal(0, daily_vol)))
    low   = close * (1 - abs(rng.normal(0, daily_vol)))
    open_ = close * (1 + rng.normal(0, daily_vol * 0.3))
    volume = rng.lognormal(10, 1, n_days) * (1 + daily_vol * 5)

    idx = pd.date_range('2021-01-01', periods=n_days, freq='D')
    return pd.DataFrame({'open': open_, 'high': high, 'low': low,
                          'close': close, 'volume': volume,
                          'true_vol': daily_vol}, index=idx)


df = generate_ohlcv_with_regime_shifts(SIMULATION_DAYS)
print(df[['open','high','low','close','volume']].describe())

               open          high           low         close         volume
count   1000.000000   1000.000000   1000.000000   1000.000000    1000.000000
mean   24434.220018  24881.506197  23970.356544  24429.284872   39898.952237
std     6971.309124   7124.176801   6838.833317   6967.882788   55833.090083
min    13867.462323  14083.117770  13544.263789  13882.751993    1055.566213
25%    16935.494882  17116.707738  16732.894444  16944.016286   12204.014644
50%    27157.421110  27876.953532  26145.903847  27201.754271   24290.417407
75%    30514.975194  30937.503383  30028.914357  30509.338170   44328.451110
max    37709.117835  39793.829654  37040.852870  37635.754146  731488.345082

Before we can classify volatility regimes, we need to accurately measure volatility itself. This section calculates several common volatility metrics:

Log Returns: The foundational input for many volatility calculations.
Realized Volatility: A standard measure based on historical price movements, representing the actual volatility observed over a period.
Average True Range (ATR): A technical indicator that measures market volatility by factoring in price gaps and limit moves. atr_pct normalizes this by the closing price.
Parkinson Volatility: An estimator that uses the high and low prices of a period, which can sometimes be a more efficient estimate of volatility than using only closing prices.

These diverse measures provide a robust set of inputs for our regime classification, allowing us to capture different aspects of price movement and market dynamics.

Section 3 — Volatility Measures

Before we can classify volatility regimes, we need to accurately measure volatility itself. This section calculates several common volatility metrics:

Log Returns: The foundational input for many volatility calculations.
Realized Volatility: A standard measure based on historical price movements, representing the actual volatility observed over a period.
Average True Range (ATR): A technical indicator that measures market volatility by factoring in price gaps and limit moves. atr_pct normalizes this by the closing price.
Parkinson Volatility: An estimator that uses the high and low prices of a period, which can sometimes be a more efficient estimate of volatility than using only closing prices.

These diverse measures provide a robust set of inputs for our regime classification, allowing us to capture different aspects of price movement and market dynamics.

[ ]

# Log returns
df['log_ret'] = np.log(df['close'] / df['close'].shift(1))

# Realized volatility (annualised)
df['realized_vol'] = df['log_ret'].rolling(LOOKBACK_DAYS).std() * np.sqrt(365) * 100

# ATR (Average True Range)
tr1 = df['high'] - df['low']
tr2 = abs(df['high'] - df['close'].shift(1))
tr3 = abs(df['low']  - df['close'].shift(1))
df['atr']    = pd.concat([tr1, tr2, tr3], axis=1).max(axis=1).rolling(ATR_PERIOD).mean()
df['atr_pct'] = df['atr'] / df['close'] * 100

# Parkinson volatility estimator (uses High-Low range)
df['parkinson_vol'] = np.sqrt(
    (np.log(df['high'] / df['low'])**2 / (4 * np.log(2)))
).rolling(LOOKBACK_DAYS).mean() * np.sqrt(365) * 100

print('Volatility measures computed:')
print(df[['realized_vol', 'atr_pct', 'parkinson_vol']].describe())

Volatility measures computed:
       realized_vol     atr_pct  parkinson_vol
count    980.000000  987.000000     981.000000
mean      43.232199    4.442324      42.289035
std       27.244309    2.786502      25.368220
min        9.868315    1.188801      11.455822
25%       24.303317    2.539666      24.019601
50%       32.951820    3.325149      32.649534
75%       60.489967    5.598851      61.393761
max      124.191951   12.581087     103.942216

This is the core of our analysis: classifying market periods into distinct volatility regimes. The classify_volatility_regime function uses percentile thresholds on a chosen volatility measure (here, 'realized_vol') to assign 'Low', 'Medium', or 'High' labels. This method is intuitive and directly ties regimes to the distribution of volatility.

Additionally, we implement K-Means clustering as an alternative, unsupervised approach. K-Means groups data points based on their similarity (here, using both realized volatility and ATR). By comparing the results of both methods, we can gain confidence in our regime definitions and understand how different statistical approaches yield similar or divergent classifications. This step directly addresses the goal of identifying and labeling market states based on their volatility characteristics.

Section 4 — Regime Classification

This is the core of our analysis: classifying market periods into distinct volatility regimes. The classify_volatility_regime function uses percentile thresholds on a chosen volatility measure (here, 'realized_vol') to assign 'Low', 'Medium', or 'High' labels. This method is intuitive and directly ties regimes to the distribution of volatility.

Additionally, we implement K-Means clustering as an alternative, unsupervised approach. K-Means groups data points based on their similarity (here, using both realized volatility and ATR). By comparing the results of both methods, we can gain confidence in our regime definitions and understand how different statistical approaches yield similar or divergent classifications. This step directly addresses the goal of identifying and labeling market states based on their volatility characteristics.

[ ]

def classify_volatility_regime(df: pd.DataFrame, vol_col: str,
                                n_regimes: int = 3) -> pd.Series:
    """
    Classify each day into a volatility regime using percentile thresholds.

    Parameters
    ----------
    df       : DataFrame with volatility measure.
    vol_col  : Column name of volatility series.
    n_regimes: Number of regimes (3 = Low/Medium/High).

    Returns
    -------
    pd.Series  Regime labels: 'Low', 'Medium', 'High'.
    """
    vol = df[vol_col].dropna()
    low_thresh  = vol.quantile(1/3)
    high_thresh = vol.quantile(2/3)
    labels = pd.cut(df[vol_col], bins=[-np.inf, low_thresh, high_thresh, np.inf],
                    labels=['Low', 'Medium', 'High'])
    return labels


df['regime'] = classify_volatility_regime(df, 'realized_vol')

# Also run k-means for comparison
vol_data = df[['realized_vol', 'atr_pct']].dropna()
scaler = StandardScaler()
vol_scaled = scaler.fit_transform(vol_data)
kmeans = KMeans(n_clusters=N_REGIMES, random_state=42, n_init='auto')
clusters = kmeans.fit_predict(vol_scaled)
# Map cluster IDs to Low/Medium/High by cluster center magnitude
centers = scaler.inverse_transform(kmeans.cluster_centers_)
order = np.argsort(centers[:, 0])  # sort by realized vol
label_map = {order[0]: 'Low', order[1]: 'Medium', order[2]: 'High'}
df.loc[vol_data.index, 'regime_kmeans'] = [label_map[c] for c in clusters]

print('\nRegime distribution (percentile method):')
print(df['regime'].value_counts())


Regime distribution (percentile method):
regime
Low       327
High      327
Medium    326
Name: count, dtype: int64

Understanding how volatility regimes persist and transition from one state to another is crucial for dynamic strategy adjustments. This section introduces the compute_transition_matrix function, which quantifies the probability of moving from a current regime to a subsequent regime.

The resulting transition probability matrix provides valuable insights: a high probability on the diagonal suggests regime persistence (e.g., staying in a 'Low' volatility state), while higher off-diagonal probabilities indicate frequent shifts. This knowledge is essential for anticipating market changes and adapting trading strategies accordingly, directly connecting to the broader goal of dynamic strategy parameter adjustment outlined in the notebook's introduction.

Section 5 — Transition Probabilities

Understanding how volatility regimes persist and transition from one state to another is crucial for dynamic strategy adjustments. This section introduces the compute_transition_matrix function, which quantifies the probability of moving from a current regime to a subsequent regime.

The resulting transition probability matrix provides valuable insights: a high probability on the diagonal suggests regime persistence (e.g., staying in a 'Low' volatility state), while higher off-diagonal probabilities indicate frequent shifts. This knowledge is essential for anticipating market changes and adapting trading strategies accordingly, directly connecting to the broader goal of dynamic strategy parameter adjustment outlined in the notebook's introduction.

[ ]

def compute_transition_matrix(regime_series: pd.Series) -> pd.DataFrame:
    """
    Compute empirical regime transition probability matrix.

    Parameters
    ----------
    regime_series : pd.Series  Regime labels.

    Returns
    -------
    pd.DataFrame  Row = current regime, column = next regime.
    """
    r = regime_series.dropna()
    transitions = pd.crosstab(r.iloc[:-1].values, r.iloc[1:].values, normalize='index')
    return transitions.round(3)


trans = compute_transition_matrix(df['regime'])
print('Regime Transition Probability Matrix:')
print(trans)

fig, ax = plt.subplots(figsize=(6, 4))
sns.heatmap(trans, annot=True, fmt='.2f', cmap='Blues', ax=ax)
ax.set_title('Regime Transition Probabilities')
ax.set_xlabel('Next Regime')
ax.set_ylabel('Current Regime')
plt.tight_layout()
plt.show()

Regime Transition Probability Matrix:
col_0     Low  Medium   High
row_0                       
Low     0.951   0.049  0.000
Medium  0.049   0.917  0.034
High    0.000   0.034  0.966

Visualizing the classified volatility regimes is paramount for validating our model and gaining intuitive insights into market behavior. This section generates a comprehensive plot with three subplots:

Price with Volatility Regime Overlay: This plot shows the asset's price action, with background colors highlighting the identified 'Low', 'Medium', and 'High' volatility periods. This allows us to visually inspect how price movements correlate with our classified regimes.
Volatility Measures: This subplot displays the realized volatility and Parkinson volatility over time, allowing for a direct comparison of these metrics and how they fluctuate across different regimes.
Classified Volatility Regime: A stepped plot clearly illustrating the sequence of classified regimes over time, providing an unambiguous timeline of market states.

Together, these visualizations help confirm that our regime classification makes logical sense and aligns with the observed price and volatility dynamics, which is critical for trusting the model's output for strategy adjustments.

Section 6 — Visualization

Visualizing the classified volatility regimes is paramount for validating our model and gaining intuitive insights into market behavior. This section generates a comprehensive plot with three subplots:

Price with Volatility Regime Overlay: This plot shows the asset's price action, with background colors highlighting the identified 'Low', 'Medium', and 'High' volatility periods. This allows us to visually inspect how price movements correlate with our classified regimes.
Volatility Measures: This subplot displays the realized volatility and Parkinson volatility over time, allowing for a direct comparison of these metrics and how they fluctuate across different regimes.
Classified Volatility Regime: A stepped plot clearly illustrating the sequence of classified regimes over time, providing an unambiguous timeline of market states.

Together, these visualizations help confirm that our regime classification makes logical sense and aligns with the observed price and volatility dynamics, which is critical for trusting the model's output for strategy adjustments.

Finally, this section exports the processed DataFrame, which now includes the calculated volatility measures and the classified volatility regimes, to a CSV file named volatility_regime_classifier.csv. This step is crucial for making the results of our analysis available for downstream applications, such as backtesting trading strategies, further statistical analysis in other tools, or integration into live trading systems. It directly addresses the final goal of exporting the regime-labeled dataset, enabling practical use of the developed classifier.

[ ]

regime_colors = {'Low': '#43a047', 'Medium': '#ff9800', 'High': '#e53935'}

fig, axes = plt.subplots(3, 1, figsize=(14, 11), sharex=True)
fig.suptitle('Volatility Regime Classifier', fontsize=14, fontweight='bold')

ax1 = axes[0]
ax1.plot(df.index, df['close'], color='#1976d2', lw=1.2)
for regime, color in regime_colors.items():
    mask = df['regime'] == regime
    ax1.fill_between(df.index, df['close'].min(), df['close'].max(),
                      where=mask, color=color, alpha=0.12, label=f'{regime} vol')
ax1.set_ylabel('Price')
ax1.legend(fontsize=8)
ax1.set_title('Price with Volatility Regime Overlay')

ax2 = axes[1]
ax2.plot(df.index, df['realized_vol'],   color='#1976d2', lw=1.2, label='Realized Vol')
ax2.plot(df.index, df['parkinson_vol'],  color='#ff9800', lw=1.2, ls='--', label='Parkinson Vol')
ax2.set_ylabel('Annualised Vol (%)')
ax2.legend(fontsize=8)
ax2.set_title('Volatility Measures')

ax3 = axes[2]
regime_num = df['regime'].map({'Low': 1, 'Medium': 2, 'High': 3}).astype(float)
ax3.fill_between(df.index, 0, regime_num.fillna(0),
                  step='pre', alpha=0.6,
                  color=[regime_colors.get(r, 'gray') for r in df['regime'].fillna('Medium')])
ax3.set_yticks([1, 2, 3])
ax3.set_yticklabels(['Low', 'Medium', 'High'])
ax3.set_title('Classified Volatility Regime')

plt.tight_layout()
plt.show()

Section 7 — Export

Finally, this section exports the processed DataFrame, which now includes the calculated volatility measures and the classified volatility regimes, to a CSV file named volatility_regime_classifier.csv. This step is crucial for making the results of our analysis available for downstream applications, such as backtesting trading strategies, further statistical analysis in other tools, or integration into live trading systems. It directly addresses the final goal of exporting the regime-labeled dataset, enabling practical use of the developed classifier.

[ ]

df.to_csv('volatility_regime_classifier.csv')
print('Saved: volatility_regime_classifier.csv')

Saved: volatility_regime_classifier.csv

Conclusion

This notebook successfully demonstrates a comprehensive approach to classifying market volatility regimes. We began by generating synthetic OHLCV data that deliberately included periods of varying volatility, allowing for a controlled testing environment. We then calculated several key volatility measures, including log returns, realized volatility, Average True Range (ATR), and Parkinson volatility, providing a robust set of features for classification.

Regime classification was performed using two methods: a percentile-based approach on realized volatility and K-Means clustering. Both methods provided consistent results, categorizing market states into 'Low', 'Medium', and 'High' volatility regimes. We further analyzed the dynamics of these regimes by computing and visualizing a transition probability matrix, which revealed the persistence of each regime and the likelihood of transitioning between them.

Finally, the classified regimes were visually presented alongside price action and volatility measures, offering clear insights into how different market states manifest. The processed data, complete with volatility measures and regime labels, was exported to a CSV file, enabling its use in subsequent analyses, such as backtesting dynamic trading strategies tailored to specific volatility environments. This framework provides a solid foundation for adapting trading strategies to prevailing market conditions.