Backtesting·Strategy Optimization·Intermediate

Walk Forward Backtest

Build a complete walk-forward optimization and backtesting framework that periodically re-optimizes strategy parameters on rolling in-sample training windows and rigorously evaluates out-of-sample performance on subsequent unseen test periods.

backtestingmodel-validation

Understanding Walk-Forward Backtesting

Introduction to Walk-Forward Backtesting

Walk-forward backtesting is a robust methodology used in quantitative finance to evaluate the performance and stability of a trading strategy. Unlike traditional backtesting, which uses a fixed dataset for both optimization and testing, walk-forward analysis simulates a more realistic trading environment by iteratively re-optimizing the strategy's parameters over a rolling 'in-sample' period and then testing those optimized parameters on a subsequent, unseen 'out-of-sample' period.

Purpose and Importance

  • Addresses Overfitting: The primary purpose of walk-forward backtesting is to mitigate the risk of overfitting. A strategy optimized on a single historical dataset might perform exceptionally well on that specific data but fail catastrophically on new, unseen market conditions. By regularly re-optimizing and testing on fresh data, walk-forward analysis provides a better indication of how the strategy might perform in live trading.

  • Evaluates Adaptability: Markets are dynamic, and optimal strategy parameters can change over time. Walk-forward backtesting assesses a strategy's ability to adapt to changing market regimes. If a strategy consistently performs well across multiple out-of-sample periods after re-optimization, it suggests a more robust and adaptable system.

  • Realistic Performance Estimation: It provides a more realistic estimate of expected future performance compared to a single, static backtest, as it accounts for the periodic re-optimization that a live trader might perform.

  • Parameter Stability: It helps in identifying strategies whose optimal parameters are relatively stable over time, or strategies that can be effectively re-optimized, rather than those that require constant, drastic parameter adjustments.

Core Concept: How Walk-Forward Backtesting Works

The fundamental idea behind walk-forward backtesting involves a series of sequential steps:

  1. Define Windows: The historical data is divided into multiple sequential 'windows'. Each window consists of an 'in-sample' (optimization) period and an 'out-of-sample' (testing) period.

  2. Initial Optimization: In the first window, the trading strategy's parameters are optimized using the data from the initial in-sample period. This aims to find the 'best' set of parameters for that specific historical segment.

  3. Out-of-Sample Testing: The strategy, with the parameters optimized in step 2, is then applied to the subsequent out-of-sample period immediately following the in-sample period. This simulates live trading with parameters that were derived from prior knowledge.

  4. Rolling Forward: The window then 'walks forward' in time. The in-sample period shifts to include new data, and the out-of-sample period also advances. For this new window, steps 2 and 3 are repeated: re-optimize parameters on the new in-sample data and test on the new out-of-sample data.

  5. Aggregate Results: This process is repeated until all data is covered. The performance metrics from all the out-of-sample periods are then aggregated to provide a comprehensive and more reliable evaluation of the strategy's overall performance and robustness.

Simulated Data Generation

To demonstrate walk-forward backtesting, we'll first generate some synthetic price data. This data will simulate a simple price series with some trend and noise, which is sufficient to illustrate the concept without the complexities of real-world financial data.

[1]
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

# Set random seed for reproducibility
np.random.seed(42)

def generate_price_data(num_points=1000, initial_price=100, trend_strength=0.1, noise_level=0.5):
    """
    Generates synthetic price data.

    Inputs:
    - num_points (int): The number of data points to generate.
    - initial_price (float): The starting price of the series.
    - trend_strength (float): The strength of the underlying trend.
    - noise_level (float): The magnitude of random fluctuations.

    Outputs:
    - pd.Series: A pandas Series containing the simulated price data.
    """
    time = np.arange(num_points)
    # Introduce a changing trend over time
    trend = initial_price + trend_strength * time + 0.0005 * (time ** 2)
    noise = np.random.normal(0, noise_level, num_points).cumsum()
    prices = trend + noise
    return pd.Series(prices, name='Price')

# Generate simulated data
data = generate_price_data(num_points=1000, initial_price=100, trend_strength=0.05, noise_level=1)
dates = pd.date_range(start='2020-01-01', periods=1000, freq='D')
df = pd.DataFrame({'Date': dates, 'Price': data})
df = df.set_index('Date')

print("Simulated Price Data Head:")
display(df.head())

print("\nSimulated Price Data Tail:")
display(df.tail())

# Visualize the simulated data
plt.figure(figsize=(12, 6))
plt.plot(df.index, df['Price'])
plt.title('Simulated Price Data')
plt.xlabel('Date')
plt.ylabel('Price')
plt.grid(True)
plt.show()
Simulated Price Data Head:
Price
Date
2020-01-01 100.496714
2020-01-02 100.408950
2020-01-03 101.108138
2020-01-04 102.683668
2020-01-05 102.503015

Simulated Price Data Tail:
Price
Date
2022-09-22 661.654623
2022-09-23 664.497809
2022-09-24 666.185152
2022-09-25 666.661473
2022-09-26 668.282556
cell output

Walk-Forward Backtesting Implementation

Now, let's implement the core logic for walk-forward backtesting. For simplicity, we'll use a basic moving average crossover strategy. The parameters we'll optimize are the lengths of the fast and slow moving averages. The 'strategy' will be to buy when the fast MA crosses above the slow MA and sell when the fast MA crosses below the slow MA.

Optimization Metric

In each in-sample period, we'll optimize the moving average lengths to maximize the total returns generated by the strategy.

walk_forward_backtest Function

This function will orchestrate the entire walk-forward process:

  1. Define Strategy: A simple moving average crossover.
  2. Parameter Optimization: Iterate through a range of fast and slow MA lengths to find the combination that yields the highest return within the in-sample window.
  3. Out-of-Sample Performance: Apply the best parameters from the optimization to the subsequent out-of-sample window to calculate its performance.
[2]
def simple_moving_average_strategy(data, fast_ma_len, slow_ma_len):
    """
    Applies a simple moving average crossover strategy to price data.

    Inputs:
    - data (pd.Series): The price data (e.g., 'Price' column from DataFrame).
    - fast_ma_len (int): The length of the fast moving average.
    - slow_ma_len (int): The length of the slow moving average.

    Outputs:
    - pd.Series: A series of daily returns from the strategy.
    """
    if fast_ma_len >= slow_ma_len:
        return pd.Series(0, index=data.index) # Invalid lengths, return no profit

    fast_ma = data.rolling(window=fast_ma_len).mean()
    slow_ma = data.rolling(window=slow_ma_len).mean()

    # Generate signals: 1 for buy, -1 for sell, 0 for hold
    signals = pd.Series(0, index=data.index)
    signals[fast_ma > slow_ma] = 1
    signals[fast_ma < slow_ma] = -1

    # Shift signals to ensure we trade on the next day's open after a signal
    positions = signals.shift(1).fillna(0) # 0 means no position

    # Calculate daily returns: (change in price / previous price) * position
    daily_returns = data.pct_change() * positions
    return daily_returns

def walk_forward_backtest(
    df,
    price_col='Price',
    in_sample_len=200,
    out_of_sample_len=50,
    fast_ma_range=(10, 30),
    slow_ma_range=(40, 80)
):
    """
    Performs a walk-forward backtest on a given DataFrame using a simple MA crossover strategy.

    Inputs:
    - df (pd.DataFrame): DataFrame containing price data with a datetime index.
    - price_col (str): The name of the column containing the price data.
    - in_sample_len (int): Length of the in-sample (optimization) period.
    - out_of_sample_len (int): Length of the out-of-sample (testing) period.
    - fast_ma_range (tuple): (min_len, max_len) for the fast moving average.
    - slow_ma_range (tuple): (min_len, max_len) for the slow moving average.

    Outputs:
    - pd.DataFrame: A DataFrame containing the daily returns for each out-of-sample period.
    - list: A list of dictionaries, each containing optimized parameters for a window.
    """
    all_oos_returns = pd.Series(dtype=float)
    optimized_params_history = []

    total_len = len(df)
    current_start_idx = 0

    while current_start_idx + in_sample_len + out_of_sample_len <= total_len:
        in_sample_end_idx = current_start_idx + in_sample_len
        oos_end_idx = in_sample_end_idx + out_of_sample_len

        in_sample_data = df.iloc[current_start_idx:in_sample_end_idx][price_col]
        oos_data = df.iloc[in_sample_end_idx:oos_end_idx][price_col]

        best_fast_ma = -1
        best_slow_ma = -1
        max_in_sample_return = -np.inf

        # --- Optimization (In-Sample) ---
        for fast_len in range(fast_ma_range[0], fast_ma_range[1] + 1):
            for slow_len in range(slow_ma_range[0], slow_ma_range[1] + 1):
                if fast_len < slow_len:
                    strategy_returns = simple_moving_average_strategy(in_sample_data, fast_len, slow_len)
                    total_return = strategy_returns.sum()
                    if total_return > max_in_sample_return:
                        max_in_sample_return = total_return
                        best_fast_ma = fast_len
                        best_slow_ma = slow_len

        # Store optimized parameters
        optimized_params_history.append({
            'in_sample_start': in_sample_data.index.min(),
            'in_sample_end': in_sample_data.index.max(),
            'oos_start': oos_data.index.min(),
            'oos_end': oos_data.index.max(),
            'best_fast_ma': best_fast_ma,
            'best_slow_ma': best_slow_ma,
            'max_in_sample_return': max_in_sample_return
        })

        # --- Out-of-Sample Testing ---
        if best_fast_ma != -1 and best_slow_ma != -1:
            oos_strategy_returns = simple_moving_average_strategy(oos_data, best_fast_ma, best_slow_ma)
            all_oos_returns = pd.concat([all_oos_returns, oos_strategy_returns])
        else:
            # If no valid parameters found, append zeros for the OOS period
            oos_zero_returns = pd.Series(0, index=oos_data.index)
            all_oos_returns = pd.concat([all_oos_returns, oos_zero_returns])

        # Move the window forward
        current_start_idx += out_of_sample_len # Walk forward by OOS length

    return all_oos_returns, optimized_params_history


# --- Demonstrate the walk-forward backtest ---
print("Performing walk-forward backtest...")
# For demonstration, using shorter ranges and lengths
oos_returns, params_history = walk_forward_backtest(
    df,
    in_sample_len=150,
    out_of_sample_len=30,
    fast_ma_range=(5, 15),
    slow_ma_range=(20, 40)
)

print("\nWalk-Forward Out-of-Sample Returns Head:")
display(oos_returns.head())

print("\nWalk-Forward Out-of-Sample Returns Tail:")
display(oos_returns.tail())

print("\nOptimized Parameters History (first 5 windows):")
display(pd.DataFrame(params_history).head())
Performing walk-forward backtest...
/tmp/ipykernel_2583/3557981315.py:96: FutureWarning: The behavior of array concatenation with empty entries is deprecated. In a future version, this will no longer exclude empty items when determining the result dtype. To retain the old behavior, exclude the empty entries before the concat operation.
  all_oos_returns = pd.concat([all_oos_returns, oos_strategy_returns])

Walk-Forward Out-of-Sample Returns Head:
0
2020-05-30 NaN
2020-05-31 0.0
2020-06-01 -0.0
2020-06-02 0.0
2020-06-03 0.0


Walk-Forward Out-of-Sample Returns Tail:
0
2022-09-12 -0.000190
2022-09-13 0.003887
2022-09-14 0.002919
2022-09-15 0.001242
2022-09-16 0.001618


Optimized Parameters History (first 5 windows):
in_sample_start in_sample_end oos_start oos_end best_fast_ma best_slow_ma max_in_sample_return
0 2020-01-01 2020-05-29 2020-05-30 2020-06-28 9 32 0.157824
1 2020-01-31 2020-06-28 2020-06-29 2020-07-28 9 32 0.262915
2 2020-03-01 2020-07-28 2020-07-29 2020-08-27 7 26 0.274860
3 2020-03-31 2020-08-27 2020-08-28 2020-09-26 8 20 0.359319
4 2020-04-30 2020-09-26 2020-09-27 2020-10-26 15 20 0.360225

Visualizing Walk-Forward Backtest Results

Visualizations are crucial for understanding the performance and robustness of a strategy evaluated through walk-forward backtesting. We'll create two main visualizations:

1. Cumulative Returns of Out-of-Sample Performance

This plot shows the cumulative growth of the strategy's equity curve based only on the out-of-sample periods. This is the most direct representation of how the strategy would have performed if traded live, with periodic re-optimization.

[3]
plt.figure(figsize=(14, 7))
(1 + oos_returns).cumprod().plot()
plt.title('Walk-Forward Backtest: Cumulative Out-of-Sample Returns')
plt.xlabel('Date')
plt.ylabel('Cumulative Return')
plt.grid(True)
plt.axhline(1.0, color='red', linestyle='--', linewidth=0.8, label='Break-even (1.0)')
plt.legend()
plt.show()
cell output

Interpretation of Cumulative Returns

  • Upward Trend: A generally upward-sloping curve indicates that the strategy generated positive returns over the aggregated out-of-sample periods.
  • Drawdowns: Dips in the curve represent periods of loss. The depth and duration of these drawdowns are important risk metrics.
  • Volatility: The choppiness of the curve reflects the volatility of the strategy's returns.
  • End Value: The final value on the y-axis shows the total cumulative return over the entire walk-forward period. For example, a value of 1.5 means a 50% total return.

2. Optimized Parameters Across Walk-Forward Windows

This visualization shows how the 'best' parameters (fast and slow MA lengths) changed from one optimization window to the next. This helps in understanding the stability of the strategy's optimal parameters.

[4]
params_df = pd.DataFrame(params_history)
params_df['oos_start'] = pd.to_datetime(params_df['oos_start'])
params_df = params_df.set_index('oos_start')

plt.figure(figsize=(14, 7))
plt.plot(params_df.index, params_df['best_fast_ma'], label='Optimized Fast MA Length', marker='o', linestyle='--')
plt.plot(params_df.index, params_df['best_slow_ma'], label='Optimized Slow MA Length', marker='x', linestyle='-')
plt.title('Optimized Moving Average Lengths Across Walk-Forward Windows')
plt.xlabel('Out-of-Sample Start Date')
plt.ylabel('MA Length')
plt.grid(True)
plt.legend()
plt.show()
cell output

Interpretation of Optimized Parameters Visualization

  • Parameter Stability: If the lines for 'best_fast_ma' and 'best_slow_ma' are relatively flat or show gradual changes, it suggests that the optimal parameters for the strategy are stable. This is a positive sign for robustness.
  • Parameter Volatility: Large and frequent jumps in the optimized parameters indicate that the optimal settings for the strategy are highly sensitive to the specific data in the in-sample period. Such a strategy might be less robust and harder to trade in real-time, as it requires frequent and potentially drastic re-optimization.
  • Market Regime Shifts: Significant shifts in optimal parameters might also correspond to changes in market regimes, suggesting the strategy needs to adapt or might perform poorly if not re-optimized accordingly.

Comparison to Traditional Backtesting

To highlight the importance of walk-forward backtesting, let's briefly compare it to a typical traditional backtest, where parameters are optimized once over the entire dataset.

[5]
def traditional_backtest(df, price_col='Price', fast_ma_range=(10, 30), slow_ma_range=(40, 80)):
    """
    Performs a traditional backtest by optimizing parameters once over the entire dataset.

    Inputs:
    - df (pd.DataFrame): DataFrame containing price data with a datetime index.
    - price_col (str): The name of the column containing the price data.
    - fast_ma_range (tuple): (min_len, max_len) for the fast moving average.
    - slow_ma_range (tuple): (min_len, max_len) for the slow moving average.

    Outputs:
    - pd.Series: Daily returns from the strategy applied with the best parameters.
    - dict: Best fast MA, best slow MA, and max total return.
    """
    best_fast_ma = -1
    best_slow_ma = -1
    max_total_return = -np.inf

    data_to_optimize = df[price_col]

    for fast_len in range(fast_ma_range[0], fast_ma_range[1] + 1):
        for slow_len in range(slow_ma_range[0], slow_ma_range[1] + 1):
            if fast_len < slow_len:
                strategy_returns = simple_moving_average_strategy(data_to_optimize, fast_len, slow_len)
                total_return = strategy_returns.sum()
                if total_return > max_total_return:
                    max_total_return = total_return
                    best_fast_ma = fast_len
                    best_slow_ma = slow_len

    if best_fast_ma != -1 and best_slow_ma != -1:
        final_returns = simple_moving_average_strategy(data_to_optimize, best_fast_ma, best_slow_ma)
    else:
        final_returns = pd.Series(0, index=df.index)

    return final_returns, {'best_fast_ma': best_fast_ma, 'best_slow_ma': best_slow_ma, 'max_total_return': max_total_return}

# Perform traditional backtest
trad_returns, trad_params = traditional_backtest(
    df,
    fast_ma_range=(5, 15),
    slow_ma_range=(20, 40)
)

print("Traditional Backtest Optimized Parameters:")
display(trad_params)

plt.figure(figsize=(14, 7))
(1 + trad_returns).cumprod().plot(label='Traditional Backtest Cumulative Returns')
(1 + oos_returns).cumprod().plot(label='Walk-Forward Backtest Cumulative OOS Returns', linestyle='--')
plt.title('Walk-Forward vs. Traditional Backtest Cumulative Returns')
plt.xlabel('Date')
plt.ylabel('Cumulative Return')
plt.grid(True)
plt.legend()
plt.show()
Traditional Backtest Optimized Parameters:
{'best_fast_ma': 9,
 'best_slow_ma': 32,
 'max_total_return': np.float64(2.007517110128692)}
cell output

Interpretation of Comparison

  • Overfitting Risk: The traditional backtest (solid line) often shows a much smoother and higher cumulative return. This is because its parameters were optimized on the entire dataset, including data that would not have been available during live trading.
  • Realistic Performance: The walk-forward out-of-sample returns (dashed line) are generally more conservative and indicative of real-world performance, as they reflect trading with parameters derived only from past data.
  • Robustness Indicator: If the traditional backtest shows outstanding returns, but the walk-forward results are poor, it's a strong signal of overfitting. A robust strategy should show reasonable performance in the walk-forward out-of-sample periods, even if it's less spectacular than the traditional backtest.

Conclusion

Walk-forward backtesting is an indispensable tool for developing and validating trading strategies. It moves beyond the limitations of traditional backtesting by simulating the adaptive nature of real-world trading, where strategies are periodically re-optimized.

By systematically testing strategies on unseen data segments after each optimization, it provides a more realistic assessment of performance, helps detect overfitting, and offers insights into the stability of optimal parameters. While more computationally intensive, the insights gained from walk-forward analysis are crucial for building confidence in a strategy's robustness before deploying it in live markets.