Bootstrap Confidence Intervals
Generate bootstrap confidence intervals using both standard and block bootstrap resampling methods for strategy performance metrics including Sharpe ratio, maximum drawdown, and win rate to rigorously quantify estimation uncertainty and avoid the false precision of point estimates computed on limited historical return samples.
Bootstrap Confidence Intervals for Trading Metrics — Statistical Analysis
Category: Statistical Analysis | Subcategory: Distributions
What This Notebook Does
Bootstrapping is a non-parametric resampling method that constructs confidence intervals for any statistic without assuming a specific distribution. This is essential for trading because:
- Return distributions are NOT normal — analytical CIs can be wrong
- Sample sizes are often small (limited backtest history)
- We need to know if our Sharpe ratio is statistically significant or just noise
Bootstrap procedure:
for b in range(B):
sample_b = resample(data, replace=True)
stat_b = compute_statistic(sample_b)
CI = [percentile(stats, α/2), percentile(stats, 1-α/2)]
This notebook constructs bootstrap CIs for:
- Sharpe ratio — is it statistically different from zero?
- Maximum drawdown — what is the true uncertainty of our worst drawdown?
- Win rate — is our win rate significantly above 50%?
- Annual return — what range of outcomes can we expect?
- Block bootstrap — proper time-series bootstrap that preserves autocorrelation
!pip install numpy pandas matplotlib seaborn scipy --quietimport numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from scipy import stats
import warnings
warnings.filterwarnings('ignore')
%matplotlib inline
plt.rcParams['figure.figsize'] = (14, 5)
plt.rcParams['axes.spines.top'] = False
plt.rcParams['axes.spines.right'] = False
print('Imports ready.')Imports ready.
Section 1 — Configuration
This section sets up the key parameters for the bootstrap analysis, including the number of bootstrap replications (N_BOOTSTRAP), the desired confidence level (CONFIDENCE), the block size for the block bootstrap (BLOCK_SIZE), and the number of simulation days for data generation (SIMULATION_DAYS).
N_BOOTSTRAP = 5_000
CONFIDENCE = 0.95
BLOCK_SIZE = 20 # for block bootstrap
SIMULATION_DAYS = 500
print('Config ready.')Config ready.
Section 2 — Data
This section generates synthetic trading data, specifically daily returns (rets) and equity curve (equity), based on a Student's t-distribution to simulate realistic financial data with fatter tails. It also calculates and displays the point estimate of the Sharpe ratio for the generated data.
rng = np.random.default_rng(42)
rets = rng.standard_t(df=4, size=SIMULATION_DAYS) * 0.018 + 0.0008
equity = np.cumprod(1 + rets) * 10_000
print(f'Point estimate Sharpe: {rets.mean() / rets.std() * np.sqrt(365):.3f}')Point estimate Sharpe: 0.713
Section 3 — Bootstrap Functions
This section defines the core bootstrap functions: bootstrap_ci for independent and identically distributed (IID) data, and block_bootstrap_ci for time-series data to preserve autocorrelation. It also defines helper functions for common trading metrics like Sharpe ratio, annual return, and maximum drawdown. Finally, it computes and prints the confidence intervals for these metrics using both IID and block bootstrap methods.
def bootstrap_ci(data: np.ndarray, statistic_fn, n_boot: int = 5000,
confidence: float = 0.95, seed: int = 42) -> tuple:
"""
IID bootstrap confidence interval for any statistic.
Parameters
----------
data : np.ndarray Sample data.
statistic_fn : callable Function that computes the statistic from a sample.
n_boot : int Number of bootstrap replications.
confidence : float Confidence level (0-1).
Returns
-------
tuple (point_estimate, lower_ci, upper_ci, bootstrap_distribution)
"""
rng = np.random.default_rng(seed)
point = statistic_fn(data)
boot_stats = np.array([
statistic_fn(rng.choice(data, size=len(data), replace=True))
for _ in range(n_boot)
])
alpha = 1 - confidence
ci_lo = np.percentile(boot_stats, alpha/2 * 100)
ci_hi = np.percentile(boot_stats, (1 - alpha/2) * 100)
return point, ci_lo, ci_hi, boot_stats
def block_bootstrap_ci(data: np.ndarray, statistic_fn, n_boot: int = 5000,
block_size: int = 20, confidence: float = 0.95) -> tuple:
"""
Block bootstrap CI that preserves time-series autocorrelation.
Parameters
----------
data : np.ndarray Time series data.
block_size : int Length of consecutive blocks.
Returns
-------
tuple (point_estimate, lower_ci, upper_ci, bootstrap_distribution)
"""
rng = np.random.default_rng(42)
n = len(data)
n_blocks = n // block_size + 1
point = statistic_fn(data)
boot_stats = []
for _ in range(n_boot):
starts = rng.integers(0, n - block_size + 1, size=n_blocks)
sample = np.concatenate([data[s:s+block_size] for s in starts])[:n]
boot_stats.append(statistic_fn(sample))
boot_stats = np.array(boot_stats)
alpha = 1 - confidence
return point, np.percentile(boot_stats, alpha/2*100), \
np.percentile(boot_stats, (1-alpha/2)*100), boot_stats
def sharpe(r): return r.mean() / (r.std() + 1e-9) * np.sqrt(365)
def annual_return(r): return (1 + r.mean()) ** 365 - 1
def max_drawdown(r):
eq = np.cumprod(1 + r)
pk = np.maximum.accumulate(eq)
return ((eq - pk) / pk).min()
print('Computing bootstrap CIs...')
metrics = {
'Sharpe Ratio': (bootstrap_ci(rets, sharpe, N_BOOTSTRAP, CONFIDENCE), None),
'Annual Return': (bootstrap_ci(rets, annual_return, N_BOOTSTRAP, CONFIDENCE), None),
'Max Drawdown': (bootstrap_ci(rets, max_drawdown, N_BOOTSTRAP, CONFIDENCE), None),
'Block Sharpe': (block_bootstrap_ci(rets, sharpe, N_BOOTSTRAP, BLOCK_SIZE, CONFIDENCE), '(block bootstrap)'),
}
alpha_str = f'{(1-CONFIDENCE)*100:.0f}%'
print(f'\n{CONFIDENCE*100:.0f}% Confidence Intervals:')
for name, (result, note) in metrics.items():
pt, lo, hi, _ = result
note_str = f' {note}' if note else ''
print(f' {name:20s}: {pt:.4f} CI=[{lo:.4f}, {hi:.4f}]{note_str}')Computing bootstrap CIs... 95% Confidence Intervals: Sharpe Ratio : 0.7130 CI=[-1.0031, 2.3786] Annual Return : 0.4410 CI=[-0.3973, 2.3885] Max Drawdown : -0.3023 CI=[-0.7065, -0.2279] Block Sharpe : 0.7130 CI=[-0.7076, 2.0893] (block bootstrap)
Section 4 — Visualization
This section visualizes the bootstrap distributions and their corresponding confidence intervals for the calculated trading metrics. It uses histograms to show the distribution of bootstrap statistics and vertical lines to indicate the point estimate and the upper and lower bounds of the confidence intervals.
fig, axes = plt.subplots(1, 3, figsize=(15, 5))
fig.suptitle('Bootstrap Confidence Intervals for Trading Metrics', fontsize=12, fontweight='bold')
for ax, (name, (result, _)), color in zip(axes,
list(metrics.items())[:3],
['#1976d2', '#43a047', '#e53935']):
pt, lo, hi, boot_dist = result
ax.hist(boot_dist, bins=60, density=True, color=color, alpha=0.6)
ax.axvline(pt, color='black', lw=2, label=f'Point: {pt:.3f}')
ax.axvline(lo, color='red', ls='--', lw=1, label=f'CI: [{lo:.3f}, {hi:.3f}]')
ax.axvline(hi, color='red', ls='--', lw=1)
if 'Sharpe' in name or 'Return' in name:
ax.axvline(0, color='gray', ls=':', lw=0.8)
ax.set_title(name)
ax.legend(fontsize=7)
plt.tight_layout()
plt.show()
Section 5 — Export
This section exports the generated synthetic returns and equity data into a CSV file named bootstrap_confidence_intervals.csv for potential external use or further analysis.
pd.DataFrame({'returns': rets, 'equity': equity}).to_csv('bootstrap_confidence_intervals.csv', index=False)
print('Saved: bootstrap_confidence_intervals.csv')Saved: bootstrap_confidence_intervals.csv
Conclusion
This notebook demonstrates how to use bootstrapping (both IID and block bootstrap) to construct confidence intervals for various trading metrics. This method provides a robust way to assess the statistical significance and uncertainty of performance statistics, especially when dealing with non-normal return distributions and limited data.