Position Snapshot History
Store end-of-day portfolio position snapshots with complete state including all asset holdings, notional exposures, margin utilization, unrealized PnL, and collateral balances for each trading account to build a full historical audit trail and enable multi-period performance attribution analysis.
Compliance & Audit: Store Daily Position Snapshots
This notebook demonstrates a robust system for capturing, storing, and auditing daily financial position snapshots. Maintaining an immutable audit trail of positions is a critical requirement for regulatory compliance (e.g., MiFID II, SEC Rule 17a-4).
Key Concepts
| Concept | Description |
|---|---|
| Snapshotting | Capturing the state of all holdings at a specific point in time (EOD). |
| State Management | Using dictionaries and functional updates to track system status. |
| Immutability | Ensuring that once a snapshot is recorded, it serves as a permanent record. |
| Retries | Implementing exponential backoff for resilience in distributed systems. |
!pip install pandas matplotlib seabornRequirement already satisfied: pandas in /usr/local/lib/python3.12/dist-packages (2.2.2) Requirement already satisfied: matplotlib in /usr/local/lib/python3.12/dist-packages (3.10.0) Requirement already satisfied: seaborn in /usr/local/lib/python3.12/dist-packages (0.13.2) Requirement already satisfied: numpy>=1.26.0 in /usr/local/lib/python3.12/dist-packages (from pandas) (2.0.2) Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.12/dist-packages (from pandas) (2.9.0.post0) Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.12/dist-packages (from pandas) (2025.2) Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.12/dist-packages (from pandas) (2026.2) Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (1.3.3) Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (0.12.1) Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (4.63.0) Requirement already satisfied: kiwisolver>=1.3.1 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (1.5.0) Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (26.2) Requirement already satisfied: pillow>=8 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (11.3.0) Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (3.3.2) Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.12/dist-packages (from python-dateutil>=2.8.2->pandas) (1.17.0)
import logging
import time
import random
from datetime import datetime, timedelta
from collections import deque
from typing import Dict, List, Any, Optional
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
# Configure logging
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
logger = logging.getLogger(__name__)Function Name: create_audit_state
Initializes the state dictionary for the compliance system.
Parameters:
retention_days(int): Number of days to keep snapshots in the rolling window.
Returns:
- (dict): Initial state dictionary.
def create_audit_state(retention_days: int = 30) -> Dict[str, Any]:
"""
Initializes the state for the position snapshot system.
Parameters
----------
retention_days : int
How many snapshots to keep in the rolling window.
Returns
-------
dict
Initial state containing snapshots and configuration.
"""
logger.info(f"Initializing audit state with {retention_days} days retention.")
return {
"snapshots": deque(maxlen=retention_days),
"config": {"retention_days": retention_days},
"metrics": {"total_captured": 0, "failures": 0}
}Function Name: capture_daily_snapshot
Simulates capturing current positions and appending them to the audit trail with retry logic.
Parameters:
state(dict): Current system state.positions(dict): Current position data to snapshot.
Returns:
- (dict): Updated state dictionary.
def capture_daily_snapshot(state: Dict[str, Any], positions: Dict[str, float]) -> Dict[str, Any]:
"""
Captures a new position snapshot into the state with retry logic.
Parameters
----------
state : dict
Current system state.
positions : dict
Dictionary of tickers and quantities.
Returns
-------
dict
Updated state with the new snapshot.
"""
max_retries = 3
base_delay = 0.1
for attempt in range(max_retries):
try:
# Simulate potential API failure
if random.random() < 0.1:
raise ConnectionError("Failed to connect to Position Service")
timestamp = datetime.now().isoformat()
snapshot = {
"timestamp": timestamp,
"data": positions.copy(),
"checksum": hash(frozenset(positions.items()))
}
state["snapshots"].append(snapshot)
state["metrics"]["total_captured"] += 1
logger.info(f"Successfully captured snapshot at {timestamp}")
return state
except Exception as e:
state["metrics"]["failures"] += 1
jitter = random.uniform(0, 0.1)
delay = (base_delay * (2 ** attempt)) + jitter
logger.warning(f"Attempt {attempt + 1} failed: {e}. Retrying in {delay:.2f}s...")
time.sleep(delay)
logger.error("Max retries reached. Could not capture snapshot.")
return stateFunction Name: summarize_audit_trail
Converts the rolling snapshots into a pandas DataFrame for analysis.
Parameters:
state(dict): Current system state.
Returns:
- (pd.DataFrame): Tabular view of all snapshots.
def summarize_audit_trail(state: Dict[str, Any]) -> pd.DataFrame:
"""
Aggregates snapshot data into a flat DataFrame.
Parameters
----------
state : dict
The current audit state.
Returns
-------
pd.DataFrame
DataFrame containing historical positions.
"""
records = []
for snap in state["snapshots"]:
for ticker, qty in snap["data"].items():
records.append({
"timestamp": pd.to_datetime(snap["timestamp"]),
"ticker": ticker,
"quantity": qty
})
df = pd.DataFrame(records)
logger.debug(f"Generated summary with {len(df)} rows.")
return dfDemonstration and Visualization
In this section, we simulate 10 days of cryptocurrency trading activity for assets like Bitcoin (BTC), Ethereum (ETH), and Solana (SOL). The primary goal is to demonstrate how daily position snapshots are captured and stored, forming an immutable audit trail. This is crucial for financial institutions dealing with volatile assets to maintain regulatory compliance by providing clear, auditable records of holdings at specific points in time.
# 1. Initialize State
audit_state = create_audit_state(retention_days=15)
# 2. Simulate 10 days of crypto position changes
# Using crypto assets: Bitcoin, Ethereum, and Solana
base_positions = {"BTC": 1.5, "ETH": 25.0, "SOL": 150.0}
for i in range(10):
# Add random volatility typical of crypto markets
current_positions = {k: round(v + random.uniform(-v*0.05, v*0.05), 4) for k, v in base_positions.items()}
# Capture snapshot
audit_state = capture_daily_snapshot(audit_state, current_positions)
# 3. Generate Summary
history_df = summarize_audit_trail(audit_state)
display(history_df.head(10))| timestamp | ticker | quantity | |
|---|---|---|---|
| 0 | 2026-06-09 08:22:20.270453 | BTC | 1.4946 |
| 1 | 2026-06-09 08:22:20.270453 | ETH | 24.3614 |
| 2 | 2026-06-09 08:22:20.270453 | SOL | 142.6465 |
| 3 | 2026-06-09 08:22:20.270487 | BTC | 1.4760 |
| 4 | 2026-06-09 08:22:20.270487 | ETH | 25.0296 |
| 5 | 2026-06-09 08:22:20.270487 | SOL | 153.1987 |
| 6 | 2026-06-09 08:22:20.270501 | BTC | 1.5228 |
| 7 | 2026-06-09 08:22:20.270501 | ETH | 24.0227 |
| 8 | 2026-06-09 08:22:20.270501 | SOL | 144.1460 |
| 9 | 2026-06-09 08:22:20.270512 | BTC | 1.5654 |
# 4. Visualization
plt.figure(figsize=(12, 6))
sns.lineplot(data=history_df, x='timestamp', y='quantity', hue='ticker', marker='o')
plt.title("Daily Position Snapshots Over Time")
plt.xlabel("Date")
plt.ylabel("Quantity Held")
plt.grid(True, linestyle='--', alpha=0.6)
plt.legend(title='Asset')
plt.show()Production Considerations
| Best Practice | Implementation Strategy |
|---|---|
| Persistent Storage | Offload snapshots from in-memory deque to an immutable database (e.g., AWS QLDB or S3 with Object Lock). |
| Data Integrity | Implement HMAC or digital signatures for every snapshot to prevent tampering. |
| Observability | Export loggers to a centralized logging system like ELK or CloudWatch. |
| Reconciliation | Periodically compare snapshots against clearing firm statements. |
Conclusion
We have implemented a functional audit snapshot system that:
- Initializes state using modular dictionaries.
- Captures data with built-in resilience (retries and jitter).
- Logs activity for transparency.
- Visualizes trends to identify potential anomalies in position reporting.