Websocket Price Feed
Implement a high-reliability real-time WebSocket market data feed client that maintains concurrent live order book and trade tick streams from multiple cryptocurrency exchanges with automatic reconnection logic, heartbeat monitoring, sequence number gap detection, and normalized data output.
WebSocket Real-time Price Feed: A Comprehensive Guide
Introduction
In the world of financial markets, information is critical, and its timeliness can directly impact trading decisions and investment strategies. A WebSocket real-time price feed provides a mechanism for receiving market data, such as cryptocurrency or stock prices, as soon as it becomes available, without the need for constant polling.
What are WebSockets?
WebSockets represent a fundamental shift from traditional HTTP request-response communication. Instead of short-lived connections where a client requests data and a server responds, WebSockets establish a persistent, full-duplex communication channel over a single TCP connection. This means both the client and the server can send and receive data independently and simultaneously, making it ideal for applications requiring real-time data exchange.
Why are they important for Price Feeds?
For real-time price feeds, WebSockets offer several advantages:
- Low Latency: Data is pushed from the server to the client instantly, reducing delays inherent in polling mechanisms.
- Reduced Overhead: After the initial handshake, WebSocket frames are smaller than HTTP requests, leading to more efficient network usage.
- Efficiency: A single, long-lived connection avoids the overhead of establishing new connections for each data update.
This notebook will guide you through understanding, implementing, and visualizing a WebSocket real-time price feed using Python.
Setting Up the Environment
We will use the websocket-client library in Python to interact with WebSocket APIs. This library provides a straightforward way to establish connections and handle incoming messages.
# Install the websocket-client library
!pip install websocket-client
!pip install websockets # Install websockets for the mock server
# Import necessary libraries
import websocket
import json
import time
import threading
from collections import deque
import matplotlib.pyplot as plt
import matplotlib.animation as animation
from datetime import datetime
# Imports for mock WebSocket server
import asyncio
import websockets
import randomRequirement already satisfied: websocket-client in /usr/local/lib/python3.12/dist-packages (1.9.0) Requirement already satisfied: websockets in /usr/local/lib/python3.12/dist-packages (15.0.1)
Connecting to a WebSocket Price Feed (Mock Data)
Due to persistent geographical restrictions preventing access to public cryptocurrency exchange WebSockets from this environment, we will now implement a local mock WebSocket server to simulate a real-time price feed. This allows us to proceed with demonstrating the core mechanics of connecting, receiving data, and visualization.
Our mock server will emit simulated price data, mimicking the structure of a real exchange's trade stream. The general process remains the same:
- Define callback functions (
on_message,on_error,on_close). - Create and start a
websocket.WebSocketAppinstance to connect to our local mock server. - Run the mock server to generate and send data.
- Run the client to establish the connection and start listening for messages.
This block defines the core callback functions for the WebSocket client:
price_data_queue: Adeque(double-ended queue) to efficiently store the latest price data points. This is useful for real-time visualization as it automatically discards older entries when themaxlenis reached.on_message: This function is called every time a new message is received from the WebSocket server. It parses the incoming JSON message, extracts the price and event timestamp, converts the timestamp to adatetimeobject, and appends the data toprice_data_queue.on_error: Handles any errors that occur during the WebSocket connection.on_close: Is invoked when the WebSocket connection is closed.on_open: Is called when the WebSocket connection is successfully established.
### Global Data Storage and WebSocket Client Callbacks
# A deque to store the last N prices for visualization
price_data_queue = deque(maxlen=100) # Store up to 100 data points
def on_message(ws, message):
"""
Callback function to handle incoming WebSocket messages.
Parses the JSON message and stores relevant price data.
Args:
ws: The WebSocketApp instance.
message: The raw message string received from the WebSocket.
"""
try:
json_message = json.loads(message)
# For our mock server, we will use 'p' for price and 'E' for event time
if 'p' in json_message and 'E' in json_message:
price = float(json_message['p'])
event_time_ms = int(json_message['E'])
# Convert milliseconds to seconds for datetime object
event_time = datetime.fromtimestamp(event_time_ms / 1000)
price_data_queue.append({'time': event_time, 'price': price})
# print(f"Time: {event_time.strftime('%H:%M:%S')}, Price: {price}") # Uncomment to see live prints
except Exception as e:
print(f"Error parsing message: {e}")
def on_error(ws, error):
"""
Callback function to handle WebSocket errors.
Args:
ws: The WebSocketApp instance.
error: The error object.
"""
print(f"WebSocket Error: {error}")
def on_close(ws, close_status_code, close_msg):
"""
Callback function to handle WebSocket closure.
Args:
ws: The WebSocketApp instance.
close_status_code: The status code of the closure.
close_msg: The close message.
"""
print("### closed ###")
def on_open(ws):
"""
Callback function when the WebSocket connection is opened.
Args:
ws: The WebSocketApp instance.
"""
print("### opened ###")The run_websocket_client function encapsulates the logic for initiating and managing the WebSocket client connection. It sets up the websocket.WebSocketApp with the defined callback functions, starts it in a separate thread (threading.Thread) to prevent blocking the main program, and maintains the connection for a specified duration using time.sleep() before closing it.
### WebSocket Client Execution Function
def run_websocket_client(url, duration_seconds=30):
"""
Runs a WebSocket client for a specified duration.
Args:
url (str): The WebSocket URL to connect to.
duration_seconds (int): The duration in seconds to keep the connection open.
"""
ws = websocket.WebSocketApp(url,
on_open=on_open,
on_message=on_message,
on_error=on_error,
on_close=on_close)
# Run in a separate thread to allow control flow in the main thread
wst = threading.Thread(target=ws.run_forever)
wst.daemon = True # Allow the main program to exit even if server thread is running
wst.start()
print(f"Listening for price updates for {duration_seconds} seconds...")
time.sleep(duration_seconds) # Keep the connection open for the specified duration
ws.close()
print("WebSocket client stopped.")This section provides the implementation for a mock WebSocket server. Due to potential restrictions on accessing external WebSocket APIs in this environment, a local server is created to simulate a real-time price feed. It includes:
_mock_server,_mock_server_thread,_server_loop: Global variables to manage the server instance and its execution thread.time_sync_mock_server: An asynchronous function that acts as the handler for client connections. It generates simulated price data with random fluctuations and sends updates every 100 milliseconds.start_server: An asynchronous function to actually start thewebsocketsserver.start_mock_server_in_thread: This function initiates the mock server in its own daemon thread. This allows the server to run concurrently with the client without blocking the main execution flow of the notebook.stop_mock_server_in_thread: This function gracefully stops the mock server by closing its connections and terminating its event loop.
### Mock WebSocket Server Implementation
import asyncio
import websockets
import random
import threading
import time
# Global variables
_mock_server = None
_mock_server_thread = None
_server_loop = None
async def time_sync_mock_server(websocket): # REMOVED the 'path' parameter
"""Mock WebSocket server that generates simulated price data.
Sends price updates every 100ms for a duration.
"""
current_price = 100.0 # Starting price for simulation
print("Mock WebSocket client connected.")
try:
for i in range(200): # Emit data points for a duration (e.g., 200 data points)
# Simulate price fluctuation
current_price += random.uniform(-0.5, 0.5)
# Ensure price doesn't go too low for realism
if current_price < 90.0:
current_price = 90.0 + random.uniform(0, 1)
message_data = {
"E": int(time.time() * 1000), # Event time in milliseconds
"p": f"{current_price:.2f}" # Price as string with 2 decimal places
}
await websocket.send(json.dumps(message_data))
await asyncio.sleep(0.1) # Simulate real-time updates every 100ms
except websockets.exceptions.ConnectionClosed:
print("Mock WebSocket server connection closed by client.")
except Exception as e:
print(f"Mock WebSocket server error: {e}")
finally:
print("Mock WebSocket client disconnected.")
async def start_server(host, port):
"""Start the WebSocket server."""
return await websockets.serve(time_sync_mock_server, host, port)
def start_mock_server_in_thread(host="127.0.0.1", port=8765):
"""
Starts the mock WebSocket server in a separate thread with its own event loop.
"""
global _mock_server, _mock_server_thread, _server_loop
if _mock_server:
print("Mock server already running.")
return
print(f"Attempting to start mock WebSocket server on ws://{host}:{port}")
def run_server():
global _mock_server, _server_loop
# Create a new event loop for this thread
_server_loop = asyncio.new_event_loop()
asyncio.set_event_loop(_server_loop)
# Start the server
_mock_server = _server_loop.run_until_complete(start_server(host, port))
print(f"Mock WebSocket server listening on ws://{host}:{port}")
# Run the event loop forever
_server_loop.run_forever()
# Start server in a daemon thread
_mock_server_thread = threading.Thread(target=run_server, daemon=True)
_mock_server_thread.start()
# Give the server time to start
time.sleep(1)
def stop_mock_server_in_thread():
"""
Stops the mock WebSocket server running in a separate thread.
"""
global _mock_server, _mock_server_thread, _server_loop
if _mock_server:
print("Stopping mock WebSocket server...")
def shutdown():
_mock_server.close()
_server_loop.stop()
# Schedule shutdown in the server's event loop
if _server_loop and _server_loop.is_running():
_server_loop.call_soon_threadsafe(shutdown)
# Wait a bit for cleanup
time.sleep(0.5)
_mock_server = None
_mock_server_thread = None
_server_loop = None
print("Mock WebSocket server stopped.")
else:
print("Mock WebSocket server not running.")This block orchestrates the entire process of running both the mock WebSocket server and the client:
- Define URL: Sets the local URL for the mock WebSocket server.
- Clear Data: Clears any previously collected data from
price_data_queueto ensure a fresh collection. - Start Server: Calls
start_mock_server_in_thread()to launch the mock server. - Wait:
time.sleep(2)provides a brief pause to allow the server to fully initialize. - Run Client: Calls
run_websocket_client()to connect to the mock server and collect price data for a specified duration (15 seconds in this case). - Stop Server: After the client has finished collecting data,
stop_mock_server_in_thread()is called to shut down the mock server. - Summary: Prints confirmation messages and the total number of price points collected, demonstrating the successful interaction between the mock server and client.
### Orchestrate Mock Server and Client Execution
# Define the local WebSocket server URL
MOCK_WS_SERVER_URL = "ws://127.0.0.1:8765"
# Clear previous data before starting new collection
price_data_queue.clear()
# Start the mock server in a separate thread
print("Starting mock WebSocket server...")
start_mock_server_in_thread(host="127.0.0.1", port=8765)
# Give the server time to fully initialize
time.sleep(2)
# Run the WebSocket client to collect data from the mock server
print("Running WebSocket client to connect to mock server...")
run_websocket_client(MOCK_WS_SERVER_URL, duration_seconds=15)
# Stop the mock server
stop_mock_server_in_thread()
print("Mock server and client execution complete.")
# Display collected data count
print(f"\nCollected {len(price_data_queue)} price points.")Starting mock WebSocket server... Attempting to start mock WebSocket server on ws://127.0.0.1:8765 Mock WebSocket server listening on ws://127.0.0.1:8765 Running WebSocket client to connect to mock server... Listening for price updates for 15 seconds... Mock WebSocket client connected. ### opened ### ### closed ### WebSocket client stopped. Stopping mock WebSocket server... Mock WebSocket server stopped. Mock server and client execution complete. Collected 100 price points.
Interpreting the Collected Data
After running the WebSocket client, the price_data_queue will contain a series of dictionaries, each representing a trade with its timestamp and price. Let's look at a few examples of the collected data.
print(f"Collected {len(price_data_queue)} price points.")
if price_data_queue:
print("First 5 collected data points (Mock Data):")
for i, data in enumerate(list(price_data_queue)[:5]):
print(f" {i+1}. Time: {data['time'].strftime('%H:%M:%S.%f')[:-3]}, Price: {data['price']}")
print(f"\nLast 5 collected data points (Mock Data):")
for i, data in enumerate(list(price_data_queue)[-5:]):
print(f" {i+1}. Time: {data['time'].strftime('%H:%M:%S.%f')[:-3]}, Price: {data['price']}")
else:
print("No data collected. Please ensure the mock server is running and the client connected successfully.")Collected 100 price points. First 5 collected data points (Mock Data): 1. Time: 09:34:10.338, Price: 99.3 2. Time: 09:34:10.438, Price: 99.29 3. Time: 09:34:10.539, Price: 99.41 4. Time: 09:34:10.639, Price: 99.53 5. Time: 09:34:10.740, Price: 99.15 Last 5 collected data points (Mock Data): 1. Time: 09:34:19.894, Price: 98.98 2. Time: 09:34:19.994, Price: 98.65 3. Time: 09:34:20.095, Price: 98.47 4. Time: 09:34:20.196, Price: 98.8 5. Time: 09:34:20.296, Price: 99.25
Visualizing Real-time Price Trends
Visualizing the price feed helps us understand market movements and identify patterns or volatility. We will plot the collected prices over time. Since we collected data for a short period, this plot will show the price fluctuations during that specific interval.
if price_data_queue:
# Extract times and prices
times = [data['time'] for data in price_data_queue]
prices = [data['price'] for data in price_data_queue]
# Create the plot
fig, ax = plt.subplots(figsize=(12, 6))
ax.plot(times, prices, marker='.', linestyle='-', color='skyblue',
markersize=4, linewidth=1.5, label='Price')
# Add styling
ax.set_title('Real-time Price Feed Simulation', fontsize=14, fontweight='bold')
ax.set_xlabel('Time', fontsize=12)
ax.set_ylabel('Price (USD)', fontsize=12)
ax.grid(True, linestyle='--', alpha=0.3)
ax.legend()
# Format x-axis for dates
fig.autofmt_xdate()
# Add some statistics
avg_price = sum(prices) / len(prices)
min_price = min(prices)
max_price = max(prices)
# Add text box with stats
stats_text = f'Avg: ${avg_price:.2f}\nMin: ${min_price:.2f}\nMax: ${max_price:.2f}'
ax.text(0.02, 0.98, stats_text, transform=ax.transAxes,
verticalalignment='top', bbox=dict(boxstyle='round', facecolor='wheat', alpha=0.5))
plt.tight_layout()
plt.show()
# Print statistics
print(f"\nPrice Statistics:")
print(f" Average Price: ${avg_price:.2f}")
print(f" Minimum Price: ${min_price:.2f}")
print(f" Maximum Price: ${max_price:.2f}")
print(f" Price Range: ${max_price - min_price:.2f}")
else:
print("Cannot create plot: No price data available.")Price Statistics: Average Price: $98.05 Minimum Price: $95.74 Maximum Price: $99.81 Price Range: $4.07
Visualization Interpretation (Mock Data)
The line plot above displays the simulated price data generated by our mock WebSocket server over the 30-second collection period. Each point on the graph represents a simulated trade price at a specific timestamp. We can observe:
- Simulated Price Fluctuations: The data shows a random walk behavior, mimicking the small, rapid price changes seen in real markets.
- Simulated Trend: Depending on the random fluctuations, a slight upward, downward, or flat trend might appear.
- Simulated Volatility: The degree of up and down movement reflects the simulated market volatility.
This visualization, though based on synthetic data, effectively demonstrates how real-time price feeds allow us to capture and analyze transient market dynamics, which is the core educational goal of this section.
Key Considerations for Real-time Price Feeds
While connecting to a WebSocket is straightforward, building robust applications requires considering several factors:
- Error Handling and Reconnection Logic: Connections can drop. Implement mechanisms to detect disconnections and automatically attempt to reconnect.
- Message Format and Parsing: Different exchanges might use varying JSON structures. Robust parsing logic is essential.
- Subscription Management: APIs often require explicit subscription messages to specify which assets or data types you want to receive.
- Rate Limits and Throttling: Even WebSockets can have rate limits on how many connections you can open or how many messages you can send (e.g., for subscriptions).
- Data Volume: High-frequency data streams can generate a massive amount of data. Efficient processing and storage are crucial.
- Scalability: For multiple assets or high-volume strategies, consider distributed architectures.
Practical Applications
Real-time price feeds are indispensable for a wide range of applications:
- Algorithmic Trading Bots: Bots rely on the latest prices to execute trades based on predefined strategies.
- Live Charting Applications: Websites and software that display dynamic price charts in real-time.
- Portfolio Trackers: Applications that update the value of a user's holdings instantly.
- Arbitrage Opportunities: Identifying price discrepancies across different exchanges in milliseconds.
- Market Sentiment Analysis: Analyzing high-frequency trade data to gauge market momentum.
Conclusion
WebSockets provide a powerful and efficient way to access real-time financial data, a cornerstone for modern trading and analytical applications. By understanding their fundamental principles and knowing how to implement them in Python, you can unlock a wealth of opportunities for building dynamic, responsive, and data-driven systems. Always remember to consider robust error handling, efficient data processing, and adherence to API guidelines when working with live market feeds.