MLOps·Feature Engineering Pipeline·Advanced
Feast Feature Store Setup
Set up a Feast feature store for centralized ML feature management and serving with point-in-time correct historical feature retrieval for training dataset generation, online feature serving for real-time model inference, and cross-model feature reuse and consistency enforcement across the ML project portfolio.
data-storagefeature-engineeringmlops
Setting up a Feature Store with Feast
Introduction to Feast
Feast (Feature Store) is an open-source feature store for machine learning that allows you to manage and serve machine learning features consistently across training and serving environments. It acts as a centralized repository for features, ensuring that the features used to train a model are the same ones used to make predictions in production.
Why Use a Feature Store?
In machine learning, features are the independent variables or attributes that are used as input to a model. The process of creating and managing these features can be complex and often leads to several challenges:
- Feature Inconsistency (Training-Serving Skew): Features are often computed differently between offline training and online serving environments, leading to discrepancies in model performance.
- Feature Duplication: Data scientists often re-implement similar feature engineering logic across different projects, leading to wasted effort and potential errors.
- Feature Discoverability: It can be difficult to find and reuse existing features within an organization.
- Data Management: Managing the lifecycle of features, including backfills, versioning, and access control, becomes challenging at scale.
How Feast Solves These Problems
Feast addresses these challenges by providing a framework to:
- Define Features: Declare features and their transformations once, using a consistent API.
- Store Features: Materialize features into both an offline store (for historical data and model training) and an online store (for real-time serving).
- Serve Features: Provide a unified API to retrieve features for both training and online inference, ensuring consistency.
- Manage Feature Lifecycle: Handle feature versioning, backfills, and data consistency.
Setting up the Feast Environment
Before we dive into defining features, we need to install Feast and its dependencies. We'll also create a working directory for our Feast feature repository.
[12]
import os
import shutil # Import shutil for rmtree
# Install Feast. Note: May require restarting the runtime.
!pip install 'feast[sqlite]' pandas scikit-learn matplotlib numpy
# Define the path for our feature repository
FEAST_REPO_PATH = 'my_feature_repo'
# Clean up previous repo if it exists to ensure a fresh start
if os.path.exists(FEAST_REPO_PATH):
shutil.rmtree(FEAST_REPO_PATH)
print(f"Removed existing directory: {FEAST_REPO_PATH}")
# Initialize a Feast repository directly in FEAST_REPO_PATH.
# This creates FEAST_REPO_PATH/feature_store.yaml and FEAST_REPO_PATH/repo.py
print(f"Initializing Feast repository in {FEAST_REPO_PATH}...")
!feast init {FEAST_REPO_PATH} --template local
print(f"Feast repository initialized in {FEAST_REPO_PATH}")Requirement already satisfied: pandas in /usr/local/lib/python3.12/dist-packages (2.2.2)
Requirement already satisfied: scikit-learn in /usr/local/lib/python3.12/dist-packages (1.6.1)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.12/dist-packages (3.10.0)
Requirement already satisfied: numpy in /usr/local/lib/python3.12/dist-packages (2.0.2)
Requirement already satisfied: feast[sqlite] in /usr/local/lib/python3.12/dist-packages (0.63.0)
[33mWARNING: feast 0.63.0 does not provide the extra 'sqlite'[0m[33m
[0mRequirement already satisfied: click<9.0.0,>=7.0.0 in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (8.4.1)
Requirement already satisfied: colorama<1,>=0.3.9 in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (0.4.6)
Requirement already satisfied: dill~=0.3.0 in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (0.3.8)
Requirement already satisfied: protobuf>=4.24.0 in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (5.29.6)
Requirement already satisfied: Jinja2<4,>=2 in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (3.1.6)
Requirement already satisfied: jsonschema in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (4.26.0)
Requirement already satisfied: mmh3 in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (5.2.1)
Requirement already satisfied: pyarrow>=21.0.0 in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (24.0.0)
Requirement already satisfied: pydantic>=2.10.6 in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (2.12.3)
Requirement already satisfied: pygments<3,>=2.12.0 in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (2.20.0)
Requirement already satisfied: PyYAML<7,>=5.4.0 in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (6.0.3)
Requirement already satisfied: requests in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (2.32.4)
Requirement already satisfied: SQLAlchemy>1 in /usr/local/lib/python3.12/dist-packages (from SQLAlchemy[mypy]>1->feast[sqlite]) (2.0.50)
Requirement already satisfied: tabulate<1,>=0.8.0 in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (0.9.0)
Requirement already satisfied: tenacity<9,>=7 in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (8.5.0)
Requirement already satisfied: toml<1,>=0.10.0 in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (0.10.2)
Requirement already satisfied: tqdm<5,>=4 in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (4.67.3)
Requirement already satisfied: typeguard>=4.0.0 in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (4.5.2)
Requirement already satisfied: fastapi>=0.68.0 in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (0.136.3)
Requirement already satisfied: uvicorn<=0.34.0,>=0.30.6 in /usr/local/lib/python3.12/dist-packages (from uvicorn[standard]<=0.34.0,>=0.30.6->feast[sqlite]) (0.34.0)
Requirement already satisfied: uvicorn-worker in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (0.3.0)
Requirement already satisfied: gunicorn in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (26.0.0)
Requirement already satisfied: dask>=2024.2.1 in /usr/local/lib/python3.12/dist-packages (from dask[dataframe]>=2024.2.1->feast[sqlite]) (2026.3.0)
Requirement already satisfied: prometheus_client in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (0.25.0)
Requirement already satisfied: psutil in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (5.9.5)
Requirement already satisfied: bigtree>=0.19.2 in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (1.4.1)
Requirement already satisfied: pyjwt in /usr/local/lib/python3.12/dist-packages (from feast[sqlite]) (2.13.0)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.12/dist-packages (from pandas) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.12/dist-packages (from pandas) (2025.2)
Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.12/dist-packages (from pandas) (2026.2)
Requirement already satisfied: scipy>=1.6.0 in /usr/local/lib/python3.12/dist-packages (from scikit-learn) (1.16.3)
Requirement already satisfied: joblib>=1.2.0 in /usr/local/lib/python3.12/dist-packages (from scikit-learn) (1.5.3)
Requirement already satisfied: threadpoolctl>=3.1.0 in /usr/local/lib/python3.12/dist-packages (from scikit-learn) (3.6.0)
Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (1.3.3)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (4.63.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (1.5.0)
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (26.2)
Requirement already satisfied: pillow>=8 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (11.3.0)
Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (3.3.2)
Requirement already satisfied: cloudpickle>=3.0.0 in /usr/local/lib/python3.12/dist-packages (from dask>=2024.2.1->dask[dataframe]>=2024.2.1->feast[sqlite]) (3.1.2)
Requirement already satisfied: fsspec>=2021.09.0 in /usr/local/lib/python3.12/dist-packages (from dask>=2024.2.1->dask[dataframe]>=2024.2.1->feast[sqlite]) (2025.3.0)
Requirement already satisfied: partd>=1.4.0 in /usr/local/lib/python3.12/dist-packages (from dask>=2024.2.1->dask[dataframe]>=2024.2.1->feast[sqlite]) (1.4.2)
Requirement already satisfied: toolz>=0.12.0 in /usr/local/lib/python3.12/dist-packages (from dask>=2024.2.1->dask[dataframe]>=2024.2.1->feast[sqlite]) (0.12.1)
Requirement already satisfied: starlette>=0.46.0 in /usr/local/lib/python3.12/dist-packages (from fastapi>=0.68.0->feast[sqlite]) (0.52.1)
Requirement already satisfied: typing-extensions>=4.8.0 in /usr/local/lib/python3.12/dist-packages (from fastapi>=0.68.0->feast[sqlite]) (4.15.0)
Requirement already satisfied: typing-inspection>=0.4.2 in /usr/local/lib/python3.12/dist-packages (from fastapi>=0.68.0->feast[sqlite]) (0.4.2)
Requirement already satisfied: annotated-doc>=0.0.2 in /usr/local/lib/python3.12/dist-packages (from fastapi>=0.68.0->feast[sqlite]) (0.0.4)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.12/dist-packages (from Jinja2<4,>=2->feast[sqlite]) (3.0.3)
Requirement already satisfied: annotated-types>=0.6.0 in /usr/local/lib/python3.12/dist-packages (from pydantic>=2.10.6->feast[sqlite]) (0.7.0)
Requirement already satisfied: pydantic-core==2.41.4 in /usr/local/lib/python3.12/dist-packages (from pydantic>=2.10.6->feast[sqlite]) (2.41.4)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.12/dist-packages (from python-dateutil>=2.8.2->pandas) (1.17.0)
Requirement already satisfied: greenlet>=1 in /usr/local/lib/python3.12/dist-packages (from SQLAlchemy>1->SQLAlchemy[mypy]>1->feast[sqlite]) (3.5.1)
Requirement already satisfied: mypy>=0.910 in /usr/local/lib/python3.12/dist-packages (from SQLAlchemy[mypy]>1->feast[sqlite]) (2.1.0)
Requirement already satisfied: h11>=0.8 in /usr/local/lib/python3.12/dist-packages (from uvicorn<=0.34.0,>=0.30.6->uvicorn[standard]<=0.34.0,>=0.30.6->feast[sqlite]) (0.16.0)
Requirement already satisfied: httptools>=0.6.3 in /usr/local/lib/python3.12/dist-packages (from uvicorn[standard]<=0.34.0,>=0.30.6->feast[sqlite]) (0.8.0)
Requirement already satisfied: python-dotenv>=0.13 in /usr/local/lib/python3.12/dist-packages (from uvicorn[standard]<=0.34.0,>=0.30.6->feast[sqlite]) (1.2.2)
Requirement already satisfied: uvloop!=0.15.0,!=0.15.1,>=0.14.0 in /usr/local/lib/python3.12/dist-packages (from uvicorn[standard]<=0.34.0,>=0.30.6->feast[sqlite]) (0.22.1)
Requirement already satisfied: watchfiles>=0.13 in /usr/local/lib/python3.12/dist-packages (from uvicorn[standard]<=0.34.0,>=0.30.6->feast[sqlite]) (1.2.0)
Requirement already satisfied: websockets>=10.4 in /usr/local/lib/python3.12/dist-packages (from uvicorn[standard]<=0.34.0,>=0.30.6->feast[sqlite]) (15.0.1)
Requirement already satisfied: attrs>=22.2.0 in /usr/local/lib/python3.12/dist-packages (from jsonschema->feast[sqlite]) (26.1.0)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /usr/local/lib/python3.12/dist-packages (from jsonschema->feast[sqlite]) (2025.9.1)
Requirement already satisfied: referencing>=0.28.4 in /usr/local/lib/python3.12/dist-packages (from jsonschema->feast[sqlite]) (0.37.0)
Requirement already satisfied: rpds-py>=0.25.0 in /usr/local/lib/python3.12/dist-packages (from jsonschema->feast[sqlite]) (2026.5.1)
Requirement already satisfied: charset_normalizer<4,>=2 in /usr/local/lib/python3.12/dist-packages (from requests->feast[sqlite]) (3.4.7)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.12/dist-packages (from requests->feast[sqlite]) (3.18)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.12/dist-packages (from requests->feast[sqlite]) (2.5.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.12/dist-packages (from requests->feast[sqlite]) (2026.5.20)
Requirement already satisfied: mypy_extensions>=1.0.0 in /usr/local/lib/python3.12/dist-packages (from mypy>=0.910->SQLAlchemy[mypy]>1->feast[sqlite]) (1.1.0)
Requirement already satisfied: pathspec>=1.0.0 in /usr/local/lib/python3.12/dist-packages (from mypy>=0.910->SQLAlchemy[mypy]>1->feast[sqlite]) (1.1.1)
Requirement already satisfied: librt>=0.11.0 in /usr/local/lib/python3.12/dist-packages (from mypy>=0.910->SQLAlchemy[mypy]>1->feast[sqlite]) (0.11.0)
Requirement already satisfied: ast-serialize<1.0.0,>=0.3.0 in /usr/local/lib/python3.12/dist-packages (from mypy>=0.910->SQLAlchemy[mypy]>1->feast[sqlite]) (0.5.0)
Requirement already satisfied: locket in /usr/local/lib/python3.12/dist-packages (from partd>=1.4.0->dask>=2024.2.1->dask[dataframe]>=2024.2.1->feast[sqlite]) (1.0.0)
Requirement already satisfied: anyio<5,>=3.6.2 in /usr/local/lib/python3.12/dist-packages (from starlette>=0.46.0->fastapi>=0.68.0->feast[sqlite]) (4.13.0)
Initializing Feast repository in my_feature_repo...
/usr/local/lib/python3.12/dist-packages/matplotlib/_fontconfig_pattern.py:64: PyparsingDeprecationWarning: 'oneOf' deprecated - use 'one_of'
prop = Group((name + Suppress("=") + comma_separated(value)) | oneOf(_CONSTANTS))
/usr/local/lib/python3.12/dist-packages/matplotlib/_fontconfig_pattern.py:85: PyparsingDeprecationWarning: 'parseString' deprecated - use 'parse_string'
parse = parser.parseString(pattern)
/usr/local/lib/python3.12/dist-packages/matplotlib/_fontconfig_pattern.py:89: PyparsingDeprecationWarning: 'resetCache' deprecated - use 'reset_cache'
parser.resetCache()
/usr/local/lib/python3.12/dist-packages/matplotlib/_mathtext.py:45: PyparsingDeprecationWarning: 'enablePackrat' deprecated - use 'enable_packrat'
ParserElement.enablePackrat()
In /usr/local/lib/python3.12/dist-packages/matplotlib/mpl-data/stylelib/classic.mplstyle: 'parseString' deprecated - use 'parse_string'
In /usr/local/lib/python3.12/dist-packages/matplotlib/mpl-data/stylelib/classic.mplstyle: 'resetCache' deprecated - use 'reset_cache'
Creating a new Feast repository in [1m[32m/content/my_feature_repo[0m.
Feast repository initialized in my_feature_repo
Feast Core Concepts
To effectively use Feast, it's crucial to understand its core building blocks:
1. Entities
Entities are the primary identifiers for the data you want to retrieve features for. They are unique keys that link different features together. Examples include user_id, driver_id, customer_id, or product_id. Every feature in Feast is associated with one or more entities.
2. Data Sources
Data Sources define where your raw feature data resides. Feast supports various data sources like files (CSV, Parquet), databases (PostgreSQL, BigQuery, Snowflake), and stream processing systems (Kafka, Kinesis). When defining a feature view, you specify which data source it draws its raw data from.
3. Feature Views
Feature Views are the central concept in Feast. They represent a collection of features that are logically grouped together, typically based on a common entity and a shared data source. A Feature View defines:
- Entities: The entities that identify the rows of feature data.
- Features: The specific attributes that constitute the feature view.
- Data Source: The origin of the raw data for these features.
- Timestamp Column: A column in the data source that indicates when the feature value was valid. This is crucial for point-in-time correctness.
- Online/Offline Storage: How the features will be stored for online serving and offline training.
4. Feature Repository
The feature repository is a directory containing all the definitions of your Feast objects (entities, data sources, feature views). It's typically managed as code, allowing for version control and collaborative development. The repository usually contains:
feature_store.yaml: The configuration file for your feature store (e.g., where the online and offline stores are located).- Python files (e.g.,
repo.py): Python code that defines yourEntity,FeatureView, andDataSourceobjects.
Defining a Sample Data Source
For demonstration purposes, we'll create a mock Pandas DataFrame that simulates historical user data. This DataFrame will serve as our offline data source.
[2]
import pandas as pd
import numpy as np
from datetime import datetime, timedelta
# Generate mock data
num_users = 10
num_days = 30
# Create a list of user IDs
user_ids = [f'user_{i}' for i in range(num_users)]
# Generate timestamps for 30 days
start_time = datetime.now() - timedelta(days=num_days)
timestamps = [start_time + timedelta(days=i) for i in range(num_days)]
# Create an empty list to store feature data
feature_data = []
for user_id in user_ids:
for ts in timestamps:
# Simulate some user features
daily_transactions = np.random.randint(0, 10)
avg_transaction_value = np.random.uniform(10, 500)
is_premium_user = np.random.choice([0, 1], p=[0.8, 0.2])
feature_data.append({
'event_timestamp': ts,
'user_id': user_id,
'daily_transactions': daily_transactions,
'avg_transaction_value': avg_transaction_value,
'is_premium_user': is_premium_user
})
# Create a Pandas DataFrame
user_df = pd.DataFrame(feature_data)
user_df['event_timestamp'] = pd.to_datetime(user_df['event_timestamp'])
print("Sample User Features DataFrame:")
display(user_df.head())
print(f"DataFrame shape: {user_df.shape}")Sample User Features DataFrame:
| event_timestamp | user_id | daily_transactions | avg_transaction_value | is_premium_user | |
|---|---|---|---|---|---|
| 0 | 2026-05-11 07:30:56.002737 | user_0 | 5 | 244.693286 | 0 |
| 1 | 2026-05-12 07:30:56.002737 | user_0 | 4 | 250.290790 | 0 |
| 2 | 2026-05-13 07:30:56.002737 | user_0 | 8 | 206.920441 | 0 |
| 3 | 2026-05-14 07:30:56.002737 | user_0 | 3 | 54.251218 | 1 |
| 4 | 2026-05-15 07:30:56.002737 | user_0 | 6 | 240.758897 | 0 |
DataFrame shape: (300, 5)
Configuring the Feature Store (feature_store.yaml)
The feature_store.yaml file defines the overall configuration of your Feast feature store, including the type and location of your online and offline stores. For this example, we'll use SQLite for both offline and online stores for simplicity.
[19]
import os
# The Feast repository initializer 'feast init' creates a nested directory structure.
# The actual Feast project root containing feature_store.yaml and repo.py will be
# FEAST_REPO_PATH/feature_repo/.
# We'll define a variable for this actual project directory.
FEAST_PROJECT_ROOT = os.path.join(FEAST_REPO_PATH, 'feature_repo')
# Ensure the data directory within the actual Feast project root exists for the registry and stores
data_dir_within_project = os.path.join(FEAST_PROJECT_ROOT, 'data')
if not os.path.exists(data_dir_within_project):
os.makedirs(data_dir_within_project)
feast_yaml_content = f"""
project: my_feature_repo
registry: data/registry.db
provider: local
offline_store:
type: local
path: data/offline_store.db
online_store:
type: sqlite
path: data/online_store.db
"""
with open(os.path.join(FEAST_PROJECT_ROOT, 'feature_store.yaml'), 'w') as f:
f.write(feast_yaml_content)
print(f"Created {os.path.join(FEAST_PROJECT_ROOT, 'feature_store.yaml')} with content:")
print(feast_yaml_content)Created my_feature_repo/feature_repo/feature_store.yaml with content:
project: my_feature_repo
registry: data/registry.db
provider: local
offline_store:
type: local
path: data/offline_store.db
online_store:
type: sqlite
path: data/online_store.db
Defining Features (repo.py)
Now, we'll define our entities, data sources, and feature views in a Python file (e.g., repo.py) within our feature repository. This is where you specify the schema of your features and how they are derived from your data source.
[14]
repo_py_content = """
from datetime import timedelta
from feast import Entity, FeatureView, Field, FileSource, ValueType
from feast.types import Float32, Int32, String
# Define an entity for users
user_entity = Entity(
name="user_id",
value_type=ValueType.STRING,
description="The ID of the user"
)
# Define a data source for our user features (using a local file for simplicity)
# In a real-world scenario, this would point to a Parquet file, BigQuery table, etc.
user_data_source = FileSource(
path="./data/user_features.parquet", # This path will be relative to the repo directory
timestamp_field="event_timestamp",
)
# Define a FeatureView for user daily features
user_daily_feature_view = FeatureView(
name="user_daily_features",
entities=[user_entity],
ttl=timedelta(days=1), # Features are valid for 1 day
schema=[
Field(name="daily_transactions", dtype=Int32),
Field(name="avg_transaction_value", dtype=Float32),
Field(name="is_premium_user", dtype=Int32),
],
source=user_data_source,
)
"""
# Write repo.py into the actual Feast project root
with open(os.path.join(FEAST_PROJECT_ROOT, 'repo.py'), 'w') as f:
f.write(repo_py_content)
print(f"Created {os.path.join(FEAST_PROJECT_ROOT, 'repo.py')}")
# Save our sample DataFrame to a Parquet file, which will be used as the data source.
# Make sure the 'data' directory exists inside the Feast project root
data_dir_for_source = os.path.join(FEAST_PROJECT_ROOT, 'data')
if not os.path.exists(data_dir_for_source):
os.makedirs(data_dir_for_source)
user_df.to_parquet(os.path.join(data_dir_for_source, 'user_features.parquet'))
print(f"Saved user_df to {data_dir_for_source}/user_features.parquet")Created my_feature_repo/feature_repo/repo.py Saved user_df to my_feature_repo/feature_repo/data/user_features.parquet
Applying the Feature Store and Materializing Features
After defining your feature repository, you need to 'apply' it to Feast. This registers your entities and feature views with the Feast registry. Then, you can 'materialize' features, which means loading historical data from your offline data source into the online store, making it available for real-time lookups.
[29]
import os
from feast import FeatureStore
from datetime import datetime, timedelta
# Ensure the data directory within the actual Feast project root exists for the registry and stores
# This ensures the necessary directory structure is in place for the feature store files.
data_dir_within_project = os.path.join(FEAST_PROJECT_ROOT, 'data')
if not os.path.exists(data_dir_within_project):
os.makedirs(data_dir_within_project)
# Explicitly re-write the feature_store.yaml to ensure the correct configuration is active.
# This step prevents issues if a previous cell (like `feast init`) created an incorrect
# configuration or if the notebook cells were run out of order.
feast_yaml_content = f"""
project: my_feature_repo
registry: data/registry.db
provider: local
"""
feature_store_yaml_path = os.path.join(FEAST_PROJECT_ROOT, 'feature_store.yaml')
with open(feature_store_yaml_path, 'w') as f:
f.write(feast_yaml_content)
print(f"Ensured {feature_store_yaml_path} contains the correct configuration:")
print(feast_yaml_content)
# Initialize the feature store client with the correct project root
store = FeatureStore(repo_path=FEAST_PROJECT_ROOT)
# Apply the feature store definitions (this registers entities, feature views, etc.)
print("Applying Feast feature store definitions...")
!cd {FEAST_PROJECT_ROOT} && feast apply
print("Feast apply complete.")
# Materialize features into the online store
# We'll materialize features for the last 7 days for our demonstration.
print("Materializing features into the online store...")
end_date = datetime.now()
start_date = end_date - timedelta(days=7)
# Use the full path for materialization if running from outside the repo directory
!cd {FEAST_PROJECT_ROOT} && feast materialize {start_date.isoformat()} {end_date.isoformat()}
print("Feast materialize complete.")Ensured my_feature_repo/feature_repo/feature_store.yaml contains the correct configuration: project: my_feature_repo registry: data/registry.db provider: local Applying Feast feature store definitions...
/usr/local/lib/python3.12/dist-packages/jupyter_client/session.py:203: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC). return datetime.utcnow().replace(tzinfo=utc)
/usr/local/lib/python3.12/dist-packages/matplotlib/_fontconfig_pattern.py:64: PyparsingDeprecationWarning: 'oneOf' deprecated - use 'one_of'
prop = Group((name + Suppress("=") + comma_separated(value)) | oneOf(_CONSTANTS))
/usr/local/lib/python3.12/dist-packages/matplotlib/_fontconfig_pattern.py:85: PyparsingDeprecationWarning: 'parseString' deprecated - use 'parse_string'
parse = parser.parseString(pattern)
/usr/local/lib/python3.12/dist-packages/matplotlib/_fontconfig_pattern.py:89: PyparsingDeprecationWarning: 'resetCache' deprecated - use 'reset_cache'
parser.resetCache()
/usr/local/lib/python3.12/dist-packages/matplotlib/_mathtext.py:45: PyparsingDeprecationWarning: 'enablePackrat' deprecated - use 'enable_packrat'
ParserElement.enablePackrat()
In /usr/local/lib/python3.12/dist-packages/matplotlib/mpl-data/stylelib/classic.mplstyle: 'parseString' deprecated - use 'parse_string'
In /usr/local/lib/python3.12/dist-packages/matplotlib/mpl-data/stylelib/classic.mplstyle: 'resetCache' deprecated - use 'reset_cache'
/content/my_feature_repo/feature_repo/feature_definitions.py:27: DeprecationWarning: Entity value_type will be mandatory in the next release. Please specify a value_type for entity 'driver'.
driver = Entity(name="driver", join_keys=["driver_id"])
Applying changes for project my_feature_repo
/usr/local/lib/python3.12/dist-packages/feast/feature_store.py:690: RuntimeWarning: On demand feature view is an experimental feature. This API is stable, but the functionality does not scale well for offline retrieval
warnings.warn(
Created project [1m[32mmy_feature_repo[0m
Created entity [1m[32mdriver[0m
Created entity [1m[32muser_id[0m
Created feature view [1m[32mdriver_hourly_stats_fresh[0m
Created feature view [1m[32mdriver_hourly_stats[0m
Created feature view [1m[32muser_daily_features[0m
Created on demand feature view [1m[32mtransformed_conv_rate_fresh[0m
Created on demand feature view [1m[32mtransformed_conv_rate[0m
Created feature service [1m[32mdriver_activity_v2[0m
Created feature service [1m[32mdriver_activity_v1[0m
Created feature service [1m[32mdriver_activity_v3[0m
Created sqlite table [1m[32mmy_feature_repo_driver_hourly_stats_fresh[0m
Created sqlite table [1m[32mmy_feature_repo_driver_hourly_stats[0m
Created sqlite table [1m[32mmy_feature_repo_user_daily_features[0m
Feast apply complete.
Materializing features into the online store...
/usr/local/lib/python3.12/dist-packages/matplotlib/_fontconfig_pattern.py:64: PyparsingDeprecationWarning: 'oneOf' deprecated - use 'one_of'
prop = Group((name + Suppress("=") + comma_separated(value)) | oneOf(_CONSTANTS))
/usr/local/lib/python3.12/dist-packages/matplotlib/_fontconfig_pattern.py:85: PyparsingDeprecationWarning: 'parseString' deprecated - use 'parse_string'
parse = parser.parseString(pattern)
/usr/local/lib/python3.12/dist-packages/matplotlib/_fontconfig_pattern.py:89: PyparsingDeprecationWarning: 'resetCache' deprecated - use 'reset_cache'
parser.resetCache()
/usr/local/lib/python3.12/dist-packages/matplotlib/_mathtext.py:45: PyparsingDeprecationWarning: 'enablePackrat' deprecated - use 'enable_packrat'
ParserElement.enablePackrat()
In /usr/local/lib/python3.12/dist-packages/matplotlib/mpl-data/stylelib/classic.mplstyle: 'parseString' deprecated - use 'parse_string'
In /usr/local/lib/python3.12/dist-packages/matplotlib/mpl-data/stylelib/classic.mplstyle: 'resetCache' deprecated - use 'reset_cache'
Materializing [1m[32m3[0m feature views from [1m[32m2026-06-03 07:51:01+00:00[0m to [1m[32m2026-06-10 07:51:01+00:00[0m into the [1m[32msqlite[0m online store.
[1m[32mdriver_hourly_stats_fresh[0m:
[1m[32mdriver_hourly_stats[0m:
[1m[32muser_daily_features[0m:
Feast materialize complete.
Retrieving Features
Feast provides a unified API to retrieve features for both offline training and online serving. This is a core benefit, ensuring consistency.
1. Offline Feature Retrieval (for Model Training)
For model training, you typically need a batch of historical features. You provide Feast with 'entity rows' (a DataFrame containing entity IDs and their corresponding event timestamps), and Feast will join the correct historical feature values, ensuring point-in-time correctness.
[33]
from feast import FeatureStore
from datetime import datetime, timedelta
import pandas as pd
# Initialize the feature store client with the correct project root
store = FeatureStore(repo_path=FEAST_PROJECT_ROOT)
# Create a set of entity rows for which we want to retrieve historical features.
# This DataFrame should contain the entity_id (user_id) and an event_timestamp.
# The event_timestamp tells Feast at which point in time we want the features.
# Let's get features for a few users on specific dates for demonstration.
entity_rows = pd.DataFrame({
'user_id': ['user_1', 'user_2', 'user_1', 'user_3'],
'event_timestamp': [
datetime.now() - timedelta(days=5),
datetime.now() - timedelta(days=3),
datetime.now() - timedelta(days=2),
datetime.now() - timedelta(days=1)
]
})
print("Entity Rows for Offline Retrieval:")
display(entity_rows)
# Define the features we want to retrieve
feature_vector = store.get_historical_features(
entity_df=entity_rows, # Corrected argument name from entity_dataframe to entity_df
features=[
"user_daily_features:daily_transactions",
"user_daily_features:avg_transaction_value",
"user_daily_features:is_premium_user"
] # Corrected feature reference format
)
print("\nRetrieved Historical Features:")
display(feature_vector.to_df())Entity Rows for Offline Retrieval:
/usr/local/lib/python3.12/dist-packages/jupyter_client/session.py:203: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC). return datetime.utcnow().replace(tzinfo=utc)
| user_id | event_timestamp | |
|---|---|---|
| 0 | user_1 | 2026-06-05 07:57:50.702910 |
| 1 | user_2 | 2026-06-07 07:57:50.702920 |
| 2 | user_1 | 2026-06-08 07:57:50.702922 |
| 3 | user_3 | 2026-06-09 07:57:50.702924 |
Retrieved Historical Features:
/usr/local/lib/python3.12/dist-packages/jupyter_client/session.py:203: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC). return datetime.utcnow().replace(tzinfo=utc)
| user_id | event_timestamp | daily_transactions | avg_transaction_value | is_premium_user | |
|---|---|---|---|---|---|
| 0 | user_1 | 2026-06-05 07:57:50.702910+00:00 | 1 | 147.692392 | 1 |
| 1 | user_2 | 2026-06-07 07:57:50.702920+00:00 | 5 | 216.515473 | 0 |
| 2 | user_1 | 2026-06-08 07:57:50.702922+00:00 | 0 | 331.462977 | 0 |
| 3 | user_3 | 2026-06-09 07:57:50.702924+00:00 | 8 | 100.477556 | 0 |
2. Online Feature Retrieval (for Real-time Inference)
For real-time model inference, you need the latest feature values for a specific entity. Feast's online store provides low-latency access to these features.
[34]
from feast import FeatureStore
import pandas as pd
# Initialize the feature store client with the correct project root
store = FeatureStore(repo_path=FEAST_PROJECT_ROOT)
# Define a list of entity keys for which to retrieve online features.
# Each dictionary represents one entity and its key(s).
entity_keys = [
{"user_id": "user_0"},
{"user_id": "user_5"},
{"user_id": "user_9"}
]
# Retrieve the latest online features
# Note: The features must have been materialized to the online store already.
online_features = store.get_online_features(
features=[
"user_daily_features:daily_transactions",
"user_daily_features:avg_transaction_value",
"user_daily_features:is_premium_user",
],
entity_rows=entity_keys,
).to_dict()
print("\nRetrieved Online Features:")
# The output is a dictionary. We can convert it to a DataFrame for better readability.
online_features_df = pd.DataFrame(online_features)
display(online_features_df)Retrieved Online Features:
/usr/local/lib/python3.12/dist-packages/jupyter_client/session.py:203: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC). return datetime.utcnow().replace(tzinfo=utc)
| user_id | is_premium_user | daily_transactions | avg_transaction_value | |
|---|---|---|---|---|
| 0 | user_0 | 0 | 3 | 406.343414 |
| 1 | user_5 | 1 | 6 | 458.339996 |
| 2 | user_9 | 0 | 4 | 483.822235 |
Visualizing Feature Distributions
Visualizations help us understand the characteristics of our features. Here, we'll visualize the distribution of daily_transactions and avg_transaction_value from our sample data.
[8]
import matplotlib.pyplot as plt
import seaborn as sns
import numpy as np
# Ensure the historical features DataFrame is available for plotting
# We'll use the user_df directly as it represents our raw feature data before Feast processing.
fig, axes = plt.subplots(1, 2, figsize=(15, 6))
fig.suptitle('Distribution of User Features', fontsize=16)
# Plot 1: Distribution of Daily Transactions
sns.histplot(user_df['daily_transactions'], bins=np.arange(0, user_df['daily_transactions'].max() + 2) - 0.5, kde=True, ax=axes[0])
axes[0].set_title('Distribution of Daily Transactions')
axes[0].set_xlabel('Number of Daily Transactions')
axes[0].set_ylabel('Frequency')
axes[0].grid(axis='y', linestyle='--', alpha=0.7)
axes[0].set_xticks(np.arange(0, user_df['daily_transactions'].max() + 1))
# Plot 2: Distribution of Average Transaction Value
sns.histplot(user_df['avg_transaction_value'], bins=20, kde=True, ax=axes[1])
axes[1].set_title('Distribution of Average Transaction Value')
axes[1].set_xlabel('Average Transaction Value')
axes[1].set_ylabel('Frequency')
axes[1].grid(axis='y', linestyle='--', alpha=0.7)
plt.tight_layout(rect=[0, 0.03, 1, 0.95]) # Adjust layout to prevent suptitle overlap
plt.show()
print("\nInterpretation:")
print("The histograms show the frequency of different values for 'daily_transactions' and 'avg_transaction_value'.")
print("For daily transactions, we see a distribution centered around lower values, with fewer users having a high number of transactions.")
print("The average transaction value shows a more spread-out distribution, indicating a variety of transaction sizes among users.")
Interpretation: The histograms show the frequency of different values for 'daily_transactions' and 'avg_transaction_value'. For daily transactions, we see a distribution centered around lower values, with fewer users having a high number of transactions. The average transaction value shows a more spread-out distribution, indicating a variety of transaction sizes among users.
/usr/local/lib/python3.12/dist-packages/jupyter_client/session.py:203: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC). return datetime.utcnow().replace(tzinfo=utc)
Visualizing Feature Trends Over Time
Understanding how features change over time is crucial, especially for time-series models. Here, we'll visualize the average daily transactions and average transaction value for a single user over our simulated period.
[9]
import matplotlib.pyplot as plt
import seaborn as sns
# Select a single user for time-series visualization
sample_user_id = 'user_1'
single_user_df = user_df[user_df['user_id'] == sample_user_id].sort_values('event_timestamp')
fig, axes = plt.subplots(2, 1, figsize=(14, 10), sharex=True)
fig.suptitle(f'Feature Trends for {sample_user_id} Over Time', fontsize=16)
# Plot 1: Daily Transactions Over Time
sns.lineplot(x='event_timestamp', y='daily_transactions', data=single_user_df, marker='o', ax=axes[0])
axes[0].set_title('Daily Transactions')
axes[0].set_ylabel('Number of Daily Transactions')
axes[0].grid(True, linestyle='--', alpha=0.7)
# Plot 2: Average Transaction Value Over Time
sns.lineplot(x='event_timestamp', y='avg_transaction_value', data=single_user_df, marker='o', color='orange', ax=axes[1])
axes[1].set_title('Average Transaction Value')
axes[1].set_xlabel('Date')
axes[1].set_ylabel('Average Transaction Value')
axes[1].grid(True, linestyle='--', alpha=0.7)
plt.xticks(rotation=45)
plt.tight_layout(rect=[0, 0.03, 1, 0.95])
plt.show()
print("\nInterpretation:")
print(f"These line plots show the daily fluctuations in 'daily_transactions' and 'avg_transaction_value' for '{sample_user_id}'.")
print("Such visualizations help in identifying trends, seasonality, or anomalies in individual user behavior, which can be critical for feature engineering or model diagnostics.")
print("For example, a sudden drop or spike might indicate a change in user activity that could impact model predictions.")
Interpretation: These line plots show the daily fluctuations in 'daily_transactions' and 'avg_transaction_value' for 'user_1'. Such visualizations help in identifying trends, seasonality, or anomalies in individual user behavior, which can be critical for feature engineering or model diagnostics. For example, a sudden drop or spike might indicate a change in user activity that could impact model predictions.
/usr/local/lib/python3.12/dist-packages/jupyter_client/session.py:203: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC). return datetime.utcnow().replace(tzinfo=utc)
Conclusion and Cleanup
This notebook has provided a step-by-step guide on setting up a feature store with Feast, from defining features and data sources to materializing and retrieving them for both offline training and online inference. By centralizing feature management, Feast helps ensure consistency, reduces engineering overhead, and accelerates ML development.
To clean up the created Feast repository and data, you can run the command below. This will delete the my_feature_repo directory and its contents.
[10]
import shutil
# Clean up the Feast repository directory
if os.path.exists(FEAST_REPO_PATH):
shutil.rmtree(FEAST_REPO_PATH)
print(f"Cleaned up: Removed directory {FEAST_REPO_PATH}")
else:
print(f"Directory {FEAST_REPO_PATH} does not exist. No cleanup needed.")Cleaned up: Removed directory my_feature_repo