Trading Pipelines — End-to-End Financial ML Framework
A proof-of-concept project that proposes an end-to-end architecture for a financial machine learning framework. It covers the full lifecycle — from raw market data ingestion and feature engineering to model training, hyperparameter optimization, model serving, and interactive visualization — all orchestrated as reproducible pipelines.
Architecture
The system is composed of five main layers:
Data Ingestion
Raw financial data is loaded into a local DuckDB database using dlt incremental pipelines, orchestrated by Apache Airflow:





