V1-V2 Core Rules
- Data must be loaded before any analysis or feature calculation.
- Stock IDs range from 0-199, date IDs represent trading days.
- Target variable is price movement at market close.
- V2 features are Numba-accelerated for performance.
- Time-series CV uses a purge gap to prevent leakage.
- Zero-sum adjustment ensures market-neutral predictions.
- Export/verify follows Write-Then-Verify pattern.
- LightGBM is the primary model with MAE objective.
V3 Advanced ML Rules
- SHAP values require trained model and test data.
- SHAP importance can differ from gain-based importance.
- Optuna uses TPE Bayesian optimization, ideally with CV.
- Stacking ensembles rely on OOF predictions to avoid leakage.
- Walk-forward backtests simulate production conditions.
- Regime analysis identifies performance clusters.
V4-V5 Monitoring + Parallel Rules
- PSI < 0.1 indicates stability; PSI > 0.25 is significant drift.
- KS test uses alpha=0.05 by default for drift detection.
- Concept drift reflects feature-target relationship changes.
- Covariate shift uses a domain classifier (AUC > 0.6 indicates shift).
- Missing value spikes can signal pipeline issues.
- Parallel CV and backtests rely on ThreadPoolExecutor.
- max_workers should not exceed available CPU cores.
V6 Synthesis Rules
- Synthesis profiles: quick (explore), standard (model), comprehensive (full).
- Quick profile: load, summary, target analysis, report.
- Standard profile: features, model, basic report.
- Comprehensive profile: parallel features, CV, backtest, monitoring, full report.
- Deployment decisions: deploy, defer, reject, or retrain.
- Five deployment criteria: data quality, drift, CV MAE, Sharpe, alerts.
- Reports follow structured templates for analysis, model, and monitoring.