This is a sophisticated statistical arbitrage system for futures pairs trading, focusing on mean-reverting relationships with optimal entry/exit timing and rigorous risk management.

The system identifies and exploits temporary mispricings between cointegrated instruments, using various statistical methods and ML enhancements to improve trading signals. It includes functionality for:

• Pair selection and cointegration testing
• Spread calculation with dynamic hedge ratios
• Signal generation with ML-based enhancements
• Risk management with adaptive position sizing
• Paper trading with performance monitoring
• Comprehensive backtesting with transaction costs

For detailed system documentation, refer to these resources:

• System Architecture Diagram: Visual representation of system components
• PAIRS Design Document: Overall system design and architecture
• Data Flow Architecture: Data flow between system components
• Docker Architecture: Containerized distributed processing

• Next Steps: Current priorities and immediate focus items
• Implementation Status: Current state of implementation
• Implementation Notes: Notes on implementation details

• Cointegration Framework: Comprehensive cointegration framework design
• Statistical Methods: Statistical methods for cointegration testing
• Johansen Test Implementation: Detailed Johansen test implementation
• Engle-Granger Test Implementation: Detailed Engle-Granger test implementation
• Statistical Validation Methods: Methods for validating cointegration relationships
• Z-Score Strategy Implementation: Comprehensive Z-Score strategy documentation

• Cointegration Framework: Detailed component interactions
• Statistical Methods: Mathematical foundations
• Statistical Validation: Ensuring robustness

• Z-Score Strategy: Core strategy implementation
• Strategy Variants: Different strategy variations

• Backtest Implementation Guide: Detailed implementation guide
• Intraday Backtest: Intraday backtesting specifics

• Paper Trading Guide: Guide for paper trading setup and usage
• Configuration Guide: Configuration options and parameters
• Intraday ML System User Guide: End-user guide for ML system
• Troubleshooting Guide: Solutions for common issues
• IB Connection Troubleshooting: Interactive Brokers connectivity help

For a complete overview of all documentation, refer to the Documentation Guide.

The system provides a unified command-line interface through main.py with the following commands:

Find and analyze cointegrated pairs from your futures data:

python main.py analyze-pairs --tickers GC SI ZB ZN --min-correlation 0.7 --timeframe 1hour

Test trading strategies on historical data:

python main.py backtest --pairs GC SI ZB ZN --start-date 2023-01-01 --end-date 2023-12-31

Test intraday-specific strategies with enhanced features:

python main.py intraday-backtest --pairs GC SI --start-date 2023-01-01 --end-date 2023-12-31 --timeframe 5min --use-ml

Train machine learning models to enhance trading signals:

python main.py train-models --pair GC_SI --start-date 2023-01-01 --end-date 2023-12-31 --timeframe 5min

Train market regime classifier for adaptive parameter selection:

python main.py train-regime-classifier --tickers GC SI ZB ZN --timeframe 1day --n-regimes 3

Optimize strategy parameters for different market regimes:

python main.py optimize-parameters --pairs-file output/pairs_analysis.json --start-date 2023-01-01 --end-date 2023-12-31 --n-regimes 3

Test your strategy in a simulated environment:

python main.py paper-trade --config config/paper_trading.json --capital 100000 --test-mode

Prepare data for analysis and backtesting:

python main.py process-data --symbols GC SI ZB ZN --start-date 2020-01-01 --end-date 2023-12-31 --timeframe 5min

Launch the web API for monitoring and control:

python main.py api --host 0.0.0.0 --port 8000

Start a Celery worker for background tasks:

python main.py worker --concurrency 4 --queue default

1. Clone the repository
2. Install dependencies: pip install -r requirements.txt
3. Process your historical data: python main.py process-data --symbols GC SI ZB ZN --start-date 2020-01-01 --end-date 2023-12-31
4. Analyze pairs: python main.py analyze-pairs --tickers GC SI ZB ZN
5. Run a backtest: python main.py intraday-backtest --pairs GC SI --start-date 2023-01-01 --end-date 2023-12-31
6. Train ML models: python main.py train-models --pair GC_SI --start-date 2023-01-01 --end-date 2023-12-31
7. Start paper trading: python main.py paper-trade --config config/paper_trading.json --test-mode

A typical workflow might look like:

1. Process your futures data
2. Find cointegrated pairs with the pair analyzer
3. Run backtests to validate the strategy
4. Train ML models to enhance signal generation
5. Optimize parameters for different market regimes
6. Run paper trading with the optimized strategy
7. Monitor performance and refine the strategy

The system expects futures data in the following structure:

• /data/processed/: Processed futures data
• /data/models/: Trained ML models
• /data/results/: Backtest results and analysis
• /config/: System and strategy configuration files

The system consists of several components:

1. Asset Classes: Abstractions for futures, equities, and other asset types
2. Pair Trading: Pair selection, spread analytics, and cointegration testing
3. Signal Generation: Signal processing, z-score calculation, and filtering
4. ML Enhancements: Feature engineering, model training, intraday signal enhancement
5. Paper/Live Trading: Order execution, position management, performance tracking
6. Risk Management: Position sizing, stop loss management, exposure control
7. Backtesting: Backtesting engine, strategy optimization, performance metrics
8. Infrastructure: Docker containers, Celery tasks, monitoring dashboard

For a visual representation of the system architecture, see System Architecture Diagram.

The system uses Docker containers for distributed task processing:

# Start the Docker containers
./scripts/start-containers.ps1

# Stop the Docker containers
./scripts/stop-containers.ps1

# Submit a task to the system
./scripts/submit-task.ps1 -TaskType train-models -Pair GC_SI -Timeframe 1hour

For details on the Docker-based architecture, see Docker Architecture.

We are actively working on improving code quality by addressing technical debt in these areas:

1. Large Files Refactoring:

   • Breaking down large files into smaller, focused modules
   • Implementing design patterns to improve code organization

2. Complex Functions Simplification:

   • Extracting helper methods from complex functions
   • Applying the single responsibility principle

3. Duplicate Code Elimination:

   • Using template method pattern and inheritance
   • Creating base classes and mixins for common functionality

For details on our technical debt resolution plan, see Technical Debt Analysis.

Guidelines for contributing to the project:

1. Use consistent naming conventions with existing code
2. Add comprehensive tests for new features
3. Document your changes in the appropriate documentation files
4. Follow the code structure and patterns established in the project
5. Run tests before submitting changes

This project is proprietary and not licensed for public use. All rights reserved.