Market Sentiment Trading Orchestrator (MSTO)

A sophisticated trading system that monitors stock market movements, analyzes news sentiment, and executes trades based on multiple strategies. The system runs continuously, monitoring selected tickers for unusual price drops and correlating them with news sentiment to generate trading signals.

Table of Contents

• Overview
• Features
• Architecture
• Prerequisites
• Installation
• Configuration
• Usage Examples
• Strategy Development
• Deployment Guide
• Monitoring & Operations
• Troubleshooting
• Contributing
• License

Overview

MSTO is designed to automate trading decisions based on market sentiment and price movements. It:

• Monitors multiple stock tickers in parallel
• Analyzes price movements for unusual patterns
• Fetches and analyzes news sentiment
• Classifies market events
• Generates trading signals based on configurable strategies
• Executes trades through TradingView integration

Key Benefits

• Automated Trading: Reduce emotional trading with systematic strategies
• Sentiment Analysis: Leverage news sentiment for trading decisions
• Scalable Architecture: Handle multiple tickers and strategies in parallel
• Cloud-Ready: Deploy to AWS with built-in monitoring
• Extensible: Easy to add new strategies and data sources

Features

Core Functionality

Price Monitoring

• Real-time stock price tracking via yfinance
• Configurable lookback periods
• Unusual price movement detection
• Technical indicator support
  • Moving averages
  • Volume analysis
  • Price patterns

Sentiment Analysis

• News article fetching and processing
• NLTK-based sentiment scoring
• Event classification:
  • Earnings reports
  • Mergers & acquisitions
  • Management changes
  • Product launches
  • Legal events
  • Market movements

Trading Strategies

1. Fundamental Event-Driven Strategy

   # Example configuration
   {
       "min_impact_threshold": 0.3,
       "max_pe_ratio": 30.0,
       "min_drop_threshold": -5.0,
       "position_sizing": {
           "base_size": 100,
           "impact_multiplier": true,
           "max_position": 1000
       }
   }

   • Analyzes fundamental metrics:
     • P/E ratio evaluation
     • Price drops analysis
     • News impact assessment
   • Position sizing based on:
     • Impact magnitude
     • Drop significance
     • Market conditions

2. Simple Volatility Strategy

   # Example configuration
   {
       "min_drop_threshold": -2.0,
       "min_sentiment_threshold": -0.5,
       "position_size": 10,
       "max_positions": 3
   }

   • Volatility-based triggers:
     • Sudden price drops
     • High volume events
   • Sentiment correlation:
     • News sentiment scoring
     • Event impact analysis

Technical Features

Health Monitoring

• Real-time health checks
• Metric tracking:

  {
      "status": "healthy",
      "metrics": {
          "uptime_seconds": 3600,
          "total_checks": 1200,
          "errors": 0,
          "signals_generated": 15
      }
  }

Parallel Processing

• Multi-threaded ticker processing
• Strategy parallelization
• Configurable worker pools:

  # Example configuration
  {
      "max_parallel_tickers": 10,
      "strategy_workers": 4,
      "queue_size": 100
  }

Logging System

• Structured JSON logging
• Log levels:

  {
      "timestamp": "2023-12-21T10:30:00Z",
      "level": "INFO",
      "component": "strategy",
      "message": "Signal generated",
      "details": {
          "ticker": "AAPL",
          "action": "BUY",
          "quantity": 100
      }
  }

Architecture

System Components

msto/
├── core/                   # Core functionality
│   ├── analytics.py       # Market analysis and sentiment
│   │   ├── detect_unusual_drop()
│   │   ├── sentiment_analysis()
│   │   └── estimate_impact()
│   ├── data_sources.py    # Data fetching utilities
│   │   ├── fetch_stock_data()
│   │   ├── fetch_news()
│   │   └── get_fundamental_metrics()
│   ├── execution.py       # Trade execution
│   │   └── TradingViewIntegration
│   ├── health.py         # Health monitoring
│   │   ├── HealthStatus
│   │   └── HealthCheckHandler
│   └── orchestrator.py    # Main coordination logic
│       └── Orchestrator
├── strategies/            # Trading strategies
│   ├── base.py           # Strategy base class
│   │   └── Strategy
│   ├── fundamental_event_driven.py
│   │   └── FundamentalEventDrivenStrategy
│   └── simple_volatility.py
│       └── SimpleVolatilityStrategy
├── utils/                 # Utility modules
│   └── config.py         # Configuration management
└── cli.py                # Command-line interface

deploy/                   # Deployment configuration
├── ecs-task-definition.json
└── deploy.sh

tests/                    # Test suite
├── unit/                 # Unit tests
├── integration/          # Integration tests
└── conftest.py          # Test configuration

Data Flow

1. Data Collection

   graph LR
       A[Stock Data] --> C[Orchestrator]
       B[News Data] --> C
       C --> D[Analytics]
       D --> E[Strategies]
       E --> F[Execution]
   

   Loading

2. Signal Generation

   graph TD
       A[Price Drop Detection] --> D[Signal Generation]
       B[Sentiment Analysis] --> D
       C[Strategy Evaluation] --> D
       D --> E[Signal Validation]
       E --> F[Execution]
   

   Loading

Prerequisites

Local Development

• Python 3.10+

  python --version  # Should be 3.10 or higher

• Docker and Docker Compose

  docker --version
  docker-compose --version

• PostgreSQL 14+

  psql --version

AWS Deployment

1. AWS CLI configured with appropriate permissions:

   aws configure

2. Required IAM Permissions:

   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Effect": "Allow",
               "Action": [
                   "ecs:*",
                   "ecr:*",
                   "efs:*",
                   "logs:*",
                   "ssm:*"
               ],
               "Resource": "*"
           }
       ]
   }

3. AWS Services Setup:

   • ECS Fargate cluster
   • ECR repository
   • EFS filesystem
   • CloudWatch log groups
   • Systems Manager parameters

Installation

Local Development Setup

1. Clone the repository:

   git clone https://github.com/cenab/MSTO.git
   cd MSTO

2. Create and activate a virtual environment:

   # Linux/macOS
   python -m venv venv
   source venv/bin/activate
   
   # Windows
   python -m venv venv
   .\venv\Scripts\activate

3. Install development dependencies:

   pip install -e ".[dev]"

4. Install pre-commit hooks:

   pre-commit install

Docker Setup

1. Build the image:

   docker build -t msto:latest .

2. Run with docker-compose:

   # Development mode
   docker-compose up --build
   
   # Production mode
   docker-compose -f docker-compose.yml -f docker-compose.prod.yml up

3. Run specific configurations:

   # Custom tickers
   docker-compose run --rm msto --tickers AAPL MSFT --strategies all
   
   # Override environment
   POSTGRES_PASSWORD=custom docker-compose up

Configuration

Environment Variables

1. Development setup:

   cp .env.local .env

2. Configuration categories:

Core Settings

# Environment configuration
ENV=dev                    # dev/prod
TRADING_MODE=paper         # paper/live
DROP_LOOKBACK_DAYS=60     # Days to analyze

# Performance tuning
PARALLEL_WORKERS=4         # Number of worker threads
BATCH_SIZE=100            # Batch size for processing

API Keys

# External services
TRADINGVIEW_WEBHOOK_URL=https://your-webhook-url
NEWS_API_KEY=your-api-key

# Optional integrations
SLACK_WEBHOOK_URL=your-slack-webhook    # For notifications
TELEGRAM_BOT_TOKEN=your-bot-token      # For alerts

Database Configuration

# PostgreSQL settings
POSTGRES_USER=msto_user
POSTGRES_PASSWORD=secure_password
POSTGRES_DB=msto_dev
DB_CONNECTION_STRING=postgresql://msto_user:secure_password@localhost:5432/msto_dev

# Connection pool
DB_POOL_SIZE=5
DB_MAX_OVERFLOW=10

Strategy Parameters

# Common parameters
DEFAULT_MIN_IMPACT_THRESHOLD=0.3
MIN_DROP_THRESHOLD=-5.0

# Fundamental strategy
MAX_PE_RATIO=30.0
MIN_MARKET_CAP=1000000000

# Volatility strategy
MIN_SENTIMENT_THRESHOLD=-0.5
VOLATILITY_WINDOW=20

Monitoring Configuration

# Health checks
HEALTH_CHECK_PORT=8080
HEALTH_CHECK_INTERVAL=30

# Logging
LOG_LEVEL=INFO
LOG_FORMAT=json
LOG_FILE=/app/logs/msto.log

# Market hours
MARKET_HOURS_ONLY=true
MARKET_TIMEZONE=America/New_York

Usage Examples

Basic Usage

1. Run with default settings:

   msto --tickers AAPL MSFT GOOGL

2. Run specific strategies:

   msto --tickers AAPL --strategies fundamental volatility

3. Debug mode with detailed logging:

   msto --tickers AAPL --log-level DEBUG --mode once

Advanced Usage

1. Custom configuration file:

   msto --config custom.env --tickers AAPL MSFT

2. Multiple strategies with parameters:

   msto --tickers AAPL MSFT \
        --strategies fundamental volatility \
        --fundamental-threshold 0.4 \
        --volatility-window 30

3. Continuous monitoring with custom interval:

   msto --tickers AAPL MSFT GOOGL \
        --mode continuous \
        --interval 600 \
        --max-signals 5

Docker Usage

1. Development environment:

   docker-compose up --build

2. Production deployment:

   docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d

3. Custom configuration:

   docker-compose run --rm msto \
       --tickers AAPL MSFT \
       --strategies all \
       --log-level INFO

Strategy Development

Creating a New Strategy

1. Create a new strategy file:

   # msto/strategies/my_strategy.py
   from typing import List, Dict, Any, Optional
   from msto.strategies.base import Strategy
   
   class MyStrategy(Strategy):
       def __init__(self,
                   name: str = "MyStrategy",
                   min_impact_threshold: float = 0.3):
           super().__init__(name, min_impact_threshold)
           # Add custom initialization
   
       def process_data(self, data: Dict[str, Any]) -> Optional[List[Dict[str, Any]]]:
           if not self._validate_data(data):
               return None
   
           # Implement strategy logic
           signals = []
           # Generate signals based on data
           return signals

2. Add configuration parameters:

   # msto/utils/config.py
   MY_STRATEGY_THRESHOLD = float(os.getenv("MY_STRATEGY_THRESHOLD", "0.5"))

3. Register strategy in CLI:

   # msto/cli.py
   def get_strategies(config: dict, selected_strategies: List[str]) -> List[Strategy]:
       strategies = {
           "my_strategy": lambda: MyStrategy(
               min_impact_threshold=float(config.get("MY_STRATEGY_THRESHOLD", 0.5))
           )
       }

Testing Strategies

1. Unit tests:

   # tests/unit/test_my_strategy.py
   def test_my_strategy():
       strategy = MyStrategy()
       data = {
           "ticker": "AAPL",
           "drop": -3.0,
           "avg_sentiment": -0.5
       }
       signals = strategy.process_data(data)
       assert len(signals) == 1
       assert signals[0]["action"] == "BUY"

2. Integration tests:

   # tests/integration/test_strategies.py
   def test_strategy_integration(mock_data):
       strategy = MyStrategy()
       orchestrator = Orchestrator([strategy])
       orchestrator.process_ticker("AAPL")

Deployment Guide

AWS ECS Deployment

1. Initial setup:

   # Configure AWS credentials
   aws configure
   
   # Create ECR repository
   aws ecr create-repository --repository-name msto
   
   # Create ECS cluster
   aws ecs create-cluster --cluster-name msto-cluster

2. Store secrets:

   # Store API keys
   aws ssm put-parameter \
       --name /msto/tradingview_webhook_url \
       --value "your_url" \
       --type SecureString
   
   aws ssm put-parameter \
       --name /msto/news_api_key \
       --value "your_key" \
       --type SecureString

3. Deploy:

   # Run deployment script
   ./deploy/deploy.sh

Monitoring Setup

1. CloudWatch Logs:

   # Create log group
   aws logs create-log-group --log-group-name /ecs/msto
   
   # Set retention
   aws logs put-retention-policy \
       --log-group-name /ecs/msto \
       --retention-in-days 30

2. Alarms:

   # Create CPU utilization alarm
   aws cloudwatch put-metric-alarm \
       --alarm-name msto-cpu-utilization \
       --metric-name CPUUtilization \
       --namespace AWS/ECS \
       --statistic Average \
       --period 300 \
       --threshold 80 \
       --comparison-operator GreaterThanThreshold

Monitoring & Operations

Health Checks

1. Endpoint information:

   curl http://localhost:8080/health

   Response:

   {
       "status": "healthy",
       "last_check_time": "2023-12-21T10:30:00Z",
       "metrics": {
           "uptime_seconds": 3600,
           "total_checks": 1200,
           "errors": 0
       }
   }

2. Metrics available:

   • System health
   • Processing statistics
   • Strategy performance
   • Error rates

Logging

1. Log format:

   {
       "timestamp": "2023-12-21T10:30:00Z",
       "level": "INFO",
       "component": "strategy",
       "message": "Processing ticker",
       "details": {
           "ticker": "AAPL",
           "strategy": "fundamental",
           "duration_ms": 150
       }
   }

2. Log levels:

   • DEBUG: Detailed debugging information
   • INFO: General operational information
   • WARNING: Warning messages
   • ERROR: Error conditions
   • CRITICAL: Critical conditions

Performance Monitoring

1. System metrics:

   • CPU usage
   • Memory utilization
   • Network I/O
   • Disk usage

2. Application metrics:

   • Processing time per ticker
   • Strategy execution time
   • Signal generation rate
   • Error rates

Troubleshooting

Common Issues

1. Connection Issues

   # Check database connection
   psql $DB_CONNECTION_STRING -c "\conninfo"
   
   # Test API endpoints
   curl -v $TRADINGVIEW_WEBHOOK_URL

2. Performance Issues

   # Check system resources
   docker stats msto
   
   # Monitor logs
   tail -f logs/msto.log | jq .

3. Strategy Issues

   # Enable debug logging
   export LOG_LEVEL=DEBUG
   msto --tickers AAPL --strategies fundamental

Debug Tools

1. Health check:

   curl http://localhost:8080/health

2. Log analysis:

   # Search for errors
   grep -i error logs/msto.log | jq .
   
   # Monitor real-time
   tail -f logs/msto.log | jq 'select(.level=="ERROR")'

3. Database debugging:

   # Connect to database
   psql $DB_CONNECTION_STRING
   
   # Check signals table
   SELECT * FROM signals ORDER BY created_at DESC LIMIT 10;

Contributing

Development Workflow

1. Fork and clone:

   git clone https://github.com/cenab/MSTO.git
   cd MSTO

2. Set up development environment:

   python -m venv venv
   source venv/bin/activate
   pip install -e ".[dev]"
   pre-commit install

3. Create feature branch:

   git checkout -b feature/my-feature

4. Make changes and test:

   # Run tests
   pytest tests/
   
   # Check code style
   black .
   flake8 .
   mypy .

5. Submit pull request:

   • Write clear description
   • Include test coverage
   • Update documentation

Coding Standards

1. Style guide:

   • Follow PEP 8
   • Use type hints
   • Write docstrings
   • Keep functions focused

2. Testing:

   • Write unit tests
   • Include integration tests
   • Maintain test coverage

3. Documentation:

   • Update README
   • Add inline comments
   • Update API documentation

License

This project is licensed under the MIT License - see the LICENSE file for details.

Third-party Licenses

• yfinance: Apache 2.0
• NLTK: Apache 2.0
• PostgreSQL: PostgreSQL License