Problem
- AWS spot markets offer cost savings, but interruption risk varies sharply across regions, instance families, and market conditions.
- Scheduling systems need better forward-looking signals than broad historical summaries.
Back to projects
Project Detail
Cloud Arbitrage Index
Forecasting when cheap cloud compute might disappear.
Cloud Arbitrage Index is my Princeton senior thesis project. It forecasts interruption risk in cloud spot markets so users can make better cost-aware compute decisions. I built data pipelines around cloud market signals, prioritized exploration with a multi-armed bandit, and trained gradient-boosted models to identify higher-risk instance pools.
What I Built
- A Python-based forecasting platform that collects cloud market data, prioritizes where to explore next, and estimates near-term interruption risk.
- A modeling workflow designed to support cost-aware compute scheduling and pool selection.
Technical Architecture
- AWS SDK-based data collection and market monitoring across spot instance pools.
- Adaptive exploration logic using multi-armed bandit prioritization to focus on volatile pools.
- Gradient-boosted tree models trained on engineered market signals and interruption targets.
Key Challenges
- Defining useful predictive targets for interruption risk instead of only describing historical spot events.
- Deciding where to allocate limited exploration effort across a large and changing set of cloud markets.
Impact / Results
- Shows an ability to combine infrastructure context, experimentation strategy, and predictive modeling in one system.
- Highlights applied ML work tied directly to real operational decisions.