The ObServatory
A production-grade data platform I built and operate on my own VPS — automated collectors, a Postgres backend, cron-scheduled pipelines, and walk-forward validated machine learning models. This page is about the engineering, not the outcomes: how the system collects data, stores it, trains against it, and protects itself from the kinds of quiet mistakes that make a model look better than it really is.
Automated collectors pull structured data on a recurring schedule, covering roughly 50 distinct market types.
A Postgres backend on a VPS, with cron-scheduled pipelines handling collection, grading, and retraining.
LightGBM, negative-binomial, and Elo-based models, all validated walk-forward instead of with a random train/test split.
From raw collection to a trusted model
Six stages, each one built to fail loudly and recover cleanly instead of silently producing bad data or bad predictions.
Collectors
Scheduled jobs pull structured market data on a recurring basis.
Postgres storage
Everything lands in a relational schema built for time-series queries and joins.
Cron pipelines
Collection, grading, and retraining all run on their own scheduled cycles.
Walk-forward models
LightGBM, negative-binomial, and Elo-based models trained and validated in strict time order.
Self-healing graders
Outcome graders retry automatically and fall back to an SFTP source if the primary feed is unavailable.
Anti-leakage audits
Structural checks and cross-model comparisons look for signals a model shouldn't have access to yet.
I wanted a real system to practice the parts of data engineering and ML that don't show up in a single Jupyter notebook: keeping a pipeline running unattended, handling upstream data that arrives late or malformed, and building validation strict enough that a model's reported performance can actually be trusted. Sports data was a convenient domain for this because it updates constantly, has a clear ground truth once an event finishes, and makes it obvious when a pipeline has quietly broken.
Automated, scheduled, and structured
Collectors run on a schedule and pull structured data across roughly 50 market types, normalizing it into a consistent shape before it ever reaches the database.
PostgreSQL as the single source of truth
Everything lands in a Postgres backend running on a VPS, with a schema designed around time-series lookups so pipelines and models can query historical state without re-deriving it.
The models are a mix of LightGBM gradient-boosted trees, negative-binomial regression for count-style outcomes, and Elo-based rating systems, chosen per problem rather than forcing one algorithm to fit every case. The part I care about most is validation: every model is evaluated walk-forward, meaning it's only ever tested against data that came after what it was trained on. A random train/test split can hide the fact that a model is quietly using information it wouldn't have had in real time — walk-forward validation is the only way I trust to catch that.
Grading a prediction means matching it against a real-world outcome once the event finishes, and that step depends on an upstream results feed that isn't always available exactly when expected. The graders are self-healing: they retry on failure and fall back to an SFTP-based data source if the primary feed is down, so a single missed pull doesn't stall the entire pipeline or leave the record permanently ungraded.
The habit I trust most in this project is auditing my own results, not just accepting them. I built a structural verification process that checks features and labels for anything a model could see that it shouldn't have access to at prediction time. As part of that process, I ran a cold cross-model audit — comparing two independently built models against each other rather than each one against itself — and it caught a real data-integrity leak that a single-model check had missed. Finding and fixing that leak mattered more to me than any single accuracy number, because it's the kind of mistake that can make a broken model look like a good one.
A backtest is only as honest as its validation method
Walk-forward validation is more work to set up than a random split, but it's the difference between a number you can trust and a number that just looks good.
Pipelines fail; the question is how gracefully
Self-healing retries and a fallback data path turned upstream outages from a pipeline-stopping problem into a non-event.
The most useful bug I found, I went looking for
The cold cross-model audit wasn't triggered by something breaking — it was a deliberate check I built because I didn't want to trust a single model's word for its own performance.
Infrastructure is part of the modeling problem
Choosing Postgres, structuring the cron schedule, and designing the grading fallback path were as important to getting reliable results as any modeling decision.