Energy Equity Research · engineering notes

Every figure on a research dashboard should be able to say where it came from, and how old it is

I started with a screen for Canadian energy stocks and ended up with a research dashboard over thousands of listings, six forecasting models and written reports from a local language model. The hard part was never the models. It was trust: free data that goes stale without saying so, analysis code that changes while its documentation does not, and generated text that will happily invent an oil price. These notes are about making each figure traceable, and about the silent failures that taught me why.

The short version
693commits
44working days, March to October 2026
3,216test functions in 45,339 lines of tests
27data-source clients behind the fallback chains
349web routes in the project's generated inventory
170numbered lessons in its retrospective archive

01Why I built it

Research on energy stocks draws on many kinds of data at once: daily prices, company fundamentals, commodity prices, interest rates and government energy statistics. Free sources for all of it exist, but each has its own limits and gaps, and Canadian listings are poorly covered by most of them.

What I kept running into was not a lack of data but a lack of trust in it. A tool built on uneven sources can show a confident number that is days old, or one computed from a column that quietly went missing. A model score, a price level or a written report is only worth reading if I can ask where it came from and get a real answer.

So the organising rule became that every figure on screen should be traceable to the data it was built from, the calculation that produced it, and how old both are. Where the system cannot support a number, it should say so instead of filling the gap. This is engineering for research, not investment advice, and these notes report no performance figures.

02Architecture at a glance

It is a Flask application in Python with a JavaScript front end of ten tabs and a separate admin panel. Data flows in one direction: scheduled jobs fetch, validate and cache, the analysis reads only the caches, and the pages read only the analysis.

Data layerFallback chains across the data-source clients, ranked by recent success and speed, with circuit breakers and daily quota counts.
Analysis and modelsFeatures from returns, fundamentals, macro sensitivities and technicals, six models, prediction intervals and a nine-gate fundamental screen.
Market contextA regime score, a hidden-Markov regime model and sector rotation layers built on long industry history.
TraceabilityStored prediction drivers, a cited per-stock narrative written without a language model, and an age on every row and report.
Outcome trackingPredictions recorded and filled in with realized returns at three, six and twelve months, feeding calibration and a scorecard against a naive baseline.
Grounded reportsOptional explanations, reports and a bull, bear and judge debate from a locally hosted model, checked number by number.
OperationsAn off-hours compute window, a run ledger with alerts, and tiered backups whose restore verifies a checksum manifest.

The one-way flow is what makes traceability possible. If a page could fetch for itself, it could show a number nothing else had validated or dated.

03Data that knows how old it is

Each kind of data has a chain of sources to try in order. The order is not fixed: sources are ranked by their recent success rate and speed, a failing source is skipped by a circuit breaker whose cooldown is saved so it survives a restart, and daily quotas are counted so a free tier is never used up by accident.

Bulk end-of-day price files go through an integrity gate before anything is written: the schema, the prices' sanity and duplicates are all checked, and an existing good file is never overwritten by a worse one.

Age is part of the figure. Each row shows when it was scored, prices carry a separate count of trading sessions since their last bar, and reports open with a banner that turns amber or red when a layer of data is past its window. During the day the site serves the overnight results rather than half-refreshed ones.

04The models, and keeping the future out

Six models estimate each stock's return over the next twelve months from monthly snapshots. Three are linear. Three are gradient-boosted trees, and two of those are trained to rank stocks rather than to predict a number, because the question being asked is mostly which names look better than which. The models are combined with weights taken from their accuracy on held-out data.

The labels overlap: a 12-month return measured each month shares eleven months with the next one. Ordinary cross-validation would let the model see the answer. So the folds leave a 12-month gap plus an embargo between training and testing.

Each forecast carries a range sized from the models' own past errors, computed separately for low, middle and high volatility, so a volatile small company gets a wider band than a utility. The range is built to adapt as real outcomes arrive, but twelve-month outcomes take a year to mature, so for now it uses a fixed setting.

Look-ahead bias was the recurring problem. An audit found fast-moving daily signals copied onto historical training rows, which let today's data flatter past predictions; they are now removed at the point of training. When validating price targets after the fact, the anchor is the last bar at or before the checkpoint, and the forward scan starts strictly after it. Getting either boundary backwards leaks the future into the answer.

05Tracing a number back

When a prediction is made, its top drivers are computed with SHAP, a method that splits a forecast into the contribution of each input, and stored on the prediction in plain words. Months later, when the realized return arrives, the stated reason and the outcome sit side by side.

For each stock, a conviction story assembles nine pillars, such as the model, the quality screen, the entry price and the sector's record, and every sentence carries the number it rests on. It uses no language model. A pillar with missing data counts as unknown, never as agreement, because an absence of evidence is not a confirmation.

Outcomes are recorded at three, six and twelve months. Calibration, a scorecard against a naive momentum baseline and a drift monitor all read from that record, and a claim that something improved has to pass a significance test that accounts for how many things were tried.

06Language models on a short leash

Written reports come from a model running locally. Left alone, a language model will produce a plausible oil price, a plausible percentage and a plausible citation. So each report is given the only figures it may cite, and every number in the output is checked against those figures. A section with an invented figure is regenerated once, and anything still unsupported is removed.

Getting that to actually work took several rounds. One daily report quoted three different oil prices in different sections. Two causes stood out. Example numbers written into the prompts were being copied as if they were data, so the examples now use placeholders. And the grounding step expected a list but was handed a table in production, so it received no real figures and checked nothing, while every unit test, built on lists, passed.

After that fix, the real oil price appeared throughout the report and every quoted value sat within a few percent of it. The rule I took from it is that a guard must be tested with the data in the shape production actually sends.

07Running at night, alerting by day

Heavy work, the model refresh, backfills, imports, reports and language-model batches, runs only in an off-hours compute window. During the day the site serves the overnight results and only light price and macro top-ups run. A click that asks for heavy work during the day is offered a choice: run now, or queue it for tonight.

Every stage of the nightly chain writes a row to a run ledger. A failure raises an alert on the first occurrence, again on the third in a row, and once on recovery, so a week-long outage is three messages rather than one a night or none at all.

Backups come in tiers with a checksum for every file, and a restore verifies them before it trusts anything. A past state can be reproduced rather than remembered.

08Keeping the analysis code honest

Analytical code decays quietly. A formula changes, the documentation does not, and nobody notices until a figure is wrong. So a status registry sits beside the code, with 112 features and 46 issues tracked by ID, and a diagnostics tool inside the app feeds it.

The registry includes a calculation audit log: each checked formula with the file and line it was checked against, and a verdict of verified, wrong or suboptimal. One of the first entries found a sign inverted in the update that adapts the prediction intervals.

The log has its own rule that every formula is checked again each quarter. Those checks date from the spring, and the code has moved since, so that pass is due. I mention it because a register that records when it was last right is only useful if it also tells you when it has gone stale.

A generated inventory lists every route, module and database migration, so the documentation can be checked against what exists rather than against memory.

09How it is tested

There are 3,216 test functions in 45,339 lines of tests. Most are fast unit tests on pure functions, which is why so much of the analysis is written to take its inputs as arguments and never raise.

The tests I value most were added after something failed in production: integration tests on realistically shaped rows, tests that feed both a table and a list to anything that can receive either, and tests that monkeypatch every data client to raise, proving a cache-only pass makes no network calls at all.

10Where it stands

Development ran from March to October 2026, most of it between March and July. The dashboard covers a daily overview, markets, industries, stocks, model predictions, risk, ETFs, a watch list and paper trading, with a shared panel that gathers everything known about one company.

The validation work is designed to mature with time. The calibration, the baseline scorecard and the drift monitor fill in as each cohort of predictions reaches its horizon, and the twelve-month cohort has not yet arrived.

Some limits are stated openly. Part of the training still applies today's fundamentals to past prices, because historical fundamentals cannot be rebuilt from free sources. An append-only log now records what is known each week, which will only pay off once it has accumulated.

11Nine lessons

The project keeps a retrospective archive of 170 numbered lessons. Nearly every expensive one shares a shape: nothing crashed, nothing looked wrong, and the product was quietly broken.

01Three weeks of empty pages under a log line that said non-fatal

I changed how cached analysis was stored, compressing it, and updated the code that wrote it. Six places that read it were not updated. On every restart the step that loads the analysis failed, logged the failure as non-fatal, and left the dashboard with no data. Two different symptoms were reported, an empty list and a missing report, with one cause.

The process kept running, so the label was technically true. The product was broken for about three weeks.

Now: Every read goes through one method that knows the format, the failure is logged as an error with a likely cause, a check asserts that the loaded analysis is not empty, and an integration test would have failed at the original commit.

a storage format change needs every reader found first, and non-fatal is a judgement about the process, not about the product.

02One hang that kept moving

After a large import, full refreshes started hanging with the process idle and progress frozen. Each time I wrapped the call that seemed stuck with a timeout, the hang reappeared somewhere else. Wrapping callers was chasing symptoms.

A stack dump of the live process showed every hang ending in the same place: a small timezone lookup inside the market-data library, made through a shared HTTP session with no default timeout.

Now: A default timeout is set on the shared session itself, so every request through it is bounded, and a test connects to an address that never answers and must fail quickly.

when a bug moves every time you fix it, look at a live stack before writing another workaround.

03One optional column dropped most of the universe

With the hang fixed, the broad pass completed and scored only 1,426 of 12,736 stocks. The feature code refused any price history without an adjusted-close column. The bulk price files do not carry one, so 11,386 of 13,463 files were dropped before a single feature was computed.

A first reproduction on six stocks passed, because those six happened to have the column from an earlier source.

Now: The code falls back to the close price when the adjusted close is absent, and records that those returns exclude dividends. A rerun on 300 never-scored stocks went from 0 to 274 produced.

a hard failure on an optional field silently amputates your input, and a reproduction must use the data that actually failed.

04A report that quoted three oil prices

One daily report cited three different oil prices, a set of sector returns that were all identical, and a share price that belonged to a different company. Some of the invented figures had been copied straight from example numbers written into the prompts.

Now: Prompts use placeholders instead of concrete example numbers, and every generator's output is checked against the real figures it was given.

a language model cannot tell an illustration from a datum, so never give it a number you do not want repeated.

05A guard that checked nothing in production

The grounding step that supplies real figures accepted only a list. In production it received a table, quietly treated it as empty, told the model no live numbers were available, and validated against nothing. Every isolated test used lists and passed. It had also been wired into only three of the eight report generators.

Now: The context builders accept either shape, both shapes are tested explicitly, and all generators go through one shared grounded call.

a passing unit test is not evidence that a guard works on the real data and the real call path.

06Eleven nights of failure that nobody heard

The nightly price fetch exited with an error on eleven weekdays in a row. The scheduler recorded it and the log said fatal, and nobody looked, because nothing was watching. Thousands of stocks quietly fell back to aging prices.

Now: Every scheduled stage writes a ledger row and alerts on the first failure, the third in a row and the recovery, and a stage that swallows a sub-step's failure must return a distinct error rather than success.

a scheduled job without an alert is a job that has already failed; writing to a log file is not monitoring.

07Two columns that were never written

While building target validation, I found the regime history table had never been filled, because its writer needed a database handle the caller never passed. Prediction conviction was empty on all 91,266 prediction rows.

Now: Both are written, and best-effort, so the fix can never break the analysis run.

a write that silently does nothing costs ground truth you can never recover, so fix it first.

08Today's signals in yesterday's training rows

An audit found that several fast-moving daily signals were being copied onto historical snapshot rows, so the model trained on past dates with information from today. Predictions looked better than they could ever be live.

Now: Those signals are no longer copied, and training clears them on the way in, which also covers snapshots already on disk.

any feature without a date of its own will eventually be read as if it had one.

09Inclusive on one side, strict on the other

Validating whether a price target held up comes down to two boundaries: the anchor is the last bar at or before the checkpoint, and the outcome is scored only on bars strictly after it. The same strict rule governs paper trading, where an order placed at a bar fills at the next bar's open.

Now: Each boundary has its own test, including one proving that a touch on the anchor bar does not count.

most look-ahead bias lives in a single comparison operator, so test the operator.

12What I would do differently

The platform does what I wanted it to. If I started again, I would spend the first weeks on the parts that only paid off after something had already gone wrong.