From Prompt to Validated Strategy: AI That's Grounded in Your Real Data

"Write me a pairs mean-reversion strategy on BTC and ETH funding" is a sentence anyone can type. Getting back Python that actually runs on a specific engine, against specific datasets, with specific column names is a different problem entirely — and it's where most "AI writes your trading code" features quietly fall apart.

The failure mode is predictable. A general-purpose model writes plausible-looking code against an API it imagined: methods that don't exist, a dataset column called funding_rate when yours is called value, an import the sandbox forbids. It compiles in the model's head and dies in your engine. The demo dazzles; the feature frustrates.

We treat AI strategy generation as a grounding-and-validation problem, not a prompting problem. The principle, stated in the module itself: "feed the model MAXIMUM grounded context — the exact authoring API, the live dataset catalog, and REAL sample rows from the chosen dataset — then validate the output against the same sandbox the engine uses and do one automatic repair round on failure."

Grounding #1: the exact authoring API, rendered from source

The model isn't told about the API in prose someone wrote once. It's handed the API as it exists right now, rendered straight from the same source-introspection catalog the editor uses:

def render_api_reference() -> str:
    """Authoring surface rendered straight from apidoc.build_api_catalog so it
    stays in lockstep with the engine."""
    from development.fund.backtest.apidoc import build_api_catalog
    cat = build_api_catalog()
    ...

So when the model reaches for a helper, it's choosing from the real list of methods and their real signatures — not from its training-data memory of what a trading API "usually" looks like. The authoring contract is explicit and strict:

- Define a module-level `class Strategy(BaseStrategy)`. There is ONE base class.
- Allowed imports ONLY: pandas, numpy, math, ... development.fund.backtest.base, .indicators.
- No file/network/os access, no eval/exec, no dunder attribute tricks. It runs sandboxed.
- implement on_bar(self) and issue orders with helpers. Anything you DON'T touch is HELD.

Grounding #2: the live catalog and real sample rows

The bigger lever is data grounding. The model is shown the live dataset catalog — names, kinds, intervals, column maps — and then actual rows pulled from the dataset you chose:

def _dataset_context(fund_engine, t1_engine, dataset):
    """Catalog summary + REAL sample rows for the chosen dataset (best-effort)."""
    ...
    # Real sample rows so the model sees actual columns + value shapes.

This is the difference between hallucination and accuracy. When the model can see that the funding dataset's values live in a column called value, that symbols look like BTCUSDT, and what a row's magnitudes actually are, it stops guessing. It writes code against the data you have, not the data it assumes. It even gets a "pick one of these" hint of real available symbols, so it won't invent a ticker you don't carry.

Validation: the same sandbox the engine uses

Grounding raises the hit rate; it doesn't guarantee correctness. So nothing the model writes is trusted. The output is validated against the identical sandbox that gates hand-written strategies:

from development.fund.backtest.sandbox import validate_code

def validate_strategy(code, params=None, ...):
    ast_violations = validate_code(code)
    ...

If the AI tries a forbidden import or a dunder trick, it's caught by the same AST validator that would catch a human doing it. There's no privileged "AI path" with weaker rules — the machine's code clears exactly the bar yours does. And beyond the static check, the candidate is smoke-run: instantiated and stepped through enough synthetic bars past its warmup that on_bar is actually exercised under the engine's full-window gate. Code that parses but throws on the first bar is rejected before you ever see it.

One automatic repair round

When validation or the smoke-run fails, the system doesn't just surface an error — it feeds the failure back to the model for one repair attempt. The first generation's grounded context is cached, so the repair round (and any further generations in the cache window) is fast and cheap. Most first-try misses are mechanical — a wrong signature, a missed warmup — and a single grounded repair round catches the bulk of them.

It's also honest about its own limits. A response truncated at the token ceiling produces syntactically broken code, so that case is detected and surfaced as a clean error rather than handed back as a silently-broken strategy:

if getattr(msg, "stop_reason", None) == "max_tokens":
    raise RuntimeError("AI response was truncated (hit max_tokens) — retry or raise the limit")

Why this is the right kind of AI feature

The market is full of "AI writes your strategy" buttons that are really "AI writes a strategy" — for some imaginary engine, against some imaginary data. They demo well and deliver friction, because the gap between plausible code and runnable code is exactly the gap the demo skips.

Closing that gap is unglamorous work: render the real API, pull real sample rows, validate against the real sandbox, smoke-run it, repair once, fail loudly when truncated. None of that fits on a landing page. All of it is the difference between a toy and a tool.

The result is an AI feature you can actually lean on. Describe the idea in a sentence; get back code that imports what's allowed, calls methods that exist, references columns you have, and survives the first bar. Not because the model is magic — because it was grounded in your reality and checked against your rules.

---

Describe a strategy in plain English and get validated, ready-to-backtest code grounded in your own datasets. Generate a strategy →