 |
algotradelib 0.0.0.9000
C++ header-only library for algorithmic trading
|
|
algotradelib 0.0.0.9000
C++ header-only library for algorithmic trading
|
A backtester is the easiest place in a library to produce confident, wrong numbers. This page is about what this one assumes, what it refuses to claim, and which of its figures you should distrust first.
See Writing a strategy for the strategy side and Binding a broker for running one live.
The third configuration of Runner, after DryRunner and the session-backed one. A strategy's source is identical in all three; if it were not, the surface would be wrong.
A fill never comes from the bar that produced the order. NextBarFillModel::decide takes the next bar and nothing else — it has no way to see the signal's own bar, so an engine wanting to cheat would have to pass the wrong one deliberately.
That is why Runner's executor accepts an order without filling it. The ordering inside feed is the whole correctness argument:
Reversing 1 and 5 is exactly the lookahead defect. Doing 3 before 2 would value a portfolio not yet updated by its own fills.
An order submitted on the last bar never fills, and is counted in caveats_.unfilled_. There is no next bar, and inventing one would be inventing data.
| Behaviour | Why | |
|---|---|---|
| Market order | fills at the next bar's open | the only price the next bar states before trading |
| Buy limit | fills if the bar's low reached it | a bar that never touched the price is one the order sat through |
| Sell limit | fills if the bar's high reached it | the same, mirrored |
| Limit price | the limit itself under AT_LIMIT — or, when the limit was marketable through the whole bar, the bar's worst price instead (a buy its high, a sell its low), which is better than the limit but never better than anything the bar traded | see below; a price the bar never showed is not a fill |
| Slippage on a limit | never carries the fill through its limit or out of its bar | a limit order that could only have traded worse did not trade worse; caveats_.limit_fills_bounded_ counts both |
| Missed limit | stays queued for the rest of its session, then expires unfilled — on daily bars, after the one bar it can trade against | discarding it at once would turn every limit into fill-or-kill; keeping it past the close would make it good-till-cancelled, which a DAY order is not |
The limit-price choice is the one to understand. A bar opening at 69,000 through a buy limit of 70,000 would often fill better in reality. Claiming that improvement assumes the order was in the book before the auction and that volume existed at that level — neither of which a bar can tell you. Refusing it understates a result; claiming it overstates one, and a backtester that flatters is worse than one that disappoints.
A DAY order lives one session, as it does at a broker. Its session is that of the first bar it can trade against: a decision at a daily bar's close trades in the next session, and so does a decision part way through a daily bar — a quote at 10:00 — because that bar's range includes the hours before it. That is a one-session shift, the same one a market order decided there takes to the next open, and the order is listed as working until that shifted session ends: a quote handler that places only when nothing is working therefore restates every other session on daily bars, where live it would restate every session — replay such a strategy on minute bars. The order expires when a bar ends its session: any bar of a day or longer, or a minute bar ending at or after the SessionWindow's close; a minute replay missing its last bar expires it before the next session's first bar can fill it. report.caveats_.expired_day_orders_ counts them. So a limit strategy on daily bars fills less often than if a missed order stayed queued, and a protective stop has to be restated each session, as it does live. Three limits:
So AT_LIMIT, the default, never credits the better open. It is still not "never better than the limit": a limit marketable through the whole bar is credited the bar's worst price — a buy limited at 72,000 on a bar whose high is 71,000 fills at 71,000 — because the limit itself is a price nobody traded. report.caveats_.limit_fills_bounded_ counts every fill the bar moved.
LimitFillPrice::AT_OPEN_IF_BETTER is the optimistic variant: it credits the open when the bar opened through the limit, opt-in because the assumption is yours to make. Whichever you use, report.caveats_.limit_fill_ says which.
A stop-limit order is dormant until a bar reaches its trigger, then the limit it becomes:
| Behaviour | Why | |
|---|---|---|
| Release | a sell stop when the bar's low reaches the trigger; a buy stop when its high does | the universal convention; neither broker documents it, and caveats_.stop_releases_ counts how often it was used |
| Opened through the trigger | priced as a limit against the whole bar, under your LimitFillPrice, and dated from the open when the open was marketable | the whole bar followed the release |
| Any stop fill | credited a price inside the bar and never through the limit — a sell no lower than the higher of the low and its limit, a buy no higher than the lower of the high and its limit | an AT_LIMIT credit, or slippage, could otherwise name a price the bar never showed |
| Released inside the bar, limit marketable at the trigger | fills on that bar at the worst price the bar traded that the limit allows — a sell the higher of its limit and the low, a buy the lower of its limit and the high | the price at release was the trigger; crediting the limit itself could name a price the bar never reached |
| Released inside the bar, limit not marketable there | fills at the limit on that bar only if the close is at or through it, else rests for the rest of its session — on daily bars it expires with that bar | the close is the one price known to come after the release |
| A bar with no open | a release is always "inside the bar" | a close cannot stand in for an open the bar did not state |
| Slippage | never carries a stop fill out of its bar or through its limit — the same bound as a plain limit | a limit order that could only have traded worse did not trade |
caveats_.stop_fills_intrabar_ counts the fills that relied on a release inside the bar. Each assumes the price moved continuously through the trigger; a jump past both trigger and limit would have left the stop released and unfilled. A working stop is shown to the strategy as a stop with its trigger, released or not, because neither broker's listing says which.
Backtester::feed takes a strategy::QuoteEvent as well as a BarEvent, so on_quote fires in a replay. It did not before: QuoteEvent appeared nowhere in backtest/, while PollKind::QUOTES is a supported live cadence — so a strategy that acts between bars ran live having been evaluated against nothing. Interleave quotes and bars in timestamp order yourself, exactly as you already interleave bars across instruments.
Sketched — bar and quote stand for events you build from your own data:
A quote does the same thing in both modes, because both go through the same function. Runner::feed(const QuoteEvent&) sets the price a market order is valued at, advances the decision clock to the quote's own instant, and calls the handler; live, the quote poll calls exactly that. What a replay arranges around it is an approximation of the rest of a live iteration, and the last three rows below say which way each one errs:
| Live | Replay | |
|---|---|---|
| Updates the price a market order is valued at | yes | yes, the same call |
| Advances the clock a decision is stamped with | yes | yes, the same call |
| Re-derives the market phase at that instant | yes, before every due poll | yes |
| Announces an order placed from the handler | when the working-order poll is due | yes, always |
| Marks the book, records an equity point, fills anything | no ledger exists live | no |
The phase row matters more than it looks: RiskLimits::permitted_phases_ defaults to {OPEN}, so a replay that derived the phase only from bars would have shown a mid-session quote the previous close's phase and refused an order a live run would have sent. A change is announced before on_quote here and after all polls live, which is the ordering feed(BarEvent) has always used; an order placed from on_phase is therefore visible to on_quote in a replay and would not yet exist live. And the announcement of an order is unconditional here and conditional live, on the working-order poll being due at all.
So an order placed from on_quote, if it fills at all, fills against a later bar, at that bar's price — and, where this engine can establish when a bar's period began, specifically against one that began at or after the decision. For a quote part-way through a session that is not that session's bar but the next one. A period that began earlier contains the decision, and its open is a price from before the strategy decided anything. Today's daily bar opened at 09:00, and crediting a 09:00 price to a 10:30 decision would be lookahead with the sign flipped. Live, the same order reaches the broker immediately and can trade at once. ReportCaveats::quotes_ is non-zero on any run that fed a quote, and that is what it is telling you. A fill model that executes against a quote is a different design with its own argument, and this is not it.
A calendar period boundary the engine cannot establish makes the check impossible, and the report says so rather than guessing. It happens on either side of the comparison. A close-stamped weekly, monthly or yearly bar does not state when its period began — and nothing in feed's contract lets the engine derive it, as three attempts to infer it from a previous bar each proved. An open-stamped bar of any calendar interval does not state when its period ended either: that is bar_period_end's nominal 24 hours, 7, 31 or 366 days, which is what every shipped adapter produces for every non-minute bar. So an adapter-fetched monthly or yearly replay carries this on most of its fills — though not all: the nominal end is an upper bound, so wherever the next period's start reaches it the check still passes, which every Monday-to-Monday week does.
Where it does not run: the fill is allowed on the older rule that an order never fills against the bar that produced it, and ReportCaveats::unchecked_fill_causality_ counts every such decision. Non-zero means those fill prices are not guaranteed to come from after the decision, and any dated order can be one of them — a quote is the obvious way in, and a bars-only replay reaches it too.
You can discharge it yourself: with no quotes, one interval per instrument, and series whose periods do not overlap, the fills were causal whatever this figure says. The engine simply could not prove it from the events it was handed.
ReportCaveats::volume_ignored is always true, and present so you are told rather than left to assume:
If a strategy's edge is small relative to these, the backtest cannot tell you whether the edge is real.
A replay advances its clock to each bar's period end and derives the market phase there. For daily bars stamped at local midnight — every shipped adapter's convention — that is the next midnight, which is outside the session, so with the default permitted_phases_ = {OPEN} a daily replay refuses every opening order (phase_not_permitted) and trades nothing. Every replay example sets permitted_phases_ = std::nullopt for this reason: the order is decided at the bar's end and filled at the next bar's open, which is what a daily strategy means. Keep that setting in the replay only. Live, the default limits already carry a daily target to the next session's open (see "Running a strategy live"), and std::nullopt there sends every decision at midnight.
For the same reason on_phase fires on a daily replay only when the phase at one bar end differs from the last — in practice once — not once a session as it does live. A strategy that acts in on_phase (vanilla::EqualWeightRebalance, vanilla::TimedLiquidation) needs intraday bars to be replayed faithfully.
The live path can ask the broker what may be spent and what may be sold. A replay has no broker, so it answers from its own ledger — and the two answers are different quantities wearing the same field:
With enforce_buying_power_ off, its default, a replay borrows without limit. It is off by default because live the broker is what decides whether a buy is funded; in a replay nothing does, so a buy the ledger's cash cannot pay for is filled and the cash goes below zero. A fixed-quantity strategy whose price rose is then leveraged without having asked to be, and its return, Sharpe and drawdown are a leveraged strategy's. ReportCaveats::overdrawn_bars_ counts the bars that closed with cash below zero, first_overdrawn_ says when it started and lowest_cash_ how far it went; read them before a ranking. They measure cash in the replay's currency only: a short's proceeds count as cash, so leverage through shorts is not in them (see "Borrow availability" above).
Borrowing can lose more than the opening equity. A period whose preceding recorded equity is zero or negative has no return (EquityPoint::period_return_ is missing); returns resume once equity is positive again. If any recorded equity point before the last is zero or negative, summarize reports sharpe_ and sortino_ as math::undefined() for the whole run, and annualized_sharpe_ too when a period count is given. A non-positive last point alone does not invalidate the ratios. Reducing the curve's returns yourself (analytics::sharpe over a slice) skips the missing ones and applies no such rule, so check the equity each return is measured from.
Turning the check on refuses such buys, but a market buy is checked at the reference price, without commission, and filled at the next bar's open, so a gap up can still overdraw a little. Live, whether a buy the cash cannot cover is refused depends on the account: a Korean account with 미수 (credit) enabled can buy past its cash, and the broker's buying-power inquiry is what says how far.
The consequence worth keeping in mind: a strategy sized against the sellable check passes more easily in a replay than live, because the whole holding is an upper bound on what a broker will let you sell. For buying power there is no such direction — ledger cash may be above or below what a broker would answer — so the difference is real but not one you can lean on either way. Either way, with enforce_sellable_quantity_ on (the default) or enforce_buying_power_ enabled, a clean backtest is not evidence the live account will accept the same orders; a settlement model in the replay would close the sellable half, and the library does not have one yet.
ReplayConfig::currency_ names the currency the report is in, and the equity curve is ledger.equity_in() of it at every bar.
It counts only positions in the currency being asked about (adding a won-quoted holding into a dollar curve at a rate of one would corrupt every derived metric), and the ledger records each position's currency from the fill that opened it.
The engine books each fill in the instrument's own currency, from the reference data the replay already needs (ReplayConfig::currency_ is the currency of the starting cash, not of every fill).
What that means for a multi-currency replay. Each currency is tracked apart and correctly, and a replay that opens with won and trades a dollar instrument runs its dollar cash negative from zero, because funding it would need a conversion this library does not do. The report still curves one currency, so read it as one currency's result rather than the account's. ReportCaveats::other_currency_executions_ counts the executions the curve leaves out; when it is nonzero and equals executions_, nothing traded is in the curve, so the total return, Sharpe and annualized Sharpe, Sortino and maximum drawdown are missing rather than a flat result (the opening and closing equity stay the curve's): set currency_ to the instruments' currency.
A strategy evaluated at zero cost can look excellent and lose money at any real one. So costs are models you state, and analytics deliberately carries none of its own:
Slippage always moves against the order — slippage that helped would not be slippage, and a model cutting both ways would be right on average and flattering in exactly the cases a strategy trades most. A tick means a tick at that price level, through the instrument's PriceRule, so it is 100 won at 70,000 and 50 at 30,000; an unknown grid contributes nothing rather than a guessed step. Several ticks walk the grid one step at a time: three ticks up from 4,990 are 4,995, 5,000 and 5,010, and three down from 5,010 are 5,000, 4,995 and 4,990, so from a price on the KRX grid the tick walk stays on it across a band boundary (basis points, when combined with ticks, are added after the walk and can leave the grid). The tick walk never steps a sell to zero or below; the report's slippage_ticks_ is the count you configured, and fewer apply where the grid has no further step.
Charging nothing is itself a model, and the report says so. A zero-cost run should be your informed choice, not an oversight you cannot detect.
Or cost each execution from a schedule. backtest::CommissionPolicy::scheduled(schedule, venue, dating_market) resolves a rate per execution, against that execution's own market, the date your replay's session gives it, and the venue you assert — so a replay spanning a rate change charges each rate on its own dates instead of one rate throughout, and a KRX fill and an NXT fill can differ. The schedule can be the one a broker published (Toss does; KIS publishes none for domestic stock, and backtest::commission_model_from(schedule, country, date) remains for the older one-date shape) or one you state from your own fee page.
Five things to know before you rely on it:
ReportCaveats::applied_rates_ then lists the rates that actually charged something, each with its origin and how many executions it charged — not the rows the schedule happens to contain.
Report::commission_paid_ is optional, and there are three reasons it can be absent. Two are ordinary and one is not:
slippage_paid_ is optional for the same three reasons, and slippage_by_currency_ is the answer whenever a scalar cannot be — slippage is money, and a replay holding a won fill and a dollar fill has no single number for it.
institution_fees_paid_ is absent for those three reasons and a fourth: any rate that stated a single figure without splitting the components, counted in inclusive_commission_executions_.
No tax rate of the library's own, deliberately: Korean securities transaction tax has changed repeatedly, and a hard-coded number that moves would silently misprice every run after it changed. State the rate for the period you replay — it is charged on sales only, which a commission rate cannot express, and on KRX it is more than ten times a typical online commission, so a high-turnover strategy replayed without it is flattered in proportion to its turnover:
It is truncated to whole currency units, charged to the account with the commission (and, like it, not in realized profit), and reported apart from it: Execution::tax_, Report::tax_paid_ / tax_by_currency_, and ReportCaveats::sell_tax_. Only sales in the stated market pay it. A replay spanning a change in the rate is charged one rate throughout, and instruments exempt in that market — Korean ETFs and ETNs pay no transaction tax — need a replay of their own without it.
summarize's periods_per_year defaults to none, which leaves the annualized Sharpe ratio unobserved, and analytics::annualize_ratio requires one: neither guesses your bar frequency. The number is 245 for daily bars on KRX and 95,550 for minute bars over the same year.
backtest::periods_per_year(interval, session, trading_days) derives it. A calendar interval — weekly, monthly — returns nothing rather than a plausible constant, because a week's divisor depends on the market's week.
The trap, which example-backtest-offline demonstrates: the same strategy over the same path at two granularities reports an identical per-bar Sharpe of 0.308, annualizing to 4.82 daily and 95.21 at minute frequency. Both are arithmetically right. Mislabel your bar frequency and the annualized figure is wrong by the square root of the ratio. And annualizing assumes returns are serially uncorrelated, which a trending strategy violates by construction — so treat an annualized Sharpe as an order of magnitude, not a measurement.
Absent a period count, annualized_sharpe_ stays unobserved rather than being scaled by a guess.
The metrics keep analytics's two kinds of absence, and the difference is diagnostic:
A report that showed 0.0 where a figure did not exist would be the worst kind of wrong.
caveats_.multiple_instruments_ is set whenever more than one instrument traded. Broker reference data lists instruments that exist now, so a universe assembled from it silently excludes everything delisted during the period — and what gets delisted is disproportionately what did badly. A cross-sectional result built that way is flattered by an amount nothing in the engine can measure.
This is a property of the data, not of the replay, so the engine states it rather than correcting it. Single-instrument backtests are unaffected.
Read the flag as a caution, not a finding. The bias comes from how you assembled the universe, and the engine cannot see that — it only sees how many instruments arrived, so the flag is a proxy and it has false positives: a hand-picked pair you know was listed for the whole period sets it and carries no survivorship problem at all.
That is also why the report still computes every metric instead of refusing. Refusing would block a replay that is perfectly sound in order to prevent a mistake the engine has no way to detect. You know how you picked the universe; the engine does not, so it tells you what it saw and leaves the judgement with you.
The engine is interval-agnostic by construction, and that is under test: the same strategy replayed at daily and minute granularity is asserted equal to itself on execution count, position, cash and realized profit. Nothing multiplies by 245 or divides by 24 hours; the clock advances by each bar's own period_end_, and phases come from a SessionWindow.
What limits intraday work today is data, not the engine. KIS serves minute bars one trading day at a time and crossing days needs a calendar walk that does not exist yet. A broker that serves intraday history, or your own bar files, replays without any engine change.
A gap between sessions is counted in caveats_.gaps_ and not interpolated across — nothing happened in it that a strategy should be told about.
In rough order:
And the honest summary: this engine is good for telling a broken strategy from a plausible one, and for comparing two strategies under identical assumptions. It is not good for predicting a return.
1.18.0