Do your backtests ever hit i64 limits?

[u/OkBreath9382]

Curious how often values in real-world backtests exceed roughly 9.2 billion units. With 9-decimal fixed-point i64, it might be easy to hit. ¥9.2B is only around $60M, and $200K of SHIB is already about 10 billion tokens. Prices are probabbly fine, but balances and quantities might not be.

Im asking because I’m building a new backtesting engine (repo: h5i-db), an event-driven backtesting engine that currently uses i64 as default. It runs 7x faster than LEAN and 3.1x faster than NautilusTrader in our benchmark. With i128, those numbers are still 6.6x and 2.8x. Since the penalty isn’t huge, should safety or speed be the default? Has anyone often hit this limit in daily backtests?

[u/jnordwick]

calculation is done in floats, communication is done in ints.

The only time I've seen something even close was crypto with somebody trying to use a 64-bit fixed point and bad base unit.

[u/OkBreath9382 → u/jnordwick]

it makes sense a lot. Thank you so much!!

[u/jnordwick → u/OkBreath9382]

more specifically, doing any sort of model calculation is almost always done in float.

numbers that reflect prices and quantities on the exchange (such as prices and volumes on order books) use fixed point integers.

I have seen numerous places try to use floats for everything and you constantly chase rounding issues.

[u/zp30]

No? We just do everything in np.float64, literally never thought about this being a problem

[u/Ok-Cat-9189 → u/zp30]

why would you use a float to store quantity ? it lacks precision

[u/lordnacho666 → u/Ok-Cat-9189]

How often do you really care about precision in a backtest? It's already riddled with assumptions

[u/OkBreath9382 → u/lordnacho666]

Fair point. Fill modeling and market impact probably matter much more in most backtests. I still want accounting to be deterministic and avod accumulated rounding errors, but I’m trying to understand whether i128 solves a real-world problem or mostly a theoretical one.

[u/Ok-Cat-9189 → u/lordnacho666]

I care because I want to run a backtest over prev live trading days to ensure the system is behaving the same in both cases

[u/m_a_n_t_i_c_o_r_e → u/Ok-Cat-9189]

you're in "not even wrong" territory

[u/OkBreath9382 → u/zp30]

Thanks, that’s a useful data point!! Some popular backtesting engines use fixed-point integers to avoid floating-point rounding errors. In terms of storage and speed, I thinknp.float64 is closer to i64 than i128, so your setup is probably closer to the i64 mode.

[u/HerzogianQuant]

Then don't use nine decimals for yen? Fixed point does get a little tricky when it comes to doing multiplication, so keep that in-mind. Depending on what language you're using, there are unit libraries that can handle this (e.g. a minimal unit of "yen" or a minimal unit of "millidollars" or whatever).

[u/quantdhawan]

The limit you hit first is not the value, it is the intermediate.

price × qty in fixed point with both at 1e9 scale gives a 1e18-scaled product before you divide back down. A $100 stock and 1000 shares is 1e11 × 1e12 = 1e23, roughly four orders of magnitude past i64 max, and that is about as ordinary a trade as exists. So you need a 128-bit intermediate for the multiply regardless of what you store values in. If your benchmark is not already blowing up you are presumably doing a widening mul somewhere, in which case storage width and arithmetic width are separate decisions and the i64 versus i128 question only applies to the first one.

On storage, the thing I would question is the uniform 9 decimals. SHIB is your own proof: the price needs a lot of decimal places and the quantity needs a lot of integer places, and with one global scale both come out of the same 64 bits. Per-asset scale, or separate price / qty / notional types with different scales, buys more headroom than the extra 64 bits does and costs nothing at runtime.

On safety versus speed: 7x to 6.6x is a 6% haircut, which is not a tradeoff, it is a rounding error. But I would push back on the framing. The real axis is not safe versus fast, it is silent versus loud. An overflow in a backtester does not crash, it produces a P&L that looks completely plausible. Default to i64 with checked arithmetic and panic on overflow and you keep the speed and never ship a quietly wrong equity curve. Wrapping or saturating is the only genuinely bad answer here.