REPLYOPSAI RESEARCH NOTE
FAILURE ANALYSIS

Trading Bot Losing Money? Diagnose It in This Order

When a bot starts losing, changing thresholds immediately destroys evidence. Diagnose where the loss is coming from before modifying the system.

1. Verify the system

Confirm the intended strategy version is running, data is current, orders or simulated fills are recorded correctly, and exits match the configured logic.

2. Separate gross edge from costs

Compare results before and after fees and modeled execution costs. A strategy with a tiny gross advantage may have no net advantage.

3. Inspect loss distribution

Do not focus on win rate alone. Check average win, average loss, losing streaks, concentration by market and whether simultaneous positions share the same exposure.

Build the loss map before touching parameters

QUESTIONWHAT TO COMPARE
Did signal quality change?Entry frequency, direction and outcome by market and period
Did payoff change?Average win, average loss, tail losses and exit reasons
Did costs increase?Gross vs net PnL, turnover and execution assumptions
Did exposure concentrate?Concurrent positions, direction and correlated markets
Did operations fail?Missing data, restarts, API errors and stale state

The goal is to locate the layer where behavior changed. A threshold adjustment made before this comparison can hide the original failure.

4. Check regime dependence

Determine whether losses cluster in a specific volatility, trend or liquidity environment. That is evidence for investigation, not permission to retroactively delete those trades.

5. Version the fix

If you change material logic, start a new version and preserve the old record.

See how we keep versions separate →