Orion Investigation — FLAG-037 Reconciler Conservative Phantom Fill¶
Vesper — Q1–Q5 findings per your tasking. Flagging 4 implementation deviations (D1–D4) that need your ruling before I write code. Happy to proceed the moment you green-light.
Branch: fix/reconciler-disappeared-order-conservative
Current HEAD: feat/directional-drift-guard (pending your review).
Q1 — Phantom fill decision point¶
File: neo_engine/ledger_reconciler.py
Function: _handle_disappeared_active_order(order, engine, result, reconciler) — line 652.
Current flow (line 681–718):
- If
order.cancel_tx_hash is not None→ cancel race, incrementresult.cancel_races += 1, return. - Else → phantom fill path:
_record_disappeared_order_anomaly(order, state=reconciler._state, action_taken="phantom_fill_applied")— line 704–708._log_fill_detected(...)— line 709–717._apply_full_fill(order, order.quantity, engine, result)— line 718.
The age gate must insert between the cancel-race guard (line 681–692) and the anomaly record (line 704). _apply_full_fill is the side-effecting call that credits the phantom fill; gating must happen before it.
Q2 — Order age calculation¶
Helper exists: _age_seconds_since(created_at_iso: str) -> Optional[int] at ledger_reconciler.py:391.
- Uses
datetime.fromisoformat(created_at_iso), normalises naive → UTC, returnsint((now - created) / 1_000_000)seconds since epoch math (UTC-aware). - Returns
Noneif the string fails to parse. - Already called at line 744 inside
_record_disappeared_order_anomalyto populate theage_secondscolumn — no duplicate call needed if I restructure slightly (see D2 below).
order.created_at is populated on insert (StateManager writes the ISO timestamp at creation), so None would only surface on corrupted rows.
Deviation D2: age_seconds is None needs a policy. See Deviations section.
Q3 — reconciler_anomaly_log write path¶
Writer: StateManager.record_reconciler_anomaly(...) — state_manager.py:1615 (confirmed prior session).
action_takenis a free-textstrcolumn. No schema migration needed.- Precedent:
action_taken="phantom_fill_applied"already persisted in D2.2/audit-log branch. - New value
"held_pending_review"will round-trip fine.
_record_disappeared_order_anomaly (line 721) already:
- Emits WARNING log with all operator-facing context (order_id, side, quantity_rlusd, xrp_equivalent, age_seconds, market context).
- Best-effort DB write (swallows exceptions, ERROR-logs on failure).
For held_pending_review, I want to:
- Change the WARNING log message from "applying phantom fill" to a neutral phrasing so it serves both action_taken values, OR
- Split into two log messages (one per action_taken).
I lean toward splitting — "held_pending_review — NOT applying fill" is a sharper operator signal than a neutral message. See D3.
Q4 — _enter_degraded_mode availability¶
This is the blocker and needs your ruling.
LedgerReconciler.__init__(self, execution_engine, state_manager) at line 144–150 — holds _engine: ExecutionEngine and _state: StateManager. No NEOEngine reference. _enter_degraded_mode lives on NEOEngine at main_loop.py:1290.
Current DEGRADED flow from reconciler → main loop:
ReconciliationResult.engine_signal: EngineStatusalready exists (models.py:438).- Reconciler can return
engine_signal=EngineStatus.DEGRADED. - But main loop at
main_loop.py:2259–2266currently only logs a warning on DEGRADED — it does NOT call_enter_degraded_mode.
Two viable patterns. I recommend (A):
Pattern A (recommended) — add discriminator field to ReconciliationResult¶
Add held_pending_review: int = 0 (count) to ReconciliationResult (models.py:438). Reconciler sets engine_signal=DEGRADED AND held_pending_review += 1 for each held order. Main loop at main_loop.py:2259 checks:
if recon_result.engine_signal == EngineStatus.DEGRADED:
log.warning("Reconciler returned DEGRADED — continuing conservatively", ...)
if recon_result.held_pending_review > 0:
self._enter_degraded_mode("reconciler_held_pending_review")
Pros: no new interface surface; _enter_degraded_mode is idempotent so the one-shot flag lives inside it (via already_degraded check); existing DEGRADED cases (cancel_races, ambiguous) preserved as warn-only, matching their current semantics.
Cons: new field on ReconciliationResult is a model change. Additive only, no DB touch.
Pattern B (rejected) — inject callback into LedgerReconciler¶
Pass enter_degraded_mode: Callable[[str], None] into LedgerReconciler.__init__. Reconciler calls it directly.
Cons: couples reconciler to NEOEngine's DEGRADED machinery; breaks existing test harness that constructs LedgerReconciler(execution_engine, state_manager) without a callback; inverted control vs. the existing engine_signal return-value pattern.
My vote: Pattern A. Requesting your ruling.
Q5 — Config access in reconciler¶
Current: LedgerReconciler.__init__(execution_engine, state_manager) — no config.
Proposed:
- New dataclass
ReconcilerConservativeConfiginneo_engine/config.py: - Add field to
StrategyConfig→reconciler_conservative: ReconcilerConservativeConfig. - YAML under
strategy:in all 3 files (config.yaml,config_live_stage1.yaml,config.example.yaml): - Extend
LedgerReconciler.__init__to acceptconservative_config: ReconcilerConservativeConfig:Default-construct when omitted so existing tests that pass onlydef __init__(self, execution_engine, state_manager, conservative_config=None): self._conservative = conservative_config or ReconcilerConservativeConfig()(execution_engine, state_manager)still compile. This mirrors the anchor/drift guard wiring. - Main loop constructor (
main_loop.py:159) updates to:
No config precedent issue — StrategyConfig already houses anchor_saturation_guard and directional_drift_guard blocks from the last two branches.
Deviations¶
D1 — Additive held_pending_review: int field on ReconciliationResult (Pattern A)¶
Per Q4. Additive only, no DB touch, no breaking change to existing consumers. Flagging for confirmation — tasking memo didn't specify the signal-up-stack mechanism.
D2 — age_seconds is None handling¶
_age_seconds_since returns None on parse failure. My proposal: treat None as "cannot prove safe — hold pending review" (fail-closed conservative). Alternative: treat as "unknown → young → apply phantom" (preserves current behavior). Fail-closed matches the Atlas invariant "if the engine cannot prove alignment with reality, it does not act." Wanting your ruling.
D3 — WARNING log message split¶
Current log message (line 747): "[RECONCILER_ANOMALY] disappeared active order — applying phantom fill". For held_pending_review, I propose emitting a second message: "[RECONCILER_ANOMALY] disappeared active order — held_pending_review, NOT applying fill". Routed by action_taken. Alternative: keep one neutral message and let operators read action_taken from the extras. I prefer the split.
D4 — Branch stacking¶
Current HEAD is feat/directional-drift-guard (pending your review). Two options:
- Stack off drift guard (my preference) — matches anchor→drift stacking pattern, avoids rebase if drift merges clean. If drift review surfaces fixes, I rebase this branch on top.
- Off main — independent, but then drift's _evaluate_directional_drift_guard changes are not available for test fixtures. (Not actually needed here — FLAG-037 tests don't touch the drift guard.)
Flagging for your call. Defaulting to off drift unless you prefer off main.
Proposed branch plan¶
3 commits (matches tasking):
| # | Subject | Files |
|---|---|---|
| C1 | feat(config): ReconcilerConservativeConfig + YAML defaults (FLAG-037) |
neo_engine/config.py, 3 × YAML |
| C2 | fix(reconciler): age-gate disappeared orders, held_pending_review path (FLAG-037) |
neo_engine/models.py (+1 field), neo_engine/ledger_reconciler.py (age gate + held path), neo_engine/main_loop.py (constructor + DEGRADED wire-in) |
| C3 | test(reconciler): FLAG-037 — ≥8 tests covering age gate, held path, DEGRADED wire-in |
tests/test_reconciler_conservative.py (new) |
Test plan (≥8, targeting ~10):¶
Config (2):
1. Defaults OK (enabled=True, age=300.0).
2. Invalid age_threshold_seconds ≤ 0 rejected in validator.
Age-gate logic (6):
3. Age < threshold → phantom fill applied (current behavior preserved).
4. Age ≥ threshold → held_pending_review, no _apply_full_fill called, engine_signal=DEGRADED, held_pending_review=1.
5. age_seconds=None (corrupt created_at) → held_pending_review (per D2 ruling).
6. enabled=False → all orders use phantom path regardless of age (feature-flag off).
7. Exactly at threshold (age == threshold) → held (≥ is inclusive per tasking).
8. cancel_tx_hash is not None → cancel race path unchanged (not gated by age).
Integration (2):
9. Held path writes reconciler_anomaly_log row with action_taken='held_pending_review' and emits WARNING log.
10. Main loop calls _enter_degraded_mode("reconciler_held_pending_review") on held_pending_review > 0. (MagicMock(spec=NEOEngine) shape, matching drift/anchor test style.)
Regression sweep¶
python -m pytest tests/test_reconciler_conservative.py \
tests/test_reconciler_anomaly_log.py tests/test_directional_drift_guard.py \
tests/test_anchor_saturation_guard.py tests/test_ledger_reconciler.py \
tests/test_flag_036_wallet_truth_reconciliation.py tests/test_config.py \
tests/test_config_invariants.py tests/test_anchor_error_stat.py -q
Open questions for your ruling¶
- Pattern A (discriminator field on
ReconciliationResult) — confirm? age_seconds is None→ hold_pending_review (fail-closed) — confirm?- WARNING log split (two messages routed by action_taken) — confirm?
- Branch stacked on
feat/directional-drift-guard— confirm, or prefer off main? - Test count target 10 (≥8 required) — any specific cases missing from my plan?
Standing by for rulings before any code is written.
— Orion