Skip to content

Orion Investigation — FLAG-037 Reconciler Conservative Phantom Fill

Vesper — Q1–Q5 findings per your tasking. Flagging 4 implementation deviations (D1–D4) that need your ruling before I write code. Happy to proceed the moment you green-light.

Branch: fix/reconciler-disappeared-order-conservative Current HEAD: feat/directional-drift-guard (pending your review).


Q1 — Phantom fill decision point

File: neo_engine/ledger_reconciler.py Function: _handle_disappeared_active_order(order, engine, result, reconciler) — line 652.

Current flow (line 681–718):

  1. If order.cancel_tx_hash is not None → cancel race, increment result.cancel_races += 1, return.
  2. Else → phantom fill path:
  3. _record_disappeared_order_anomaly(order, state=reconciler._state, action_taken="phantom_fill_applied") — line 704–708.
  4. _log_fill_detected(...) — line 709–717.
  5. _apply_full_fill(order, order.quantity, engine, result) — line 718.

The age gate must insert between the cancel-race guard (line 681–692) and the anomaly record (line 704). _apply_full_fill is the side-effecting call that credits the phantom fill; gating must happen before it.


Q2 — Order age calculation

Helper exists: _age_seconds_since(created_at_iso: str) -> Optional[int] at ledger_reconciler.py:391.

  • Uses datetime.fromisoformat(created_at_iso), normalises naive → UTC, returns int((now - created) / 1_000_000) seconds since epoch math (UTC-aware).
  • Returns None if the string fails to parse.
  • Already called at line 744 inside _record_disappeared_order_anomaly to populate the age_seconds column — no duplicate call needed if I restructure slightly (see D2 below).

order.created_at is populated on insert (StateManager writes the ISO timestamp at creation), so None would only surface on corrupted rows.

Deviation D2: age_seconds is None needs a policy. See Deviations section.


Q3 — reconciler_anomaly_log write path

Writer: StateManager.record_reconciler_anomaly(...)state_manager.py:1615 (confirmed prior session).

  • action_taken is a free-text str column. No schema migration needed.
  • Precedent: action_taken="phantom_fill_applied" already persisted in D2.2/audit-log branch.
  • New value "held_pending_review" will round-trip fine.

_record_disappeared_order_anomaly (line 721) already: - Emits WARNING log with all operator-facing context (order_id, side, quantity_rlusd, xrp_equivalent, age_seconds, market context). - Best-effort DB write (swallows exceptions, ERROR-logs on failure).

For held_pending_review, I want to: - Change the WARNING log message from "applying phantom fill" to a neutral phrasing so it serves both action_taken values, OR - Split into two log messages (one per action_taken).

I lean toward splitting — "held_pending_review — NOT applying fill" is a sharper operator signal than a neutral message. See D3.


Q4 — _enter_degraded_mode availability

This is the blocker and needs your ruling.

LedgerReconciler.__init__(self, execution_engine, state_manager) at line 144–150 — holds _engine: ExecutionEngine and _state: StateManager. No NEOEngine reference. _enter_degraded_mode lives on NEOEngine at main_loop.py:1290.

Current DEGRADED flow from reconciler → main loop:

  • ReconciliationResult.engine_signal: EngineStatus already exists (models.py:438).
  • Reconciler can return engine_signal=EngineStatus.DEGRADED.
  • But main loop at main_loop.py:2259–2266 currently only logs a warning on DEGRADED — it does NOT call _enter_degraded_mode.

Two viable patterns. I recommend (A):

Add held_pending_review: int = 0 (count) to ReconciliationResult (models.py:438). Reconciler sets engine_signal=DEGRADED AND held_pending_review += 1 for each held order. Main loop at main_loop.py:2259 checks:

if recon_result.engine_signal == EngineStatus.DEGRADED:
    log.warning("Reconciler returned DEGRADED — continuing conservatively", ...)
    if recon_result.held_pending_review > 0:
        self._enter_degraded_mode("reconciler_held_pending_review")

Pros: no new interface surface; _enter_degraded_mode is idempotent so the one-shot flag lives inside it (via already_degraded check); existing DEGRADED cases (cancel_races, ambiguous) preserved as warn-only, matching their current semantics.

Cons: new field on ReconciliationResult is a model change. Additive only, no DB touch.

Pattern B (rejected) — inject callback into LedgerReconciler

Pass enter_degraded_mode: Callable[[str], None] into LedgerReconciler.__init__. Reconciler calls it directly.

Cons: couples reconciler to NEOEngine's DEGRADED machinery; breaks existing test harness that constructs LedgerReconciler(execution_engine, state_manager) without a callback; inverted control vs. the existing engine_signal return-value pattern.

My vote: Pattern A. Requesting your ruling.


Q5 — Config access in reconciler

Current: LedgerReconciler.__init__(execution_engine, state_manager) — no config.

Proposed:

  1. New dataclass ReconcilerConservativeConfig in neo_engine/config.py:
    @dataclass(frozen=True)
    class ReconcilerConservativeConfig:
        enabled: bool = True
        age_threshold_seconds: float = 300.0
    
  2. Add field to StrategyConfigreconciler_conservative: ReconcilerConservativeConfig.
  3. YAML under strategy: in all 3 files (config.yaml, config_live_stage1.yaml, config.example.yaml):
    strategy:
      reconciler_conservative:
        enabled: true
        age_threshold_seconds: 300.0
    
  4. Extend LedgerReconciler.__init__ to accept conservative_config: ReconcilerConservativeConfig:
    def __init__(self, execution_engine, state_manager, conservative_config=None):
        self._conservative = conservative_config or ReconcilerConservativeConfig()
    
    Default-construct when omitted so existing tests that pass only (execution_engine, state_manager) still compile. This mirrors the anchor/drift guard wiring.
  5. Main loop constructor (main_loop.py:159) updates to:
    self._reconciler = LedgerReconciler(
        self._execution, self._state,
        conservative_config=config.strategy.reconciler_conservative,
    )
    

No config precedent issue — StrategyConfig already houses anchor_saturation_guard and directional_drift_guard blocks from the last two branches.


Deviations

D1 — Additive held_pending_review: int field on ReconciliationResult (Pattern A)

Per Q4. Additive only, no DB touch, no breaking change to existing consumers. Flagging for confirmation — tasking memo didn't specify the signal-up-stack mechanism.

D2 — age_seconds is None handling

_age_seconds_since returns None on parse failure. My proposal: treat None as "cannot prove safe — hold pending review" (fail-closed conservative). Alternative: treat as "unknown → young → apply phantom" (preserves current behavior). Fail-closed matches the Atlas invariant "if the engine cannot prove alignment with reality, it does not act." Wanting your ruling.

D3 — WARNING log message split

Current log message (line 747): "[RECONCILER_ANOMALY] disappeared active order — applying phantom fill". For held_pending_review, I propose emitting a second message: "[RECONCILER_ANOMALY] disappeared active order — held_pending_review, NOT applying fill". Routed by action_taken. Alternative: keep one neutral message and let operators read action_taken from the extras. I prefer the split.

D4 — Branch stacking

Current HEAD is feat/directional-drift-guard (pending your review). Two options: - Stack off drift guard (my preference) — matches anchor→drift stacking pattern, avoids rebase if drift merges clean. If drift review surfaces fixes, I rebase this branch on top. - Off main — independent, but then drift's _evaluate_directional_drift_guard changes are not available for test fixtures. (Not actually needed here — FLAG-037 tests don't touch the drift guard.)

Flagging for your call. Defaulting to off drift unless you prefer off main.


Proposed branch plan

3 commits (matches tasking):

# Subject Files
C1 feat(config): ReconcilerConservativeConfig + YAML defaults (FLAG-037) neo_engine/config.py, 3 × YAML
C2 fix(reconciler): age-gate disappeared orders, held_pending_review path (FLAG-037) neo_engine/models.py (+1 field), neo_engine/ledger_reconciler.py (age gate + held path), neo_engine/main_loop.py (constructor + DEGRADED wire-in)
C3 test(reconciler): FLAG-037 — ≥8 tests covering age gate, held path, DEGRADED wire-in tests/test_reconciler_conservative.py (new)

Test plan (≥8, targeting ~10):

Config (2): 1. Defaults OK (enabled=True, age=300.0). 2. Invalid age_threshold_seconds ≤ 0 rejected in validator.

Age-gate logic (6): 3. Age < threshold → phantom fill applied (current behavior preserved). 4. Age ≥ threshold → held_pending_review, no _apply_full_fill called, engine_signal=DEGRADED, held_pending_review=1. 5. age_seconds=None (corrupt created_at) → held_pending_review (per D2 ruling). 6. enabled=False → all orders use phantom path regardless of age (feature-flag off). 7. Exactly at threshold (age == threshold) → held (≥ is inclusive per tasking). 8. cancel_tx_hash is not None → cancel race path unchanged (not gated by age).

Integration (2): 9. Held path writes reconciler_anomaly_log row with action_taken='held_pending_review' and emits WARNING log. 10. Main loop calls _enter_degraded_mode("reconciler_held_pending_review") on held_pending_review > 0. (MagicMock(spec=NEOEngine) shape, matching drift/anchor test style.)

Regression sweep

python -m pytest tests/test_reconciler_conservative.py \
  tests/test_reconciler_anomaly_log.py tests/test_directional_drift_guard.py \
  tests/test_anchor_saturation_guard.py tests/test_ledger_reconciler.py \
  tests/test_flag_036_wallet_truth_reconciliation.py tests/test_config.py \
  tests/test_config_invariants.py tests/test_anchor_error_stat.py -q
Target: all green. Expected ~120 tests.


Open questions for your ruling

  1. Pattern A (discriminator field on ReconciliationResult) — confirm?
  2. age_seconds is None → hold_pending_review (fail-closed) — confirm?
  3. WARNING log split (two messages routed by action_taken) — confirm?
  4. Branch stacked on feat/directional-drift-guard — confirm, or prefer off main?
  5. Test count target 10 (≥8 required) — any specific cases missing from my plan?

Standing by for rulings before any code is written.

— Orion