Learning faster with the symmetry
Reinforcement learning and its inverse, with and without the imbalance equivalence — and what happens when the symmetry is only approximately true. Companion to On a Simple Relationship Between Order Imbalance, Skew and Width in Over-The-Counter Trading, §6.
The flow regime cycles through four imbalances (q = 0.40, 0.50, 0.60, 0.70, switching every 400 enquiries). Identical tabular Q-learners quote into this market. The baseline learns a separate policy per regime; each table sees a quarter of the data. The symmetry learner keeps one policy in the balanced frame and maps it through the skew shift δ(q) = (w/2) log(q/(1−q)): every regime’s experience trains the same table. The robust learner adds a small per-regime residual table on top of the shared one, so it can absorb whatever the symmetry gets wrong. The oracle quotes the exact optimal policy for the true generative model.
The win curve selector changes the generative model of the best competing quote. Exponential is the paper’s model: constant hazard, symmetry exact. Logistic is the shape Fermanian–Guéant–Pu fit to real RFQ data — increasing hazard — plus competitors who crowd the flow-heavy side, so the symmetry becomes only approximately true. The δ the learners use is deliberately NOT recalibrated: it is the misspecified, exponential-model formula.
Look for: under the exponential model, the symmetry learner reaching in ten thousand enquiries a level the baseline never reaches. Then switch to logistic — and notice the symmetry learner barely degrades. Acting is self-correcting: the shared table quietly re-centers around the misspecified δ. The place the deviation really bites is the inverse problem below.
The inverse problem, where robustness earns its keep
Now watch instead of act. A dealer quotes optimally across the same regimes; an observer sees noisy quotes and wants her skew curve S(x) — by the paper’s identities, the slope of her inventory cost. Three estimators: per-regime uses only the target regime’s observations. Pooled de-tilts everything by δ(q) and averages — unbeatable when the symmetry is exact, but it has no way to re-center when it isn’t: unlike the forward learner, an estimator cannot absorb its own bias. Shrunk is the robust design: random-effects partial pooling, in which the between-regime dispersion of the de-tilted means estimates the size of the symmetry violation and sets the pooling weight automatically — the same reliability-sets-the-dial mechanism as the Schur damping γ*.
Look for: under the exponential model, pooled wins and shrunk tracks it. Under the logistic model, pooled becomes the WORST estimator at large samples — its bias never averages away — while shrunk quietly slides toward per-regime. The dial sets itself; you never have to know which world you are in.
Environment and solver: mm_core.js, rl_core.js. Roadmap entries P3, P7 and P9 in papers/ROADMAP.md.