Repository navigation
Skip or cancel the queued cross-zero second part once its order is canceled - #10214
quantconnect-kronos[bot] wants to merge 2 commits into
Conversation
…nceled After the first part of a cross-zero order fills, its second part is placed from a queued task. A cancel acknowledged in between, against the filled first part's brokerage id, closed the Lean order while the task still placed the second part, leaving a new opening order working at the brokerage. Canceled and invalid order events now mark the queued second part: the task skips placing it when the order closed before submission, and cancels it when the order closed while it was being placed. Closes #10210 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Requesting changes: when a cancel is requested while the second part is being placed, the second part can still be left working while Lean shows the order Canceled. On TradeStation, the brokerage the issue reproduces, that is what happens every time.
The tracking ends before a cancel that targets the first part can be acknowledged. The task drops the order's entry right after PlaceCrossZeroOrder returns (Brokerage.cs:967). A cancel that read the order's brokerage ids before the second part's id was added still targets the filled first part. If its Canceled arrives after that point, nothing marks the order, so the second part keeps working while the order shows Canceled. That's the result #10210 asks to prevent ("cancellation must account for that order rather than closing Lean based solely on the completed first-half ID").
On TradeStation this isn't a narrow timing window, it's what always happens:
TradeStationBrokerage.PlaceCrossZeroOrderplaces the second part inside_messageHandler.WithLockedStream.CancelOrderreadsorder.BrokerId.Last()(the first part's id), then waits on that same lock.- So a cancel requested during placement is only sent after placement returns. It's soft-rejected with
Not an open order.an HTTP round trip later, well after the entry is gone.
Alpaca and Tastytrade don't hold a lock while placing, but a cancel sent against the first part's id just before placement returns ends the same way.
I reproduced this locally with OrderCrossingBrokerageTests' PhonyBrokerage. I added a lock that the second part's placement holds, the way TradeStation's does. CancelOrder reads its id first, then takes the lock and waits 50 ms before emitting Canceled. The cancel is started on another thread from PlacingCrossZeroSecondPart. Result: both parts placed, only the first part's id canceled, order Canceled. The new CancelAcknowledgedWhileSecondPartIsSubmittedCancelsTheSecondPart passes only because the phony CancelOrder runs synchronously inside the placement, which TradeStation's locking can't do.
The cause is the same for every brokerage: a cancel can be acknowledged against the first part's id after the second part is placed. Whatever closes this needs a test where the cancel is requested on another thread during placement and acknowledged after it returns. Possible directions, your call:
- Keep the order's entry until a cancel can no longer target the first part's id.
- Have the cancel path pick its target id once the second part's placement has finished, for example TradeStation resolving the id inside the stream lock.
The end-to-end run in the description should also request the cancel while the second part is in flight, the way an algorithm would, not from inside the placement.
The skip-before-submission path looks right. It also helps Alpaca, where a cancel of the filled first part emits Invalid, which now stops the second part from being placed.
I set risk:high. The change touches thread coordination in the shared order-placement path used by four live brokerages, and the case above is still open.
…t is acknowledged late A cancel can read the filled first part's brokerage id before the second part's id is added and be acknowledged only after the second part was placed. Any canceled or invalid event for a Lean order whose second part is still open now requests, once, the cancellation of that second part. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks. The case is real, and 9bd9299 fixes it in the shared One correction on TradeStation: it sets
|
There was a problem hiding this comment.
Approving. 9bd9299 fixes the case from my earlier review. A cancel that targets the filled first part and is acknowledged after the second part was placed now cancels the second part by its own brokerage id. The hand-off between the event and the submitting task is atomic: OnOrderEvents flips the flag with TryUpdate(false → true), and the task reads and removes it with TryRemove after the lock, once the second part's id is already in LeanOrderByZeroCrossBrokerageOrderId. Either way it ends up being canceled, whichever side of the placement the acknowledgement lands on. The automatic cancel targets the second part's id in TradeStation, Alpaca and Tastytrade, since each one's CancelOrder uses the order's last brokerage id.
I ran it locally. OrderCrossingBrokerageTests passed 19/19 in 5 runs out of 5. With Brokerage.cs from 828ccd9, CancelRequestedWhileSecondPartIsSubmittedAndAcknowledgedAfterCancelsTheSecondPart fails, as the description says.
CI never ran on 9bd9299: that commit has no workflow runs. Every run on 828ccd9 passed. Marking the pull request ready for review should start CI, and it has to be green before this merges.
The label stays risk:high. This changes thread coordination in the order path that four live brokerages share, and CI hasn't run on this commit.
There was a problem hiding this comment.
Requesting changes: a delayed compensating cancel can overwrite a confirmed second-part fill with Canceled. The exact Lean base finishes Filled for the same schedule; this head with current TradeStation master finishes Canceled. Executed quantity is still recorded correctly. Details and the reproduction are inline.
@Martin-Molinero Decision needed: should this change require TradeStation's existing cancellation-state protection, or remain limited to preventing submission of a canceled queued part?
I verified that the already-open QuantConnect/Lean.Brokerages.TradeStation#132 (7781c40) resolves this schedule: its cancel path rereads the order under the stream lock and preserves an already-closed order. That would introduce a brokerage-version prerequisite rather than requiring another shared-API redesign.
- A. Require the TradeStation protection first (recommended): retain the full fix, add this regression test, and ensure the compatible brokerage version is used with it.
- B. Narrow this change to skipping queued submissions: remove the new compensating-cancel path for now. This avoids the new terminal-status regression, but leaves cancellation during or after submission unresolved.
Reply with the option you choose, or another approach.
Validation: Release build succeeded; the full CI-filtered suite passed 39,848, failed 0, skipped 100. All 19 cross-zero tests passed. Twelve additional lifecycle checks passed in ten repeated runs. Six offline TradeStation production-path checks passed, alongside the failing post-fill schedule, exact-base comparison, and successful PR132 comparison. These used controlled placement/transport, not a new simulator end-to-end run.
CI: this head has no workflow runs or check runs; CI is missing for the commit authored at 2026-11-20 14:34 UTC.
Other implementations: Alpaca and Tastytrade remove second-part tracking before normal terminal notifications. Tradier's existing first-filled-ID cancellation failure is separate and is now tracked in QuantConnect/Lean.Brokerages.Tradier#62; it does not block this change.
Risk: risk:high — the new asynchronous cancellation can replace a confirmed fill's terminal status.
| #pragma warning disable CA1031 // Do not catch general exception types | ||
| try | ||
| { | ||
| isCanceled = CancelOrder(leanOrder); |
There was a problem hiding this comment.
The mapping can disappear after this task is queued, but the captured order is still canceled unconditionally. I reproduced this through TradeStation's production stream handler and CancelOrder, controlling only placement/transport and pausing the automatic invocation before it enters CancelOrder:
- A cancel of the filled first ID emits
Canceledand queues this task. - The second ID's
FLLarrives before the task proceeds: Lean receivesFilled, andTryGetOrRemoveCrossZeroOrderremoves the mapping. - The queued task nevertheless cancels that completed second ID. TradeStation returns
Not an open order., and its current soft-reject path emits anotherCanceled.
This head produces PartiallyFilled, Canceled, Filled, Canceled, final status Canceled, total fill −10. With the exact base Brokerage.cs, it produces PartiallyFilled, Canceled, Filled, final status Filled, the same fill quantity. BrokerageTransactionHandler accepts the final cancellation and overwrites Filled (BrokerageTransactionHandler.cs:1316–1318), so this is not just a test-provider artifact. Please cover the second part completing before the compensating cancellation executes, and preserve that confirmed fill. The bounded existing prerequisite is described in the review body.
There was a problem hiding this comment.
Agreed: the base never sends this cancel, so a second part that fills before the compensating cancel lands ends Filled there and Canceled here. That comes from this change.
Checking LeanOrderByZeroCrossBrokerageOrderId again just before CancelOrder would cover your exact schedule, where the fill is processed before the task runs. It wouldn't cover a fill that arrives while the cancel request is in flight. Keeping that fill needs either TradeStation's cancel to leave an already closed order alone (QuantConnect/Lean.Brokerages.TradeStation#132) or dropping the compensating cancel. So I'm leaving the code as it is until the decision in your review is answered, and I'll build whichever option is chosen, with this schedule as a test.
Closes #10210
What changed
When the first part of a cross-zero order fills,
Brokerage.TryHandleRemainingCrossZeroOrderemitsPartiallyFilledand places the second part from aTask.Run. A cancel that targets the already filled first part can be acknowledged by the brokerage asCanceled(for example a soft reject with "Not an open order."). Onmasterthat closes the order in Lean while the second part is still placed and keeps working.Brokerage.OnOrderEventsnow checks everyCanceledorInvalidevent against the order's second part:The cancel is requested once per second-part brokerage id, through
CancelOrderwith the order whose last brokerage id is the second part's. If it returnsfalseor throws, aCrossZeroCancelFailedwarning is sent. The second part counts as open while it's inLeanOrderByZeroCrossBrokerageOrderId. Brokerages remove it there throughTryGetOrRemoveCrossZeroOrderbefore emitting its own fill or cancel event, so a cancel that targeted the second part itself doesn't trigger another.The cancel is requested from a
Task.Run, never from the thread raising the event. The event can be raised while the brokerage holds its stream lock, which itsCancelOrdertakes again. The queued-order marking is a lock-freeConcurrentDictionaryupdate.Orders with no cancel during the cross-zero lifecycle behave as before. This changes no public API. Alpaca, Tastytrade, Tradier and TradeStation all place the second part through this shared helper, so the fix applies to all four without changes in their repositories.
Tests
New tests in
OrderCrossingBrokerageTests. The phony brokerage now holds a placement lock while it places or cancels, and its cancel reads the brokerage id before taking that lock:CancelAcknowledgedBeforeQueuedSecondPartIsNotSubmitted: a cancel is acknowledged when thePartiallyFilledevent arrives, before the queued task runs. Expected result: the second part is never placed, the order staysCanceled, and no cross-zero brokerage id is left tracked.CancelAcknowledgedWhileSecondPartIsSubmittedCancelsTheSecondPart: a cancel is acknowledged while the second part is being placed. Expected result: a cancel is then sent for the second part's brokerage id.CancelRequestedWhileSecondPartIsSubmittedAndAcknowledgedAfterCancelsTheSecondPart: a cancel is requested on another thread while the second part is being placed. It targets the first part's id and is acknowledged after placement returns. Expected result: a cancel is then sent for the second part's brokerage id.All three fail on
master. The third also fails on this pull request's first commit. All pass with this change.OrderCrossingBrokerageTests: 19/19 passed.dotnet build /p:Configuration=Release QuantConnect.Lean.sln, thendotnet test ./Tests/bin/Release/QuantConnect.Tests.dll --filter "TestCategory!=TravisExclude&TestCategory!=ResearchRegressionTests"): Passed 39848, Failed 0, Skipped 100.Tested end to end against the TradeStation brokerage built on this change, using the issue's schedule. Holding +5 AAPL, a sell of 10 sends a 5-share closing part, and its
FLLframe arrives. While the second part is being placed, a cancel is requested from another thread. It targets the first part's id and is soft-rejected withNot an open order.after placement returned, which closes the order asCanceled.masterand on this pull request's first commit, the second part kept working.TradeStation enables concurrent order submission, so its stream lock doesn't serialize the cancel behind the second part's placement. The cancel is sent while placement is in flight, and the race is between the two HTTP requests.
Known limitation
When the second part is canceled this way, the brokerage may report one more
Canceledevent for an order Lean already shows asCanceled. Tradier'sCancelOrderemitsCanceleditself without going throughTryGetOrRemoveCrossZeroOrder. So a regular cancel of a placed second part on Tradier can be followed by one redundant cancel request, which Tradier rejects because the order is already closed.