Repository navigation
Add unrequested securities by symbol instead of by ticker - #10222
quantconnect-kronos[bot] wants to merge 2 commits into
Conversation
GetOrAddUnrequestedSecurity() added equities, forex, CFDs and crypto by their ticker, discarding the security identifier of the symbol it was given. Holdings restored from the live-holdings brokerage data carry the security's first ticker, so resolving it today could add a different security, such as FB for META, and book the position on it. Add the security by its symbol so the position stays on the security identifier it was saved with. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Requesting changes. One thing in the approach needs a decision first.
Adding the restored holding by symbol does book it on the right security, which is the fix the issue asked for. But AddSecurity(Symbol) also caches symbol.Value (SecurityService.CreateSecurity → SymbolCache.Set(symbol.Value, symbol)). The restored symbol's Value is the security's first ticker, so after the restore, that ticker resolves to the held security for the rest of the deployment.
I reproduced it locally by adding three lines to the end of the new test, after setup: SymbolCache.TryGetSymbol("GOOG"), algorithm.AddEquity("GOOG").Symbol and an implicit Symbol s = "GOOG" all return GOOG T1AZ164W5VTX (GOOGL). Today's GOOG is GOOCV VP83T1ZUHROL. On master they return today's GOOG, because the restore cached the ticker it re-resolved.
For the reported cases, after META is restored as FB, AddEquity("FB") and MarketOrder("FB", ...) act on META instead of the FB ETF, and "NB" acts on BAC. If the algorithm added the ETF in Initialize, the restore overwrites that cache entry, so later string-based orders go to the other security. That narrows the issue's wrong-security problem, but it's a new path to orders on the wrong security, and it can only be closed by changing which ticker the restored symbol carries.
@Martin-Molinero Decision needed: should the restored security keep its first ticker as Symbol.Value, given that this ticker then resolves to it for string-based calls in the same deployment?
This pull request adds unrequested securities by Symbol instead of by ticker, as you suggested on #9736, so restored holdings no longer land on whichever security owns the first ticker today. That works: there's a test that fails on master, and an end-to-end run. The open question is whether the first-ticker side effect described above is acceptable, or whether the ticker should be remapped now, which the issue left as a separate change.
- A. Remap the ticker before adding (recommended: it closes the new path to the wrong security and also fixes the stale ticker the issue reports): rebuild the symbol with the identifier's current ticker from the map file (as
MapToCurrentTickerin #9738 did) beforeAddSecurity. ThenFB V6OIPNZEM8V9is added asMETA, and"FB"keeps meaning the ETF. It needs map file access in the restore path, plus a test. - B. Merge as is: positions land on the right security. For the rest of the deployment, the first ticker means the restored security in string-based calls (
AddEquity, implicit conversions, string-based orders), which differs from master. The displayed ticker stays the first one.
Reply with the option you choose, or another way to go.
Everything else checks out. Open orders at start-up and brokerage-side order events use symbols that the brokerages build with Symbol.Create from current tickers, so they resolve to the same security as before. Errors that the ticker overload used to swallow (returning a null security) now throw, but every caller already failed on a null security (security.Exchange in holdings, algorithm.Securities[order.Symbol] in AddOpenOrder) or catches the exception (Brokerage.OnNewBrokerageOrderNotification), so nothing that starts on master stops starting with this change.
I set risk:high. The change alters which security string tickers resolve to in live algorithms, which needs the decision above.
| security = algorithm.AddSecurity(symbol.SecurityType, symbol.Value, resolution, symbol.ID.Market, fillForward, leverage, extendedHours); | ||
| // add by symbol, not by ticker: the ticker can be outdated, such as a deserialized symbol's first ticker, | ||
| // and resolving it today could land on a different security than the security identifier | ||
| security = algorithm.AddSecurity(symbol, resolution, fillForward, leverage, extendedHours); |
There was a problem hiding this comment.
AddSecurity(Symbol) caches symbol.Value in SymbolCache. For a restored holding, Value is the first ticker (FB for META), so string-based calls with that ticker (AddEquity("FB"), implicit Symbol conversion, string-based orders) now resolve to this security instead of the one that trades under that ticker today. I reproduced it in the new test: after setup, AddEquity("GOOG") returns GOOGL's identifier. See the review body for the decision.
There was a problem hiding this comment.
Remapped to the current ticker before adding, as decided, in 809482d: the restored security is META, and the test checks that AddEquity of the old ticker resolves to today's security.
|
Replying to #10222 (review): I own this PR, so I will make this decision for Martin. Option A. Remap the restored symbol to the identifier's current ticker from the map file before |
A restored equity symbol can carry an outdated ticker, such as FB for
META. Adding it as is cached that ticker for it, so string-based calls
like AddEquity("FB") resolved to the restored security instead of the
one trading as FB today. Remap the symbol to its current ticker from the
map file before adding it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Closes #9736
Description
GetOrAddUnrequestedSecurity()(Common/Extensions.cs) added equities, forex, CFDs and crypto throughAddSecurity(symbol.SecurityType, symbol.Value, ...), which drops theSymbol'sSecurityIdentifierand resolves the ticker again as of today. Holdings restored from thelive-holdingsbrokerage data carry the security's first ticker (for exampleFBfor META,NBfor BAC,ATHfor ELV). When that ticker now belongs to another security, the position was booked on it, or liquidated when that security is delisted.It now calls
AddSecurity(symbol, resolution, fillForward, leverage, extendedHours), so the security is added for the identifier it was given. Before adding an equity, the symbol is remapped to the ticker its identifier trades under today, from the map file (FB V6OIPNZEM8V9is added asMETA), so the old ticker isn't cached for it andAddEquity("FB"), implicit string conversions and string-based orders keep resolving to the security that trades asFBtoday. Options and futures already went throughAddOptionContract/AddFutureContractwith the symbol and are unchanged; the other security types resolve to the same identifier either way, so only equities whose saved ticker is outdated behave differently.Decision (@jhonabreul, on the review's question): remap the restored symbol to its current ticker from the map file before adding it, with a test.
The same path is used for unrequested securities from open orders at start up and from brokerage order events; those symbols are built from the brokerage's current ticker, so they resolve to the same security as before.
How Has This Been Tested?
BrokerageSetupHandlerTests.LoadsHoldingsForTheSecurityIdentifierWhenTheTickerIsOutdated: a GOOGL holding saved with its first tickerGOOG(which today resolves to a different security in the test data) is restored on GOOGL's identifier asGOOGL, no security is added for today'sGOOG, andAddEquity("GOOG")afterwards resolves to today'sGOOG. It fails on master (Expected: True But was: False) and without the remap (Expected: "GOOGL" But was: "GOOG"), and passes with the change.dotnet build /p:Configuration=Release ...thendotnet test ./Tests/bin/Release/QuantConnect.Tests.dll --filter "TestCategory!=TravisExclude&TestCategory!=ResearchRegressionTests"):Passed! - Failed: 0, Passed: 39858, Skipped: 100, Total: 39958.FB(FB V6OIPNZEM8V9):FB YTLZP82EVU5H(the ETF now trading as FB), with anFBdata subscription, and META has no holding.FB V6OIPNZEM8V9with tickerMETA(47 @ 648.98), its data subscription isMETA, andAddEquity("FB")then resolves toFB YTLZP82EVU5H.Types of changes
Checklist: