Repository navigation
[C++] Segmentation fault at SparseUnionType destructor when arrow.dll is unloaded via ODBC driver manager #48019
Description
Describe the bug, including details regarding any error messages, version, and platform.
We found that when arrow.dll is unloaded on Windows MinGW Msys and attached the back trace below. There is a segmentation fault occurring at SparseUnionType deallocation. However, our code does not use SparseUnionType so I am not sure what is the cause. Could it have to do with arrow.dll being loaded and unloaded too quickly? Because the lesser the tests that use arrow.dll, the more likely the segfault seems to occur.
Context: we develop a Flight SQL ODBC driver dll #47971 which has a dependency on the arrow.dll. The ODBC driver manager unloads the driver after use, and it also unloads all dependency dlls including arrow.dll. When arrow.dll is unloaded, we get segmentation fault from the SparseUnionType destructor.
Environment: Windows MinGW CLANG64 Msys Shell.
Stack Trace from gdb:
Thread 1 received signal SIGSEGV, Segmentation fault.
0x00007ff891c5cf31 in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
(gdb) bt
#0 0x00007ff891c5cf31 in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#1 0x00007ff89252d364 in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#2 0x00007ff89252be44 in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#3 0x00007ff8925ab11b in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#4 0x00007ff8925abe68 in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#5 0x00007ff8925a933b in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#6 0x00007ff8925a736d in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#7 0x00007ff8925a71fc in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#8 0x00007ff8925b0b30 in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#9 0x00007ff8925ab06a in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#10 0x00007ff8925aab8c in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#11 0x00007ff8925ab244 in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#12 0x00007ff8925ab28e in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#13 0x00007ff8925abf08 in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#14 0x00007ff8925abdd0 in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#15 0x00007ff8925ab1e3 in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#16 0x00007ff8925a3eb7 in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#17 0x00007ff8925abeb8 in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#18 0x00007ff8925abd70 in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#19 0x00007ff8925ab183 in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#20 0x00007ff89369d782 in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#21 0x00007ff8b51a2b99 in ucrtbased!_initterm_e () from C:\WINDOWS\SYSTEM32\ucrtbased.dll
#22 0x00007ff8b51a25c5 in ucrtbased!_initterm_e () from C:\WINDOWS\SYSTEM32\ucrtbased.dll
#23 0x00007ff8b51a2717 in ucrtbased!_initterm_e () from C:\WINDOWS\SYSTEM32\ucrtbased.dll
#24 0x00007ff8b51a2d44 in ucrtbased!_execute_onexit_table () from C:\WINDOWS\SYSTEM32\ucrtbased.dll
#25 0x00007ff89339aae9 in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#26 0x00007ff89339c2dd in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#27 0x00007ff89339c0b7 in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#28 0x00007ff89339c44f in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#29 0x00007ff89339c5c2 in arrow!??1SparseUnionType@arrow@@UEAA@XZ () from C:\Users\path-to\cpp\debug-build\debug\Debug\arrow.dll
#30 0x00007ff9c44ff89e in ntdll!RtlEncodeRemotePointer () from C:\WINDOWS\SYSTEM32\ntdll.dll
#31 0x00007ff9c43abcae in ntdll!RtlRaiseException () from C:\WINDOWS\SYSTEM32\ntdll.dll
#32 0x00007ff9c4437a48 in ntdll!LdrGetProcedureAddressForCaller () from C:\WINDOWS\SYSTEM32\ntdll.dll
#33 0x00007ff9c443679e in ntdll!LdrGetProcedureAddress () from C:\WINDOWS\SYSTEM32\ntdll.dll
#34 0x00007ff9c4436942 in ntdll!LdrGetProcedureAddress () from C:\WINDOWS\SYSTEM32\ntdll.dll
#35 0x00007ff9c44365e4 in ntdll!LdrGetProcedureAddress () from C:\WINDOWS\SYSTEM32\ntdll.dll
#36 0x00007ff9c43b4730 in ntdll!LdrUnloadDll () from C:\WINDOWS\SYSTEM32\ntdll.dll
#37 0x00007ff9c1b67f19 in KERNELBASE!FreeLibrary () from C:\WINDOWS\System32\KernelBase.dll
#38 0x00007ff9aca41e9e in ValidateErrorQueue () from C:\WINDOWS\SYSTEM32\odbc32.dll
#39 0x00007ff9aca5588c in ODBCGetTryWaitValue () from C:\WINDOWS\SYSTEM32\odbc32.dll
#40 0x00007ff9aca019a7 in SQLFreeHandle () from C:\WINDOWS\SYSTEM32\odbc32.dll
#41 0x00007ff659b92004 in arrow::flight::sql::odbc::ODBCRemoteTestBase::FreeEnvConnHandles (this=0x6abc10)
at C:/path-to/cpp/src/arrow/flight/sql/odbc/tests/odbc_test_suite.cc:85
#42 0x00007ff659b91e98 in arrow::flight::sql::odbc::ODBCRemoteTestBase::Disconnect (this=0x6abc10)
at C:/path-to/cpp/src/arrow/flight/sql/odbc/tests/odbc_test_suite.cc:76
#43 0x00007ff659b9462c in arrow::flight::sql::odbc::FlightSQLODBCMockTestBase::TearDown (this=0x6abc10)
at C:/path-to/cpp/src/arrow/flight/sql/odbc/tests/odbc_test_suite.cc:417
#44 0x00007ff92287a475 in ?? () from C:\msys64\ucrt64\bin\libgtest.dll
#45 0x00007ff922860a1b in ?? () from C:\msys64\ucrt64\bin\libgtest.dll
#46 0x00007ff922860fec in ?? () from C:\msys64\ucrt64\bin\libgtest.dll
#47 0x00007ff922865ca1 in ?? () from C:\msys64\ucrt64\bin\libgtest.dll
#48 0x00007ff92286c610 in ?? () from C:\msys64\ucrt64\bin\libgtest.dll
#49 0x00007ff92286cf7a in ?? () from C:\msys64\ucrt64\bin\libgtest.dll
#50 0x00007ff9adc22304 in ?? () from C:\msys64\ucrt64\bin\libgmock_main.dll
#51 0x00007ff659b910c9 in __tmainCRTStartup () at D:/W/B/src/mingw-w64/mingw-w64-crt/crt/crtexe.c:236
--Type <RET> for more, q to quit, c to continue without paging--
#52 0x00007ff659b91416 in mainCRTStartup () at D:/W/B/src/mingw-w64/mingw-w64-crt/crt/crtexe.c:122
Component(s)
C++, FlightRPC
Activity
alinaliBQ commentedon Nov 3, 2025
I recall that unloading is not supported by the Arrow dll files in general, and it might be the reason why we are seeing this issue.
[-][C++] Segmentation fault at `SparseUnionType` destructor when arrow.dll is unloaded[/-][+][C++] Segmentation fault at `SparseUnionType` destructor when arrow.dll is unloaded via ODBC driver manager[/+]alinaliBQ commentedon Nov 4, 2025
ODBC tests are failing due to segmentation fault at teardown on MinGW Windows (CI run), while the tests pass on MSVC from my local testing. The good news is that MSVC is the main platform for ODBC builds so most end users will not be impacted by this segmentation fault error. (Though it is worth noting we currently don't have MSVC build in the CI as we do for C++ MinGW Windows.)
According to stack trace, the segmentation fault is from Arrow library DLLs. This is understandable as Arrow DLLs don't support unloading.
I propose to ignore the segmentation fault at ODBC test teardown in MinGW CI, so the CI isn't broken by the ODBC tests. Please let me know your thoughts (and feel free to tag more ppl to join this conversation, as I plan to bring this up in the Arrow community meeting this week as well). @lidavidm @kou
kou commentedon Nov 4, 2025
How about disabling ODBC in MinGW CI?
alinaliBQ commentedon Nov 4, 2025
I am okay with disabling ODBC in MinGW CI for the time being. Our team will test ODBC on MSVC locally. And when the tests stop breaking the CI, we can re-enable ODBC
kou commentedon Nov 5, 2025
How about enabling ODBC in MSVC CI?
alinaliBQ commentedon Nov 5, 2025
You're referring to the MSVC CI at cpp_windows.yml, right? Sure my team can look into enabling ODBC in MSVC
kou commentedon Nov 5, 2025
Yes. If we focus on only MSVC support, it's better that we use MSVC CI job instead of MinGW CI job.
Remove ODBC from MinGW build
Remove ODBC from MinGW build
alinaliBQ commentedon Nov 19, 2025
Here is an update:
Tl;dr ODBC in MSVC CI are blocked by 2 things, 1) segmentation fault from Arrow.DLL library during unloading, and 2) Flight and Flight SQL tests in MSVC CI need to be enabled first.
I find there are segmentation fault errors from Arrow library unloading in the MSVC CI job and local MSVC as well. Didn't catch this earlier because the segmentation fault occurs after the test teardown, and the google test executable doesn't print out "segmentation fault" upon exiting tests, and we needed to run
Write-Host "Exit code: $LASTEXITCODE"(PS command) to see exit code which is the only indicator on MSVC that segmentation fault has occurred. More time is needed to finalize the segmentation fault fix but I don't have capacity right now. 😔I ran into build errors with enabling tests in MSVC. I was able to resolve some errors in draft PR GH-48019 : [C++][FlightRPC] work-in-progress Enable ODBC in MSVC CI #48067 but there are still some build issues with linking VCPKG dependencies in the test example. Some work is still needed to enable Flight and Flight SQL and tests on MSVC CI. Currently I don't have capacity for this item either.
alinaliBQ commentedon Nov 26, 2025
Raise issues for the blockers above:
GH-48834 [C++][FlightRPC][Doc] Add instructions to run ODBC tests in …
apacheGH-48834 [C++][FlightRPC][Doc] Add instructions to run ODBC tes…