Skip to content

Avoid NaN real components in complex tanh after overflow - #1435

Draft
agourakis82 wants to merge 1 commit into
xtensor-stack:masterfrom
agourakis82:fix/complex-tanh-overflow-1432
Draft

agourakis82 wants to merge 1 commit into
xtensor-stack:masterfrom
agourakis82:fix/complex-tanh-overflow-1432

Conversation

@agourakis82

Copy link
Copy Markdown

Complex tanh can return a NaN real component for finite inputs once cosh(2 * real(z)) overflows. The existing formula divides sinh(2 * real(z)) by that denominator, producing an infinity/infinity quotient. The issue #1432 examples (356, 0) and (-400, 1) for double, and (45, 0) and (-100, 2) for float, reproduce this failure.

When the denominator is infinite, select the correctly signed saturated real component. Keep the ordinary finite path, imaginary-component formula and fast-math path unchanged. A NaN denominator is not treated as an overflow saturation case.

Add mixed-lane float/double tests with both real signs, maximum finite real magnitude, zero/nonzero imaginary parts and ordinary finite controls. They check finite classification and component values against C++23 std::tanh with floating-point tolerances; they do not assert zero sign bits.

Validation from upstream ebe9991:

  • Native Apple clang 21, C++23, arm64/NEON64: the same new regression fails on unchanged headers for both types, then passes with this correction. All four cases in the complete complex hyperbolic unit pass (96 assertions).
  • That unit also passes AddressSanitizer and UndefinedBehaviorSanitizer.
  • Full native CMake/CTest suite passes on arm64: 432 doctest cases, 77,716 assertions; CTest 1/1.
  • Native GCC 13.3, Linux x86-64, C++23: identical before/after comparisons and the complete complex hyperbolic unit pass in SSE2 and AVX2 builds (Slurm 18071, exit 0:0).
  • The original issue reproducer was compiled unchanged on the base and candidate. The four reported NaN real components become the correct signed finite values. The full comparison still reports some signed-zero differences in the imaginary component, which are retained as an explicit limitation rather than presented as fixed.

This addresses only the tanh real-overflow cases in #1432 and should not close that issue. Imaginary signed-zero/underflow behavior, floating-point exception flags, pole cases, fast-math and the remaining complex functions are outside this change. The branch starts from upstream master independently of #1433 and #1434.

AI assistance supported investigation, implementation and native validation.

When the denominator cosh(2*real(z)) + cos(2*imag(z)) overflows, return the correctly signed saturated real component instead of the inf/inf quotient. Keep the ordinary finite path, imaginary formula and fast-math path unchanged.

Add mixed-lane float/double regression tests. Validate unchanged-base failure and corrected results in C++23 on arm64, SSE2 and AVX2, focused ASan/UBSan and the complete arm64 CMake suite. Signed-zero discrepancies and floating-point exception behavior remain outside this fix.

Assisted-by: OpenAI Codex

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant