Repository navigation
Add a performance benchmark for forward mode autodiff - #1664
reach2sayan wants to merge 4 commits into
Conversation
|
@reach2sayan : It is indeed of interest. Could you post the benchmark results on this thread? |
|
Google Benchmark's complexity fit over all orders 1–32: Results on an AMD Ryzen 7 PRO 6850U, GCC 15.2,
|
|
@reach2sayan : If you don't mind I'm going to try to get this fixed in this PR-I see you "allow edit by maintainers". |
Co-Authored-By: Claude <noreply@anthropic.com>
O(N^2) multiplications at order N, not the O(N^3) of Horner. Co-Authored-By: Claude <noreply@anthropic.com>
|
@reach2sayan: you are right. I pushed two commits to your branch: multivariate workloads for the benchmark, then the recurrences for Apple M-series, clang One variable, by derivative order:
Several variables, by order in each variable:
Against 100-digit arithmetic, the worst relative error in the 33 coefficients of Transcendental at order 32 goes from 2e8 ε to 14 ε. One tolerance in |
Skip zero coefficients; integer powers and non-finite roots take the old path; sin and cos share one recursion. Co-Authored-By: Claude <noreply@anthropic.com>
Adds
reporting/performance/autodiff_performance.cpp, a Google Benchmark forfvar(forward mode autodiff).exp(sin(x)) * log(1 + x²) / sqrt(x)example/autodiff_black_scholes.cpp(log,erfc)exp,log,sqrt,pow, and a double loop of mixedfvar/doublearithmetic)Proposition
Differentiating$y = \exp(x)$ with respect to $\varepsilon$ gives $y' = x' y$ . Expanding both sides as power series gives
The terms contributing to$\varepsilon^{k-1}$ are those with $j = k - m$ . Equating the coefficients of $\varepsilon^{k-1}$ on both sides therefore yields
Since each$y_k$ depends only on $y_0, \dots, y_{k-1}$ , the coefficients are computed in increasing order of $k$ , and the $k$ -th one costs $k$ multiplications. The total cost
is therefore
multiplications and a single scalar$$\exp(x) = \sum_{i=0}^{N} \frac{e^{x_0}}{i!} (x - x_0)^i$$ by Horner's scheme. This performs $N$ full series products of $\Theta(N^2)$ multiplications each, for a total of $\Theta(N^3)$ .
exp. The present implementation, by comparison, evaluates@NAThompson Would this be of interest? Otherwise I guess a benchmark is good enough