The new structure matches that of Arm Optimized-routines,
to facilitate files traceability.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Vector variants of the new C23 powr routines.
These provide same maximum error error as pow by virtue of
relying on shared approximation techniques and sources.
Note: Benchmark inputs for powr(f) are identical to pow(f).
Performance gain over pow on V1 with GCC@15:
- SVE powr: 10-12% on subnormal x, 12-13% on x < 0.
- SVE powrf: 15% on all x < 0.
- AdvSIMD powr: for x < 0, 40% if x subnormal, 60% otherwise.
- AdvSIMD powrf: 4% on x subnormals or x < 0.
Optimize handling of subnormal x and/or negative x.
Some cleanup in attributes, macros and improving overall consistency.
Move core computation to header
Introduce config parameter to turn sign_bias on/off.
Performance improvement on V1 with GCC@15:
- AdvSIMD pow: 10-15% on subnormals.
- AdvSIMD powf: 30 to 70% on subnormals or x < 0, <=3% on x > 0.
- SVE pow: 10-15% on subnormals, <=3% otherwise.
- SVE powf: no significant variations in codegen/perf.
powf:
Update scalar special case function to best use new interface.
pow:
Make specialcase NOINLINE to prevent str/ldr leaking in fast path.
Remove depency in sv_call2, as new callback impl is not a
performance gain.
Replace with vectorised specialcase since structure of scalar
routine is fairly simple.
Throughput gain of about 5-10% on V1 for large values and 25% for subnormal `x`.
Reviewed-by: Wilco Dijkstra <Wilco.Dijkstra@arm.com>
Fixed svld1rq using incorrect predicates (BZ #33642).
Next to no performance variations (tested on V1).
Reviewed-by: Wilco Dijkstra <Wilco.Dijkstra@arm.com>
Fix a bug in predicate logic introduced in last change.
A slight performance improvement from relying on all true
predicates during conversion from single to double.
This fixes BZ #33299.
Reviewed-by: Wilco Dijkstra <Wilco.Dijkstra@arm.com>
Polynomial order was unnecessarily high, unlocking multiple
optimizations.
Max error for new SVE expf is 0.88 +0.5ULP.
Max error for new SVE coshf is 2.56 +0.5ULP.
Performance improvement on Neoverse V1: expf (30%), coshf (26%).
Reviewed-by: Wilco Dijkstra <Wilco.Dijkstra@arm.com>
Add inline helper for expm1 and rearrange operations so MOV
is not necessary in reduction or around the special-case handler.
Reduce memory access by using more indexed MLAs in polynomial.
Speedup on Neoverse V1 for expm1 (19%), sinh (8.5%), and tanh (7.5%).
Add inline helper for log1p and rearrange operations so MOV
is not necessary in reduction or around the special-case handler.
Reduce memory access by using more indexed MLAs in polynomial.
Speedup on Neoverse V1 for log1p (3.5%), acosh (7.5%) and atanh (10%).
Remove spurious ADRP and a few MOVs.
Reduce memory access by using more indexed MLAs in polynomial.
Align notation so that algorithms are easier to compare.
Speedup on Neoverse V1 for log10 (8%), log (8.5%), and log2 (10%).
Update error threshold in AdvSIMD log (now matches SVE log).
Remove spurious ADRP. Improve memory access by shuffling constants and
using more indexed MLAs.
A few more optimisation with no impact on accuracy
- force fmas contraction
- switch from shift-aided rint to rint instruction
Between 1 and 5% throughput improvement on Neoverse
V1 depending on benchmark.
Pierre Blanchard