rust-decimal: `powd` overflow

We got a similar issue with the following code:

>> let v = Decimal::from_f64(13.815517970299580976037625513).unwrap();
>> v.exp()

We are also using Decimal in the blockchain.

Of course, I can always do:

        for n in 2..=27 {
            term = term.checked_mul(self.div(Decimal::new(n, 0)))?;
            let next = result + term;
            let diff = (next - result).abs();
            result = next;
            if diff <= tolerance {
                break;
            }
        }

basically distributing the powers over the factorial.

But I guess this opens up its own can of worms, in terms of precision.

Do you think this method has merit? Problems as I see it are:

• More expensive / less performant.
• Gives wildly inaccurate results for larger exponents (e^100 or so); instead of just overflowing.
• powd is less precise for large bases, too.

Nice thing is that the number of terms can be easily adjusted. And, results are pretty good for small exponents (which is my case in particular).

As @schungx correctly points out, it’s due to the current implementation attempting to maintain precision. This “overflowing” precision is not uncommon in simple multiplication and division scenarios - i.e. typically we’d call this an “underflow” and automatically round the number to make it fit into the allocated bits. Of course, when we have more complex scenarios whereby a multiplication is just a small part of the formula you’ll start losing precision later down the track each time you round - especially as smaller numbers trend to zero.

The current pow implementation was purposely taking a lazy approach by way of leveraging checked_mul. The underlying implementation of checked_mul actually uses 192 bits (e.g. 96 + 96) to calculate the multiplication product before “shrinking” it back to the required 96 bits to help handle the underflow scenarios. I remember as I was implementing powd I started to expose the raw product however the size of the change quickly spiraled out of control - consequently I used the checked_ functions as a shortcut.

Anyway, one possible solution would be to keep the 196 bit precision product in tact until the calculation is complete - i.e. only rounding to 96 bits at the last moment. Unfortunately, this requires a fair bit of refactoring to get going since other operations would also need to be able to support Buf24 data structures. The other solution (which @schungx mentions above) is to “normalize” the product as you go effectively ignoring the overflow and saturating the result (by rounding). The risk of course is that you lose precision - especially since it can trend to zero.

The better solution is to of course keep as much precision, for as long as possible. This is definitely a large chunk of work which makes it tempting to wait until v2 for (since variable precision could make this much easier). The easier solution (for now) would be to provide saturating logic in the power function - perhaps as a separate function. It’d still require a bit of fiddling around to get going but could perhaps solve your requirement.

https://github.com/paupino/rust-decimal/blob/master/src/maths.rs#L316-L323

This is where it calculates the pow by converting it into exp, and this may be the place where it overflows.

Since you know the exponent will be <1, I suggest you break down the steps and display intermediate values to find out which step it is overflowing. Then you may simply add normalize calls?