Proper use of NSDecimalNumberHandler

Question

Created Oct ’17

Replies 2

Boosts 0

Participants 2

Morning All:

I'm using NSDecimalNumbers for arithmetic in my application. I'm having some problems with rounding. If I want to do something like:

25 / 99 * 99 - 25. Obviously it's a repeating decimal and there's going to be some change left at the end of the application. After the 'divide', by result is:

0.252525252525252525252525252525252526

After the multiply:

25.000000000000000000000000000000000074

And after the subtraction:

0.000000000000000000000000000000000074

I'm using a behaviour defined as follows to try and eliminate these rounding errors

aHandler = [NSDecimalNumberHandler decimalNumberHandlerWithRoundingMode:NSRoundUp
                                                                          scale:kRoundingScale  <= 36
                                                               raiseOnExactness:NO
                                                                raiseOnOverflow:NO
                                                               raiseOnUnderflow:NO
                                                            raiseOnDivideByZero:NO];

I've tried NSDecimalNoScale to try and see what that might change, and when I do the multiply, I get the following result:

24.9999999999999999999999999999999999998

The divide returns:

0.25252525252525252525252525252525252525.

Can someone provide some insight as to how to best define a rounding mode to best eliminate the extraneous bits at the end of a calculation?

Thanks in advance for your time

Best Regards

John

Boost

Answer 1

Claude31 OP

Oct ’17

I tried this in playgroud :

let calc1 : Double = 25 / 99
let calc2 : Double = calc1 * 99
let calc3 = calc2 - 25

I get calc3 = 0

I tried with an explicit Double

let calc1bis : Double = Double(25) / Double(99)
let calc2bis : Double = calc1bis * Double(99)
let calc3bis = calc2bis - 25

Finally

let calc = 25 / 99 * 99 - 25

gives also 0

But

let calc1 = 25 / 99 // 0 here

let calc2 = calc1 * 99 // 0 here

let calc3 = calc2 - 25

0

Answer 2

IntelJohnny OP

Oct ’17

Thanks Claude.

I originally was using doubles (actually double_ts), but found that those didn't quite work with the requirements. NSDecimalNumbers helped alot, except for the 'change' problem.

I think that I'm not correctly setting up the NSDecimalHandler properly to handle the rounding.

I was originally going to whip up a couple of test harnesses to see if I can get a handler that meets the requirements.

Have a good day

John

0