Poor precision with fract in MSL fast-math mode

Question

delphifirst OP

Created Feb ’24

Replies 2

Boosts 0

Participants 2

Here is an example fragment shader code (Rendering a cube with texCoord in [0, 1]):

colorSample.x = in.texCoord.x;

Which produce this result:

However, if I make a small change to the code like this:

colorSample.x = fract(ceil(0.1 + in.texCoord.x * 0.8) * 1000000) + in.texCoord.x;

Then it will produce this result:

If I disable fast-math in the second case, then it will produce the same image as in the first case. It seems that in fast-math mode, large parameter for fract() will affect precision of other operand in the same expression.

Is this a bug in fast-math mode? How should I circumvent this problem?

Boost

Answer 1

endecotp OP

Feb ’24

Here's my guess as to what it is doing:

Note that fract(x) = x - floor(x).

let C = ceil(0.1 + in.texCoord.x * 0.8) * 1000000 = 1000000
colorSample.x = fract(C) + in.texCoord.x
= C - floor(C) + in.texCoord.x
= ( C + in.texCoord.x ) - floor(C)     <--- this is what fast-math allows

The range of C + in.texCoord.x will be 1000000 to 1000001.

This reduces the available precision for the fractional part.

Is this a bug? Probably not, it's what fast-math is for. Don't enable fast-math unless you have a good reason.

0

Answer 2

delphifirst OP

Feb ’24

I found that using precise::fract can solve this problem. This is good since I can enable fast-math for other parts of my program.

Wish apple can provide more fine-grain fast-math control, e.g. function level, statment block level...

0