Floor and Ceil versus Denormals on CPU and GPU

(asawicki.info)

32 points | by ibobev 4 days ago

4 comments

  • yosefk 1 hour ago
    Flush denormals to zero. Even their inventor had trouble writing correct code in their presence - see the Appendix to that "what every programmer should know..." paper
    • loicd 1 hour ago
      > Even their inventor had trouble writing correct code in their presence

      I didn't know that. Could you provide a more specific reference?

    • kevmo314 2 hours ago
      > This is not the first time we can see Nvidia taking shortcuts to achieve maximum performance of their GPUs

      Why is implementing it correctly not performant? For context I have no idea how rounding is typically implemented anyways.

      • crote 3 hours ago
        Another thing to keep in mind is that CPU processing of denormals tends to be extremely slow - I vaguely recall running into something like a 10x slowdown a decade ago.

        For a lot of applications the difference between a denormal and zero is small enough to be irrelevant, so if you expect near-zero values to be common, enabling a denormals-to-zero compiler flag might give you a pretty nice performance boost for free.

        • adgjlsfhk1 1 hour ago
          cpus that aren't Intel are plenty fast on denormals. Intel is the only one where denormals are 100x slower. (and Intel has fixed that on their new cpus, but only on their e cores)
          • andrepd 32 minutes ago
            More like 100x, but not sure how true that is nowadays.
          • andrepd 29 minutes ago
            It's one of several issues with the design of IEEE floats, unfortunately. I wish we could start thinking more seriously about a new design, to complement if not replace IEEE in the long term. Posits are an example https://github.com/andrepd/posit-rust