When the fractional part of a float fixes your shader

(crocidb.com)

52 points | by vinhnx 1 day ago

6 comments

  • Sharlin 9 minutes ago
    Should be noted that `fract(x)` is importantly only equivalent to `x - floor(x)` when x >= 0. Which it was in this case, but often negative values are also possible/expected and the latter should be used instead. This is an easy mistake to make.
  • earth-tattoo 3 hours ago
    Nice article. What a joy to read. Reminds me of what hacker news was until a few years ago!

    I haven't worked with opengl/shaders in a while, but was thinking about some algorithm I wrote a few years ago, while in shower yesterday. It brought back the memories of how difficult it was for me to first understand the whole concept of shaders. There's basically no main function, no for loops etc. Your shader is called for each pixel, for every frame. I was wondering how easy it would have been in today's world. I spent like 2 months on something that is just a prompt now.

    • mitxela 54 minutes ago
      If you prompted a shader today, you still wouldn't understand shaders. The LLM also has absolutely no understanding of whether a graphical effect looks good.
  • flohofwoe 46 minutes ago
    It turned out to be something different, but another much more common source of shader output differences is when a float value is clamped to integer (with the value being very close to the closest integer), and then use the clamped value as index (or tex coord with an unfiltered sampler). This may result in an off-by-one error on some gpu/driver combos but not others. Completely understandable why it happens, but hard to catch unless testing on a wide range of GPUs and drivers.

    TL;DR: don't expect that floating point operations on GPUs are strictly IEEE-754 compatible

  • xyzsparetimexyz 3 minutes ago
    Good investigation! The whole webgl stack is such a piece of shit huh
  • kg 30 minutes ago
    The classic HLSL compiler is infamous for being full of bugs (and slow, for that matter) so I wasn't surprised to see it come up here...

    Fantastic seeing the author lay out all the steps involved in getting a renderdoc capture of a browser and what's involved in debugging shader issues in a webpage.

    It's interesting that the issue only manifested on the geforce 40xx if it was an fxc issue though... wonder if it's fxc combined with a driver issue and not just one or the other?

    • CrociDB 26 minutes ago
      hey! author here. I think I missed the opportunity to mention that, it wasn't only reproducible on the 40xx. it was my first hypothesis, but after I figured out it was a DirectX11 issue, i was able to reproduce on another windows machine with a different configuration. I just didn't have enough windows machines available to begin with. :)
  • LoganDark 27 minutes ago
    > Root cause: Nvidia driver 595.79 (RTX 4070) miscompiles GLSL fract() usage in this shader’s context for large-magnitude operands (~100–1000): the returned fractional part is temporally discontinuous (moves smoothly, then jumps).

    It looks like the LLM hallucinated a diagnosis of a miscompilation based on which fix worked. I would not treat its claim as accurate without further investigation.

    Edit: Read the rest of the article and it looks like further investigation was performed, and it had nothing to do with the GPU driver after all. Good stuff.