Explanation of INT8 ConvRot (FP8 is no longer needed)

3 comments

• liuliu 28 minutes ago
  One thing is not obvious to me is how ConvRot can be applicable beyond diffusion models. Especially for LLM decoding, as each ConvRot would be more expensive for a given decoding vector, and it is required now, so you cannot easily get the benefit for prefill only, while maintaining the same decoding performance.
• augment_me 29 minutes ago
  Alternative title: "Our invention makes INT8 better on older consumer cards without FP8 support"
• peter_d_sherman 1 hour ago
  >"FP8, which has been used as the standard until now, was only supported by the RTX 40/50 series. In contrast,

  INT8 ConvRot

  is attracting attention because it can benefit from hardware support even on the RTX 20/30 series."

  Related:

  https://blog.comfy.org/p/minimax-h3-day-0-support-in-comfyui

  [-]
  • spottedmarley 1 hour ago
    Just read that before finding this. Makes me feel like we're all on the same brain-wavelength. Im actually in the middle of pulling and setting up H3 locally (int8 convrot)