Huawei Zurich Lab’s New Open-Source Tech Lets LLMs Run on Consumer GPUs · TechNode
technode.com
Huawei’s Zurich Computing Systems Laboratory has released SINQ (Sinkhorn Normalization Quantization), an open-source quantization method that reduces the
  • wizzor@sopuli.xyzEnglish
    4·
    2 days ago

    Little sparse on detail, I regularly run LLMs on 5 year old CPUs so no problem there, I wonder how the approach compares in memory requirements to existing quantization methods.