Wherever I wander I wonder whether I’ll ever find a place to call home…

  • 1 Post
  • 89 Comments
Joined 21 days ago
Cake day: December 31st, 2025
  • wonderingwanderer@sopuli.xyztoTechnology@lemmy.world•AI boom could falter without wider adoption, Microsoft chief Satya Nadella warnsEnglish
    2·
    6 hours ago

    See, you have more experience in the matter than I do, hence the caveat that I’m not an expert. Thanks for sharing your experience.

    Then again, I’d consider 128GB of memory to be fairly serious hardware, but if that’s common among hobbyists then I stand corrected. I was operating on the assumption that 64GB of RAM is already a lot

    All in all, 106 billion parameters on 128GB of memory with quantization doesn’t surprise me all that much. But again, I’m just going off of the vague notions I’ve gathered from reading about it.

    The focus of my original comment was more on the fact that self-hosting is an option, I wasn’t trying to be too precise with the specs. My bad if it came off that way

  • wonderingwanderer@sopuli.xyztoTechnology@lemmy.world•AI boom could falter without wider adoption, Microsoft chief Satya Nadella warnsEnglish
    3·
    10 hours ago

    If Microsoft cared about privacy then they wouldn’t have made windows practically spyware. Even if they install AI locally in the OS, it’s still proprietary software that constantly sends data back to the mothership, consuming your electricity and RAM to do so. Linux has so many options, there’s really no reason not to switch.

    Small LLMs already exist for local self-hosting, and there are open-source options which won’t steal your data and turn you into a product.

    https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/

    Bear in mind that the number of parameters your system can handle is limited by how much memory is available, and using a quantized version can increase the number of parameters you can handle with the same amount of memory.

    Unless you have some really serious hardware, 24 billion parameters is probably the maximum that would be practical for self-hosting on a reasonable hobbyist set-up. But I’m no expert, so do some research and calculate for yourself what your system can handle.

  • wonderingwanderer@sopuli.xyztoEurope@feddit.org•Trump’s bizarre letter to Norway’s prime minister in fullEnglish
    7·
    2 days ago

    “Wahhh, you didn’t give me the Nobel Peace Prize, so now I’m making thinly veiled threats to invade our mutual ally. Also, look I got a FIFA peace prize and I stole a medal from the Nobel laureate.”

    Except he probably doesn’t know the word laureate…

    And there literally are documents recording Denmark’s rights to Greenland. But by trump’s logic of “Only cause boats landed there a few hundred years ago,” I guess the US doesn’t have rights to the land it’s on?

    Also, trump has done nothing for NATO except weaken it. Other NATO countries are gearing up their commitments in spite of him, not because of him.