Hello,

I’m trying to decide whether I should upgrade to a Z790-P with Dasahro firmware. The motherboard with Dasharo firmware is about 50% free currently (not sure if it can entirely be freed, IME is also still present and can’t be fully removed like the older boards). I’m only thinking about getting this board so that I can run larger LLMs locally. My OS environment is 100% FOSS, I put anything non-free into a virtual machine and use it in there.

I’m planning on adding Libreboot support for the Intel DX38BT 82X38 and the Gigabyte GA-P43T-ES3G. Both boards can be freed entirely, IME isn’t present on either of these boards. The Intel board is capable of utilizing up to 8GB DDR3 ECC memory, so I can use it as my server board. The Gigabyte board can use up to 16GB DDR3 and can use CPUs like the QX9650 or even the QX9770, I plan to use it as my main desktop. I do light gaming, I already have a 7900 XTX, so I’m just going to plop it into this board once I get support added.

I already use LibreCMC and an OpenWRT network switch. I’m nearly complete, but man, I don’t think I can give up using LLMs. I’m afraid of being left behind in my field. I just started working in IT and I am a bit worried if I were to restrict myself from using it. I definitely don’t want to stick with using ChatGPT, hence the reason why I’m thinking about running one locally and putting it on a VPS so I can access it while away from home. Even then, I understand I would have to trust the VPS provider not to do anything bad.

I don’t have a crazy threat model or anything, I’m just a nerd that likes to have control over my tech. What are your thoughts? Should I give up my FOSS environment or no?

  • just_another_person@lemmy.world
    0·
    4 months ago

    If you’re that dependent in LLMs, and you like your current setup, why not just build a dedicated box for LLMs? You don’t need a GPU like that to get decent speed, and you can probably get a used one that works just fine to plop in another box, or an AMD APU.

    • Time@sh.itjust.worksOP
      0·
      4 months ago

      I don’t want to have anything proprietary on my network. I thought about putting the computer on a separate VLAN and all, but the NSA can probably still infiltrate any defense I have on my network. Regardless, there are devices on my main network my family uses that are proprietary. So, maybe I should keep the Z790-P for dedicated tasks like LLMs? I would have 3 computers running, my server, LLM machine, and my main desktop.