A software developer and Linux nerd, living in Germany. I’m usually a chill dude but my online persona doesn’t always reflect my true personality. Take what I say with a grain of salt, I usually try to be nice and give good advice, though.
I’m into Free Software, selfhosting, microcontrollers and electronics, freedom, privacy and the usual stuff. And a few select other random things as well.
I’m a bit below 20W. But I custom-built the computer a long time ago with an energy-efficient mainboard and a PicoPSU. I think other options for people who don’t need a lot of harddisks or a graphics card include old laptops or Mini-PCs. Those should idle at somewhat like 10-15W. It stretches the definition of “desktop pc” a bit, but I guess you could place them on a desk as well 😉
There’s another community for this: !localllama@sh.itjust.works
Though we mostly discuss the news and specific questions there, beginner questions are a bit more rare.
I think you already got a lot of good answers here, LMStudio, OpenWebUI, LocalAI…
I’d like to add KoboldCpp that’s kind of made for gaming/dialogue, but it can do everything. And from my experience it’s very easy to set up and bundles everything into one program.
Thanks for your insight. Reading these stories always makes me feel data should stay on own premises with extra security measures. And yes, on my VPS, imaging the storage is one click and I believe it’s done online without any interruption of service. Not that I do a lot of illegal stuff on the internet. But with the current situation in the US and the general overboarding surveillance, I think i’d like to keep their government and agencies out of my emails and personal stuff… (And maybe even what I do publicly and within legal limits.)
Though I didn’t ask about privacy here, but anonymity. And I guess selfhosting stuff at home isn’t an option either. Everyone can tell my ISP and location to like 30km with that. And link the IP to other activities.
I think they should be roughly in a similar range for selfhosting?! They’re both power-efficient. And probably have enough speed for the average task. There might be a few perks with the ThinkCentre Tiny. I haven’t looked it up but I think you should be able to fit an SSD and a harddrive and maybe swap the RAM if you need more. And they’re sometimes on sale somewhere and should be cheaper than a RasPI 5 plus required extras.