Build anything small into a container on your laptop, push it to DockerHub or the Github package registry then host it on fly.io for free.

Great write up, thanks. For video learners, Wolfgang does a good step-by-step on YouTube

Guide to Self Hosting LLMs with Ollama.

• Download and run Ollama
• Open a terminal, type ollama run llama3.2

Love the effort you’ve put into this question. You’ve clearly done some quality research and thinking.

When I asked myself this same question a couple of years ago, I ended up just buying a second hand Synology NAS to use alongside my mini-pc. That would meet your criteria, and avoids the (I’m not sure what magnitude) reliability risk of using disks connected over USB. It’s more proprietary than I’d like, but it’s battle tested and reliable for me.