↓ Skip to main content
SchifanoIA

SchifanoIA

For some time now, I’ve had an artificial intelligence at home that runs entirely on my own machines, in my homelab. No subscriptions, no data leaving my network. It didn’t arrive all at once: I started with a PC set up one way, then it evolved into something else, and so on. This blog was born to tell the story of that evolution and, above all, of what I’ve built on top of it.

I’m a computer scientist, and AI caught my attention right from the start, both as a subject of study and as a working tool. The turning point, for me, came with open-weight models, especially the Chinese ones: for the first time, a model you could download and run locally was capable enough to be genuinely useful, and not just a lab curiosity or a toy for nerds. That’s where the idea came from to build my own private AI, without spending a fortune.

It’s natural to wonder why anyone would go to all this trouble, when a subscription to Anthropic or OpenAI gives you the best models in the world with zero maintenance. It’s a fair question, and for many people the right answer is indeed the subscription. But there are good reasons to take the other road. The first is privacy: there’s data I would never hand over to an external service, and locally the problem simply doesn’t arise. The second is control: I choose the model, the version and the configuration, and nobody can change them behind my back, shut down the service, or double the price overnight. The third, perhaps the most tempting for a computer scientist, is the chance to understand: a model running at home can be observed, measured, put to the test. As for costs, it’s best to be honest: hardware has a price and electricity isn’t free, especially here in Italy. The advantage lies not so much in saving money as in knowing exactly how much you’re spending, with no surprises.

You also need to know what not to expect. A local model on fairly modest hardware won’t match a frontier model, and locally you almost always use quantized models, that is, models compressed to take up less memory, with an inevitable drop in quality compared to the original. The software orchestrating the model can partly make up for this, but the gap remains. And yet progress is extremely fast: today, models of relatively small size can handle tasks that seemed out of reach only a few months earlier. Versions 3.6 and 3.8 of the Qwen model family are a striking example.

Objectively assessing the quality of a model is far from a trivial problem, and I don’t claim to solve it. Here I’ll adopt a simpler and, I believe, more honest criterion: a model is good enough if it makes the use cases I care about work systematically and reliably. Leaderboards matter little; what matters is that the job gets done, every time it’s needed.

In the pages that follow, then, I’ll talk about three things: hardware, and how it has evolved over time while trying to keep spending in check; software, which tools I use and how I configure them; and use cases, what I’ve built and how well it actually works. On this last point I have a clear position. Online you see plenty of examples where an LLM is called upon to deliver the Hacker News headlines or the weather forecast every morning: nice demos, but for things like that a script is enough. I’m interested in something else. I’m interested in the tedious tasks that eat up time, and in those where AI offers real help, something I wouldn’t have done without it, or would have done worse. For example, managing the utility bills for several apartments, automatically collecting their data and amounts into a database where I have everything under control, is one of those things that saves me precious time every month.

If the idea sparks your curiosity, let’s start with this.

Recent

I hate paperwork: how an AI agent manages my rentals

·1583 words·8 mins
Application Llm-Wiki Hermes Karpathy Rag Nextcloud Telegram Automation
I admit it, I hate paperwork, and I suspect many of you feel the same way. Still, we have to live with it: contracts, invoices, utility bills and documents of every kind are part of real life. This is surely where AI can lend a hand: documents are essentially text, and LLMs are pretty good at handling text. Will local models be too? We’ll find out soon.

Proxmox, vLLM and Hermes: the software on my local AI PC

·1549 words·8 mins
Software Proxmox Vllm Qwen Hermes Benchmark
The perfect software, like the perfect hardware, doesn’t exist: it depends on what you want to achieve. I favour stability and reliability, and I always use open-source software or, in the case of LLMs, open-weight models. In AI everything changes at breakneck speed: chasing every new release is impossible and not very useful either, but you often find yourself having to try a new version of something or tweak a delicate parameter. It takes next to nothing to break everything, beyond repair. Without a solid safety net, the PC stays out of action until you find the time to fix it.

A cheap but effective computer

·1400 words·7 mins
Hardware
Like every computer scientist or enthusiast, over the years I’ve piled up an impressive number of components: CPUs, memory, disks, cases, power supplies and boards of every kind. Anything more than 10 years old is considered obsolete, and yet these components are still perfectly functional and can run almost anything decently. Does that apply to AI models too? Definitely not! But there is a loophole: what matters is the GPU, because the LLM runs entirely there. The rest of the machine doesn’t need to be particularly sophisticated.