[{"content":"","date":"18 October 2026","externalUrl":null,"permalink":"/en/categories/application/","section":"Categories","summary":"","title":"Application","type":"categories"},{"content":"","date":"18 October 2026","externalUrl":null,"permalink":"/en/tags/automation/","section":"Tags","summary":"","title":"Automation","type":"tags"},{"content":"","date":"18 October 2026","externalUrl":null,"permalink":"/tags/automazione/","section":"Tags","summary":"","title":"Automazione","type":"tags"},{"content":"","date":"18 October 2026","externalUrl":null,"permalink":"/en/blog/","section":"Blog","summary":"","title":"Blog","type":"blog"},{"content":"","date":"18 October 2026","externalUrl":null,"permalink":"/en/categories/","section":"Categories","summary":"","title":"Categories","type":"categories"},{"content":"","date":"18 October 2026","externalUrl":null,"permalink":"/en/tags/hermes/","section":"Tags","summary":"","title":"Hermes","type":"tags"},{"content":"I admit it, I hate paperwork, and I suspect many of you feel the same way. Still, we have to live with it: contracts, invoices, utility bills and documents of every kind are part of real life.\nThis is surely where AI can lend a hand: documents are essentially text, and LLMs are pretty good at handling text. Will local models be too? We\u0026rsquo;ll find out soon.\nThe result, right away # Before telling you how I got there, here\u0026rsquo;s what I get today. I write a question to my agent on Telegram and get the answer within a few seconds, sources included (personal and/or confidential information has been redacted):\nUser: in which room Anna ***** was and in which year? How much was her rent? check the affitti wiki . Assistant: Done — answer from the wiki (log entry appended): Anna *****, at via ** **********, **, in 2019–2020 (contract 01/10/2019–30/09/2020, terminated early by mutual agreement on 30/06/2020). Room: no individual room — her contract was for the entire apartment, let jointly (in solido) to five students (A, B, C, D, E). Per-room (\u0026#34;stanza\u0026#34;) contracts only began from Nov 2020. Rent: €****/month (€*****/year) for the whole apartment, shared by the five; deposit €****. Sources: [[tenant-anna]], [[lease-xx-xxxxxxxxx-2019]], [[property-xx-xxxxxxxxx-xx]] (registered contract, Agenzia Entrate 15/10/2019, cedolare secca). Notice one detail: the question started from a wrong assumption (a \u0026ldquo;room\u0026rdquo; that didn\u0026rsquo;t exist), and the agent corrected it, explaining when the per-room contracts began. Finding the same information by hand would have meant opening one PDF after another.\nThe problem # I took on the management of the rental contracts for a few apartments, which take up a considerable amount of my family\u0026rsquo;s (my wife\u0026rsquo;s) time. One of these apartments in particular, let to students, generates most of the paperwork: in fact, there\u0026rsquo;s a contract for each room. As we all know, students come and go; there are constant terminations, new contracts and renewals, and with 4 rooms the amount of paperwork is remarkable, and so is the effort of remembering all the deadlines.\nWhat I want is a system that, whenever the documents I mentioned are added, builds and updates a knowledge base that can be queried. It\u0026rsquo;s not a database in the traditional sense, but something similar to a Retrieval-augmented generation (RAG) approach, without the need for (and the complexity of) embeddings.\nA. Karpathy, one of the most influential researchers in the field, proposed the llm-wiki pattern. Instead of retrieving documents for every question as in traditional RAG, it\u0026rsquo;s the LLM itself that builds and maintains a knowledge base in markdown, persistent and growing richer over time. In practice, you put the raw sources in a folder and the model writes and indexes a wiki that you can then query, producing summaries of the sources, entries on concepts, articles and backlinks. Outputs such as slides, charts and answers are also filed in the wiki as new articles, so knowledge accumulates instead of starting from scratch at every session.\nWhat I did # Let me say it candidly right away: I didn\u0026rsquo;t invent anything, I simply connected a few pieces together.\nThe heart of the system is Hermes, the agentic platform that runs in a dedicated VM on my local AI PC and uses Qwen3.8-27B as its model; I described the whole stack in the previous post.\nThe PDF documents are first stored in my Nextcloud. Every 6 hours, a cron job on the Hermes VM copies that folder into the \u0026lsquo;affitti/\u0026rsquo; folder in the user\u0026rsquo;s home:\n# cloning affitti folder: 0 */6 * * * rclone copy remote-cloud:\u0026#34;\u0026lt;Nextcloud_path\u0026gt;\u0026#34; ~/affitti Hermes comes with a ready-to-use \u0026rsquo;llm-wiki\u0026rsquo; skill. I opened a chat session and started asking Hermes to do what I wanted. Here are a few excerpts from the chat; personal and/or confidential information has been redacted.\nA single line is enough to create the wiki structure:\nUser: go to the \u0026#39;affitti/\u0026#39; folder and create an llm-wiki. Assistant: Done. Initialized the llm-wiki at /home/hermes/affitti: affitti/ ├── SCHEMA.md # domain: rentals — contracts, tenants, landlords, market data ├── index.md # sectioned catalog (currently 0 pages) ├── log.md # init entry logged ├── raw/{articles,papers,transcripts,assets}/ ├── entities/ # properties, tenants, landlords ├── concepts/ # lease terms, rent increases, maintenance rules, utilities ├── comparisons/ # e.g. rent levels across properties └── queries/ Then I asked it to link the PDFs into the wiki via symbolic links, without copying them, and not to start the import right away. The interesting part is how it replied: before giving me its opinion, it went and looked at how the folders were organized and figured out on its own that this was a synced folder, one it must never write into.\nAssistant: Good question. Let me look at the actual nextcloud layout before I commit to a recommendation. `Appartamento XYZ` — via *******, 4 PDFs (contract, registration, 2 proroghe) `Appartamento ABC` — via *******, ~50 PDFs organized by year (affitto 2017 … affitto 2026) There\u0026#39;s also a .sync_*.db file, confirming this is a managed sync dir — so we must never write into it. My recommendation: yes, symlinks are the right solution — better than copying: [...] Read-only stays read-only. [...] anything written into the nextcloud folder (including a symlink created inside it) could be picked up by the sync client and re-uploaded. So the links go in the wiki, never in nextcloud. It created the links, added the rule to SCHEMA.md, and only then waited for my go-ahead. I started with the smaller apartment:\nUser: start with \u0026#39;Appartamento XYZ\u0026#39; Assistant: ****** ingested. All 4 PDFs were read through the symlink (none copied/modified); the main contract\u0026#39;s body is a scanned image with no text layer, but page 1 (asseverazione) plus the 3 Entrate receipts contained all the key data. Created 7 pages: [...] Open questions (flagged on the pages, worth answering when you see the signed proroghe): Did the rent change at the proroghe? The receipts don\u0026#39;t restate it — assumed still €***. Which landlord opted for cedolare secca, and does it carry to the proroghe? 2+1 renewal/proroga rules and deadlines — the receipts don\u0026#39;t contain them. The \u0026ldquo;open questions\u0026rdquo; are what I appreciated most: instead of making up what isn\u0026rsquo;t in the documents, the agent says so and flags it on the pages. I answered the first two questions, and the wiki pages were updated, citing my confirmation as the source.\nBy default, the skill generates text in English; you have to tell Hermes that all documents must be written in Italian: sorry not to have mentioned before, but the wiki text must be in Italian as the input documents.\nI carried on like this until it had digested all the documents available at the time: 60 PDFs turned into 54 wiki pages, in about 4 hours.\nMaking it automatic # I want Hermes to regularly check for new documents and update the wiki accordingly. I also want checks to be run so that all the knowledge stays consistent (the lint or health-check feature typical of the llm-wiki). Any updates or inconsistencies must be reported to me via Telegram, so I can look at them right away and step in.\nOnce again I used cron, but not the system (Unix) one. Hermes has its own cron subsystem, which it manages by itself. In the picture you can see the prompt the default agent is invoked with, the days and time it runs, the skills to include and the folder the agent will have access to. Following the same principle, I set up a second cron job that runs on Saturday mornings to lint the wiki.\nWhy do I write the instructions in English? I almost always write prompts, skills and agent instructions in English, even though my native language is Italian, for two good reasons:\nmost of the text used to train a model is in English (or Chinese!), so I expect it to respond better in this language; English is usually more direct and concise than Italian, so you save tokens the GPUs would otherwise have to process for nothing, which also saves time. How do you use it? # In his original template, Karpathy used Obsidian to browse the wiki. In fact, you can use anything capable of reading markdown files.\nI take a different approach: I don\u0026rsquo;t look at the wiki at all, I only use the chat with the agent, mostly via Telegram. When I need something, I just ask for it, as in the example you saw at the beginning.\nLimitations and improvements # Since it relies on markdown documents, the scalability of the classic llm-wiki approach is limited. Realistically, up to ~100 source files and hundreds of wiki pages, performance is good even with a modestly sized local LLM like Qwen3.8-27B, which is exactly the one I use. So here\u0026rsquo;s the answer to the opening question: yes, for this task a local model is more than adequate. Beyond that threshold, you need something better than an index file.\nConclusions # For my specific case, the wiki is more than enough. I no longer have to rummage through folders of PDFs to find a date, an amount or the name of a tenant from a few years ago: I ask on Telegram and the answer arrives within a few seconds, with sources. And when a deadline is approaching, or a new document doesn\u0026rsquo;t add up with the rest, Hermes reminds me every Saturday morning.\nI still hate paperwork, but at least now someone else takes care of it.\n","date":"18 October 2026","externalUrl":null,"permalink":"/en/blog/2026/10/18/affitti_wiki/","section":"Blog","summary":"I admit it, I hate paperwork, and I suspect many of you feel the same way. Still, we have to live with it: contracts, invoices, utility bills and documents of every kind are part of real life.\nThis is surely where AI can lend a hand: documents are essentially text, and LLMs are pretty good at handling text. Will local models be too? We’ll find out soon.\n","title":"I hate paperwork: how an AI agent manages my rentals","type":"blog"},{"content":"","date":"18 October 2026","externalUrl":null,"permalink":"/en/tags/karpathy/","section":"Tags","summary":"","title":"Karpathy","type":"tags"},{"content":"","date":"18 October 2026","externalUrl":null,"permalink":"/en/tags/llm-wiki/","section":"Tags","summary":"","title":"Llm-Wiki","type":"tags"},{"content":"","date":"18 October 2026","externalUrl":null,"permalink":"/en/tags/nextcloud/","section":"Tags","summary":"","title":"Nextcloud","type":"tags"},{"content":"","date":"18 October 2026","externalUrl":null,"permalink":"/en/tags/rag/","section":"Tags","summary":"","title":"Rag","type":"tags"},{"content":"For some time now, I’ve had an artificial intelligence at home that runs entirely on my own machines, in my homelab. No subscriptions, no data leaving my network. It didn’t arrive all at once: I started with a PC set up one way, then it evolved into something else, and so on. This blog was born to tell the story of that evolution and, above all, of what I’ve built on top of it.\nI’m a computer scientist, and AI caught my attention right from the start, both as a subject of study and as a working tool. The turning point, for me, came with open-weight models, especially the Chinese ones: for the first time, a model you could download and run locally was capable enough to be genuinely useful, and not just a lab curiosity or a toy for nerds. That’s where the idea came from to build my own private AI, without spending a fortune.\nIt’s natural to wonder why anyone would go to all this trouble, when a subscription to Anthropic or OpenAI gives you the best models in the world with zero maintenance. It’s a fair question, and for many people the right answer is indeed the subscription. But there are good reasons to take the other road. The first is privacy: there’s data I would never hand over to an external service, and locally the problem simply doesn’t arise. The second is control: I choose the model, the version and the configuration, and nobody can change them behind my back, shut down the service, or double the price overnight. The third, perhaps the most tempting for a computer scientist, is the chance to understand: a model running at home can be observed, measured, put to the test. As for costs, it’s best to be honest: hardware has a price and electricity isn’t free, especially here in Italy. The advantage lies not so much in saving money as in knowing exactly how much you’re spending, with no surprises.\nYou also need to know what not to expect. A local model on fairly modest hardware won’t match a frontier model, and locally you almost always use quantized models, that is, models compressed to take up less memory, with an inevitable drop in quality compared to the original. The software orchestrating the model can partly make up for this, but the gap remains. And yet progress is extremely fast: today, models of relatively small size can handle tasks that seemed out of reach only a few months earlier. Versions 3.6 and 3.8 of the Qwen model family are a striking example.\nObjectively assessing the quality of a model is far from a trivial problem, and I don’t claim to solve it. Here I’ll adopt a simpler and, I believe, more honest criterion: a model is good enough if it makes the use cases I care about work systematically and reliably. Leaderboards matter little; what matters is that the job gets done, every time it’s needed.\nIn the pages that follow, then, I’ll talk about three things: hardware, and how it has evolved over time while trying to keep spending in check; software, which tools I use and how I configure them; and use cases, what I’ve built and how well it actually works. On this last point I have a clear position. Online you see plenty of examples where an LLM is called upon to deliver the Hacker News headlines or the weather forecast every morning: nice demos, but for things like that a script is enough. I’m interested in something else. I’m interested in the tedious tasks that eat up time, and in those where AI offers real help, something I wouldn’t have done without it, or would have done worse. For example, managing the utility bills for several apartments, automatically collecting their data and amounts into a database where I have everything under control, is one of those things that saves me precious time every month.\nIf the idea sparks your curiosity, let’s start with this.\n","date":"18 October 2026","externalUrl":null,"permalink":"/en/","section":"SchifanoIA","summary":"","title":"SchifanoIA","type":"page"},{"content":"","date":"18 October 2026","externalUrl":null,"permalink":"/en/tags/","section":"Tags","summary":"","title":"Tags","type":"tags"},{"content":"","date":"18 October 2026","externalUrl":null,"permalink":"/en/tags/telegram/","section":"Tags","summary":"","title":"Telegram","type":"tags"},{"content":"","date":"1 July 2026","externalUrl":null,"permalink":"/en/tags/benchmark/","section":"Tags","summary":"","title":"Benchmark","type":"tags"},{"content":"","date":"1 July 2026","externalUrl":null,"permalink":"/en/tags/proxmox/","section":"Tags","summary":"","title":"Proxmox","type":"tags"},{"content":" The perfect software, like the perfect hardware, doesn\u0026rsquo;t exist: it depends on what you want to achieve.\nI favour stability and reliability, and I always use open-source software or, in the case of LLMs, open-weight models. In AI everything changes at breakneck speed: chasing every new release is impossible and not very useful either, but you often find yourself having to try a new version of something or tweak a delicate parameter. It takes next to nothing to break everything, beyond repair. Without a solid safety net, the PC stays out of action until you find the time to fix it.\nThe operating system # Proxmox is my choice of operating system (OS) for the inference PC. The ability to isolate specific applications or services in LXC containers or VMs, together with snapshots and backups, keeps you reasonably safe from the problems I mentioned. Snapshots and backups can be stored locally, on the same disk where Proxmox is installed. For greater reliability, however, it\u0026rsquo;s advisable to use a remote share on a NAS or, better still, a dedicated backup service such as Proxmox Backup Server (PBS). I went for the latter, creating a VM dedicated to PBS on my Synology NAS.\nProxmox is a fairly complex system, and I\u0026rsquo;m the first to admit I don\u0026rsquo;t use many of its advanced features. For example, I\u0026rsquo;ve disabled all the High Availability (HA) services: even though I have several machines running Proxmox, I don\u0026rsquo;t use clustering, let alone HA. For a practical overview of how to install Proxmox on your AI server, I refer you to the guide by digitalspaceport.\nInference engines # There are two kinds of inference software:\naimed at personal inference, that is, for a single user. Llama.cpp and its derivatives are the typical example.\naimed at enterprise inference, typically in datacenters. Here the priority is performance with many parallel requests. Hardware support targets mainly high-end GPUs, where NVIDIA is at home. vLLM and SGLang are the reference solutions in this category.\nI adopted vLLM because it natively supports NVFP4 quantization on Blackwell GPUs and tensor parallel across multiple GPUs, and because it has a larger community. For efficiency, vLLM runs inside an LXC container (Debian 13) that is granted access to the system\u0026rsquo;s GPUs. This way access is more direct, faster and cheaper: there are no extra software layers such as the kernel and drivers of a VM with its own OS. To start the inference server with the selected model I use Docker, with a docker-compose.yml file for each model or model version I want to try. You\u0026rsquo;ll find the docker-compose.yml file to use as a template on my git.\nLLM # As I explained previously, the model I adopted is Qwen3.8-27B. I prefer a single model, very capable for its size, with the largest possible context (KV cache), which comes in handy for agent sessions. This means that even the simplest requests always invoke an LLM that takes up all the resources of the local system. Qwen3.8 handles image-to-text, video-to-text and text-to-text: image generation, text-to-audio and audio-to-text are therefore left out, unless implemented on the CPU.\nAlternatively, you can take a different, more sophisticated approach: a compact LLM acts as a router and dispatches requests to models of different sizes, capabilities and specializations. This solution, however, requires extra resources (GPUs or NPUs) that my hardware doesn\u0026rsquo;t have at the moment.\nSince the two RTX 5060 Ti cards in the inference machine are Blackwell class, the Qwen quantization I adopted is NVFP4. There are dozens of NVFP4 quantizations of this model, and I\u0026rsquo;ve tested several, including:\nnvidia/Qwen3.8-27B-NVFP4: 21.9 GB unsloth/Qwen3.8-27B-NVFP4: 23.4 GB gittensor-model-hub/Qwen3.8-27B-NVFP4-RTX5090: 17.9 GB It\u0026rsquo;s no accident that I listed the size of each model. On average, an NVFP4 quantization takes up more space than an INT4 one, and 22-24 GB starts to be too much if you also want a generous KV cache. That\u0026rsquo;s why I chose the gittensor model: although it was designed for a single RTX 5090, it works perfectly in a tensor parallel setup. In my configuration I can\u0026rsquo;t fully use the 32 GB of VRAM, because each GPU has to keep room for the CUDA context data structures, the NCCL buffers and the CUDA Graphs. As a result, I don\u0026rsquo;t reach 262K tokens of KV cache, but a respectable 237K.\nI won\u0026rsquo;t go into the details of how Gittensor managed to shrink the NVFP4 model while keeping quality on par with (or better than) its competitors, at least according to the benchmarks. On this my approach is pragmatic: a model is effective if it systematically gets done the tasks I give it.\nHermes # I adopted Hermes as my local agentic platform. You\u0026rsquo;ll find countless resources online on how to install, configure and use it.\nFor security reasons, I installed Hermes in a VM (Ubuntu 26.04.1 LTS) with a non-sudoer user. Besides the local Qwen3.8-27B, I defined an agent (or profile) in it that has access to Claude Code through my Pro subscription. This feature is very interesting because it lets Hermes use Claude Code, the console program, directly, and therefore Anthropic\u0026rsquo;s models without going through the expensive APIs.\nPutting it all together # flowchart TB subgraph HW[\"Inference PC: Intel i7-3770 + 2x RTX 5060 Ti 16 GB (Blackwell)\"] subgraph PVE[\"Proxmox\"] subgraph LXC[\"LXC container: Debian 13 (direct GPU access)\"] subgraph DOCKER[\"Docker (one docker-compose.yml per model)\"] VLLM[\"vLLM\ntensor parallel on 2 GPUs\"] LLM[\"Qwen3.8-27B NVFP4 (gittensor, 17.9 GB)\nKV cache: 237K tokens\"] VLLM --- LLM end end subgraph VM[\"VM: Ubuntu 26.04.1 LTS (non-sudoer user)\"] HERMES[\"Hermes\nagentic platform\"] CC[\"Claude Code\"] HERMES -- \"Claude profile\" --\u003e CC end end end HERMES -- \"local profile\n(vLLM API)\" --\u003e VLLM CC -- \"Pro subscription\" --\u003e ANT[\"Anthropic models\n(cloud)\"] subgraph NAS[\"Synology NAS\"] PBS[\"Proxmox Backup Server VM\"] end PVE -. \"snapshots and backups\nof LXC and VMs\" .-\u003e PBS The diagram shows the overall architecture of the system and summarizes the hardware and software involved.\nLet\u0026rsquo;s test the model # Finally, after all this talk, let\u0026rsquo;s look at some numbers. I distinguished two extreme cases:\ncold: every prompt is different, so the prefill is fully recomputed and vLLM\u0026rsquo;s prefix caching doesn\u0026rsquo;t help. It simulates completely independent requests, that is, the worst case. warm: all requests share the prefix and only a suffix of S tokens changes (S = 1K). It simulates an agent turn, with a tool result and a new message. There are three parameters of interest:\nTime to first token (TTFT): the median time needed to generate the first token. Generation speed: tokens generated per second (1000 divided by the median time per token, in ms). Prefill speed: the context size divided by the TTFT. In the warm case it\u0026rsquo;s an effective speed, because most of the prompt is already cached. Each has its own chart, with measurements as the context grows from 1K to 225K tokens (from 2K in the warm case). You\u0026rsquo;ll find my test script on my git as well.\nWhat the numbers say # Prefix caching makes all the difference. In the cold case TTFT grows with the context, and beyond 64K even more than proportionally: 0.6 seconds at 1K, 10 seconds at 16K, 50 seconds at 64K, over two minutes at 128K and about 5 minutes at 225K. In the warm case, on the other hand, the first token arrives in under 2 seconds up to 64K and in 5-7 seconds even with the longest contexts. This is what makes agentic sessions usable: at each turn the model reprocesses only the new part, not the whole conversation.\nGeneration degrades gently. Speed stays around 42 tokens per second up to 8K, drops to about 39 at 64K and to 32 at 225K: for every generated token the model has to consult an ever-larger KV cache. Cold and warm are practically identical, as expected: the cache speeds up reading the prompt, not writing the answer.\nCold prefill is the real bottleneck. Without the cache, the two RTX 5060 Ti cards process about 1,500-1,700 tokens per second up to 32K; then speed drops to 1,300 at 64K, 1,000 at 128K and about 800 at 225K. In the warm case the effective speed climbs to 25,000-40,000 tokens per second, but that\u0026rsquo;s no hardware miracle: simply, almost the entire prompt is already cached.\nIn practice: pasting in a long document from scratch is expensive, but a conversation that grows one turn at a time stays smooth even with hundreds of thousands of tokens of context. So it pays to avoid anything that invalidates the cache, such as changing the system prompt or the list of tools mid-session.\nFinal thoughts # Proxmox, vLLM and Hermes make up a stack that may not be trivial to set up, but it\u0026rsquo;s solid: every component is isolated, protected by snapshots and backups, and replaceable without risking the rest of the system. I can try a new model or a new version of vLLM knowing that, if something goes wrong, rolling back is a matter of minutes.\nOn the performance front, a 27B model on two mid-range GPUs generates between 32 and 42 tokens per second and, thanks to prefix caching, responds within a few seconds even with contexts over 200K tokens. For personal and agentic use it\u0026rsquo;s a more than satisfying result, especially considering the cost and limitations of the hardware.\n","date":"1 July 2026","externalUrl":null,"permalink":"/en/blog/2026/07/01/aipc_software/","section":"Blog","summary":" The perfect software, like the perfect hardware, doesn’t exist: it depends on what you want to achieve.\nI favour stability and reliability, and I always use open-source software or, in the case of LLMs, open-weight models. In AI everything changes at breakneck speed: chasing every new release is impossible and not very useful either, but you often find yourself having to try a new version of something or tweak a delicate parameter. It takes next to nothing to break everything, beyond repair. Without a solid safety net, the PC stays out of action until you find the time to fix it.\n","title":"Proxmox, vLLM and Hermes: the software on my local AI PC","type":"blog"},{"content":"","date":"1 July 2026","externalUrl":null,"permalink":"/en/tags/qwen/","section":"Tags","summary":"","title":"Qwen","type":"tags"},{"content":"","date":"1 July 2026","externalUrl":null,"permalink":"/en/categories/software/","section":"Categories","summary":"","title":"Software","type":"categories"},{"content":"","date":"1 July 2026","externalUrl":null,"permalink":"/en/tags/vllm/","section":"Tags","summary":"","title":"Vllm","type":"tags"},{"content":"Like every computer scientist or enthusiast, over the years I\u0026rsquo;ve piled up an impressive number of components: CPUs, memory, disks, cases, power supplies and boards of every kind.\nAnything more than 10 years old is considered obsolete, and yet these components are still perfectly functional and can run almost anything decently. Does that apply to AI models too? Definitely not!\nBut there is a loophole: what matters is the GPU, because the LLM runs entirely there. The rest of the machine doesn\u0026rsquo;t need to be particularly sophisticated.\nThe plan # So the idea is simple: we can reuse old components we may already own and focus the spending on the GPU. That way we can afford a personal AI without spending a fortune.\nActually, even the GPU could be second-hand. You can read pretty much everywhere that NVIDIA\u0026rsquo;s RTX 3090 is still an excellent choice, but it keeps very high prices (around €1000 today), and its power draw is just as high. Besides, a single 3090 might be a tight fit with its 24GB of VRAM.\nNVIDIA isn\u0026rsquo;t the only option, but it makes things simpler: all the software, especially for machine learning, inference and everything else you need for local AI, has the best support on NVIDIA hardware, since it relies on the proprietary CUDA libraries.\nRequirements and choices # To build my local inference machine, I first need to define what I consider the minimum requirements to run a model capable enough for my needs.\nOn August 14, 2026 the weights of Qwen3.8-27B were released. At the same size as the previous version 3.6, it improves performance even further and is still unbeaten among models of its size, at least in my personal opinion. Qwen3.8-27B is therefore the model I chose for local inference.\nHosting a 27B LLM means taking up ~18-23GB of VRAM on the GPU, depending on the quantization used (int4, fp4). On top of that comes the KV cache, which, if you want to support the full native context window (262K tokens), amounts to ~8 GiB = 8.6 GB. It follows that 32GB of VRAM is the minimum requirement.\nThe problem is that NVIDIA\u0026rsquo;s only consumer product with 32GB is the RTX 5090, which, besides being impossible to find, has a prohibitive price I won\u0026rsquo;t even bother to mention. So I fell back on a dual-GPU setup with two 16GB RTX 5060 Ti cards used in parallel (\u0026rsquo;tensor parallel\u0026rsquo;). We\u0026rsquo;ll see later that this setup has both advantages and disadvantages. Still, I consider it the best compromise between performance and power consumption (180W TDP and 3W at idle), and being a \u0026lsquo;Blackwell\u0026rsquo; generation card it natively supports NVFP4 quantization. I refer you to NVIDIA\u0026rsquo;s documentation to appreciate the differences between the various 4-bit floating-point formats.\nThe final configuration # The PC I used was originally built as a gaming PC, but given how much it was used, or perhaps I should say not used, I repurposed it for local inference.\nThe configuration and costs are summarized in the table.\nComponent Model Already owned Cost (€) CPU Intel Core i7-3770 \u0026#x2705; 0 CPU cooler Thermalright sl-100 \u0026#x2705; 0 Motherboard Gigabyte GA-Z77X-UD3H \u0026#x274c; 79 RAM 32GB DDR3 (4x8GB) \u0026#x2705; 0 SSD Samsung 860 EVO mSATA 500GB \u0026#x2705; 0 PSU be quiet! Pure Power 12M 650W \u0026#x2705; 0 Case very old Cooler Master case \u0026#x2705; 0 GPU (x2) Asus DUAL RTX 5060 Ti OC \u0026#x274c; 580*2 Making the PC fit for inference cost €1239. That\u0026rsquo;s similar to, or less than, a single-card solution from AMD or Intel, with all the complications that come with it. I chose to swap the motherboard for one (GA-Z77X-UD3H) that could allocate 8 lanes to both PCIe x16 slots. These are PCIe 3.0 rather than the 5.0 supported by the RTX 5060.\nIs it a lot? Is it little? It depends. Justifying an expense is a function of many parameters, each weighted by your personal situation. In my case, this expense brings no financial return at all: it\u0026rsquo;s driven by passion and personal interest, and I believe it\u0026rsquo;s the minimum possible spend.\nPower consumption # For a server in a home lab, idle power consumption matters a lot: most of the time, in fact, the machine isn\u0026rsquo;t doing anything.\nAlthough this post isn\u0026rsquo;t about software, it\u0026rsquo;s worth knowing that this machine runs Proxmox. Power consumption was measured with the following running:\n1 LXC container (Debian GNU/Linux 13 trixie) for the inference engine, with access to the GPUs 1 Linux Ubuntu 26.04.1 LTS VM for the agent environment (Hermes) Above: RTX 5060 Ti idle power consumption.\nAccording to \u0026rsquo;nvtop\u0026rsquo;, the 5060 Ti cards draw very little: 3W per GPU. Maximum power has been capped at 160W (vs. 180W stock).\nAbove: whole-system idle power consumption.\nThe whole system\u0026rsquo;s idle consumption, measured at the power supply input, comes to 44W. Considering that all CPU C-states were enabled in the BIOS, and that the audio and the Marvell 88SE9172 SATA controller were disabled, I don\u0026rsquo;t think it\u0026rsquo;s a low figure in absolute terms for doing nothing. Maybe my main server (an HP Elitedesk 800 mini G9), which draws 6-8W at idle, has spoiled me!\nFor those who want more, you might squeeze out a bit by turning off all the SATA ports (only one is used by the SSD) and disabling the Intel iGPU, making the system headless. Still, you can\u0026rsquo;t expect miracles from a 2012(!) system with DDR3 RAM.\nOn the other hand, the maximum consumption I\u0026rsquo;ve observed during inference is 294W. I\u0026rsquo;ve never seen a single card go above 120W, so the GPUs\u0026rsquo; power could be capped further, down to 140W, without hurting performance.\nCost of power # Let\u0026rsquo;s assume the system is always idle and that the hours spent on inference are negligible in a home setting. Let\u0026rsquo;s also assume a cost of €0.25 per kWh, including taxes, fees and whatnot. We get:\n44 W⋅24 h1000⋅30 days=31.68 kWh \\frac{44\\,\\text{W} \\cdot 24\\,\\text{h}}{1000} \\cdot 30\\,\\text{days} = 31.68\\,\\text{kWh} 100044W⋅24h​⋅30days=31.68kWh31.68 kWh⋅0.25 EUR/kWh=7.92 EUR 31.68\\,\\text{kWh} \\cdot 0.25\\,\\text{EUR/kWh} = 7.92\\,\\text{EUR} 31.68kWh⋅0.25EUR/kWh=7.92EUR €7.92 a month is the minimum electricity cost of having a local AI with this machine. With a solar power system, the cost roughly halves on average over the year. The comparison with my Anthropic Pro subscription is merciless: €18 a month. Little more than double, for several orders of magnitude more capability! But if you want a local AI, you have to look beyond the bare numbers.\nFinal thoughts and limitations # With €1239 and a PC from 2012 I got a machine with 32GB of VRAM, able to host Qwen3.8-27B in NVFP4 with a generous KV cache and modest idle power consumption. There is a trade-off, though, and it should be stated clearly: the potential bottleneck is the PCIe 3.0 bus.\nThe RTX 5060 Ti cards use 8 PCIe 5.0 lanes, which works out to about 32 GB/s per direction. On the GA-Z77X-UD3H the same 8 lanes run at PCIe 3.0: about 8 GB/s, a quarter of the bandwidth the cards were designed for.\nHow much does this limitation weigh? Less than you might think, but it\u0026rsquo;s there:\nDuring generation, the model and the KV cache live entirely in VRAM, so the bandwidth that matters is the GPU\u0026rsquo;s memory bandwidth, not the bus\u0026rsquo;s. Tensor parallel, on the other hand, forces the two GPUs to exchange intermediate results at every layer. Consumer cards have no NVLink, so this traffic goes over the PCIe bus and through the CPU\u0026rsquo;s memory as well (the NCCL library doesn\u0026rsquo;t use P2P mode, but SHM). For each generated token, little data is exchanged and latency is what matters most. During the prefill of a long prompt, however, the volumes grow, and that\u0026rsquo;s where PCIe 3.0 might make itself felt. Loading the model is slow because of the mSATA SSD (about 550 MB/s): it takes tens of seconds to read almost 18GB. But this only happens when the inference server starts. I could have avoided the problem with a more modern platform, but I would have had to replace the CPU and RAM too, at a very different cost. For personal use, where having a capable model and a large context matters more than top speed, I consider the trade-off acceptable.\nImpressions count for little, though: we need numbers. In the next post, dedicated to software, I\u0026rsquo;ll put this configuration to the test and we\u0026rsquo;ll see how much these limitations really weigh on performance.\n","date":"5 June 2026","externalUrl":null,"permalink":"/en/blog/2026/06/05/game2aipc/","section":"Blog","summary":"Like every computer scientist or enthusiast, over the years I’ve piled up an impressive number of components: CPUs, memory, disks, cases, power supplies and boards of every kind.\nAnything more than 10 years old is considered obsolete, and yet these components are still perfectly functional and can run almost anything decently. Does that apply to AI models too? Definitely not!\nBut there is a loophole: what matters is the GPU, because the LLM runs entirely there. The rest of the machine doesn’t need to be particularly sophisticated.\n","title":"A cheap but effective computer","type":"blog"},{"content":"","date":"5 June 2026","externalUrl":null,"permalink":"/en/categories/hardware/","section":"Categories","summary":"","title":"Hardware","type":"categories"},{"content":"My name is Gian Paolo Jesi, and it all started with a triangle.\nAs a kid I had a Commodore 64. Programming fascinated me, even though I had a rather muddled idea of what it was. One day my best friend, armed with a few scraps of BASIC learned at school, wrote me a program that calculated the area of a triangle. I was convinced that programming meant \u0026ldquo;making the computer think\u0026rdquo;, a bit like on Star Trek. Instead, all it took was base * height / 2. I was left speechless, and I\u0026rsquo;ve never stopped since.\nThe road wasn\u0026rsquo;t a straight one, though. I went to a liceo classico, the Italian high school built around Latin, Ancient Greek and philosophy. Then I earned a degree in Computer Science and a PhD in distributed systems, between Bologna and Amsterdam. I spent thirteen years in research, in Italy and abroad, and in 2018 I moved to the private sector, still in research and development, but finally closer to home.\nToday, with artificial intelligence, computers look a little more like the ones I imagined as a kid. That\u0026rsquo;s why I wanted to build one at home: to understand how it really works and what it can do. That\u0026rsquo;s what this blog is about. You\u0026rsquo;ll find the project described on the home page, and the story behind the name in Why Schifanoia?.\nAway from the keyboard, you\u0026rsquo;ll find me riding my motorcycle, with a book in hand, or watching a film, preferably on the couch at home with the sound system on.\nHow I work on this blog # I write the articles myself, by hand. I use AI to proofread and edit them and to help me run the site: given the topic, it seemed only fair to say so.\nContact # Questions, corrections, ideas or just curiosity: write to me at gp.jesi [at] gmail [dot] com. The code mentioned in the articles is on my GitHub.\n","externalUrl":null,"permalink":"/en/about/","section":"SchifanoIA","summary":"","title":"About","type":"page"},{"content":"","externalUrl":null,"permalink":"/en/authors/","section":"Authors","summary":"","title":"Authors","type":"authors"},{"content":"","externalUrl":null,"permalink":"/en/series/","section":"Series","summary":"","title":"Series","type":"series"},{"content":" Above: Schifanoia palace.\nIn Ferrara, my hometown, right in the heart of the historic centre, at number 23 of Via Scandiana, stands Palazzo Schifanoia. It was built in 1385 at the behest of Alberto V d\u0026rsquo;Este, and its name says it all: schivar la noia, \u0026ldquo;to escape boredom\u0026rdquo;. It was neither a seat of government nor a fortress, but a place designed for the rest and leisure of the court, a retreat where one could step away from affairs of state and devote oneself to what makes life more pleasant.\nA few decades later, Borso d\u0026rsquo;Este enlarged it and had the Hall of the Months frescoed, one of the most extraordinary pictorial cycles of the Renaissance: twelve months in which everyday life, the gods of Olympus and the constellations of the zodiac intertwine. But the place I\u0026rsquo;m most fond of is another: the garden. Sitting at a café table, in the shade of centuries-old trees and surrounded by flowering plants, I always get the same feeling: the mind grows calm, and at the same time the urge to imagine and to build something comes alive.\nI chose this name for three reasons.\nThe first is precisely that feeling. This blog, for me, is a space for otium in the Latin sense of the word: not idleness, but time taken away from obligations and devoted to what nourishes the mind, exactly as the palace was for the Este family.\nThe second is more personal. By day I work as an employee in the IT sector, among big data pipelines and corporate processes with very little creativity to them: my negotium, to stay on theme. Schifanoia is my breath of fresh air: the place where technology goes back to being curiosity, experimentation and, yes, fun as well.\nThe third, I\u0026rsquo;ll admit, is a not-so-brilliant pun: \u0026ldquo;Schifanoia\u0026rdquo; ends with IA, which is Italian for AI. A homemade AI, born to escape boredom.\n","externalUrl":null,"permalink":"/en/schifanoia/","section":"SchifanoIA","summary":"","title":"Why Schifanoia?","type":"page"}]