--- title: "Admitting the Limits of Local LLMs Is Where Real Automation Begins" date: 2026-10-21 model: admin category: reviews summary: "What I concluded after installing and testing local LLMs on an RTX 3070 8GB. Use web AI for general work, keep local models for personal knowledge management and offline processing, and design automation around the model's limits." tags: local-llm, automation, RTX3070, web-ai, obsidian author_type: human --- For a while I was interested in local LLMs and installed and tested models myself. I ran several models on an RTX 3070 8GB, and while building a personal agent I also experimented with prompts, skills, memory structures, and many other ways to raise model performance. At first I thought that using local LLMs well would let me solve a considerable amount of work. But after using them directly and testing them many times, my thinking changed. The conclusion I reached is simple. **It is not about denying the potential of local LLMs, but about clearly recognizing their limits and deciding where to use them.** ## 1. Personal hardware has clear limits These days LLMs of many sizes, including 27B models, keep pouring out. Larger models promise better performance, but the hardware an individual can afford has limits. In an ordinary personal environment with a GPU such as the RTX 3070 8GB, you have to compromise among model size, quantization level, context length, and inference speed. Being able to run a larger model does not mean you can use it comfortably. Of course, buying a GPU with more memory can change the situation. But continuing to invest money does not mean you can match the performance of commercial AI services either. Between models operated on large-scale server infrastructure and models running on a personal PC, there are differences that are hard to explain by model size alone. I began to ask myself whether it is really reasonable to ignore this reality and keep investing in hardware. ## 2. The rise of free web AI is changing the situation In the past, installing and running a local LLM yourself was quite attractive. You could use AI while reducing cost, and you did not have to depend on an external service. But the situation is different now. From web chatbots such as Gemini to a wide range of free AI services, the number of tools ordinary users can choose from has grown greatly. Services that support creative work such as image and video production have also appeared. Especially for work where reasoning ability and context understanding matter, such as writing, ordinary coding, and analysis of complex questions, using a good web AI can be far more efficient. I am also skeptical about writing with a local model. A small model can certainly revise sentences or produce simple text. But what actually matters is not whether it can write, but how little time and cost it takes to get the result you want. If you can already use free web AI that is good enough, is there any reason to upgrade hardware, install a model, and constantly tune prompts and skills? I think that for ordinary users, the need to do so will keep shrinking. ## 3. The general-purpose value of small and mid-sized LLMs may decline I think the general-purpose value of small and mid-sized LLMs is likely to decline gradually. I do not mean that small and mid-sized models become useless. I mean that their appeal as a way to replace commercial AI in general-purpose work may shrink. The hardware an individual can use has limits, and running better models costs more. Web AI services, by contrast, remove much of the burden of hardware and runtime environments that users would otherwise have to manage themselves. Free services do carry risks such as usage limits and changes in service policy. For people who care about privacy or offline execution, local models remain useful. Even so, for general-purpose work such as ordinary questions, writing, and coding, using an already built service can be more economical than operating a local model yourself. I think the value of small and mid-sized LLMs will not disappear. Rather, it will likely shift from replacing general-purpose AI to becoming specialized components for specific tasks. ## 4. Local LLMs may fit personal knowledge management better That said, I do not think local LLMs are unnecessary. Rather, I think we should focus on the areas where local models do well. For example, you can connect one to a personal knowledge management tool such as Obsidian and have it search the documents you collected, summarize past records, classify materials, or find relationships between pieces of knowledge. Such work does not necessarily require the most powerful model. A retrieval system and ordinary programs can handle much of it, and the local LLM only needs to assist where necessary. Local models are also meaningful for processing personal files without sending them outside, performing repetitive document work, or organizing data according to fixed rules. The key is not trying to solve everything with a local LLM. I think it is far more reasonable to use a good web AI for general-purpose work and local models for personal knowledge management or task-specific work. ## 5. Automation begins with admitting the model's limits The most important lesson I took from testing local LLMs is that automation does not necessarily require the largest model. I think a great deal can be automated. What is needed is a clear separation between what the model can do and what it cannot. For example, when organizing a large number of documents, the LLM does not need to understand everything perfectly. You can design ordinary programs to process files and validate data according to rules, with the LLM assisting with classification and summarization. If the model might make mistakes, you can add a verification step. For important work, you can also control the system so that wrong results are not executed immediately. This is especially important for server administration or for modifying the code of a running service. Even commercial API models can make mistakes, so trusting model capability alone with important permissions is dangerous. In the end, the quality of automation is not determined by model size alone. How well you separate tasks, how you verify results, and how you design responses when errors occur also matter. I think it is more valuable to build a system that works reliably on the premise of the model's limits than to invest blindly in trying to eliminate those limits. ## Conclusion: using models well matters more than running them yourself I do not deny the potential of local LLMs. Having installed and tested them myself, I recognize their strengths and limits all the more clearly. Trying to match every capability of commercial AI on personal hardware has limits in cost and efficiency. As free web AI improves, the reason for ordinary users to run local models themselves for general-purpose work may also shrink. That does not mean local models should be discarded. Use them in areas where the local environment is genuinely advantageous, such as personal knowledge management, offline work, privacy, and repetitive data processing. Going forward, I will not evaluate small and mid-sized LLMs by size or benchmark scores alone. I will weigh what tasks they actually perform well, how much they cost to operate, and how efficient they are when combined with existing programs. **What I want is not to own the largest model. It is a system that chooses the right tools, understands the limits of each, and reliably completes the work I want at the lowest possible cost.** I believe that is a realistic AI strategy for an individual, and the direction automation should take.