💡

Key Points

Key Takeaways

  • 1

    Intelligence at the Edge

  • 2

    Ollama: De facto standard for running local LLM. Quantized model of several GB starts moving on your PC just by `ollama run llama3`

  • 3

    Hardware (Raspberry Pi 5): If 8GB model, 7B (7 billion parameters) class model works at practical speed. Power consumption is only few watts

  • 4

    Privacy: Medical data, personal diary, confidential code. Data that do not want to send to cloud should be fed to local LLM

  • 5

    Uncensored: Enterprise AI (ChatGPT etc.) is bound by 'Political Correctness'. Local model answers your question honestly

Introduction: Democratization of AI

If OpenAI server stops, development all over world stops. This is not healthy Internet.

In 2026, AI goes from “Thing to borrow” to “Thing to own”. Raspberry Pi driving quietly in corner of home becomes your exclusive counselor and programmer.

1. The Engine: Ollama

It is tool that can manage LLM easily like Docker. Works cross-platform with Windows, Mac, Linux, and also functions as API server.

# AI starts with just this
ollama run llama3

Can call from existing apps (Obsidian and VS Code) via http://localhost:11434.

2. The Hardware: Raspberry Pi 5 (8GB)

To run LLM, GPU of hundreds of thousands of yen is not essential (Necessary for learning, but unnecessary for inference). By Quantization technology, surprisingly smart model works even with Raspberry Pi of memory 8GB.

Search

Raspberry Pi 5 (8GB Model)

CPU performance improved more than 2 times from previous generation. NVMe SSD can also be connected by PCIe interface. If add dedicated AI HAT (Hailo-8), image recognition etc. also becomes explosively fast.

Amazon

3. Use Cases: What to use for?

Private Journaling

Let’s link Ollama to diary app. If ask “Many negative words recently?”, AI analyzes contents of diary. Not single byte is sent to external server.

Coding Assistant

Copilot is convenient, but there is risk that company’s code is used for learning data. If local LLM, code completion works even in offline environment (in airplane etc.).

4. Comparison: Cloud vs Local

項目 ChatGPT (Cloud) Ollama (Local)
Smartness Super Genius (GPT-4) Talented (Llama 3 8B)
Privacy Gray Complete (100%)
Cost Monthly 3,000 yen Electricity bill only
Availability Downtime exists Always running

Conclusion: Distributed Possession of Intelligence

What can oppose “Centralized Intelligence” monopolized by giant tech companies is “Distributed Intelligence” by billions of edge devices. Small AI in your Raspberry Pi becomes breakwater protecting future Internet.

# llm # ai # ollama # raspberry-pi # server # 2026