Local AI use cases with Ollama
Ollama is a free tool that downloads and runs open-weight models on macOS, Windows and Linux. It serves them over a local HTTP API on port 11434, and most local AI apps can use it as their backend.
Rebel BitesHome RAG assistant for embedded-systems docs on Proxmox
Ollama, llama.cpp, Open WebUI, BGE-M3 and SearXNG in a GPU-passthrough VM. Moving from 6 GB to 16 GB of VRAM changed which models could run.
Auto-tagging documents in Paperless-ngx with Ollama
Since version 3.0, Paperless-ngx suggests titles, senders, document types and tags, and can chat with your documents, using a local Ollama model.
Self-hosted bookmarks that tag themselves with local AI (Karakeep)
Karakeep (formerly Hoarder) runs on a Debian LXC, and gemma3:4b in Ollama tags each bookmark. Getting reliable tags from a small model takes one key setting.
Offline meeting minutes with whisper.cpp + Ollama
Turns a 1-hour meeting into structured minutes in under 4 minutes on an RTX 3060, with zero network requests. Bash and Python pipeline included.
Paperless AI on an 8 GB RTX 3060 Ti shared with Frigate
Gemma 3 4B with embeddinggemma answers over 1,318 documents in about 5 seconds, on a GPU that's also running Frigate and Plex.
Home Assistant Voice PE with a local LLM instead of Echo
A $60 Voice Preview Edition, an N100 box and an optional RTX 3060 replace every Echo and Nest in the house. It keeps working with the internet unplugged.
A private ChatGPT for a family of four on a Mac mini
One always-on M4 Mac mini serves Open WebUI to every phone and laptop in the house. Kids get a tutor preset, and three AI subscriptions are gone.
Hybrid workflow: local for private and repetitive, cloud for hard
A Linux sysadmin stopped trying to replace cloud LLMs. Ollama now handles private docs, overnight article summaries and VS Code autocomplete.
Fully local Home Assistant voice assistant on a GPU
Wake word to spoken answer in 2-3 seconds, with zero audio leaving the house. Built from faster-whisper, Piper and Ollama, with a cloud fallback.
Offline RAG over 268 Obsidian notes, queryable from a coding agent
nomic-embed-text, ChromaDB and mistral-nemo:12b give cited answers from a vault of internal docs, and an MCP server lets the coding agent search it too.
Replacing Google Nest Minis with local Assist on llama.cpp
After a year of Nest Minis getting dumber, a household moved to fully local Assist. Higher-quant GGUF models from Hugging Face are what finally fixed tool calling.
Self-hosted AI email assistant on Linux (n8n + Ollama)
Classifies IMAP mail with llama3.1:8b, routes it, saves reply drafts and extracts action items as JSON. Reports 85-95% classification accuracy.
mdrag: a Rust CLI to chat with an Obsidian vault
A self-contained RAG tool built on SQLite with sqlite-vec and gemma3:1b. It answers 'how did I fix that Git submodule issue?' from your own notes.
Connecting a private Obsidian vault to Ollama on macOS
Meeting notes stay off public AI services. llama3.1 and nomic-embed-text in Ollama, wired up through the Local GPT and AI Providers plugins.
Mac Studio as an always-on, LLM-driven smart-home brain
Phrases like 'it's stuffy in here' become real device calls, run by Ollama with qwen3:14b on a Mac Studio. €15 ESP32 satellites keep all audio on the LAN.
Receipt extraction and bank matching with PaperCortex
An open-source local AI layer for Paperless-ngx. It answers plain-English questions, extracts receipt line items, matches bank transactions and exports to DATEV.
Obsidian vault Q&A with Mini-RAG and llama.cpp
An Obsidian plugin that chats with a note or folder, using separate local llama.cpp servers for chat and embeddings (Ollama works too).
Security camera descriptions and search with Frigate GenAI
Frigate sends camera frames to a local vision model (qwen3-vl via Ollama or llama.cpp) for searchable event descriptions, review summaries and chat.
Email triage and morning digest with a local Ollama model (n8n)
Classifies each email as URGENT, REPLY_TODAY, FYI or JUNK using gemma4:e4b, sends urgent alerts and builds a grouped morning digest.
Auto-label Gmail and draft replies with a local model (n8n)
Sorts unread Gmail into six categories with Ollama, applies labels, drafts replies for urgent mail and logs everything to Sheets, with zero paid API costs.
Local-first meeting notes app (whisper.cpp + Ollama)
A macOS app that records your mic and the call's audio as two tracks, transcribes locally and writes a Markdown note with 'You vs. Them' speaker labels.
14 months local-first: what still breaks
An honest report from a three-machine home lab. Local handles about 85% of tasks, but four kinds of work still fail, and that cut cloud spend from $187 to about $25 a month.
Øxygen: a private desktop chat app for Ollama
Conversations, files and settings live in a file in your home directory. It talks only to Ollama on 127.0.0.1 and adds optional embeddings, vision, voice and image generation.
Branching conversation graphs with OpenYoke
A Tauri and Rust desktop app for Ollama where every exchange is a node you can branch from, and sibling branches never bleed context into each other.
Offline chat on desktop or Android via Termux with Prefrontal
A privacy-first chat UI that works with Ollama on desktop or llama.cpp anywhere, including Android through Termux. No telemetry, no subscriptions.
Offline LLM on a Raspberry Pi 5 for classification and extraction
Phi-3 mini at Q4_K_M runs at 4-6 tokens per second on an 8 GB Pi 5 with no network at all. Under GDPR, data that never leaves the board isn't transferred anywhere.
Aider + qwen2.5-coder:14b as a local Claude Code alternative
A tic-tac-toe test on an RTX 5060 Ti 16 GB. The game worked on the first pass, but the tests and input validation needed several rounds of prompting.
Copilot-style coding assistant from a headless RTX 3090 box
qwen2.5-coder:7b on a home server, reached from a laptop over an SSH tunnel and wired into VS Code through Continue for chat, edits and autocomplete.