Local AI use cases with Raspberry Pi
The Raspberry Pi 5 is a low-cost ARM single-board computer. With 8 GB of RAM it can run small quantized models on the CPU for narrow offline tasks.
Rebel BitesOffline LLM on a Raspberry Pi 5 for classification and extraction
Phi-3 mini at Q4_K_M runs at 4-6 tokens per second on an 8 GB Pi 5 with no network at all. Under GDPR, data that never leaves the board isn't transferred anywhere.