Category: Artificial Intelligence
-
Running Local LLMs on a $30 SBC
Meet Pipsqueak: A pocket-sized AI assistant powered by an Orange Pi Zero 2W and TinyLlama, proving that local LLMs don’t always need powerful hardware. You can see how few tokens per second Pipsqueak can infer… but YAY! The Hardware Setup At the heart of this experiment is the Orange Pi Zero 2W, a remarkably capable…