The fervor surrounding AI has reached such heights that the Raspberry Pi Foundation recently unveiled a new AI Hat+ add-on. Nevertheless, one does not necessarily require specialized hardware to execute AI models on Raspberry Pi locally. By utilizing the CPU, you can operate small language models on your Raspberry Pi board. While the token generation may be sluggish, there are diminutive million-parameter models that function adequately. With that exposition, let us delve into the realm of running AI models on Raspberry Pi.
Requirements
- A Raspberry Pi with a minimum of 2GB of RAM for a satisfactory user experience. I am employing my Raspberry Pi 4 in this demonstration, equipped with 4GB of RAM. Several users have even managed to run AI models on a Raspberry Pi Zero 2 W with just 512MB of RAM.
- A microSD card featuring at least 8GB of storage.
Install Ollama on Raspberry Pi
Before embarking on the installation of Ollama, ensure that your Raspberry Pi setup is complete. Subsequently, launch the Terminal and execute the command below to update all packages and dependencies.
sudo apt update && sudo apt upgradeAfter completing the aforementioned steps, proceed by running the subsequent command to install Ollama on your Raspberry Pi.
curl -fsSL https://ollama.com/install.sh | shUpon successful installation of Ollama, you will receive a notification indicating that the CPU will be utilized to run the AI model locally. You are now prepared to proceed.
Run AI Models Locally on Raspberry Pi
Post installation of Ollama, input the following command to install the tinyllama AI model on your Raspberry Pi. This model is compact, with 1.1 billion parameters and a memory consumption of merely 638MB.
ollama run tinyllamaUpon completion of the AI model installation, enter your prompt and press Enter. Patience is requisite as a response will be generated after a brief interval. In my experiments, the Raspberry Pi 4 exhibited a leisurely pace in generating responses, expected due to the modest capabilities of a single-board computer. I recommend experimenting with smollm, which boasts 135 million parameters and requires only 92MB of memory. This petite LLM is ideal for Raspberry Pi.
ollama run smollm:135mAs a final flourish, consider exploring Microsoft’s Phi model, encompassing 2.7 billion parameters and utilizing 1.6GB of memory. Should you possess a Raspberry Pi 5, this model is certainly worth exploring.
ollama run phiIn this elegant manner, you can experience the thrills of running AI models on Raspberry Pi locally with Ollama. Its user-friendly nature sets it apart. While other frameworks like Llama.cpp exist, their installation processes can be arduous. With just two simple Ollama commands, you can commence your foray into the world of LLMs on Raspberry Pi.
Should you wish to explore further, I recently harnessed my Raspberry Pi to craft a wireless Android Auto dongle. If such innovative projects spark your interest, peruse our comprehensive guide. For any queries or insights, do not hesitate to share them in the comments section below.
Support our work ❤️
If you enjoyed this article, consider leaving a tip to help us keep publishing great content.
