Run LLM on Pi5: Connecting an NVIDIA GPU to Raspberry Pi 5 via PCIe x4

After a long effort, I connected an Nvidia RTX 2060 Super to Raspberry Pi 5 running Ubuntu. I will guide you step by step through all the stages.

First, we need some hardware components.

1. GPU Riser: https://tr.aliexpress.com/item/1005007348688764.html?spm=a2g0o.productlist.main.11.6b23Ttj8Ttj89n&algo_pvid=0e7f4d42-eb23-4838-9ad5-a20ba58cc971&algo_exp_id=0e7f4d42-eb23-4838-9ad5-a20ba58cc971-5&pdp_npi=4%40dis%21TRY%21363.51%2181.21%21%21%2172.00%2116.08%21%40211b807017332135387342399edc5f%2112000040366348410%21sea%21TR%210%21ABX&curPageLogUid=ttxjSAxyY0rl&utparam-url=scene%3Asearch%7Cquery_from%3A
2. PI 5 X1010 PCle : https://tr.aliexpress.com/item/1005007550933808.html?spm=a2g0o.productlist.main.1.6e6a1aa9kDgPw1&algo_pvid=e8f9e70b-400b-460c-be1f-687b65379d69&algo_exp_id=e8f9e70b-400b-460c-be1f-687b65379d69-0&pdp_npi=4%40dis%21TRY%211108.49%211008.73%21%21%21219.56%21199.80%21%4021038df617332137527772148ec7f3%2112000041260418047%21sea%21TR%210%21ABX&curPageLogUid=Yqpdnc8PnHuM&utparam-url=scene%3Asearch%7Cquery_from%3A
3. External PSU

In the first step, connect the graphics card to the riser and attach power to both the riser and the GPU. Then, use the USB cable to connect the riser’s x4 connection to the board.

Next, you need to connect the PI5 X1010 PCIe to the Raspberry Pi and attach it to the Pi’s PCIe port. Then, the 4 pins on the front of the X1010 must make contact with the lower GPIO pins of the Pi. This allows you to jumper without using cables. Also, connect the 4-pin fan power cable to the X1010.

Your PSU will not work without a motherboard. Therefore, jumper the 4th and 5th pins from the left on the first row of the 24-pin power cable.

Next, power on the PSU, and once the GPU and PSU are running, turn on the Raspberry Pi as well. After logging into the Pi, proceed with the following installations in sequence.

Source : https://www.lifewire.com/atx-24-pin-12v-power-supply-pinout-2624578

First, download egpu-switcher-arm: egpu-switcher-arm64

Then, run the following commands:

sudo cp <downloaded-binary> /opt/egpu-switcher
sudo chmod 755 /opt/egpu-switcher
sudo ln -s /opt/egpu-switcher /usr/bin/egpu-switcher
sudo egpu-switcher enable

Once you do this, Ubuntu will recognize your GPU. Proceed with the configuration.

Next, install the drivers:

sudo apt update && sudo apt upgrade -y
sudo apt install build-essential dkms

wget https://us.download.nvidia.com/tesla/535.104.12/nvidia-driver-local-repo-ubuntu2204-535.104.12_1.0-1_arm64.deb

chmod +x NVIDIA-Linux-<ARCH>-<VERSION>.run
sudo ./NVIDIA-Linux-<ARCH>-<VERSION>.run

For checking the graphics card;

nvidia-smi 

OR

lswh -C display

Your system is now ready! You can now check out my article below to run LLM locally.

Set Up Your Own Cybersecurity-Focused AI Development, Training, and Fine-Tuning Lab at Home

Enjoy!