Tetobox 32 or 48GB vram Dual or Triple A770 Setup Guide

My Build:

• CPU: 5600x
• Any motherboard that can support 3 gpus at a high enough bandwidth x470s work well enough here with a riser
• RAM: 32GB
• 2-3x A770s 16GB (250-300ea new vs used P40s are now 300+)

This shit was surprisingly 0 fuss and just werked

1. Download Ubuntu LTS 22.04
2. Install as normal

Make sure to scroll down to the Linux 6.5 step after the first 5 steps in the install guide in step 3 here

Chose option 1 In the install Drivers section for the install guide in the step 3 here even if you have an AMD cpu just SKIP the middle sudo apt install -y intel-i915-dkms intel-fw-gpu thats in the middle of the two commands unless you have an intel CPU

Use Option 1 even if you have AMD unless you have an Intel Xeon-W/SP CPU

3. Follow this guide from Intel: https://github.com/intel/ipex-llm/blob/main/docs/mddocs/Quickstart/install_linux_gpu.md
4. Once you verify you can see all gpus go Do the setup python environment and install the ipex-llm steps.
5. Now you can follow https://github.com/intel/ipex-llm/blob/main/docs/mddocs/Quickstart/llama_cpp_quickstart.md and setup llama.cpp

1
2
3
4

Currently running Mistral-Small-24B-Instruct-2501-Q6_K_L
prompt eval time =    1993.93 ms /   979 tokens (    2.04 ms per token,   490.99 tokens per second)
       eval time =    1682.19 ms /    23 tokens (   73.14 ms per token,    13.67 tokens per second)
      total time =    3676.12 ms /  1002 tokens