KoboldAI on gfx803

KoboldAI has a 'play-rocm.sh' script that allows AMD GPU owners to use ROCm to run KoboldAI. However, in it's default configuration, it only supports cards supported by the latest version of ROCm. This is a little modification to roll the version of ROCm back to one that is compatible with gfx803 cards.

Prerequisites
How-To
1. Dockerfile changes
2. Run the script

Prerequisites

Hardware

A gfx803 GPU (also called Polaris 10)
List of cards: RX470, RX480, RX570, RX580, RX590
If you don't have one of these cards, this will not help you. If you have a newer card you are potentially reducing performance.

Operating system

ROCm requires Linux, and is officially supported on Ubuntu, CentOS, and SUSE Linux Enterprise Server
Ubuntu 20-04 LTS is what I used

Docker

You will need Docker installed. Check the errors you run into to find the packages. No I didn't keep track of that. Yes I have bloat.

If you've already attempted to start KoboldAI on the current operating system

You'll have to remove the old docker image(s). Google that, it's super simple.

How-To

Dockerfile changes

Go to /docker-rocm in the main KoboldAI-Client directory
Open Dockerfile using a text editor, and replace the contents with the following:
FROM rocm/pytorch:rocm3.9_ubuntu18.04_py3.6_pytorch
RUN git clone https://github.com/xuhuisheng/rocSPARSE && cd rocSPARSE && bash install.sh -di
RUN conda install --all -y Flask-SocketIO
RUN pip3 install git+https://github.com/finetuneanon/transformers@gpt-neo-localattention3

See https://github.com/jrcichra/rocm-pytorch-gfx803 for more info. All credit to the owners, I'm just sharing the information.

Run the script

Navigate to the main KoboldAI-Client directory and run the play-rocm.sh script
It's that easy