How to use AutoGen Studio with Text Gen (pictures included)
Owee, this one is pretty interesting. I've been trying out various other types of programs that use the openai api and using oobabooga's textgen as the backend. Today I tried out AutoGen Studio : https://microsoft.github.io/autogen/blog/2023/12/01/AutoGenStudio/
These instructions assume you are using the windows one click version of oobabooga, and you have WSL installed. (any other os configuration just requires the ip settings to be different)
1- Install autogen in WSL (you can install in windows miniconda, it will work you can talk to your models, but you might have issues with the model trying to run and execute code, idk I switched to WSL and was having much success).
conda create --name autogen python=3.11 -y
conda activate autogen
pip install autogenstudio
To run autogen studio use:
conda activate autogen
autogenstudio ui --port 8081
2- with autogen sudio running go to model and create a new model like so, here I am using http://192.168.192.1:5000/v1 because I am disconnected from the internet and this is the ip address of my windows machine (192.168.192.1) from the perspective of the WSL installation. Go to your windows command window and enter "ipconfig /all" to see the Preferred address your machine has on the network:
The important thing to note is that the format should be http://"Your Local IP HERE":5000/v1
3- in the CMD_FLAGS.txt file for obabooga text gen put this in the file:
--api --api-key 11111 --verbose --listen --listen-host 0.0.0.0 --listen-port 1234
4- load up obabooga textgen and then load your model (you can go back to autogen and your model and press the "test model" button when the model is finished loading in oobabooga's textgen, this will verify that AutoGen and your model are talking via textgen). Also when you load textgen and look at the command window you'll see that the api is running on http://0.0.0.0:5000, the 0.0.0.0 mean that anything connecting to textgen needs to use the ip of the machine on your network, don't enter http://0.0.0.0:5000 into the autogen studio model window.
5- configure your agents and workflow to use the oobabooga model
I used a multi fine-tuned model from here: https://huggingface.co/Xwin-LM/Xwin-LM-70B-V0.1 the multi finetune model is one that I have locally, the linked model is the model that I've been finetuning. I am running it with the exllama2 quantization.
All I did to do the demo was click, the "Sine Wave" button at the bottom of the screen...omg I can't believe it worked!
Also all of your code and images are saved here in the WSL installation your user number and name will be different, but this is the location in general:
\\wsl.localhost\Ubuntu-22.04\home\myselflinux\miniconda3\envs\autogen\lib\python3.11\site-packages\autogenstudio\web\files\user\198fb9b77fb10d10bae092eea5789295
Edit: Adding agents to group chats has a profound change on the output, idk this is very interesting. Here is a video that goes over the agents and agent groups, they are using chatgpt but the same ideas still hold: https://www.youtube.com/watch?v=4ZqJSfV4818