DIY - Virtual Waifu
Notice:
At the moment, this is just a draft for reference, so all information may be somewhat limited and everything is subject to change.
Also, you might as well consider this just some coomer's ramble.
Note: This is not a guide.
What would you need to create your digital waifu? Can you even do that?
Yes, this scenario has been a reality for some time now. And it's probably even possible to create a physical waifu, not just a digital one — but it's at a much higher level of complexity.
Of course, I'm talking about simplified versions of digital cunts; unfortunately, to create a waifu with a certain level of consciousness would require hardware at the level of giant corporations, and almost unlimited funding. To be honest, I'm pretty sure there's someone fucking some highly intelligent robot bitch somewhere as you read this. The only problem is that for now this level of realism is still out of reach for the common citizen.
=========
Anyway, back to the subject… On this page, for example, I'll just focus on the digital side of things; whether due to limited time or limited knowledge.
Also, I'll break everything down into three parts:
• Text: Related to dialogues or any text produced.
• Audio: Related to speech; preferably in a form that can be converted from the results of the text part.
• Visual: Something to represent the visualization of your waifu in a more dynamic way than just a static image.
Preferably, I will try to use as a reference, open-source projects or software that can be acquired without too much difficulty; also giving preference to things that can work locally, at least to some degree.
TABLE OF CONTENTS:
Again, I just want to reiterate what I said before: We are just trying to create a simplified and dumbed down version of a waifu, at best a simulation of what a waifu with some kind of consciousness would look like. Unfortunately, the projects we have related to AI at the consumer level are still not that advanced — YET.
TEXT
First thing, I would like to list some models that we currently have available for text generation that are open-source. Especially the conversational models, the ones that were fine-tuned for chat.
Models:
Most of these models obviously work in a much more limited way than the models used in closed and larger scale projects like cAI, but that doesn't mean they are unusable. On the contrary, considering the freedom you have with an open model, it's possible for you to use your creativity in far more elaborate ways. You'll just need to guide the model and tune the settings a little more closely. But over time even these inconveniences will diminish, as the trend is always towards improvement and optimization.
Now about the front-end interfaces for your models.
Interfaces:
Project | Github Page |
---|---|
KoboldAI | https://github.com/henk717/KoboldAI |
TavernAI | https://github.com/TavernAI/TavernAI |
Text Generation Web UI | https://github.com/oobabooga/text-generation-webui |
gradio-ui | https://github.com/PygmalionAI/gradio-ui |
Let me tell you something; at the moment, the algorithms for text generation are much more demanding in terms of hardware than the ones for image generation. Of course, these algorithms work completely differently, so it's understandable, but I would still like to comment on that, because to run these models locally, you'll need some pretty robust hardware, especially on the GPU side. But on that same note, it's also possible to run these models on a cloud computing service, either using paid or free services like Google Colab, for example.
Anyway, back to the interfaces...
KoboldAI:
Quick introduction, taken from the Github page:
This is a browser-based front-end for AI-assisted writing with multiple local & remote AI models. It offers the standard array of tools, including Memory, Author's Note, World Info, Save & Load, adjustable AI settings, formatting options, and the ability to import existing AI Dungeon adventures. You can also turn on Adventure mode and play the game like AI Dungeon Unleashed.
...
Here are some extra links about the project:
Stuff | Link |
---|---|
KoboldAI | https://github.com/henk717/KoboldAI |
Horde | https://lite.koboldai.net/ |
KoboldAI on Windows | https://rentry.org/ipfub |
/aids/ Prompts | https://aetherroom.club/ |
aids Wiki | https://aids.miraheze.org/wiki/KoboldAI |
Soft Prompt Guide | https://rentry.org/shiso-softprompt-quickstart |
Pygmalion + Soft Prompt | https://rentry.org/pygsoft |
TavernAI:
Quick introduction, taken from the Github page:
Atmospheric adventure chat shell for AI language models (KoboldAI, NovelAI, Pygmalion).
...
Here are some links about the project:
Stuff | Link |
---|---|
TavernAI | https://github.com/TavernAI/TavernAI |
Bot Cards for TavernAI+Pygmalion | https://booru.plus/+pygmalion |
Text Generation Web UI:
Quick introduction, taken from the Github page:
A gradio web UI for running large language models like gpt-j-6B, gpt-neo, opt, galactica, and pygmalion on your own computer. Its goal is to become the "AUTOMATIC1111/stable-diffusion-webui" of text generation.
...
Here are some links about the project:
Stuff | Link |
---|---|
Text Generation Web UI | https://github.com/oobabooga/text-generation-webui |
gradio-ui (Pygmalion):
Quick introduction, taken from the Github page:
This is a prototype Gradio-based UI which allows you to chat with the Pygmalion models.
...
To be honest, there are better options for using the Pyg models; which are the previous three options. But I'll still use this part to talk a little bit about the Pyg models, more specifically, Pygmalion 6B. This specific model is probably the closest we have, at a consumer level, to a chatbot model similar to those used on the cAI platform. Of course, I'm not talking about scale, because the difference is even difficult to calculate, but I mean in terms of training and fine-tuning.
The current stable version of Pyg6B has been fine-tuned with some data (SODA + LIGHT + CAI + other stuff) that worked great for chatbots. It's been a few days since the model has undergone another training, so the next waves of training and fine-tuning hold even more promise considering the data that is or has been collected recently.
If you want to help, but don't have enough experience or knowledge to help with the more complex stuff along with the matrixbros, you can also help by dumping your cAI logs.
Page | Link |
---|---|
CAI Dumper | https://rentry.org/chatlog-dumping |
Training Data | https://rentry.org/training-data |
CAI Dump Uploader | https://dump.nopanda.io/ |
Here are some links about the project:
Other Stuff | Link |
---|---|
gradio-ui | https://github.com/PygmalionAI/gradio-ui |
MainRentry | https://rentry.org/pygmalion-ai |
Pygmalion Local | https://rentry.org/pygmalion-local |
Pygmalion on Mobile | https://rentry.org/pygmobile |
Pygmalion Tips | https://rentry.org/PygTips |
Pygmalion Bot Prompts | https://rentry.org/pygbotprompts |
More Pygmalion Bot Prompts | https://botprompts.net/ |
Bot Cards for TavernAI+Pygmalion | https://booru.plus/+pygmalion |
Chai's Pygmalion Character Creation & Writing Tips | https://rentry.org/chai-pygmalion-tips |
How to Make a Character | https://rentry.org/create-a-character-for-fucking-idiots |
Pygmalion + Soft Prompt | https://rentry.org/pygsoft |
cAI Import To PYG | https://rentry.org/chartopyg |
Discord (god have mercy on your retarded soul) | https://discord.com/invite/ZHXEa3yywq |
AUDIO
For audio we're even more behind than text, so it's a bit trickier to deal with this shit on an open-source level; but we are getting there, and there is initiative for training and further development, so our hope is far from dead.
Project | Github Page |
---|---|
ESPnet | https://github.com/espnet/espnet |
Glow-TTS | https://github.com/jaywalnut310/glow-tts |
HiFi-GAN | https://github.com/jik876/hifi-gan |
HiFi-VC | https://github.com/tinkoff-ai/hifi_vc |
PPG-VC | https://github.com/liusongxiang/ppg-vc |
Soft-VC | https://github.com/bshall/soft-vc |
Sovits | https://github.com/Francis-Komizu/Sovits |
StarGAN | https://github.com/yl4579/StarGANv2-VC |
Tacotron 2 | https://github.com/NVIDIA/tacotron2 |
TorToiSe | https://github.com/neonbjb/tortoise-tts |
VITS | https://github.com/jaywalnut310/vits |
Whisper | https://github.com/openai/whisper |
I actually don't have much to say about the Audio side yet. I'm still researching and testing the open-source projects to see if it's viable for the development of our personal waifu.
But as I said at the beginning, this page is still under construction, so...
VISUAL
On the visual side, I've already started testing on two different platforms, and both are producing interesting results. Of course, one of these platforms is a game engine worth billions of dollars, while the other is a VR Porn platform made by a small team of developers. But either way, both are extremely sandbox and actually work really well for our situation.
Unreal Engine
Page | Link |
---|---|
Unreal Engine | https://www.unrealengine.com/en-US |
There is no need for an introduction. This is one of the biggest game engines out there, if not the biggest, and the entry level for development on the platform is relatively approachable.
I'm working on some tests at the moment, and plan to write a report detailing some of the results soon.
...
MetaHuman:
Page | Link |
---|---|
MetaHuman | https://www.unrealengine.com/en-US/metahuman |
It is a tool of the platform for creating characters with a more realistic style. It's a great tool, I must say. But it doesn't work very well for unrealistic characters. I also plan to write a lot about it in the future.
...
Virt-A-Mate
Page | Link |
---|---|
Patreon (to buy) | https://www.patreon.com/meshedvr |
F95 (to download for free) | https://f95zone.to/threads/virt-a-mate-v1-21-0-4-meshed-vr.6881/ |
HUB | https://hub.virtamate.com/resources/categories/free.4/?page=1 |
Wiki | https://hub.virtamate.com/wiki/ |
This is a software/game, aimed at creating sexual scenes in virtual reality. Or in short, porn in VR. To be honest, I've been playing around with this thing since a long time ago, when the software was much more limited than it is today. Currently, the level of sandbox the software offers allows for basically unlimited setups, and it is relatively simple to create new plugins allowing the integration of APIs and external equipment.
In my opinion, I think it's much easier to integrate the three parts (Text>Audio | Visual) using VAM than using the Unreal Engine.
I'm not saying that because I think the UE is more limited, on the contrary, the UE is an extremely robust engine and a much more complete platform. However, the problem lies with the use of the software; Unreal Engine is an engine for creating games, where you would have to build the platform to use your waifu first, while VAM is a ready-to-use platform for our case — we would just need to work on the plugins to complement it.
But not everything is perfect, despite the almost limitless possibilities, this software is very poorly optimized; to the point of being practically useless for very complex scenes or scenes with many characters.
However, I still believe that the advantages still outweigh the disadvantages; and the main advantage, in my opinion, is the possibility of creating and sharing scenes and characters, that is, you don't need to know how to create everything from scratch, you can simply download other people's creations and modify them to your liking.
...
Here is a list of how everything is normally divided and shared:
Category | Link | Description |
---|---|---|
Scenes | https://hub.virtamate.com/resources/categories/scenes.3/ | They are basically scenes already created you can play with, or templates for you to load your characters, and obviously also play with. |
Looks | https://hub.virtamate.com/resources/categories/looks.7/ | These are characters that you can download and place in your scenes. |
Clothing | https://hub.virtamate.com/resources/categories/clothing.8/ | Clothes to use on your characters. |
Hairstyles | https://hub.virtamate.com/resources/categories/hairstyles.9/ | Hairstyles to use on your characters. |
Morphs | https://hub.virtamate.com/resources/categories/morphs.10/ | These are files that you download to change specific parts of your character's body. Let's say you've downloaded a pack of morphs that affects the breasts; then with this pack you can change the shapes, sizes and other details of your characters breasts. |
Textures | https://hub.virtamate.com/resources/categories/textures.11/ | To change the texture of your characters. |
Plugins | https://hub.virtamate.com/resources/categories/plugins.12/ | With plugins you can change basically everything. Including creating new features that did not exist on the platform before. If I'm not mistaken, you can program the plugins using C#, and there are almost no limitations on what you can do within the platform. And using these plugins is our best chance of integrating the three parts (Text>Audio>Visual). |
Assets | https://hub.virtamate.com/resources/categories/assets.2/ | They are basically 3D assets converted from other platforms to be used in your scenes. |
Considering the amount of existing content, I believe that this software is one of our best options to use to create the visual representation of our waifu. There is a good amount of documentation and tutorials, so it's not too hard to get started.
Also, there are already some plugins aimed at integrating and expanding the use of texts and speech, and I would like to highlight some of them.
Plugin | Link | Info |
---|---|---|
Voice Model (Text-To-Speech, Neural Network based) | https://hub.virtamate.com/resources/voice-model-text-to-speech-neural-network-based.25807/ | |
vamX | https://hub.virtamate.com/resources/vamx-1-26-easy-save-load-presets-natural-eye-movement-desktop-mouse-thrust-ragdoll.28617/ | One of the updates to this plugin added the option to add audio and speech to your character. |
Chatbots | https://hub.virtamate.com/threads/chatbots-for-vam.10109/ | |
SpeechRecognition | https://hub.virtamate.com/resources/speechrecognition.6865/ | |
VoiceControl | https://hub.virtamate.com/resources/voicecontrol.8269/ | |
Realtime Lipsync | https://hub.virtamate.com/resources/realtime-lipsync.1286/ |
If you want to do some testing but don't know how to use the software, I'll be leaving a series of tutorials made by Captain Varghoss; It's a really good way to start.
Tutorials:
Koikatsu Party
Page | Link |
---|---|
Steam (to buy) | https://store.steampowered.com/agecheck/app/1485180/ |
F95 (to download for free) | https://f95zone.to/threads/koikatu-repack-rx16-illusion.14554/ |
It's been a while since I've played this game, so it might take a while before I can write anything about it. But I know that the game has a very active community, and I believe that it is also possible to do some kind of integration with the game, let's see...
...