Local Models Related Links

/lmg/ Accelerate
Guides
Quick Start Guide Anon's tutorial for getting models running locally
SillyTavern Guide Instructions for roleplaying via koboldcpp. Additional GNBF grammar usage
LM Tuning Guide Training, fine-tuning, and LoRA/QLoRA information
LM Settings Guide Explanation of various settings and samplers with suggestions for specific models
LM GPU Guide Current as of the 40 series. Alternatively some Anons made a few different build guides
Models
HuggingFace Best source for current quants (filter by GGUF or EXL2)
LLM VRAM Calc Tool to estimate VRAM usage for GGUF/EXL2/GPTQ quants
OpenModelDB Specifically models for upscaling images and videos
Voice Models Easily searchable list for use mainly with RVC 1/2
Models Info Table Googlesheet of models, AI labs, datasets, and various other ML info by Alan Thompson
Chat Leaderboard Closed and local models ELO rated with additional MMLU/MT-bench scores
Papers
Local Models Papers Papers and articles that I've found to be interesting with a way to search via abstracts
Arxiv ML Primary source of machine learning papers
PapersWithCode Indexer that allows sorting by GitHub stars
Semantic Scholar Scientific literature semantic search tool
Scholar Inbox ML focused paper recommendations based off personal preferences
News
AI Explained General AI news with well sourced links (Youtube)
AI News Blog Lesswrong cultist so "AI Bad" takes but does a good weekly AI news roundup (Blog)
ML Resources Broader sporadically updated list (not fully local)
Previous Threads Always good to search for previous questions before asking
Learn
LLM Course Collection of articles, videos, courses, and colabs for learning applied ML
Andre Karpathy YT In-depth videos of LLM construction from one of OpenAI's founding members
TF From Scratch Blogpost with Juypter notebook that goes step by step for coding and training a small GPT
LLM-Sampling Token Probability visualizer with support for current popular samplers
LLM Visualization Drag and pull 3D model of various LLMs with explanation for components
Intro to DNN Book format of a Neural Networks course that serves as in introduction to ML
Principles of DL Textbook that introduces the math behind Deep Learning
LLM Inferencing
Text Gen WebUI Frontend to most GPU/CPU model backends
WebUI Extensions Most notable XTTSv2 and Stable Diffusion
llama.cpp Main CPU inferencing development with GPU acceleration (GGUF models)
kobold.cpp llama.cpp fork with Kobold UI and additional features (with support for older GGML models)
exllama2 Inference library for local LLM with new quant style (70B llama2 on 24GB VRAM)
TabbyAPI FASTAPI application for exllama2 backend for use with SillyTavern
SillyTavern Frontend that is a heavily modified TavernAI fork
vllm Inference library with fast inferencing and PagedAttention for KV management
LLM Tools
Axolotl Fine-tuning tool for various architectures with integrated support for flash attention and rope scaling
Mergekit Toolkit for merging LLMs including piecewise assembly of layers
promptfoo Tool for testing and evaluating LLM output quality also with side-by-side feature
Floneum Graph/node editor for AI workflows with a focus on community made plugins
OpenRLHF Framework for RLHF generation optimized for performance and distributed models
FedModule Framework for federated learning with over 20 implemented algorithms
LLM Guiding
Langchain Set of resources to maximize LLMs Chains/tool integrations/agents/etc
llama_index Central interface to connect LLM's with external data
TextGrad Framework with API to backpropagate textual gradients with user defined loss functions
SGLang Structured generation language designed for LLM/VLMs
DSPy Composable and declarative modules for instructing LMs in a familiar Pythonic syntax
Continue Open source code assistant that works with local models
Datasets
Huggingface Best source for datasets
Wiki Embeddings Predone embeddings for various language of Wikipedia
ERP Scrapes (1)(2) Raw RP/ERP/ELIT content
VN JP/EN Scrape 60 million tokens of dialogue and actions/narration
WN JP/EN Scrape 100k chapters of webnovels paired with fan-translations
janitorai-cards 190k character cards converted to v2 format and viewable as local webpage
chub.ai Archive of various character cards from chub as well as from some other sources
Dataset Tools
augmentoolkit Generates multi-turn instruct-tuning data from input documents
dswav Audio dataset preparation tool using whisper and ffmpeg to transcribe and split inputs
lilac Dataset curation tool for RAG or tuning with annotating/clustering/labeling support
Data-Juicer Dataset preparation tool with support for multimodal data
InfoGrowth Online dataset curation framework for data cleaning and selection
Non-LLM Models
Vision/Image
ComfyUI Node based stable diffusion GUI. User submitted workflows
LDSR ComyUI Image super resolution upscaler with less artifacts than others but slower
MambaIRv2 Image restoration model that uses attentive state-space modeling for improved results
ControlNeXt 90% less parameters than ControlNet and works with other LoRA techniques
Molmo Multimodal LLMs with image/video understanding and has the VLM component fully open sourced
ColPali VLM that indexes documents from their visual features (PDF focused)
Surya OCR, layout analysis, reading order, line detection in 90+ languages
ShareCaptioner Image captioning model with lower hallucinations than LLaVa
BSQ-ViT Image/Video tokenizer with Binary Spherical quant that has best image/video restoration performance
Spandrel Library for loading various upscaling models for use with chaiNNer or SD WebUI
DiffEditor Tuning-free method for fine-grained image editing using score-based diffusion
MASA Match anything via SAM for use in finding similar objects across different domains
Depth-Anything-V2 Robust monocular depth estimation that works well with semantic segmentation
ProLab Semantic segmentation via property-level label space rather than just categories
SUPIR Image restoration and upscale method with semantic adjustment editing ability
DDColor Vivid and natural colorization for black and white photos (and possibly video)
lama-cleaner Local inpainting tool (remove or erase and replace)
TRELLIS Image-to-3D-assets generative model that uses a unified structured latent representation
Video
HunyuanVideo Video foundation model with SOTA results with NSFW output ability ComfyUI wrapper
Efficient Track Anything Allows for real time video segmentation on mobile devices or 2x FPS improvements over SAM2 on GPU
Upscale Hub Set of resources and models for image and video upscaling (anime focused)
Ground-A-Video Video Editing via Text-To-Image diffusion models with groundings/motion/depth data
EasyAnimate Text-to-Video model that maxes out at 6s usable with various framerates and resolutions
LivePortrait Real time face swap with extended controllability (eyes, lips, stitching)
MegActor Animate images from audio/image with consistent motion via diffusion
Audio/Speech
Amphion Audio/Music/Speech toolset of various models with visualization capability
Fish Speech 1.4 Text-to-Speech model with good CHN/JPN and decent ENG audio
DiffEditor Speech Editing model with improvements to OOD text output
Qwen2-Audio Audio-Language model that can voice chat and do audio analysis without specific prompting
FluxMusic Text-to-Music model with large improvements over AudioLDM2
UniMuMo Text/Music/Motion foundational model capable of mixing all modalities for output generation
GPT-SoVITS Few-shot voice cloning and Text-to-Speech WebUI (ENG/JPN/CHN) rentry guide
ControlSpeech Text-to-Speech with voice clone capability that takes in voice/style/content prompts
whisper.cpp Speech-to-Text inference library with CPU/GPU support for various whisper based models
Whisper Diarization STT via Transformers.js with word-level timestamps and speaker segmentation
STAR-Adapt ASR unlabeled finetune method that reduces WER for specific accents/noise
RVC Retrieval based Voice Conversation model
Urhythmic Unsupervised rhythm modeling for voice conversion
Descrpyt High-Fidelity audio compression with improved RVQGAN (can drop-in replace EnCodec)
DeepFilterNet Real time noise suppression using deep filtering
UVR Audio source separation GUI for various models with full Demucs and MDX23C support
AudioSR Audio super resolution (any -> 48kHz)
EAT Audio and speech classification
Other
AnythingLLM RAG and agent focused frontend with support for local and cloud models
MemoRAG RAG framework that leverages its memory model by recalling with query-specific clues
T-Ragx Translation fine-tune method that works with RAG (glossaries) and preceding text
GenTranslate Fine-tune of SeemlessM4T from N-best hypotheses dataset for MT and Speech-to-Text
Dragon+ Dual-encoder based dense retriever for use with the RA-DIT FT approach with paired LLM
Magica File content type detector model
AutoACT Automatic agent learning framework using a division-of-labor strategy
LOCUST State-space model for long document abstractive summarization
NV Embed v1 Decoder-only LLM embedding model that outperforms T5/BERT/similar models
ESPN GPUDirect Storage implementation for multi-vector embedding retrieval and bindings
FastFit Text few-shot classification fine-tuning method with high accuracy and fast training time
Prithvi-WxC Weather forecasting foundation model trained on 160 types of atmospheric data
Time-MoE Time series MoE foundation models with largest having 1.1B active parameters
Edit
Pub: 20 Mar 2023 19:58 UTC
Edit: 06 Dec 2024 21:52 UTC
Views: 79121