/lmg/ Official Best Models To Use Guide
This is a collaborative project that anyone can edit. You WILL contribute, and help keep this project up-to-date for the benefit of the whole general! The document is read-only, explanation how to contribute below!
How to contribute: Click edit in the bottom left corner, copy the whole thing (important to keep the markdown intact, copying the preview that is seen by default would break the document), make a new rentry with your edits (do not share the edit code, or edit the rentry, make new ones instead!), and post near the end of the thread for OP to put into the new one.
| Parameter size | Model | GGUF huggingface link | exl2 huggingface link |
|---|---|---|---|
| 9B | Gemma2-Gutenberg-Doppel-9B | GGUF | [exl2] could not find |
| 12B | Rocinante 12B v1.1 | GGUF | exl2 |
| 22B | Cydonia-22B-v1.2 | GGUF | IDK if the exl2 quants on HF are any good so I didn't put them here |
| 35B | c4ai-command-r-v01 | GGUF | exl2 |
| 70B | Nemotron-70B | GGUF | exl2 |
| 123B | Magnum v4 123b | GGUF | exl2 |