So, I tried training my first LORA, and let me tell you, the documentation out there is trash. Nobody knows the right learning rate, nobody properly explains parameters, and Google is useless. I thought this would be hard, but nope—just a Google account and a few clicks. Unsloth provided working notebooks. Uploaded my 20MB dataset(based on no_robots) to Hugging Face, opened Google Colab (free GPU, yay), and boom, training code was already there. Default settings? Sure, why not.

I used Mistral Nemo, but the default 2e-4 learning rate made the loss jump all over the place. Gemini in Colab was like, "That's bad, lower it," so I dropped it to 5e-5 and set 120 steps. Still, the loss was wobbly. Tested the model—prompt understanding? Awful. Did it learn one of the speech styles? Kinda. No alignment data in my dataset, but somehow it still talks like ChatGPT, all preachy. Guess Nemo's "base" model isn't so base after all. Will try with different settings later.