What is this? A private cultural benchmark suite.
What it contains: Simple pop quizzes, including video games, anime, urban dict definitions, internet culture, vibes, song lyrics, etc.
What it tests for: How diverse the training data is. How much the model can recall (which is directly correlated to the model size). How likely the model is to play along, aggressive safety-alignment.
What it does not test for: How "smart" a model is, how good it is at following instructions, coding, its effective context, creativity, slop, etc.
Why: Because many model makers remove anything they deem "useless" from the training data, or teeter on catastrophic forgetting in their attempt to achieve better STEM benchmark scores.

All tests are run on temperature 0 (greedy sampling).