Post training a model to explore Demoscenes
Summary
This is a detailed experimental write-up about post-training a language model to generate Dweets for Demoscenes. It covers dataset collection from Dwitter, continued pretraining on a 7B model, measuring surprisal and perplexity improvements, curriculum-based sampling, and a reinforcement learning from human feedback (RLHF) approach to infuse personal aesthetic preferences. The piece also includes results, visuals, and links to released datasets and models on HuggingFace.