RLHF vs DPO vs IPO vs KTO: which alignment method should you use

RLHF vs DPO vs IPO vs KTO: which alignment method should you use You have a base model, say Llama 3.2 8B, that can write poetry in any meter and pass the bar exam. It can also generate instructions for synthesizing controlled substances, roleplay as a manipulative therapist, and explain in loving de