preference-optimization

[ICML 2025] Beyond Bradley-Terry Models: A General Preference Model for Language Model Alignment (https://arxiv.org/abs/2410.02197)

PipelineLLM 是一个系统性的大语言模型（LLM）后训练学习项目，涵盖从监督微调（SFT）到偏好优化（DPO）、强化学习（RLHF/PPO/GRPO）再到持续学习（Continual Learning)的完整技术栈。

[TMLR] Triple Preference Optimization

A lightweight post-training framework for LLMs and VLMs. 51 algorithms, 38 verified models. Scales with DeepSpeed, vLLM, and Ray.

Symmetrical Visual Contrastive Optimization: Aligning Vision-Language Models with Minimal Contrastive Images

[ACL 2025] Uncovering the Impact of Chain-of-Thought Reasoning for Direct Preference Optimization: Lessons from Text-to-SQL

Code for ICLR 2025 Paper: Synthio: Augmenting Small-Scale Audio Classification Datasets with Synthetic Data

[ICML 2025] TGDPO: Harnessing Token-Level Reward Guidance for Enhancing Direct Preference Optimization

🔬 Official implementation of ExPO-HM: Learning to Explain-then-Detect for Hateful Meme Detection (ICLR 2026). Novel multimodal RL approach for interpretable and explainable content moderation.

[TMLR] Dual Caption Preference Optimization

Pre-DPO: Improving Data Utilization in Direct Preference Optimization Using a Guiding Reference Model

GenPark AI Agent Skill - Iterative self-rewarding LLM judge scoring multi-turn agent outputs and synthesizing contrastive hard-negative prompt variations.

GenPark AI Agent Skill - Direct Preference Optimization (DPO) implicit reward calculation, reference policy log-ratio tracking, and pairwise preference loss evaluation.

GenPark AI Agent Skill - Pairwise Bradley-Terry and Elo rating tournament engine for ranking multi-agent strategies, prompt mutations, and generated tool solutions.

GenPark AI Agent Skill - Prioritized experience replay buffer for agent reinforcement fine-tuning (RFT / GRPO) with advantage estimation and importance sampling weights.

GenPark AI Agent Skill - Pairwise Bradley-Terry and Elo rating tournament engine for ranking multi-agent strategies, prompt mutations, and generated tool solutions.

GenPark AI Agent Skill - Direct Preference Optimization (DPO) implicit reward calculation, reference policy log-ratio tracking, and pairwise preference loss evaluation.

GenPark AI Agent Skill - Kahneman-Tversky Optimization (KTO) loss evaluator aligning agents directly on unpaired binary feedback (thumbs-up / thumbs-down) using prospect theory loss aversion.

GenPark AI Agent Skill - Iterative self-rewarding LLM judge scoring multi-turn agent outputs and synthesizing contrastive hard-negative prompt variations.

GenPark AI Agent Skill - Prioritized experience replay buffer for agent reinforcement fine-tuning (RFT / GRPO) with advantage estimation and importance sampling weights.