TensorTonic
Study Plans
Study Plan
Canonicalize Pairwise Preference Data
Language ModelsData ProcessingReinforcement Learning
Medium

Convert complete preference rankings into winner-loser training pairs. Each record contains a prompt_id and a candidates list; each candidate has id, text, and rank fields. Rank 1 is best, and lower ranks beat higher ranks. Candidate text is context only; output pairs contain IDs.

Include every better/worse candidate combination once. Preserve prompt input order, then order each prompt’s pairs by ascending winner rank and ascending loser rank. Return a Python dictionary with pairs, a list of dictionaries containing exactly prompt_id, winner_id, and loser_id, copied from the input records.

Loading visualization...

Examples

Input: records = [{"prompt_id":"p1","candidates":[{"id":"a","text":"best","rank":1},{"id":"b","text":"middle","rank":2},{"id":"c","text":"last","rank":3}]}]

Output: {"pairs":[{"prompt_id":"p1","winner_id":"a","loser_id":"b"},{"prompt_id":"p1","winner_id":"a","loser_id":"c"},{"prompt_id":"p1","winner_id":"b","loser_id":"c"}]}

Explanation: A strict ranking of three candidates contains all three ordered preference pairs.

Input: records = [{"prompt_id":"p2","candidates":[{"id":"b","text":"second","rank":2},{"id":"a","text":"first","rank":1}]}]

Output: {"pairs":[{"prompt_id":"p2","winner_id":"a","loser_id":"b"}]}

Input: records = [{"prompt_id":"x","candidates":[{"id":"a","text":"A","rank":1},{"id":"b","text":"B","rank":2}]},{"prompt_id":"y","candidates":[{"id":"c","text":"C","rank":1},{"id":"d","text":"D","rank":2}]}]

Output: {"pairs":[{"prompt_id":"x","winner_id":"a","loser_id":"b"},{"prompt_id":"y","winner_id":"c","loser_id":"d"}]}

Hint 1

sorted accepts a key function without changing the original list.

Hint 2

itertools.combinations generates distinct two-item combinations.

Requirements

  • Use ordinary Python lists and dictionaries; no numerical library is needed.
  • Pair only candidates from the same prompt record.
  • Preserve input records and candidate-list order without modification.

Constraints

  • records is a list of prompt dictionaries in desired prompt order.
  • Every prompt contains at least two candidates.
  • Candidate IDs, texts, and ranks are unique within a prompt.
  • Ranks form the complete integer range from 1 through candidate count.
Try Similar Problems
Implement the DPO LossMediumScore a KL-Regularized Policy BatchMediumCompute Temperature-Based Data MixturesMediumCompare Sequence and Token Length NormalizationMediumNormalize GRPO Group AdvantagesMediumBuild Deterministic MinHash SignaturesMediumPerform Stable Exact DeduplicationEasyCompute a Clipped GRPO SurrogateHardExperience Replay BufferEasyTile and DiffMediumBox Encode and DecodeMediumDrop DuplicatesMedium

Accepts: array

You must run your code first.

Canonicalize Pairwise Preference Data

Language ModelsData ProcessingReinforcement Learning
Medium

Convert complete preference rankings into winner-loser training pairs. Each record contains a prompt_id and a candidates list; each candidate has id, text, and rank fields. Rank 1 is best, and lower ranks beat higher ranks. Candidate text is context only; output pairs contain IDs.

Include every better/worse candidate combination once. Preserve prompt input order, then order each prompt’s pairs by ascending winner rank and ascending loser rank. Return a Python dictionary with pairs, a list of dictionaries containing exactly prompt_id, winner_id, and loser_id, copied from the input records.

Loading visualization...

Examples

Input: records = [{"prompt_id":"p1","candidates":[{"id":"a","text":"best","rank":1},{"id":"b","text":"middle","rank":2},{"id":"c","text":"last","rank":3}]}]

Output: {"pairs":[{"prompt_id":"p1","winner_id":"a","loser_id":"b"},{"prompt_id":"p1","winner_id":"a","loser_id":"c"},{"prompt_id":"p1","winner_id":"b","loser_id":"c"}]}

Explanation: A strict ranking of three candidates contains all three ordered preference pairs.

Input: records = [{"prompt_id":"p2","candidates":[{"id":"b","text":"second","rank":2},{"id":"a","text":"first","rank":1}]}]

Output: {"pairs":[{"prompt_id":"p2","winner_id":"a","loser_id":"b"}]}

Input: records = [{"prompt_id":"x","candidates":[{"id":"a","text":"A","rank":1},{"id":"b","text":"B","rank":2}]},{"prompt_id":"y","candidates":[{"id":"c","text":"C","rank":1},{"id":"d","text":"D","rank":2}]}]

Output: {"pairs":[{"prompt_id":"x","winner_id":"a","loser_id":"b"},{"prompt_id":"y","winner_id":"c","loser_id":"d"}]}

Hint 1

sorted accepts a key function without changing the original list.

Hint 2

itertools.combinations generates distinct two-item combinations.

Requirements

  • Use ordinary Python lists and dictionaries; no numerical library is needed.
  • Pair only candidates from the same prompt record.
  • Preserve input records and candidate-list order without modification.

Constraints

  • records is a list of prompt dictionaries in desired prompt order.
  • Every prompt contains at least two candidates.
  • Candidate IDs, texts, and ranks are unique within a prompt.
  • Ranks form the complete integer range from 1 through candidate count.
Try Similar Problems
Implement the DPO LossMediumScore a KL-Regularized Policy BatchMediumCompute Temperature-Based Data MixturesMediumCompare Sequence and Token Length NormalizationMediumNormalize GRPO Group AdvantagesMediumBuild Deterministic MinHash SignaturesMediumPerform Stable Exact DeduplicationEasyCompute a Clipped GRPO SurrogateHardExperience Replay BufferEasyTile and DiffMediumBox Encode and DecodeMediumDrop DuplicatesMedium

Accepts: array

You must run your code first.