Preference learning

Preference learning is a subfield of machine learning that focuses on modeling and predicting preferences based on observed preference information. Preference learning typically involves supervised learning using datasets of pairwise preference comparisons, rankings, or other preference information.

Tasks

The main task in preference learning concerns problems in "learning to rank". According to different types of preference information observed, the tasks are categorized as three main problems in the book Preference Learning:

Label ranking

In label ranking, the model has an instance space

X

=

{

x

i

}

{\displaystyle X=\{x_{i}\}\,\!}

and a finite set of labels

Y

=

{

y

i

|

i

=

1

,

2

,

⋯

,

k

}

{\displaystyle Y=\{y_{i}|i=1,2,\cdots ,k\}\,\!}

. The preference information is given in the form

y

i

≻

x

y

j

{\displaystyle y_{i}\succ _{x}y_{j}\,\!}

indicating instance

x

{\displaystyle x\,\!}

shows preference in

y

i

{\displaystyle y_{i}\,\!}

rather than

y

j

{\displaystyle y_{j}\,\!}

. A set of preference information is used as training data in the model. The task of this model is to find a preference ranking among the labels for any instance.

It was observed that some conventional classification problems can be generalized in the framework of label ranking problem: if a training instance

x

{\displaystyle x\,\!}

is labeled as class

y

i

{\displaystyle y_{i}\,\!}

, it implies that

∀

j

≠

i

,

y

i

≻

x

y

j

{\displaystyle \forall j\neq i,y_{i}\succ _{x}y_{j}\,\!}

. In the multi-label case,

x

{\displaystyle x\,\!}

is associated with a set of labels

L

⊆

Y

{\displaystyle L\subseteq Y\,\!}

and thus the model can extract a set of preference information

{

y

i

≻

x

y

j

|

y

i

∈

L

,

y

j

∈

Y

∖

L

}

{\displaystyle \{y_{i}\succ _{x}y_{j}|y_{i}\in L,y_{j}\in Y\backslash L\}\,\!}

. Training a preference model on this preference information and the classification result of an instance is just the corresponding top ranking label.