Generative model

Generative models are a class of computational models frequently used for classification. In machine learning, it typically models the joint distribution of inputs and outputs, such as P(X,Y), or it models how inputs are distributed within each class, such as P(X∣Y) together with a class prior P(Y). Because it describes a full data-generating process, a generative model can be used to draw new samples that resemble the observed data, a process often referred to as synthetic data generation. Generative models are used for density estimation, simulation, and learning with missing or partially labeled data. In classification, they can predict labels by combining P(X∣Y) and P(Y) and applying Bayes' rule. Generative models are often contrasted with discriminative models, which focus on predicting outputs from inputs directly.

Generative model approaches which use a joint probability distribution instead, include naive Bayes classifiers, Gaussian mixture models, variational autoencoders, generative adversarial networks and others.

Definition

In statistical classification, two main approaches are called the generative approach and the discriminative approach. These compute classifiers by different approaches, differing in the degree of statistical modelling. Terminology is inconsistent, but three major types can be distinguished:

A generative model is a statistical model of the joint probability distribution

P

(

X

,

Y

)

{\displaystyle P(X,Y)}

on a given observable variable X and target variable Y; A generative model can be used to "generate" random instances (outcomes) of an observation x.