Rectified linear unit
activation function defined as the identity function for positive arguments and zero for negative arguments
In the context of artificial neural networks, the rectifier or ReLU (rectified linear unit) activation function is an activation function defined as the non-negative part of its argument, i.e., the ramp function:
ReLU
(
x
)
=
x
+
=
max
(
0
,
x
)
=
x
+
|
x
|
2
=
{
x
if
x
>
0
,
0
x
≤
0
{\displaystyle \operatorname {ReLU} (x)=x^{+}=\max(0,x)={\frac {x+|x|}{2}}={\begin{cases}x&{\text{if }}x>0,\\0&x\leq 0\end{cases}}}
where
x
{\displaystyle x}
is the input to a neuron. This is analogous to half-wave rectification in electrical engineering.
ReLU is one of the most popular activation functions for artificial neural networks, and finds application in computer vision and speech recognition using deep neural nets and computational neuroscience.
History
The ReLU was first used by Alston Householder in 1941 as a mathematical abstraction of biological neural networks.
Kunihiko Fukushima in 1969 used ReLU in the context of visual feature extraction in hierarchical neural networks.
In 1998, Gregory Woodbury demonstrated that the rectified linear function could account for a broad range of emergent properties in the visual cortex. His work showed that a single unified model could drive the joint development of refined retinotopic maps, ocular dominance columns, and orientation selectivity. By utilizing the rectifier's "cutoff" property, Woodbury achieved a close quantitative fit to biological data, matching the spatial periodicities and topographic refinement patterns observed in macaque and cat cortical maps. Furthermore, he extended this framework to adult plasticity, accurately replicating the spatial and temporal dynamics of lesion-induced cortical reorganization. This research established that the rectified linear response was a necessary mechanism for the stable self-organisation and maintenance of complex, multi-feature neural maps.
This brief starts where responsible research should: with the source description of “Rectified linear unit” as activation function defined as the identity function for positive arguments and zero for negative arguments. Everything that follows is an evidence route, not borrowed authority.
Why this record matters
The subject matters to the general reference register because the source frames it as activation function defined as the identity function for positive arguments and zero for negative arguments. Its deeper value depends on whether names, dates, institutions and citations support that framing.
The citation trail is more important than the brevity of the summary: it shows where individual claims can be examined in context. The source revision retrieved here is dated Sep 7, 2026. The linked authority identifier is Q7303176. None of the 0 selected statements returned an explicit reference. The first chronological checks are 1941, 1969 and 1998.
A concise general-reference account can conceal disagreements about scope, terminology or the weight assigned to individual sources. The lead is largely declarative, so disagreement and counter-evidence require a deliberate search beyond the opening account. Authority statements aid reconciliation but still require their own references, qualifiers and ranks to be checked.
How to read it
Use the entry as an orientation point, then follow its citations and revision history. Names, dates and institutional relationships should be checked against the original record.
- Subject orientation
- Search vocabulary
- Locating named sources
The closest primary source, responsible institution and strongest cited specialist reference.
Three-step research path
- Establish the record: confirm the title “Rectified linear unit”, its source revision and the description used here.
- Expand the search: follow Rectified linear unit primary sources, Rectified linear unit archive and Rectified research across catalogues and specialist indexes.
- Test the account: compare the strongest cited source with the responsible institution’s current record and note any disagreement.
Questions for further research
- Which source most directly establishes the central claim about “Rectified linear unit”?
- Which cited source is closest to the event, object or claim?
- What terminology or title could unlock a more precise catalogue search?
Search terms from this dossier
This entry incorporates text from “Rectified linear unit” on English Wikipedia. Contributors are listed in the page history. Text is available under the Creative Commons Attribution-ShareAlike 4.0 License. Selected authority identifiers and statements are retrieved from Wikidata under CC0; their references and qualifiers remain part of the verification path.