W-shingling
Open-knowledge reference entry
In natural language processing a w-shingling is a set of unique shingles (therefore n-grams) each of which is composed of contiguous subsequences of tokens within a document, which can then be used to ascertain the similarity between documents. The symbol w denotes the quantity of tokens in each shingle selected, or solved for.
The document, "a rose is a rose is a rose" can therefore be maximally tokenized as follows:
(a,rose,is,a,rose,is,a,rose)
The set of all contiguous sequences of 4 tokens (Thus 4=n, thus 4-grams) is
{ (a,rose,is,a), (rose,is,a,rose), (is,a,rose,is), (a,rose,is,a), (rose,is,a,rose) }
Which can then be reduced, or maximally shingled in this particular instance to
{ (a,rose,is,a), (rose,is,a,rose), (is,a,rose,is) }.
Resemblance
For a given shingle size, the degree to which two documents A and B resemble each other can be expressed as the ratio of the magnitudes of their shinglings' intersection and union, or
r
(
A
,
B
)
=
|
S
(
A
)
∩
S
(
B
)
|
|
S
(
A
)
∪
S
(
B
)
|
{\displaystyle r(A,B)={{|S(A)\cap S(B)|} \over {|S(A)\cup S(B)|}}}
where |A| is the size of set A. The resemblance is a number in the range [0,1], where 1 indicates that two documents are identical. This definition is identical with the Jaccard coefficient describing similarity and diversity of sample sets.
See also
Bag-of-words model
Jaccard index
Concept mining
k-mer
MinHash
n-gram
Rabin fingerprint
Rolling hash
Vector space model
References
Broder; Glassman; Manasse; Zweig (1997). "Syntactic Clustering of the Web". SRC Technical Note #1997-015.{{cite web}}: CS1 maint: url-status (link)
Manber (1993). "Finding Similar Files in a Large File System" (PDF). Does not yet use the term "shingling".
Begin with the source’s own compact description: “W-shingling” is open-knowledge reference entry. The dossier treats that line as a proposition to test through W-shingling, Open-knowledge and entry, not as a finished interpretation.
Why this record matters
The phrase “open-knowledge reference entry” supplies a clear boundary for inquiry. It also exposes the unanswered questions: who defined that boundary, when it became stable and which sources sit outside it.
Named sources, stable identifiers and responsible institutions provide the strongest route from overview to verifiable evidence. The source revision retrieved here is dated May 17, 2026. The linked authority identifier is Q4060694. None of the 0 selected statements returned an explicit reference. The first chronological checks are 1997 and 1993.
The absence of detail may reflect summary conventions rather than a lack of surviving documentation. The lead is largely declarative, so disagreement and counter-evidence require a deliberate search beyond the opening account. Authority statements aid reconciliation but still require their own references, qualifiers and ranks to be checked.
How to read it
Use the entry as an orientation point, then follow its citations and revision history. Names, dates and institutional relationships should be checked against the original record.
- Subject orientation
- Search vocabulary
- Locating named sources
The closest primary source, responsible institution and strongest cited specialist reference.
Three-step research path
- Establish the record: confirm the title “W-shingling”, its source revision and the description used here.
- Expand the search: follow W-shingling primary sources, W-shingling archive and W-shingling research across catalogues and specialist indexes.
- Test the account: compare the strongest cited source with the responsible institution’s current record and note any disagreement.
Questions for further research
- Which source most directly establishes the central claim about “W-shingling”?
- Which institution is responsible for the underlying evidence?
- Which cited source is closest to the event, object or claim?
Search terms from this dossier
This entry incorporates text from “W-shingling” on English Wikipedia. Contributors are listed in the page history. Text is available under the Creative Commons Attribution-ShareAlike 4.0 License. Selected authority identifiers and statements are retrieved from Wikidata under CC0; their references and qualifiers remain part of the verification path.