Zipf and Zipfian distributions

The Zipf distribution, also known as the zeta distribution or discrete Pareto distribution, models the frequency of elements in ranked data where the frequency of an element is inversely proportional to its rank. The Zipfian distribution is a variant with finite support. Both distributions arise naturally in linguistics, population distribution, web traffic analysis, and many other domains exhibiting power law behavior.

Zipf's law, named after the linguist George Zipf, was first observed in the 1930s and 1940s as a pattern in the frequency of words in natural language texts. Zipf noted that the most frequent word in a text occurs approximately twice as often as the second most frequent word, three times as often as the third most frequent word, and so on. This observation led to the formulation of Zipf's law, which has since been found to apply to a wide range of phenomena in fields such as linguistics, economics, and information science.

The historical significance of Zipf's law lies in its ability to reveal underlying regularities in complex systems. It has inspired extensive research into the mathematical and statistical properties of power-law distributions and their applications in modeling real-world data.

The Zipf distribution has one parameter $s$ where $s>1$, which represents the exponent characterizing the distribution. The probability mass function (PMF) is given by:

$$P(X=k)=\frac{1}{k^s\zeta (s)}$$

where $\zeta (s)$ is the Riemann zeta function defined as:

$$\zeta (s)=\sum _{n=1}^{\mathrm{\infty }}\frac{1}{n^s}$$

The cumulative distribution function (CDF) can be expressed as:

$$F(k)=\frac{H_k^{(s)}}{\zeta (s)}$$

where $H_k^{(s)}$ is the $k$th generalized harmonic number of order $s$.

The domain of the Zipf distribution is $k\in \{1,2,3,\dots \}$. The parameter $s$ must satisfy $s>1$ to ensure that the distribution is normalizable.

The Zipfian distribution (with finite support) has two parameters: the exponent $s$ and the size of the population $N$. Its PMF is given by:

$$P(X=k)=\frac{1/k^s}{\sum _{i=1}^{N}1/i^s}\text{for }k\in \{1,2,\dots ,N\}$$

with $s>0$ and $N$ a strictly positive integer.

The Zipf and Zipfian distributions have numerous applications across diverse fields due to their ability to model naturally occurring power-law relationships:

• In linguistics, the frequency of words in natural language texts follows Zipf's law, with the most frequent word occurring approximately twice as often as the second most frequent word, three times as often as the third most frequent word, and so on.

• In urbanization studies, the sizes of cities within a country often follow a Zipfian distribution, where the largest city is about twice the size of the second largest, three times the size of the third largest, and so on.

• In information retrieval, the frequency of web page visits and search term usage demonstrates Zipfian behavior, making this distribution crucial for search engine optimization and web analytics.

• In bibliometrics, the distribution of citations among papers typically follows a Zipf-like pattern, with a small number of papers receiving a disproportionately large number of citations.

The Zipf and Zipfian distributions have several important statistical properties:

The infinite moments of the Zipf distribution highlight its heavy-tailed nature. For instance, the mean exists only when $s>2$, and the variance only when $s>3$. This behavior is characteristic of power-law distributions and is an important consideration when modeling real-world phenomena.

The Zipf and Zipfian distributions are connected to several other distributions:

• The Zipf distribution is a discrete analog of the continuous Pareto distribution. If $X$ follows a Pareto distribution with shape parameter $\alpha$, then $\lfloor X\rfloor$ approximately follows a Zipf distribution with parameter $s=\alpha +1$.

• The Zipf-Mandelbrot distribution generalizes the Zipf distribution by introducing an additional parameter $q$ that shifts the ranks: $P(X=k)\propto \frac{1}{(k+q{)}^s}$.

• When $s\to \mathrm{\infty }$, the Zipf distribution approaches a degenerate distribution concentrated at $k=1$.

The Zipf distribution is implemented by the ZipfDistribution class. It has one constructor that takes as its only argument the exponent, which must be greater than 1.

var zipf = new ZipfDistribution(2.5);

Dim zipf = new ZipfDistribution(2.5)

No code example is currently available or this language may not be supported.

let zipf = new ZipfDistribution(2.5)

The ZipfDistribution class has one specific property: Exponent, which returns the exponent parameter $s$.

The Zipfian distribution is implemented by the ZipfianDistribution class. It has one constructors that takes two arguments: the exponent and the size of the population. The size of the population must be a positive integer, and the exponent The exponent must be greater than 0. following constructs a Zipfian distribution with exponent 2.5 and a population size of 10:

var zipfian = new ZipfianDistribution(2.5, 10);

Dim zipfian = new ZipfianDistribution(2.5, 10)

No code example is currently available or this language may not be supported.

let zipfian = new ZipfianDistribution(2.5, 10)

The ZipfianDistribution class has two specific properties: Exponent, which returns the exponent parameter $s$, and Size, which returns the size of the population (and therefore the maximum rank $N$).

Both ZipfDistribution and ZipfianDistribution also provides efficient methods for generating random samples from the distributions:

var random = new Pcg32();
int sample = zipf.Sample(random);

Dim random = new Pcg32()
Dim sample = zipf.Sample(random)

No code example is currently available or this language may not be supported.

let random = new Pcg32();
let sample = zipf.Sample(random)

The above example uses the Pcg32 class to generate uniform random numbers, which are then used to follow the Zipf distribution.

For details of the properties and methods common to all discrete probability distribution classes, see the topic on Discrete Probability Distributions.

• Zipf's Law on Wikipedia

• Powers, David M. W. (1998). Applications and explanations of Zipf's law. In New methods in language processing and computational natural language learning.

• Newman, M. E. J. (2005). Power laws, Pareto distributions and Zipf's law. Contemporary Physics, 46(5), 323-351.

Other Resources