Full text of DOI:10.1007/s100510070125

Eur. Phys. J. B 17, 459–469 (2000)
THE EUROPEAN
PHYSICAL JOURNAL B
EDP Sciences
Societa` Italiana di Fisica
Springer-Verlag 2000
c
ꢀ
Geometrical entropies. The extended entropy
J.M. Zoidoa and F. Carren˜o
Departamento de Optica, Escuela Universitaria de Optica, Universidad Complutense de Madrid, C/ Arcos de Jal´on s/n,
28037 Madrid, Spain
Received 4 November 1999
Abstract. By taking into account a geometrical interpretation of the measurement process [1,2], we define
a set of measures of uncertainty. These measures will be called geometrical entropies. The amount of
information is defined by considering the metric structure in the probability space. Shannon-von Neumann
entropy is a particular element of this set. We show the incompatibility between this element and the
concept of variance as a measure of the statistical fluctuations. When the probability space is endowed
with the generalized statistical distance proposed in reference [3], we obtain the extended entropy. This
element, which belongs to the set of geometrical entropies, is fully compatible with the concept of variance.
Shannon-von Neumann entropy is recovered as an approximation of the extended entropy. The behavior
of both entropies is compared in the case of a particle in a square-well potential.
PACS. 03.65.Bz Foundations, theory of measurement, miscellaneous theories – 89.70.+c Information
science
1 Introduction
The uncertainty contained in p(x) is coincidental with
the amount of additional information which is required
in order to specify the value of x. Thus, variance (1) is
a measure of the amount of information gained when we
obtain the value xi in a measure.
Theories of probability and mathematical statistics
provide different measures for expressing the uncertainty.
The description of an observable as a random variable
makes it possible to use these measures for expressing the
uncertainty in quantum physics. All the results obtained
in this paper for quantum probability distributions can
be easily translated to classical probability distributions.
In the framework of information theory [8–11] Shannon
entropy (or information entropy),
Let us consider a random M-dimensional variable x,
x1, ..., xM being the discrete set of the M possible mutu-
ally exclusive outcomes when performing N random inde-
pendent experiments. The probability distribution p(x) =
(p1, ..., pM ) for the possible outcomes of measurements of
x obeys to a multinomial function [1,4,5]. The uncertainty
when determining pi is given by the variance
pi(1 − pi)
σi(pi) =
·
(1)
N
This quantity provides a measure of the statistical fluctu-
ations in a random experiment [6,7].
Let X denote an observable associated with the phys-
ical variable x, |χii (i = 1, ..., M) being the correspond-
ing set of eigenfunctions with eigenvalues xi. For a state
M
X
S(S)(p) = −
pi log pi,
(3)
i=1
P
M
described by a density matrix ρ = i=1 λi |ϕii hϕi| the
probabilities for the M possible outcomes are given by
provides a quantitative measure of the uncertainty associ-
ated with the probability distribution p(x). When eigen-
values λi of a certain density matrix ρ are replaced by
probabilities in expression (3) we obtain the von Neumann
entropy [12] S(S)(ρ) = −Tr ρ ln ρ = − i=1 λi ln λi. It
measures the amount of uncertainty contained within the
density operator [2,13].
Let X, Y be non-commuting observables representing
two physical variables x and y. It has been pointed out
that the standard formulation of the uncertainty princi-
ple, expressed in terms of variances, does not properly
express the quantum uncertainty principle [14,15]. In this
pi = hχi, ρχii .
(2)
P
These probabilities are the relevant (macroscopic) vari-
ables which describe our physical system. From a micro-
scopic point of view, the probabilities appear as the statis-
tical expectation values of the set of orthogonal projectors
ξi = |χii hχi| [2]. Observable ξi is associated with the num-
ber of occurrences of xi when performing N measurements
of x. Expression (1) provides the variance of ξi.
M
a
e-mail: fiopt11@emducms1.sis.ucm.es

460
The European Physical Journal B
situation the entropic relations
σi(0) = σi(1) = 0 (no statistical fluctuations) respectively.
Its inverse function is defined by
S(S)(x) + S(S)(y) ≥ Sxy
,
(4)

ꢂ
ꢃ
ꢃ
1
2


pi, if pi ∈ 0,

provide stronger re-formulations of the Heisenberg uncer-
tainty principle [15–19]. In equation (4) Sxy is a positive
constant.
pi(σi) ≡ σi−1(σi) =
,
(5)
ꢂ

1
2

+

p , if p ∈
, 1

i
i
This paper is organized as follows: in Section 2 we pro-
pose some properties of any measure of uncertainty. Infor-
mation geometry [1,2,20,21] directly relates distance and
uncertainty. Taking into account this relation, we define in
Section 3 a set of “geometrical uncertainties”. From this
set we obtain a family of “geometrical entropies”. It is
shown in Section 4 that Shannon (von Neumann) entropy
belongs to this family. We demonstrate the incompatibility
between this entropy and the concept of variance. We pro-
pose the extended entropy, which also belongs to the set
of geometrical entropies, as a measure of the uncertainty.
Some formal properties of the extended entropy are given
in Section 5. Section 6 is devoted to apply the previous
results to the case of a particle in a square well potential.
The paper finishes with some concluding remarks.
ꢀ
ꢁ
√
with pꢁi
=
1 ꢀ 1 − 4Nσi . From this, we can define
1
2





ꢂ
ꢃ
1
2


si(σi), if pi ∈ 0,

si(σi) =
(6)
(7)
ꢂ
ꢃ
1
2
+
i
s (σ ), if p ∈
, 1 ,
i
i
where
sꢁi (σi) = si(pꢁi ).
When measuring statistical fluctuations, it is expected si
and variance σi be compatible quantities. This compati-
bility suggests us the following intuitive properties:
2 Intuitive properties of the uncertainty
– s.4. For a given value of N, si(σi) is a monotonically
increasing function of σi,
A M-dimensional random variable x can be characterized
by its probabilistic scheme Ep = {Ei; pi; xi}, i = 1, ..., M.
In this scheme E1, ..., EM represent the random events
(states in a quantum system), quantities p1, ..., pM give
their probabilities, and x1, ..., xM are the values ascribed
to the random events [22]. Let S be the total uncertainty
contained in a probabilistic scheme. Our aim is to find
a function si = si(Ei) providing the contribution to S
due to the ith event . We begin proposing some intuitive
properties of function si.
and
– s.5. smi in ≡ si(0) = si(1) = 0 (No statistical fluctua-
tions), and smi ax ≡ si(1 ) (maximum statistical fluctu-
2
ations).
In the last property si is considered as a function over pi.
The larger the difference of pi from 1/2, the lesser
the statistical fluctuations of the event Ei. Decrease in
fluctuations should be coincidental when the cases pi(1)
=
Taking into account the independence among the
events, we can state that quantity si must be indepen-
dent of probabilities pj (i = j), thus
1/2 + ∆pi, and p(2) = 1/2 − ∆pi are considered. On the
i
other hand, it is expected that the decreasing rate of the
amount of uncertainty should be equal in both cases. In
this situation the first derivative of si satisfies
– s.1. si ≡ si(Ei) = si(pi). si ≡ si(Ei) = si(pi).
– s.6. s0i(pi) > 0, if pi ∈ [0, 21 ), s0i(pi) < 0, if pi ∈ (12 , 1],
and
Total uncertainty should be obtained by adding the con-
tributions si of all the M possible events, i.e.,
M
P
– s.2. S(p) = S(p1, · · · , pM ) =
si(pi).
– s.7. s0i(12 + ∆pi) = −si0(21 − ∆pi), for all ∆pi ∈ [0, 12 ],
i=1
with s0(pi) = ds(pi)/dpi. From s.5 to s.7, we obtain the
The scheme with the most uncertainty is the one with
equally likely outcomes, pi = 1/M for all i. Moreover, the
more alternative outcomes the random experiment has,
the larger its uncertainty. From these facts, we have
followiing property:
– s.8. si(1 + ∆pi) = si(1 − ∆pi), for all ∆pi ∈ [0, 1 ],
2
2
2
M
P
which allows us to write
ꢀ
ꢁ
1
M
– s.3. Function SM
=
si
is a monotonically in-
ꢂ
ꢃ
i=1
creasing function of M.
1
4N
si(σi) = s+i (σi), ∀σi ∈ 0,
.
(8)
The following properties will be related with variance (1).
The extreme values of this function are σimax = σi(1/2) = Properties s.4 to s.8 are required in order to obtain the ex-
1/4N (maximum statistical fluctuations) and σimin
pected compatibility between variance (1) and function si.
=

J.M. Zoido and F. Carren˜o: Geometrical entropies. The extended entropy
461
3 Geometrical uncertainties and geometrical
is formally stated by the following inequality:
entropies
Vii ≥ gii,
(10)
Let P = {p(x)} be the set of multinomial probability dis-
tributions p(x) over the random variable x. These dis-
tributions are parameterized by the M-dimensional real
vector parameter p = (p1, ..., pM ). P can be treated as
a statistical model and it is known as the multinomial
statistical model [1]. When p(x) is sufficiently smooth in
p1, ..., pM , the statistical model P forms a M-dimensional
manifold embedded in the set of all the possible prob-
ability distributions, where p1, ..., pM play the role of a
coordinate system [1,20,21]. Mutual relations of distribu-
tions are then understandable as geometrical properties
of the manifold. The question is, what is the natural ge-
ometric structure to be introduced in a manifold consist-
ing of a statistical model? The answer to this problem is
given by information geometry [1,20,21]. This field studies
the geometrical structures of the manifolds of probability
distributions. An introduction to information geometry is
given in reference [21]. Reference [20] reviews the geome-
try of the manifolds of statistical models. In reference [1]
the information geometry is throughly analyzed.
As we will see in this section, information geome-
try relates in a natural way geometry and uncertainty
(or information). The results provided in this work are
based in this relation. The same idea underlies in the
work by Balian et al. where a geometrical theory of sta-
tistical physics is proposed in terms of the Riemannian
geometry [2].
When the inner product of two vectors belonging to
the tangent space is defined, the manifold P is called a
Riemannian space. In this case, the natural geometrical
structure to be introduced in the manifold of probability
distributions is given by the positive definite Riemannian
metric [1–3,6,20,21,23–26]
where Vii is the variance (uncertainty) associated with pi.
The lower bound in this equation is a particular form of
the Cramer-Rao inequality. Expression (10) relates uncer-
tainty and distance and it has been extended to quantum
mechanics [23,29–31].
The above exposed results show how information ge-
ometry provides a natural relation between uncertainty
and distance. This relation suggests us that functions si
and coefficients gii should be related quantities, i.e.,
si ≡ si(gii) = si(pi).
(11)
The second equality in (11) is a consequence of property
s.1 and it allows us to state the functional dependence
gii = gii(pi).
From relation (11) and property s.2 we obtain
M
M
X
X
S =
si =
si(gii).
(12)
i=1
i=1
This expression points out, as expected from the results
provided by information geometry, how information is ob-
tained from the metric structure defined in the probability
space. With definition (11) uncertainty acquires a clear ge-
ometrical interpretation. This result is in good agreement
with similar ideas proposed by other authors [2,32–35].
Given a metric structure {gij}, each possible distri-
bution of functions s = (si, ..., sM ) generates a different
quantity S in (12). Thus we have the set of measures of
uncertainty
(
)
M
X
Ug = Sg(s) =
si(gii) ∀s
.
(13)
i=1
M
M
X X
ds2 =
gij dpidpj.
(9)
On the other hand, by considering the possible distribu-
tions g = (g11, ..., gMM ) which it is possible to define in a
probability space, we obtain the set of sets
i=1 j=1
It has been demonstrated [1,20,21] that the inner prod-
uct is naturally defined by the covariance of two random
variables. Especially, when the inner product of two vec-
tors belonging to the natural basis of the tangent space
is considered, matrix {gij} in (9) is an important quan-
tity known as the Fisher information [1,20]. Term gii is
a measure of the amount of information due to the event
Ei [27,28]. It was Rao [25] who first proposed the Rieman-
nian structure by using the Fisher information matrix. It
is well known that the only invariant Riemannian metric
is given by the Fisher information [1,20,21]. This is called
the information metric.
When information metric is considered, the statistical
meaning of distance (9) is elucidated by the Cramer-Rao
theorem [1]. According to this theorem, the lowest vari-
ance in an estimation of pi when the remaining probabil-
ities pj (i = j) are unknown is given by gii, {gij} being
the inverse of the Fisher information matrix. This result
ꢄ
U = Ug ∀g/gii = gii(pi) .
(14)
We will refer to U as set of “geometrical uncertainties”.
The conceptual validity of each measure of uncertainty
S ⊂ Ug depends on the behavior, concerning to properties
s.1 to s.8, of function si.
Concavity is an expected property of function si [36].
Thus, given a metric tensor, we can take the concave
function
siln(gii) = −gii ln gii.
(15)
This function generates the following element in Ug:
M
M
X
X
Sg(sln) =
slin(gii) = −
gii ln gii,
(16)
i=1
i=1

462
The European Physical Journal B
with sln = (s1ln, ..., slMn ). When all the possible Riemannian
metrics are considered, expression (16) generates in (14)
the family of measures of uncertainty
ꢄ
Sln = Sg(sln) ∀g/gii = gii(pi) ⊂ U.
(17)
si(S)(σi)
If we consider the functional dependence upon gii, quanti-
ties (16) exhibit the same formal behavior than the usual
definition of entropy (3) when this quantity is considered
as a function on pi. By taking into account this formal re-
lation, we will refer to the family (17) as “geometrical en-
tropies”. Geometrical entropies will allow us to introduce
in a natural way the metric properties of the probability
space in the entropic uncertainty relations (4).
si+(S)(σi)
si-(S)(σi)
Νσi
Fig. 1. Function s(iS)(σi) plotted against Nσi. As it is shown in
equation (6), the values of this function are given by s−i (S)(σi)
when pi ∈ [0, 1/2], and by s+i (S)(σi) when pi ∈ [1/2, 1]. In
order to compute the amount of information, we have token
the natural logarithm. Thus, the units of s(iS)(σi) are given in
Nats.
4 Shannon-von Neumann entropy
and extended entropy
The metric properties of a probability space are due to
the statistical fluctuations in a finite sequence of measure-
ments [6,24–26,30–32]. In this way, the components of the
metric tensor are completely given by the uncertainties
(variances). After drawing N samples from a probability
distribution, one can estimate the probabilities as the ob-
served frequencies. As it has been pointed out in Section 1,
the probability distribution for the frequencies is given by
a multinomial distribution with variances given by (1).
When N is large enough, the local limit theorem provides
an asymptotic approximation to the multinomial distri-
bution. When this approximation is considered, the prob-
ability for the frequencies is proportional to a Gaussian
distribution with variances [5,6,23]
the result provided in reference [2], in which it is shown
how the natural metric is generated by the Shannon (von
Neumann) entropy.
We now analyze the behavior of Shannon entropy re-
garding properties proposed in Section 2. Quantities si(S)
and S(S) satisfy properties s.1 to s.3 [8]. The maxi-
mum of function s(iS)(σi), as defined in (6), occurs at
σi = (e − 1)/N e2, below 1/4N for all N ≥ 1. From this,
one conclude that this function does not satisfy s.4. By
taking into account definition (7) we have (see Fig. 1)
ꢂ
ꢃ
1
4N
s−i (S)(σi) ≥ s+(S)(σi), ∀ σi ∈ 0,
.
(21)
i
pi
σi(n)
=
·
(18)
N
This inequality reveals an unexpected result for s(iS): it is
These uncertainties generate the “natural metric” (see, for a bivaluated function on σi.
instance, Ref. [6]),
The second part of property s.5 is not satisfied by
(
function s(iS)(pi): this quantity reaches its maximum value
0
if i = j
at pi = 1/e, with smax(S) = s(iS)(1/e) = 1/e (see Fig. 2).
gnij
=
.
(19)
i
pi if i = j
According to this, the maximum amount of uncertainty at-
tached to the ith event is obtained when its corresponding
probability is 1/e. If we think in the probabilistic scheme
defined by a coin or dice, this result is not an intuitive
This tensor usually provides the metric in the manifold P
of multinomial distributions [1,2,6,23,25,28,37].
By taking gii = gnii = pi in (17), the natural metric
one. It can be verified that function s(iS) does not satisfy
properties s.6 to s.8. Inequality (21) points out how the
information (von Neumann) entropy does not verify con-
dition (8). In view of these results, we can conclude that
the standard definition of entropy is not compatible with
variance (1) as a measure of statistical fluctuations.
As it has been pointed out, the natural metric is gen-
erated by variances (18), which are recovered as an ap-
generates the element Sg (sln) in the family of geometrical
n
entropies. In this case contribution (15) due to the ith
event will be given by
s(iS)(pi) ≡ sln(gii) = −pi ln pi.
(20)
i
n
From this and expression (12) we can see that Shannon
entropy (3) is the member of the geometrical entropies
proximation of variances σi (σ(n) = lim σi). Thus, when
generated by the natural metric, i.e., Sg = S(S). This
i
pi→0
n
result can be applied to the von Neumann entropy when the natural metric is introduced in the probability space,
eigenvalues λi are replaced by probabilities pi in the previ- the metric properties are due to approximation (18) and
ous expressions. In this way, the Shannon (von Neumann) not to the true variances (1). The above mentioned mutual
entropy is in a normal manner associated with the nat- incompatibility between Shannon (von Neumann) entropy
ural metric (19). This conclusion is in agreement with and variance is a consequence of this fact.

J.M. Zoido and F. Carren˜o: Geometrical entropies. The extended entropy
463
2.5
1/e 1/2
0.4
0.3
0.2
0.1
0.0
2.0
1.5
SM
1.0
si(pi)
(S)
Shannon (von Neumann) entropy (SM
)
0.5
0.0
(g)
Extended entropy (SM
)
2
Dimension of the probability space (M)
3
4
5
6
7
8
9
10 11
Fig. 3. Function SM (in Nats) providing the maximum uncer-
tainty in a M-dimensional probability space. The results pre-
dicted by the Shannon (von Neumann) entropy, SM(S), are com-
pared with those obtained by using the extended entropy, SM(g)
0.0
0.2
0.4
0.6
0.8
1.0
Probability (pi)
.
Fig. 2. Quantity si(pi) in the case of the Shannon (von Neu-
mann) entropy (dashed line), and in the case of the extended
entropy (solid line). The units of si are given in Nats.
The measure of uncertainty (24) satisfies all the properties
listed in Section 2. Thus, quantity S(g) is fully compatible
with variance (1).
In reference [3] an asymptotic approximation to the
multinomial distribution is proposed. This approximation
maintains the usual definition of variances (1) and it en-
dows to the probability space with the “generalized sta-
tistical distance”. The corresponding contravariant metric
tensor is
By introducing expression (25) in (12) quantity S(g)
P
M
can be rewritten as S(g) = − i=1 Nσi ln Nσi. This result
points out how this measure of uncertainty depends on the
knowledge of variances while the specification of Shannon
(von Neumann) entropy only requires the knowledge of
probabilities. We will call to quantity S(g) “extended en-
tropy”. Using expression (1) when computing the diagonal
terms in (22), one finds ggii(σi) = Nσi. In this way, for a
ꢅ
0
if i = j
ggij
=
.
(22)
pi(1 − pi) if i = j
fixed number of trials N, quantity ggii provides us with
a direct measure of the variance, as it was expected by
considering inequality (10). This fact points out how the
extended entropy is fully compatible with the geometrical
interpretation of the measurement process. In a similar
way, by introducing pi(σi), as defined in (5), in expres-
sion (19) we obtain
With this distance the metric properties of the probability
space are due to the true variances (1).
If we take gii = ggii in (15) the contribution to the total
uncertainty due to the event Ei will be given by
s(ig)(pi) ≡ sln(gii) = −pi(1 − pi) ln[pi(1 − pi)].
(23)
i
g
In this case, the element Sg in (16) generated by the gen-





ꢂ
ꢃ
ꢃ
ꢀ
ꢀ
ꢁ
√
eralized statistical distance can be written as
1
2
1
2




1 − 1 − 4Nσi , if pi ∈ 0,
M
gnii(σi) =
.
(26)
X
ꢂ
S(g) ≡ Sg (sln) =
s(ig)(pi)
ꢁ
√
1
2
1
2
g
1 + 1 − 4Nσi , if pi ∈
, 1
i=1
M
X
= −
pi(1 − pi) ln pi(1 − pi).
(24)
Thus the diagonal terms of the natural metric does not
provide us with a direct measure of the variance. This
result shows how standard entropy exhibits serious incon-
sistencies with the geometrical interpretation of the mea-
surement process.
i=1
Quantities (23, 24) satisfy properties s.1 and s.2. Prop-
erty s.3 is also satisfied: the maximum of S(g) is reached
for the most random scheme (pi = 1/M ) and quantity
The maximum value of the Shannon entropy is SM(S)
=
,
P
SM(g)
=
i=1 s(ig)(1/M ) = [(M − 1)/M ] ln[M2/(M − 1)]
M
ln M. By comparing this value with the maximum, SM(g)
is a monotonically increasing function of M (see Fig. 3).
From expressions (1, 23), we obtain
obtained in the case of the extended entropy we have in-
equality SM(g) ≤ SM(S) for all integer M. Thus, in any prob-
ability space, with M > 2, endowed with the extended
entropy, the maximum uncertainty is lesser (see Fig. 3).
s(ig)(σi) = −Nσi ln Nσi,
(25)
which is a monotonically increasing function on σi. In this
In the limit pi → 0 quantities (18) can be considered
way, s(ig) satisfies property s.4. It is immediately seen that as approximations to the variances (1). From this, it be-
this quantity verifies properties s.5 to s.8 (see Fig. 2). comes obvious that the natural metric will be recovered as

464
The European Physical Journal B
an approximation to the generalized statistical distance:
and we have
gnii = lim ggii.
(27)
– p.3. Concavity. S(g)(ρ) ≥ aS(g)(ρa) + (1 − a)S(g)(ρb).
pi→0
Concavity formalizes a well known fact: uncertainty in an
By considering this result, we finally obtain
mixture always increases.
S(S) = lim S(g)
.
(28)
We now consider ρ ∈ H and 0 ∈ H00, H and H00 being
Hilbert spaces. Density matrix ρ0 = ρ ⊕ 0, which belongs
pi→0
Condition pi → 0 is satisfied for the neighboring states to the space H0 = H ⊕ H00, satisfies
to that with equally outcomes for large enough number
– p.4. Expansibility. S(g)(ρ0) = S(g)(ρ).
M of possible outcomes. If we take into account that the
larger M, the larger the uncertainty, expression (28) tells
us that Shannon entropy is recovered as an approximation
to the extended entropy when large enough values of the
uncertainty are considered.
By replacing the eigenvalues λi by probabilities pi in
expression (24), we have that the extended entropy asso-
ciated with the density operator ρ is
Extended entropy remains invariant when we add events
with vanishing probabilities.
We assume that the time evolution of probabilities in
a physical system obeys to the master equation [36]
M
X
dpi
dt
=
(wij pj − wjipi) ,
(31)
j=1
S(g)(ρ) = −Tr [ρ(1 − ρ) ln ρ(1 − ρ)]
M
wij being the transition probability, per unit time, from
the jth state to the ith one. If detailed balance holds [38],
i.e., wij = wji, the time evolution of the extended entropy
can be written as
X
= −
λi(1 − λi) ln λi(1 − λi).
(29)
i=1
M
M
dS(g)
dt
5 Some formal properties of the extended
entropy
X X
=−
wji(pj − pi)(1 − 2pi) [ln pi(1 − pi)+1].
i=1 j=1
(32)
In this section we will demonstrate some formal properties
of the extended entropy.
If we interchange subindex i and j in this equation, by
summing both results, we obtain
– p.1. Positivity. S(g)(ρ) ≥ 0, and S(g)(ρ) = 0 if, and
only if, ρ is a pure state (λi = 1 for some i).
Thus, when the system is described by a pure state there
exist measurements (those relative to a basis containing
|ϕi, if ρ = |ϕi hϕ|), whose result can be predicted with
absolute certainty.
M
M
dS(g)
dt
1
2
X X
=
wij (pi − pj)f(pi, pj),
(33)
i=1 j=1
– p.2. Equiprobability. S(g)(ρ) ≤ SM(g)(ρ) and the maxi-
mum value is achieved when λi = 1/M for all i. This
upper limit is a concave function of M and diverges as
M → ∞ (see Fig. 3).
with f(pi, pj)
=
(1 − 2pi) [ln pi(1 − pi) + 1] − (1 −
2pj) [ln pj(1 − pj) + 1]. Function f satisfies f(pi, pj) ≥ 0 if
pi ≥ pj, and f(pi, pj) < 0 if pi < pj for all pi, pj such that
0 ≤ pi, pj ≤ 1. From this property and expression (33),
we can state
From this property we obtain that the probabilistic
scheme with more uncertainty is that with equally likely
eigenvectors.
– p.5. Irreversibility (H-theorem). dS(g)/dt ≥ 0.
This property points out how the irreversibility associated
with the second law of thermodynamics remains valid for
the extended entropy.
From expression (20) we obtain the following relation
between the standard and the extended entropy:
Function s(ig)(xi) = −xi(1−xi) ln xi(1−xi) is concave
in the interval [0, 1]. Let ρ = aρa + (1 − a)ρb (a ∈ [0, 1])
be the quantum state obtained by fitting together two
mixed states ρa and ρb. From concavity of s(g) extended
i
entropy (29) satisfies [36]
– p.6. Relation with the standard entropy. S(g)(ρ) =
S(S)(ρ) + Sσ(S)(ρ) − Sc(S)(ρ), with
M
X
S(g)(ρ) =
s(ig)(hϕi |ρ| ϕii)
i=1
M
X
M
M
Sσ(S)(ρ) =
λis(S)(1 − λi),
(34)
(35)
X
X
s(ig)(hϕi |ρa| ϕii) + (1 − a)
s(ig)(hϕi |ρb| ϕii)
i
≥ a
≥ a
i=1
i=1
i=1
M
M
ꢆ
ꢆ
ꢆ
ꢆ
ꢆ
ꢆ
ꢆ
ꢆ
D
E
D
E
and
X
X
ϕ s(g)(ρ ) ϕ + (1 − a)
ϕ s(g)(ρ ) ϕ
ꢆ
ꢆ
ꢆ
ꢆ
i
a
i
i
b
i
i
i
M
X
i=1
i=1
Sc(S)(ρ) =
λis(S)(λi).
= aS(g)(ρa) + (1 − a)S(g)(ρb),
i
(30)
i=1

J.M. Zoido and F. Carren˜o: Geometrical entropies. The extended entropy
465
Let ρa and ρb be density operators, with eigenvalues µi
(i = 1, ..., L) and ηj (j = 1, ..., M) respectively, repre-
senting two uncorrelated systems a and b. The compos-
ite system ab obtained by fitting together both individual
systems will be described by the density matrix ρab with
eigenvalues
3.0
2.5
2.0
1.5
1.0
0.5
3.5
3.0
2.5
2.0
1.5
1.0
0.5
L=3 and M=5
L=5 and M=10
λk = µiηj (1 ≤ i ≤ L, 1 ≤ j ≤ M, 1 ≤ k ≤ L × M).
0.0
0.5
1.0
1.5
2.0
2.5
0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5
S(g)(ρa)+S(g)(ρb)
S(g)(ρa)+S(g)(ρb)
(36)
Fig. 4. Extended entropy for a composite system, ρa⊗ρb, plot-
ted against the sum of the extended entropies of the individual
systems, ρa and ρb. In each graph, L and M are the dimensions
of the Hilbert spaces associated with the density matrices ρa
and ρb respectively. The results shown in this figure have been
obtained by using two sets of 1000 pseudo-random distribu-
tions of eigenvalues, µ1, ..., µL for ρa, and η1, ..., ηM for ρb.
The units in both axes are given in Nats.
In this situation standard entropy is an additive quantity,
i.e. information about the total system equals the sum
of the information about its constituents. When eigenval-
ues (36) are introduced in definition (29), we obtain the
extended entropy of the composite system,
S(g)(ρab) = S(g)(ρa ⊗ ρb)
L
M
X X
= −
µiηj(1 − µiηj) ln µiηj(1 − µiηj).
i=1 j=1
gain some additional information from knowledge of vari-
ances. From expression (1) the second term in the right-
hand side of property p.6 can be rewritten as
(37)
If we take into account the additivity of von Neumann
entropy, by applying property p.6 to (37), we obtain
ꢇ
ꢈ
M
X
Nσi
pi
Sσ(S)(p) =
pis(iS)
.
(41)
– p.7. Pseudo-additivity. S(g)(ρa ⊗ ρb) = S(g)(ρa) +
i=1
S(g)(ρb) + Sσ(ρa ⊗ ρb) − Sc(ρa ⊗ ρb), with
The dependence on σi suggests us that quantity
s(iS)(Nσi/pi) is the additional amount of information
gained from the knowledge of the ith variance. Thus, ex-
pression (41) provides the average amount of information
gained by the observer due to the knowledge of the M
Sσ,c(ρa ⊗ ρb) = Sσ(S,c)(ρa ⊗ ρb) − Sσ(S,c)(ρa) − Sσ(S,c)(ρb),
(38)
L
M
variances σi. Quantity S(S)(p) + Sσ(S)(p) is the amount
of information due to the knowledge of probabilities and
variances.
X X
Sσ(S)(ρa ⊗ ρb) = −
µiηj(1 − µiηj) ln(1 − µiηj),
i=1 j=1
(39)
(40)
As it is pointed out by equation (1), variances and
probabilities are correlated variables and knowledge of pi
tends to reduce uncertainty in σi [43]. In this way, due
to this correlation, a certain amount of information con-
and
L
M
X X
tained in S(S)(p) + Sσ(S)(p) is not removed and it should
Sc(S)(ρa ⊗ ρb) = −
µ2i ηj2 ln µiηj.
be subtracted of this quantity. Term Sc(S)(p) in p.6 con-
tributes to the total uncertainty as a negative quantity
and it can be associated with the amount of uncertainty
contained in the correlation between variances and proba-
bilities. Thus, in the context of information theory, prop-
erty p.6 allows us to interpret the extended entropy as the
amount of uncertainty removed in a probabilistic scheme
due to the knowledge of probabilities and variances minus
that amount of uncertainty contained in the correlation
between both variables. The amount of information associ-
ated with the correlation between variances and probabil-
ities has been pointed out by different authors [13,43,44].
We now analyze the meaning of property p.7. From
relation (36), property p.6 allows us to write the extended
entropy of the composite system ρa ⊗ ρb as
i=1 j=1
This property clearly points out how generalized entropy
is a non-extensive quantity. S(g)(ρa ⊗ ρb) can becomes
larger or smaller than S(g)(ρa) + S(g)(ρb) in a non triv-
ial way. This behavior is checked in Figure 4. In this fig-
ure S(g)(ρa ⊗ ρb) is plotted against S(g)(ρa) + S(g)(ρb) for
two pseudo-random sets of 1000 distributions µ1, ..., µL,
and η1, ..., ηM when different values of L and M are
considered.
Properties p.1 to p.5, and p.7 are those of a general-
ized entropy. These kind of entropies have been introduced
in order to extend the Boltzmann-Gibbs thermodynamics
by generalizing the concept of entropy to non-extensive
physics [39–42].
We now analyze property p.6 from an information-
theoretical point of view. Quantity S(S)(p) is the amount
of information gained when we know the probabilities in a
random experiment. In an analogous way, it is expected to
S(g)(ρa ⊗ ρb) = S(S)(ρa ⊗ ρb)
+ Sσ(S)(ρa ⊗ ρb) − Sc(S)(ρa ⊗ ρb), (42)

466
The European Physical Journal B
with
Applying definition (35) to the individual systems ρa,
and ρb one has
L
M
X X
S(S)(ρa ⊗ ρb) = −
µiηj ln µiηj
L
M
X X
Sc(S)(ρa) + Sc(S)(ρb) = −
µ2i ηj ln µi
µiηj2 ln ηj.
i=1 j=1
i=1 j=1
= S(S)(ρa) + S(S)(ρb),
(43)
L
M
X X
−
(47)
(48)
and Sσ(S,c)(ρa ⊗ ρb) given by (39, 40). The additivity of von
Neumann entropy (43) is a consequence of the trivial re-
lation (36). In order to specify the extended entropy (42)
we need to know probabilities µiηj and their correspond-
ing variances σij. In contrast to the case of probabilities,
variances of the composite system, σij , are related with
those of the individual systems, σi and σj, in a non-trivial
way. From this fact, it is expected the extended entropy
to be a non-additive quantity.
i=1 j=1
From this result and expression (40) we arrive at
Sc(S)(ρa ⊗ ρb) ≤ Sc(S)(ρa) + Sc(S)(ρb),
i.e., function (35) is also a sub-additive quantity.
Relations (46, 48) point out how the term Sσ(S) − Sc(S)
in (42) will not be, in general, an additive quantity. This
non-additivity is the responsible of property p.7. From the
previous analysis, it can be concluded that property p.7
is a natural consequence of the contribution of variances
and correlations to the definition of the extended entropy.
We now discuss another interpretation of the extended
entropy. In a random experiment it is expected that the
non-occurrence of an event will provide us with some ad-
ditional information about the physical system. The prob-
ability of non-occurrence for the ith event is given by
The information gained due to the knowledge of vari-
ances and correlations in a composite system is given by
quantity Sσ(S)(ρ ⊗ ρb) − Sc(S)(ρa ⊗ ρb). From the previous
arguments, the anon-additivity is an expected property of
this quantity. Now we will formalize this fact. From (34)
and property Tr ρa = Tr ρb = 1 we obtain
L
M
X X
Sσ(S)(ρa) + Sσ(S)(ρb) = −
µiηj(1 − µi) ln(1 − µi)
µiηj(1 − ηj) ln(1 − ηj).
i=1 j=1
L
M
qi = 1 − pi.
(49)
X X
−
In the context of the information theory the total uncer-
tainty removed in a probabilistic scheme due to the non-
occurrence of its events will be given by the corresponding
Shannon entropy
i=1 j=1
(44)
If we take into account the trivial inequalities 1 − µiηj ≥
1 − µi, and 1 − µiηj ≥ 1 − ηj, the last expression, and
definition (39) it can be shown that the following relation
holds:
M
X
Sn(S)(p) = −
= −
(1 − pi) ln(1 − pi)
qi ln qi = S(S)(q).
i=1
M
X
− Sσ(S)(ρa ⊗ ρb) + Sσ(S)(ρa) + Sσ(S)(ρb) ≥
(50)
L
M
i=1
X X
µiηj(µi + ηj − µiηj − 1) ln(1 − µiηj). (45)
The contribution to this quantity associated with the ith
event is
i=1 j=1
When restricted to the region x, y ≥ 0, the maximum of
function h(x, y) = x + y − xy − 1 is achieved at the point
(x, y) = (1, 1), with h(1, 1) = 0. From this fact one easily
verifies that condition µi + ηj − µiηj − 1 ≤ 0 is satisfied
in (45) and we have
s(iS)(qi) = −qi ln qi = −(1 − pi) ln(1 − pi).
(51)
When Shannon entropy is considered, expressions (49, 51)
allow us to rewrite definition (34) in the form
M
X
Sσ(S)(ρa ⊗ ρb) ≤ Sσ(S)(ρa) + Sσ(S)(ρb).
(46)
Sσ(S)(p) =
pis(S)(qi).
(52)
i
i=1
This inequality points out how Sσ(S)(ρa ⊗ ρb) in (42) is
a sub-additive quantity, i.e., the amount of information
associated with the knowledge of variances of the individ-
ual constituents is greater than the amount of information
provided by the knowledge of variances of the composite
system. This result is an intuitive one: the knowledge of
the individual variances allows us to obtain the variances
of the composite system, however, it is not possible to
recover variances σi and σj from variances σij .
In this way, the amount of information Sσ(S) associated
with the knowledge of variances is coincidental with the
average amount of information gained from the knowledge
of probabilities of non-occurrence.
By using expressions (49, 51), the extended en-
tropy (24) can be expressed as
S(g)(p) = Sy(g)(p) + Sn(g)(p),
(53)

J.M. Zoido and F. Carren˜o: Geometrical entropies. The extended entropy
467
with
cally conjugate variables, r and k, in terms of the cor-
responding information theory entropies [19],
M
X
Sy(g)(p) =
qis(S)(pi),
(54)
Z
i
Sr(S) = hln ρr(r)i = −
Sk(S) = hln ρk(k)i = −
dmrρr(r) ln ρr(r),
dmkρk(k) ln ρk(k).
(58)
(59)
i=1
Z
and Sn(g)(p) ≡ Sσ(S)(p). Quantity Sy(g) is the average of
the contributions si(pi) when weighted by probabilities
of non-occurrence. We can consider this quantity as the
amount of uncertainty removed in a random experiment
due to the knowledge of the probabilities associated with
the occurrence of the different outcomes. In a similar way,
The position-space entropy, Sr(S), measures the uncer-
tainty in the localization of a particle in space. In the
same way, momentum-space entropy, Sk(S), provides the
quantity Sn(g)(p) can be associated with the amount of
information gained due to the knowledge of probabilities
of non-occurrence in a given probabilistic scheme. Thus,
the information provided by the extended entropy takes
into account the amount of information due to the prob-
abilities of occurrence and that obtained from the knowl-
edge of probabilities of non-occurrence. From an intuitive
point of view, this is an expected result: if probabilities
of non-occurrence provide information about the system,
it seems a natural fact to consider them when computing
the total information removed in a random experiment. In
uncertainty in predicting the momentum of the parti-
2
2
cle. Functions ρr(r) = |ψr(r)| and ρk(k) = |ψk(k)| are
probability densities in m−dimensional position and mo-
mentum spaces, ψr(r) and ψk(k) being the corresponding
wave functions. The total information entropy is given by
S(S) = Sr(S) + Sk(S)
.
Expressions (58, 59) provide the discrete von Neumann
entropies when continuous random variables, r and k, are
considered. In a similar way, by applying definition (29),
quantities
Sr(g) = − h(1 − ρr) ln[ρr(1 − ρr)]i
this sense, quantities Sy(g) and Sn(g) provide complemen-
tary informations. This behavior is a consequence of rela-
tion (49): a variation δpi implies a complementary change
δqi = −δpi.
Z
= − dmrρr(1 − ρr) ln[ρr(1 − ρr)],
(60)
When two quantum uncorrelated systems, ρa and ρb,
are considered, definitions (52, 54) allow us to state the
following inequalities:
and
Sk(g) = − h(1 − ρk) ln[ρk(1 − ρk)]i
Z
= − dmkρk(1 − ρk) ln[ρk(1 − ρk)],
(61)
Sy(g)(ρa ⊗ ρb) ≥ Sy(g)(ρa) + Sy(g)(ρb),
(55)
and
are the position and momentum extended entropies. The
total extended entropy is S(g) = Sr(g) + Sk(g)
.
Sn(g)(ρa ⊗ ρb) ≤ Sn(g)(ρa) + Sn(g)(ρb).
(56)
Let us consider a particle in a one-dimensional (m = 1)
box with perfectly rigid and impenetrable walls located at
points x = ꢀL/2. When pure energy states are considered,
this physical situation is described by the probability den-
sities for position x and momentum k
These relations point out how terms Sy(g) and Sn(g) are
super-additive and sub-additive quantities respectively.
This result formally expresses the complementary behav-
ior of both quantities.
Expression (53) provides an important interpretation
for the extended entropy: it is the total amount of infor-
mation gained by an observer in a random experiment by
considering the uncertainty removed due to the knowledge
of probabilities of occurrence and the information gained
from the knowledge of probabilities of non-occurrence.



2
L
2
nπx
L



cos2
sin2
, if n odd
ρx(x, n, L) =
(62)

nπx
L
, if n reven,
L
and





cos2(kL/2)

2

4n Lπ
2 , if n odd


(n2π2 − k2L2)
6 Entropies for a particle in a square well
potential
ρk(k, n, L) =
(63)
sin2(kL/2)
2
4n Lπ
2 , if n even.
(n2π2 − k2L2)
A well known entropic uncertainty relation, which belongs
to the set of inequalities (4), is [19]
By introducing these expressions in definitions (58–61)
we have computed the position, Sx(S,g)(n, L), and mo-
mentum, Sk(S,g)(n, L), entropies. The total entropies are
Sr(S) + Sk(S) ≥ m(1 + ln π).
(57)
This inequality expresses the restrictions imposed by S(S,g)(n, L) = Sx(S,g)(L)+Sk(S,g)(n, L). These entropies are
quantum theory on probability distributions of canoni- shown in Figure 5 for L = 4. The results obtained when

468
The European Physical Journal B
have demonstrated that Shannon (von Neumann) entropy
is the member of Sln obtained when the probability space
is endowed with the natural metric. This entropy does
not satisfy properties s.4 to s.8 and it becomes incompat-
ible with variances (1). This inconsistency arises from the
fact that natural metric is obtained from an asymptotic
approximation with variances given by (18), which differ
from the true variances (1). As a consequence of this result
we question the validity of the Shannon (von Neumann)
entropy as a suitable measure of uncertainty in a random
experiment.
In this situation we have used the generalized sta-
tistical distance (22). When the probability space is en-
dowed with this distance, the metric properties are given
by the true variances (1). The element of Sln generated
by the generalized statistical distance is the extended en-
tropy S(g). We have demonstrated that this measure of
uncertainty satisfies properties s.1 to s.8. Thus, when
measuring statistical fluctuations, extended entropy is
fully compatible with variances (1). Extended entropy re-
produces, in the pi → 0 limit, the standard Shannon en-
tropy.
Fig. 5. Total (position plus momentum) von Neumann and
extended entropies for a particle in a one-dimensional box com-
puted as functions on the quantum number n. The size of the
box is L = 4 arb. units. The dashed line shows the lower limit
imposed by the entropic uncertainty relation (4). The units of
entropy are Nats.
different values of L considered are qualitatively equal to
those provided in this figure. Extended and von Neumann
entropy satisfy inequality
In the context of the information theory, extended en-
tropy can be considered as the amount of information
gained in a probabilistic scheme due to the knowledge of
probabilities and variances minus the amount of uncer-
tainty contained in the correlation between both magni-
tudes. In this way, extended entropy is associated with
a second order statistics. On the other hand, extended
entropy can be considered as the total amount of infor-
S(g)(n, L) < S(S)(n, L).
(64)
It is shown in Figure 5 that total extended entropy can be
lesser than the lower bound in the right-hand side of the
entropic uncertainty relation (57). By replacing extended
entropy by von Neumann entropy, inequality (64) allows
us to obtain a stronger version of the entropic uncertainty
relation (57).
mation, Sy(g) +Sn(g), obtained by summing the information
gained due to the knowledge of probabilities of occurrence
and that provided by the knowledge of probabilities of
non-occurrence. Quantities Sy(g) and Sn(g) provide comple-
7 Conclusions
mentary information. This result can be formally stated
We have described some general properties of a suit- by considering that both functions are super-additive and
able measure of uncertainty, S, in a random experiment. sub-additive quantities respectively.
Some of these properties, s.4 to s.8, are stated by con-
sidering the expected compatibility between quantity S
and the concept of variance when measuring statistical
fluctuations.
Extended entropy satisfies the most important prop-
erties of Shannon (von Neumann) entropy, except that of
additivity, and all the properties of a generalized entropy,
including pseudo-additivity [39–42] and the H-theorem.
Taking into account the above exposed considerations,
non-additivity becomes an expected property of Sg.
The case of a particle in a square well potential has
been analyzed. From this analysis, it is shown that von
Neumann and extended entropies exhibit a similar qual-
itative behavior. The amount of uncertainty provided by
the extended entropy is always lesser than that obtained
from the von Neumann entropy. This result allows us to
obtain a stronger versions of the entropic uncertainty re-
lations (4).
From the geometrical interpretation of the measure-
ment process provided by information geometry, and for-
mally expressed by relation (10), we have defined the set U
of geometrical uncertainties. The measures of uncertainty
belonging to this set are generated by the contravariant
metric tensors {gii}. In this way, the amount of informa-
tion removed in a random experiment depends upon the
metric structure generated by the statistical fluctuations
in a finite sequence of measurements. This result provides
a geometrical interpretation of the uncertainty and it al-
lows us to introduce the metric properties of the proba-
bility space in the entropic uncertainty relations (4). The
quantum uncertainty principle can be formulated in terms
of geometrical quantities.
The authors want to express their sincere thanks to Prof. E.
Bernabeu for his constant support and encouragement. They
are also indebted to Prof. R. Barakat for his fruitful and valu-
able comments. Authors wish to express their gratitude to
In˜igo Ortiz de Urbina for his manuscript review. While work-
When the concave function slin(gii) = −gii ln gii is in-
troduced in the set of geometrical uncertainties, we obtain
the family of geometrical entropies, Sln. Each element of
this subset is generated by a different metric tensor. We ing in this paper, one of the authors, J.M. Zoido, was fellow

J.M. Zoido and F. Carren˜o: Geometrical entropies. The extended entropy
469
of the Real Colegio Complutense at Harvard, established in 18. M.H. Partovi, Phys. Rev. Lett. 50, 1883 (1983).
cooperation with Harvard University.
19. I. Bialynicki-Birula, J. Mycielski, Commun. Math. Phys.
44, 129 (1975).
20. S.I. Amari, Contemporary Math. 203, 81 (1997).
21. S. Amari, From statistical physics to statistical inference
and back 113-138, edited by P. Grassberger, J.P. Nadal
(Kluwer Academic, Dordrecht, 1994).
References
22. N. Martin, J.W. England, Mathematical theory of entropy
(Adison-Wesley, London, 1981).
23. S.L. Braunstein, C.M. Caves, Phys. Rev. Lett. 72, 3439
(1994).
24. P.C. Mahalanobis, Proc. Nat. Inst. Sc. Ind. 2, 49 (1936).
25. C.R. Rao, Calcutta Math. Soc. 37, 81 (1945).
26. C.R. Rao, Nature 159, 30 (1947).
27. J. Burbea, C.R. Rao, J. Multivariate Anal. 12, 575 (1982).
28. J. Burbea, Expo. Math. 4, 347 (1986).
29. J. Anandan, Y. Aharonov, Phys. Rev. Lett. 65, 1697
(1990).
1. S. Amari, Differential-Geometrical Methods in Statistics,
Lecture Notes in Statistics (Springer, New York, 1985).
2. R. Balian, Y. Alhassid, H. Reinhardt, Phys. Reports 131,
1 (1986).
3. J.M. Zoido, Ph.D. thesis, Complutense University of
Madrid, 1997.
4. M. Fisz, Probability Theory and Mathematical Statistics
(R.E. Krieger, Florida, 1980).
5. B.V. Gnedenko, The theory of Probability (Chelsea, New
York, 1989).
6. W.K. Wootters, Phys. Rev. D 23, 357 (1981).
7. J. Summhammer, Int. J. Theor. Phys. 33, 171 (1994).
8. C.E. Shannon, W. Weaver, The Mathematical Theory of
Communication (Univ. Illinois, Urbana, 1962).
9. A.I. Khinchin, Mathematical Foundations of Information
Theory (Dover, New York, 1957).
10. R. Ash, Information Theory (Wiley, New York, 1965).
11. S. Goldman, Information Theory (Dover, New York,
1968).
12. J. von Neumann, Mathematical Foundations of Quantum
Mechanics (Princeton Univ., New Jersey, 1996).
13. S.M. Barnett, S.J.D. Phoenix, Phys. Rev. A 40, 2404
(1989).
14. J.B.M. Uffink, J. Hilgewoord, Found. Phys. 15, 925 (1985).
15. D. Deutsch, Phys. Rev. Lett. 50, 631 (1983).
16. V. Majern´ık, L. Richterek, Eur. J. Phys. 18, 79 (1997).
17. H. Maassen, J.B.M. Uffink, Phys. Rev. Lett. 60, 1103
(1988).
30. J.P. Provost, G. Vallee, Commun. Math. Phys. 76, 289
(1980).
31. S.K. Pati, Phys. Lett. A 159, 105 (1991).
32. S. Abe, Phys. Rev. A 48, 4102 (1993).
33. T.W.B. Kibble, Commun. Math. Phys. 65, 189 (1979).
34. G.W. Gibbons, J. Geom. Phys. 8, 147 (1992).
35. S. Abe, Phys. Rev. A 46, 1667 (1992).
36. A. Wehrl, Rev. Mod. Phys. 50, 221 (1978).
37. J.A. Wheeler, Int. J. Theor. Phys. 21, 557 (1982).
38. A.M. Mariz, Phys. Lett. A 165, 409 (1992).
39. C. Tsallis, J. Stat. Phys. 52, 479 (1988).
40. E.M.F. Curado, C. Tsallis, J. Phys. A 24, L69 (1991).
41. S. Abe, Phys. Lett. A 224, 326 (1997).
42. E.P. Borges, I. Roditi, Phys. Lett. A 246, 399 (1998).
43. B.W. Schumacher, Phys. Rev. A 44, 7047 (1991).
44. A. Mann, M. Revzen, E. Santos, Phys. Lett. A 238, 85
(1998).