The Entropy and Information in a two-linked-population model

Xu Cui
cuixu@tiem.utk.edu

Introduction

We may say safely that two things must share some information when they linked, and things will be isolated if they share no information with others. It will be of great interest if we apply this idea to ecological problems, specifically, to two populations linked by migration (See Figure 1). We thus have such questions as: in what conditions do the two populations share “most” information? And in what conditions does a population share “most” information with its previous generation?

First, we have to make clear the concept of “information”. If a random variable X could have n states, with probability p_i at state i, then we define the entropy (or uncertainty) of X as

H(x)=-∑p_ilnp_i

If there is another random variable Y, we can define joint entropy as:

H( x,y)=-∑_i∑_j p(x_i,y_j) lnp(x_i,y_j)

where p(x_i,y_j)  is the probability of X to be in the state i and at the same time Y to be in the state j.

And we define the information between X and Y as:

I(X:Y)=H(X)+H(Y)-H(X,Y)

It is easy to prove that the information between two variables is zero if they are independent. And I(X:Y)=H(X)=H(Y) if they are complete dependent. From the information we can answer such a question: if we know X, how much can we know Y? Also note I(X:Y) is always be less than H(X) or H(Y). (See Figure 2.)

Now let’s return to our population problem. If we know the density of one population, can we infer the density of the other population? If we know the density of this generation, can we know the density in the future? And in what conditions do the two populations share “greatest” information? We will ask such questions to the computer with the below model.

(Back to top)

Model

In the below discrete model, two populations, A and B, is linked by migration. The whole system is "isolated" in the sense that no individuals can move into or out this system. Individuals can only migrate between the populations A and B. Emigration occurs only when the population density is higher than a certain value, the emigration critical value. Each population can also grow itself. (See Figure 3)

We use the following equations to simulate the above system:

A_n+1 = f[An – m(An) + m(Bn)]
B_n+1 = f[Bn – m(Bn) + m(An)]            (1)

where

f(X) = rX(1-X)                                      (2)

m(X) =  k(X-m₀)   if X>m₀

        =  0    if X<m₀

f(x) is the common logistic equation and m(x) is the emigration function. If the population density is higher than the emigration critical value, m₀ , some individual will emigrate. k is the emigration efficiency, ranging from 0 to 1. r is the intrinsic growth rate. Table 1 is the summary of the parameters and symbols. A₀ , B₀ and m₀ are assigned a constant value without special reasons. r is assigned as 3.99 because the population dynamics is wild, or chaotic, at r=3.99.(See Figure 4)

k is the most important parameter in this project. It can be thought as the degree of connection of population A and B. If k is zero, then there is no migration and the connection is zero; as k increases, the migration between two populations increases and thus the connection increases.

Here you may have noticed, the population density is not random variable at all. So how can we use the concept of "information", which is based on random variables? In fact, if we were really the natural population observers, we would never know the above equation and the fluctuation of the population density would be random. Just like if we know every thing, such as air condition, wind, initial condition, etc when tossing a coin, we will think which side of the coin is up is not random. Figure 4 shows how population density fluctuates at r = 3.99.

(Back to top)

Result

As k, the degree of connection between the two populations, increases, the entropy of A¹ almost remains constant at 2.8 except a sharp valley around k=0.25. (See Figure 5) Note that the maximum entropy A can reach is ln20=3(See the below section of method), thus for most values of k, the density of A seems to be a totally random variable uniformly distributed from 0 to 1. What’s most interesting is that the entropy suddenly decreases to 0.7 and then increases to the previous level, 2.8, in the same way, around k=0.25.

The information between A and B also changes in an interesting way (Figure 5). As k increases, the information increases moderately until k reaches 0.35, at which it suddenly reaches its highest possible value. At k=0.6 it drops sharply to a low level. Such behavior leaves a blank of information value in the range from 1 to 2.8. That’s to say, you almost can’t find a value of k that make the information between A and B be 1 to 2.8. This could be an analogy with phase transition. As temperature increases from –10°C-0°C, the structure of ice remain ordered; at 0°C, the ordered structure is quickly destroyed and a chaotic structure of water substitutes.

As the ratio of information to entropy, instead of the information itself, is better to be a representative of how much we know one from the other, how I/H changes as k increases is examined. (See Figure 6) As information is always no bigger than the entropy, the highest possible value of I/H is 1. We can see in Figure 6 that it’s hard to forecast the next generation’s density based the current density, for information between two generations is low for most values of k. The only exception is when k is around 0.25, where we can forecast precisely. Situation is better when we forecast B’s density based on A’s. Around k=0.25 and 0.35-0.6, we can forecast precisely B’s density, for the ratio of information between two populations to entropy is 1 in such ranges.

As the correlation coefficient, c_o, like information, is also an indicator about how two variables correlate, it is interesting to study the relationship between k and c_o. It is shown in Figure 7 that the correlation coefficient between two populations changes almost in the same way as information changes. The highest correlation occurs when k is around 0.25 and 0.3-0.6. But for that between two continuous generations of population A, the situation is quite different when k=0.3-0.6. Correlation is low but information is high in this interval.

(Back to top)

Discussion

It's necessary to study the effect of the initial conditions on how information changes. The initial conditions are A₀ = 0.2 and B₀ = 0.3 for the above work. Other conditions, A₀ = 0.8 and B₀ = 0.3,  A₀ = 0.8 and B₀ = 0.6 are chosen and it's shown in Figure 6b that information fluctuation doesn't sensitively depend on initial conditions.

Implication

Based the above result, I would guess that the emigration efficiency between populations in nature is about 0.25, if other parameters are really similar to what I chose above. Because around k=0.25, the population uncertainty is low (i.e. the population does not fluctuate so wildly), and at the same time, the information is conducted most efficiently along time and space.

(Back to top)

Method

The population density fluctuates between 0 and 1. To calculate the entropy, I divided the possible density range (0,1) to 20 small intervals (0-0.05,0.05-0.1,…,0.95-1) as 20 states. 20,000 iterations or generations were calculated. The density of one generation will fall into only one interval and I counted how many times have the density fallen into every interval thus got the distribution of population density, based on which the entropy can be calculated.

To calculate joint entropy between A and B we need 20´20 states, because we have to get the probability of A=state i and B=state j. The method is the same as above.

(Back to top)

1. The reason why I only consider the entropy of A is that the entropy of B is the same with that of A.

(Back to top)