The Ising model describes a magnetic system in terms of binary spins
. The system is governed by the energy function
where nearest neighbors interact pairwise with a constant attractive (if J > 0) coupling of strength J. The lowest energy state is reached by iterative updating of
which leads to a state where all spins point in one of two possible directions.
As mentioned above, in the Potts model we restrict the state space of the neurons such that exactly one neuron at every site is allowed to be on and represented as follows:
Thus for every i, should be 1 for only one value of a in
the set of states A, and 0 for the rest of the states.
In general, lets assume that the states of the Potts neuron can be
represented by a k-dimensional vector. Now, we could say that the
possible state of the neuron vector
associated
with neuron i is represented by the unit vector in the direction of
a, so we end up with k different states. Also, these states
,
,
,
are all
normalized and mutually orthogonal. Therefore, the number of states
available at every node is reduced from
to k. Hence, we now
have a k-state Potts model.
In short, the explicit enforcement shown in equation
(12) is known in the MFA algorithm as Neuron
Normalization and the reason behind it is to reduce the number of
states at each neuron. This concept is really the key to the Potts
model.
Before we introduce the mean field variables, we would like step back
and take a second look at the Hopfield model [33,1]. In
[33] Hopfield introduced a model with neurons having a graded
response. So given a fixed input bias to neuron
,
the weight of the connection between neurons
and
, and parameter
T to denote the temperature, the behavior of
can be stated as:
When , function (13)'s response takes
on a step-wise form.
Now, to compute the actual mean field variables we do the following:
At a finite T, (of eq. 13) can be replaced by a
mean field theory (MFT) variable
, then differentiating
to obtain the Potts MFT equations:
Substituting for
, we find that equations
(14,15) clearly satisfy equation
(12). Also the
and
are
referred to as the mean field variables.
Figure 12: A general schema for the mean field annealing algorithm.