Feedback Nerual Network(六）：Boltzmann Machine

It is a Random NN, is also a kind of  Feedback NN
It is similar with DHNN in many aspects, big difference is that Boltzmann NN is a Random NN
When the temperature parameter T→ 0

⇒S-Type Function approaches to Binary Function
⇒Random NN degenerates to Deterministic NN(Same as DHNN)
It can be used in Pattern Classification, Prediction, Combination Optimization, Planning etc.

 

Structure of Networks & Work Mode

Structure:
Single layer Feedback Networks
Same as Hopfield, has Symmetric Link-Weight Matrix, i.g
wij= wji
, and wii= 0

Function:
Can be viewed as a Multi-Layer Networks
Part of nodes are input nodes, part of nodes are output nodes and part of nodes are hide nodes
The hide nodes have no relationship with outside, only implement the high-order relationship between I/O

Work Mode:
Synchronous Mode
Asynchronous Mode

 

 

Discuss the Convergence (2)
From probability point of view, the tendency of system energy is always towards to the minimum
Some states of neurons possible take the values oppositely, consequently will increase the Energy.
Sometimes this is good for jumping out of local minimum point. This is another difference compare with Hopfield NN.

 

Discuss the Convergence (3)
Mimic the Annealing Method 模拟退火

At beginning, use higher temperature T
Curve of S-Type Function is flatter, the difference of probability of each state is small
Can accurately move to a minimum and also prevent  jumping out of this minimum
When near neighboring global minimum point, step by step reduce T
Curve of S-Type Function is more cliff  the difference of probability of each state is bigger
Easy to jump out of local minimum point and go to global minimum point

 

Summary:
Energy is lower
the probability of being a certain state is bigger
T is biggerS-Type Function is flatter
the difference of probability of occurring different state is small
Accurately move to a minimum and prevent jump out of this minimum point
T is smallerS-Type Function is more cliffy
the difference of probability of occurring different state is big

Easy jump out of Local Minimum Point and reach the Global Minimum