Subject: Project Proposal -- Neural Network

From: Saleh Elmohamed <[email-masked]>
Date: Tue, 11 Apr 2000 02:46:19 -0400 (EDT)

http://class-server.npac.syr.edu:3768/cps615spring00-docs/cc00ych/proposal.html

Hi Choonhan,

I'm not quite sure that implementing a backprop neural nets and
training it to recognize (0-9) digits would be that interesting project
for this course... For one thing, you have not said anything in your 
proposal on how you are going to deal with the aspect of parallelism.

Over the years, there have been great deal of work on this subject and
most it turned out to be not that interesting. 
If you are going to deal with, let say, a three-layer net, with input and
output each of size N and the hidden layer, let say of size log(N), and N
>=10; how are you going to carry out the parallelism on a cluster of 4 
nodes (8 processors)??

It is quite important that your application exploit parallelism, at
the message-passing level.

One hint is to deal with more than a single network and have them
connected at certain levels (either at the input, output or the hidden
level) and then make them work on parallel to speed up the training
process... During processing any interesting discovery by one network
should be communicated to others for the goal of speeding up the overall 
training process... Perhaps modifying their training parameters with the
value(s) just discovered, etc.
So, doing it this way you probably would need to increase the size of
your nets as well as increase the size of your training data... 

Anyway, this just an idea but you really need to put together an
interesting parallel application if you're going to be dealing with
neural nets... 

Thanks.

-Saleh