Network Equivalence and Uniqueness

Let #tex2html_wrap_inline3015# and #tex2html_wrap_inline3017# stand for 2 feedforward nets, for example, see Figure-#fig1#607>. Also, let the symbol (#tex2html_wrap_inline3019#) stands for #tex2html_wrap_inline3021#-equivalence in behavior, and (#tex2html_wrap_inline3023#) for #tex2html_wrap_inline3025#-equivalence in structure. Albertini, Sontag and Maillot [#5##1#] have stated that given two networks #tex2html_wrap_inline3027# and #tex2html_wrap_inline3029#, one says that they are I-O equivalent if the behavior of #tex2html_wrap_inline3031# equal to the behavior of #tex2html_wrap_inline3033#. In other words, applying the same set of inputs to both networks will yield the same output. Clearly, networks with identical structures and identical internal representations (<#613#> weight sets<#613#>) have the same I-O mappings. Also, nets with different internal representations can have the same I-O mappings. Now, what we would like to answer is the following question: When does #tex2html_wrap_inline3035# imply #tex2html_wrap_inline3037# ? Sontag [#20##1#] pointed out that Sussmann's argument takes into account the ``independence property'' (IP) (defined below), but he did not explicitly state this property in his work [#8##1#]. Also, they state that to show uniqueness of the connection weights of a given network, one needs to demonstrate that the given activation functions satisfy IP. Sussmann showed that #tex2html_wrap_inline3039# and the related sigmoid #tex2html_wrap_inline3041# satisfy IP.