#equation1723#
and the second derivative is:
#equation1726#
Now the first derivative of #tex2html_wrap_inline2855# is:
#equation1729#
#tex2html_wrap_inline2857#
From previous definitions and observations, we observe the following:
- #tex2html_wrap_inline2859#, and
-
#tex2html_wrap_inline2861#
#tex2html_wrap_inline2863#
#tex2html_wrap_inline2865#
#tex2html_wrap_inline2867#.
Also we observe some properties of #tex2html_wrap_inline2869# that are shared with
#tex2html_wrap_inline2871#:
- For all x, #tex2html_wrap_inline2875# .
- #tex2html_wrap_inline2877# is odd.
- #tex2html_wrap_inline2879# and #tex2html_wrap_inline2881# are continuous.
- For all x, #tex2html_wrap_inline2885#.
- #tex2html_wrap_inline2887# and #tex2html_wrap_inline2889#.
#tex2html_wrap_inline2891#
The second #tex2html_wrap_inline2893# function, due to Elliott [#9##1#],
is defined next.
#equation1748#
#tex2html_wrap_inline2895#
The function stated in equation (#pwrtype21#493>)
is differentiable everywhere to:
#equation1754#
and satisfies a ``general'' logistic differential equation as follows:
#equation1757#
#tex2html_wrap_inline2897#
The steps leading to equation (#pwrtype22#501>) from function
(#pwrtype21#502>) are as follows:
#equation1763#
differentiating to:
#equation1768#
and a second derivative of:
#equation1771#
Also,
#equation1774#
differentiating to:
#equation1779#
and a second derivative of:
#equation1782#
Now, from equations (#tex2html_wrap_inline2899#, #tex2html_wrap_inline2901#) we derive the
following:
#equation1785#
#tex2html_wrap_inline2903#
It is worth pointing out that #tex2html_wrap_inline2905# and #tex2html_wrap_inline2907# satisfy
an ``autonomous'' differential equation, so that calculating
#tex2html_wrap_inline2909# or #tex2html_wrap_inline2911# requires very little effort beyond
the calculation of the function itself. The aforementioned
``autonomy'' concept can be illustrated by the following example:
To evaluate functions #tex2html_wrap_inline2913# and
#tex2html_wrap_inline2915#, the only values needed are
#tex2html_wrap_inline2917# and #tex2html_wrap_inline2919#, respectively.
Regarding the activation function #tex2html_wrap_inline2921#, it seems not to satisfy
such a differential equation, but even so calculation of #tex2html_wrap_inline2923#
requires negligible additional effort. Also, networks using these three
types of activation functions, can be trained using backpropogation
algorithm.
#tex2html_wrap_inline2925#
We observe some properties of #tex2html_wrap_inline2927# shared with #tex2html_wrap_inline2929#:
- #tex2html_wrap_inline2931# and #tex2html_wrap_inline2933# are continuous.
- For all x, #tex2html_wrap_inline2937#.
- #tex2html_wrap_inline2939# and #tex2html_wrap_inline2941#.
- #tex2html_wrap_inline2943# is odd.
- For all x, #tex2html_wrap_inline2947#.
#tex2html_wrap_inline2949#
%
#figure557#
Figure 2: f1() = #tex2html_wrap_inline2951#, f2() = #tex2html_wrap_inline2953#, and f3() = #tex2html_wrap_inline2955#.
%
%