4: Minimization

4Minimization 4.1

• The basic problem is to minimize ---a function of n variables .

• Equivalently, we must find the zero of an arbitrary function of n variables.

• This arises in previous sections when we wish either to minimize or maximize the likelihood L (equivalently to minimize---).

• The methods divide according to

  (i) Can calulate f only

  (ii) Can calculate f, , and .

• (i) is extremely time consuming (i.e., it takes a lot of function evaluations to converge to desired minimum).

• In all problems I have faced, it has been simple to calculate derivatives and so I will only discuss (ii).

• 4.2Derivative Formalism for ---I
• Derivative Formalism for ---II
• Derivative Formalism for ---III
• Derivative Calculation for Maximum Likelihood---I
• Derivative Calculation for Maximum Likelihood---II
• Derivative Calculation for Maximum Likelihood---III
• 4.3The Derivative Method for ---I
• The Derivative Method for ---II
• Error Calculation from Expansion
• 4.4The Real World and Minimization---I
• The Real World and Minimization---II
• The Real World and Minimization Example of Two Variables---I
• The Real World and Minimization Example of Two Variables---II Pictures of Two Variable Case
• The Real World and Minimization
• Example of Two Variables---III
• Eigenfunction Analysis
• Transformation of M to Eigendirections
• What I Used to Do!
• Why is M Poorly Conditioned?
• Marquardt's Method
• Estimate of Marquardt
• Parameter Q
• Experience with Marquardt's Method