CPS713 APPLICATIONS OF COMPUTATIONAL SCIENCE: STATISTICS FOR PHYSICS

Geoffrey Fox

Syracuse University
Northeast Parallel Architectures Center
111 College Place
Syracuse, New York 13244-4100
(315) 443-2163
gcf@npac.syr.edu
http://www.npac.syr.edu

February, 1996

Abstract of Statistics for Physics

• These notes were developed in a set of courses taught at Caltech, 1980--1989.
• They describe statistics with a particular emphasis on parameter estimation, as occurs in many physics experimental analysis problems.
• Although the notes were developed for physics majors, they can be read by any computational scientist.
• We cover the following topics:

  basic definitions and results,

  important discrete distributions,

  estimation of parameters,

  minimization,

  goodness of fit,

  and generation of random numbers.

Useful Books for Statistics
Part of Course

• G. P. Yost, ``Lectures on Probability and Statistics'' --- copies available from G. Fox.

• W. T. Eadie, et al. (North Holland), ``Statistical Methods in Experimental Physics'' --- a little sophisticated for this course, but technically excellent (called EDJRS).

• W. H. Press, et al., ``Numerical Recipes --- The Art of Scientific Computing'', Chapter 7, 13, 14 --- Computer Related Issues. This has C, FORTRAN, and PASCAL programs available.

• J. Mathews and R. L. Walker (Benjamin), ``Mathematical Methods of Physics'', Chapter 14.

• Cramer: Mathematical Methods of Statistics, published by Princeton.

• Table of Contents
• 1: Basic Definitions and Results
  • 1.1Basic Definitions and Results Is Probability Intuitive or Mathematical?
  • Basic Definitions---Borel Sets
  • Axioms of Probability
  • Immediate Deductions from Axioms of Probability
  • From Mathematical to Intuitive Definition of Probability
  • 1.2Bayes Theorem: Conditional Probabilities---1) Frequency Approach
  • Bayes Theorem: Conditional Probabilities---2) Frequency and Probability
  • Bayes Theorem: Conditional Probabilities---3) Bayes Law
  • Estimation of Image Scanning Efficiency---I
  • Example: Estimation of Image Scanning Efficiency---II
  • Example: Analysis of Unreliable Test
  • The Bayes Controversy Naive Application for Estimation---I
  • The Bayes Controversy Naive Application for Estimation---II
  • The Bayes Controversy Naive Application for Estimation---III
  • 1.3Continuous Distributions General (Borel) Formalism
  • Gaussian Distribution---I
  • Properties of Gaussian Distribution
  • 1.4Functions of a Random Variable
  • Joint Probability Distributions
  • Example of Joint Probability Distributions: Bayes Law for Densities---I
  • Example of Joint Probability Distributions: Bayes Law for Densities---II
  • 1.5Properties of Joint Distribution Functions
  • Means---Moments, etc. for One-dimensional Distributions
  • 1.6Means--Moments--Correlations for Multidimensional Distributions I
  • Means---Moments---Correlations for Multidimensional Distributions II
  • Errors and Moment Matrices
  • Density Matrix Element Example---I
  • Density Matrix Element Example---II
  • Examples of Correlations 2D Track Finding---I
  • Examples of Correlations 2D Track Finding---II
  • Examples of Correlations Exponential Fits---I
  • Examples of Correlations Exponential Fits---II
  • Errors in Linear Combinations of Uncorrelated Variables
  • Error in Lack of Correlation Assumption
  • Parameters of a Gaussian Distribution---I
  • Parameters of a Gaussian Distribution---II
  • Addition of Independent Random Variables
  • 1.7Leading Up to the Central Limit Theorem Moment Generating Function---I
  • Moment Generating Function of a Sum of Two Random Variables
  • Moment Generating Function III
  • Central Limit Theorem Statement Laplace
  • Central Limit Theorem Statement---Liapounoff
  • Proof of Central Limit Theorem Laplace---I
  • Proof of Central Limit Theorem Laplace---II
  • Monte-Carlo Integration and the Central Limit Theorem---I
  • Monte-Carlo Integration and the Central Limit Theorem---II
  • Monte-Carlo Integration and the Central Limit Theorem---III
  • Extension of Central Limit Theorem to Correlated Random Variables---I
  • Extension of Central Limit Theorem to Correlated Random Variables---II
  • Central Limit Theorem for Quotients
  • (i) Example from Histogramming (Physics) Events---I
  • (i) Example from Histogramming (Physics) Events---II
  • (i) Example from Histogramming (Physics) Events---III
  • (ii) Analysis of Scattering Experiments---I
  • (ii) Analysis of Scattering Experiments---II
  • (iii) Example of Weighted Histograms
• 2: Feefoefum in Las Vegas: Combinatoric and Discrete Probability Overview
  • Bernoulli Trials---I
  • Bernoulli Trials II
  • Example of Bernoulli Trials
  • Poisson Limit of Binomial Distribution
  • Example of Poisson Distribution
  • Compound Poisson Distribution
  • Particle Cluster Production---I
  • Particle Cluster Production II
  • Example of Cherenkov Light
  • Additivity Property of Binomial or Poisson Distributions
  • Gaussian Limit of Binomial Distribution
  • Gaussian Limit of Binomial and Poisson Distributions---II
  • Gaussian Limit of Binomial and Poisson Distributions---III
  • Gaussian Limit of Binomial and Poisson Distributions---IV
  • 2.2Birth and Death Processes Overview---I
  • Telephone Calls as a Birth Process---I
  • Telephone Calls as a Birth Process---II
  • Basic Birth Process Formalism---I
  • Basic Birth Process Formalism---II
  • General Birth and Death Process
  • Example of Birth/Death Process---I
  • Example of Birth/Death Process---II
  • Markov Chains
• 3: Estimation of Parameters
  • Estimation of Parameters---References
  • 3Overview of Estimation of Parameters
  • 3.1Maximum Likelihood Principle Illustrated by Measurement Errors---I
  • Maximum Likelihood Principle Illustrated by Measurement Errors---II
  • Enter Mr. Bayes for Conditional Probablities---I
  • Enter Mr. Bayes for Conditional Probablities---II
  • Interpretation of Bayes Formulation of Likelihood---I
  • Interpretation of Bayes Formulation of Likelihood---II
  • The Likelihood Maximizes the Quivering Rod---I
  • The Likelihood Maximizes the Quivering Rod---II
  • The Likelihood Maximizes the Quivering Rod---III
  • General Maximum Likelihood Methods---I
  • General Maximum Likelihood Methods---II
  • Warning Example of Misuse of Likelihood Method
  • Practical Advice on Use of Maximum Likelihood---I
  • Practical Advice on Use of Maximum Likelihood---II
  • Determination of Lifetime Example of Maximum Likelihood---I
  • Determination of Lifetime Example of Maximum Likelihood---II
  • Central Limit Estimate of Error
  • Maximum Likelihood Estimate of Error in Lifetime---I
  • Maximum Likelihood Estimate of Error in Lifetime---II
  • Comparison of Two Error Estimates in Maximum Likelihood---I
  • Comparison of Two Error Estimates in Maximum Likelihood---II
  • Theorem on Asymptotic Validity of Maximum Likelihood Method---Conditions and Statement of Theorem
  • Theorem on Asymptotic Validity of Maximum Likelihood Method Conditions---First Assertion
  • Theorem on Asymptotic Validity of Maximum Likelihood Method Conditions---Second and Third Assertions
  • Proof of Maximum Likelihood Theorem Part (a)---I
  • Proof of Maximum Likelihood Theorem Part (a)---II
  • Proof of Maximum Likelihood Theorem Part (a)---III
  • Proof of Maximum Likelihood Part (b): Formula for Error---I
  • Proof of Maximum Likelihood Part (b): Formula for Error---II
  • Proof of Maximum Likelihood Part (b): Formula for Error---III
  • Multiplication of Experiments
  • 3.2Overview of Method
  • Derivation of from Maximum Likelihood
  • Mean and Standard Deviation of
  • Interpretation of Value of
  • Correlated Observations in
  • Example of Multiple Moments
  • Linear fits to Cross Section Data Example of Use of ---Overview
  • Formulation of Linear Fit for Cross Section Data---I
  • Formulation of Linear Fit for Cross Section Data---II
  • Solution and Errors from Linear
  • Estimate of Errors to be Used in Fits
  • 3.3Counter Physics---Introduction and Poisson Process---I
  • Counter Physics---Introduction and Poisson Process---II
  • Unnormalized Counter Experiment---I
  • Unnormalized Counter Experiment---II
  • Unnormalized Counter Experiment---III
  • Normalized Counter Experiment
  • Gaussian Limit of Counter Physics
  • 3.4Bubble Chamber or Spectrometer Physics---I
  • Bubble Chamber or Spectrometer Physics---II
  • Bubble/Spectrometer---Physics Individual Events with Continuous Distributions---III
  • Bubble/Spectrometer---Physics Individual Events with Continuous Distributions---IV
  • Bubble/Spectrometer---Physics Individual Events with Continuous Distributions---V
  • Individual Events Become Binned Data---I
  • Individual Events Become Binned Data---II
  • Method of Moments---I
  • Method of Moments---II
  • Method of Moments---III
  • Example of Moment Method
  • Comparison of Moment Method with Maximum Likelihood---I
  • Comparison of Moment Method with Maximum Likelihood---II
  • Several Parameters in Method of Moments---I
  • Several Parameters in Method of Moments---II
  • Remarks on Counting Errors in Moments---I
  • Remarks on Counting Errors in Moments---II
  • Remarks on Counting Errors in Moments---III
• 4: Minimization
  • 4.2Derivative Formalism for ---I
  • Derivative Formalism for ---II
  • Derivative Formalism for ---III
  • Derivative Calculation for Maximum Likelihood---I
  • Derivative Calculation for Maximum Likelihood---II
  • Derivative Calculation for Maximum Likelihood---III
  • 4.3The Derivative Method for ---I
  • The Derivative Method for ---II
  • Error Calculation from Expansion
  • 4.4The Real World and Minimization---I
  • The Real World and Minimization---II
  • The Real World and Minimization Example of Two Variables---I
  • The Real World and Minimization Example of Two Variables---II Pictures of Two Variable Case
  • The Real World and Minimization
  • Example of Two Variables---III
  • Eigenfunction Analysis
  • Transformation of M to Eigendirections
  • What I Used to Do!
  • Why is M Poorly Conditioned?
  • Marquardt's Method
  • Estimate of Marquardt
  • Parameter Q
  • Experience with Marquardt's Method
• 5: Goodness of Fit
  • Distribution for One Degree of Freedom
  • General Distribution
  • Characteristic Function
  • 5.2Confidence Level
  • Typical Confidence Level Distributions
  • Real World Confidence Levels
  • Error Estimate from Value
  • 5.3Bartlett's Function---Overview
  • Bartlett's Function for
  • 5.4Hypothesis Testing---I
  • Hypothesis Testing---II
  • Hypothesis Testing---III
  • 5.5Student's t Distribution for Hypothesis Testing---I
  • Student's t Distribution for Hypothesis Testing---II
  • Student's t Distribution for Hypothesis Testing---III
  • 5.6Robust Estimation--Motivation--I
  • Robust Estimation---Motivation---II
  • Robust Estimation---Motivation---III
  • Robust Determination of Mean---I
  • Robust Determination of Mean---II
  • Robust Determination of Mean---III
  • Robust Determination of Mean Trimmed Mean
  • Likelihood Based Robust Estimation of Means---I
  • Likelihood Based Robust Estimation of Means---II
  • Particular Robust Estimates for Likelihood---Huber
  • Particular Robust Estimates for Likelihood---Andrews
  • Comparison of Methods---I
  • Comparison of Methods---II
• 6: Generation of Random Numbers
  • 6Overview of Generation of Random Numbers
  • Generation of Random Numbers---Genuinely Random
  • Congruential Pseudorandom Numbers
  • Congruential Random Numbers in Monte Carlo
  • A Practical Congruential Pseudorandom Number Generator
  • Shift Register Pseudorandom Numbers---Overview
  • Initialization of Pseudorandom Numbers
  • Correlations
  • Decorrelation of Pseudorandom Numbers
  • Generation of Gaussian Random Numbers---I
  • Generation of Gaussian Random Numbers---II
  • Generation of General Random Number Distributions
  • Generation of Exponentially Distributed Random Numbers
  • Interpolation to Find General Distributions---I
  • Interpolation to Find General Distributions---II
  • Accept/Reject Method for General Probability Distributions
  • Accept-Reject Methods for Event Generation---I
  • Accept-Reject Methods for Event Generation---II