tailieunhanh - Ebook Advanced calculus with applications in statistics (2nd edition): Part 2

(BQ) Part 2 book "Advanced calculus with applications in statistics" has contents: Optimization in statistics, approximation of functions, orthogonal polynomials, fourier series, approximation of integrals. | CHAPTER 8 Optimization in Statistics Optimization is an essential feature in many problems in statistics. This is apparent in almost all fields of statistics. Here are few examples, some of which will be discussed in more detail in this chapter. 1. In the theory of estimation, an estimator of an unknown parameter is sought that satisfies a certain optimality criterion such as minimum variance, maximum likelihood, or minimum average risk Žas in the case of a Bayes estimator Some of these criteria were already discussed in Section . For example, in regression analysis, estimates of the parameters of a fitted model are obtained by minimizing a certain expression that measures the closeness of the fit of the model. One common example of such an expression is the sum of the squared residuals Žthese are deviations of the predicted response values, as specified by the model, from the corresponding observed response values This particular expression is used in the method of ordinary least squares. A more general class of parameter estimators is the class of M-estimators. See Huber Ž1973, 1981 The name ‘‘ M-estimator’’ comes from ‘‘generalized maximum likelihood.’’ They are based on the idea of replacing the squared residuals by another symmetric function of the residuals that has a unique minimum at zero. For example, minimizing the sum of the absolute values of the residuals produces the so-called least absolute ®alues ŽLAV. estimators. 2. Estimates of the variance components associated with random or mixed models are obtained by using several methods. In some of these methods, the estimates are given as solutions to certain optimization problems as in maximum likelihood ŽML. estimation and minimum norm quadratic unbiased estimation ŽMINQUE In the former method, the likelihood function is maximized under the assumption of normally distributed data wsee Hartley and Rao . A completely different approach is used in the latter method, which was proposed .