Essay: The Theory of Statistical Estimation

Overview
The 1922 essay by Ronald A. Fisher lays out a systematic approach to point estimation that became the backbone of classical parametric inference. It centers on using the likelihood function as the primary object for making inferences about unknown parameters, and it frames estimation as a problem of selecting parameter values that make the observed data most probable. The essay synthesizes mathematical definition, practical algorithms, and guiding principles for estimator performance.

Maximum likelihood and likelihood functions
Fisher popularized and formalized the method of maximum likelihood: treat the probability of observed data as a function of the model parameter and choose the parameter value that maximizes that function. The likelihood perspective shifts attention from treating data as random realizations to treating the parameter as the quantity to be estimated from the observed evidence. Fisher emphasized the operational simplicity and wide applicability of maximizing likelihoods across different statistical models.

Score, information, and asymptotic behavior
A key technical contribution is the introduction of the score function, the derivative of the log-likelihood with respect to the parameter, and the associated measure of "information" about the parameter contained in the data. Fisher showed how these quantities govern the variability of estimators: under regularity conditions, maximum likelihood estimators are asymptotically normal with variance approximately equal to the inverse of the Fisher information. This connection provides a principled way to compute approximate standard errors and to assess estimator precision in large samples.

Sufficiency and efficiency
The essay gives prominence to the concept of sufficient statistics as functions of the data that capture all information relevant to estimating a parameter. Fisher argued that reduction to sufficient statistics simplifies problems without loss of inferential content, and he connected sufficiency to the likelihood framework. He also introduced the notion of efficiency to compare estimators, describing an estimator as efficient if it attains the smallest possible variance (asymptotically) among a class of unbiased estimators, a notion tied to the Fisher information.

Illustrative examples and practical guidance
Fisher illustrated the general ideas with worked examples from common parametric families, showing how maximum likelihood yields familiar estimators and how asymptotic approximations perform in realistic sample sizes. He advocated analytic approximations for the distribution of estimators and for forming confidence intervals, emphasizing the role of curvature of the log-likelihood in determining estimator uncertainty. Practical recommendations included using likelihood-based diagnostics and transformations to stabilize variance or improve normal approximations.

Impact and subsequent developments
The essay crystallized principles that steered much of 20th-century statistical practice: likelihood as the centerpiece of estimation, the use of score and information for variance assessment, and sufficiency as a data-reduction principle. Later formal developments refined and extended these ideas, providing rigorous regularity conditions, finite-sample theory, and alternative optimality criteria, but Fisher's concepts remain central. The legacy is a coherent, operational framework for extracting parameter estimates and quantifying their uncertainty from probabilistic models.