Some notes on Statistics

Some of these definitions have been taken from the tutorial notes at (Lan Huong Nguyen, 2015).

Point Estimate

Point estimation involves the use of sample data to calculate a single value (known as a point estimate since it identifies a point in some parameter space) which is to serve as a “best guess” or “best estimate” of an unknown population parameter (for example, the population mean).

If $\left\{x^{(1)},\dots ,x^{(m)}\right\}$ are a set of $i.i.d.$ data, a point estimator or statistic is any function of the data:

$${\hat{\theta }}_m=g\left(x^{(1)},\dots ,x^{(m)}\right).$$

Frequentist Vs Bayesian Approach

• Frequentist:
  • The true parameter $\theta$ is fixed but unknown - therefore not a random variable.
  • The dataset is a random sample from the true distribution, and therefore is random.
  • The point estimate of the parameter $\hat{\theta }$ is a function of the dataset, and therefore is a random variable.
  • Leads to MLE.
• Bayesian:
  • The dataset is directly observed and is therefore not random.
  • The true parameter $\theta$ is unknown or uncertain and therefore is a random variable.
  • Leads to incorporation of a prior in the estimate, and estimates the proper posterior using MAP.
• Useful discussion with a video on the difference in perspectives: Are you Bayesian or Frequentist?

Lan Huong Nguyen. (2015). Statistical Inference. https://stanford.edu/~lanhuong/refresher/notes/probstat-section3.pdf