●Preface
Chapter 1 Probability and Distributions 1
1.1 Probability 1
1.1.1 Permutation,combination and binomial coefficients 1
1.1.2 Sample space 3
1.1.3 Events 4
1.1.4 Properties of probability 5
1.2 Conditional Probability 7
1.3 Bayes Theorem 9
1.4 Probability Distributions 10
1.5 Bivariate Distributions 13
1.5.1 Joint distribution 13
1.5.2 Marginal and conditional distributions 14
1.5.3 Independency of two random variables 14
1.6 Expectation,Variance and Moments 16
1.6.1 Moments 16
1.6.2 Some probability inequalities 18
1.6.3 Conditional expectation 21
1.6.4 Compound random variables 23
1.6.5 Calculation of (conditional) probability via (conditional) expectation 23
1.7 Moment Generating Function 24
1.8 Beta and Gamma Distributions 27
1.8.1 Beta distribution 27
1.8.2 Gamma distribution 29
1.9 Bivariate Normal Distribution 32
1.9.1 Univariate normal distribution 32
1.9.2 Correlation coefficient 34
1.9.3 Joint density 34
1.9.4 Stochastic representation of random variables or random vectors 38
1.10 Inverse Bayes Formulae 40
1.10.1 Three inverse Bayes formulae 40
1.10.2 Understanding the IBF 43
1.10.3 Two examples 45
1.11 Categorical Distribution 47
1.12 Zero-inflated Poisson Distribution 49
Exercise 1 53
Chapter 2 Sampling Distributions 57
2.1 Distribution of the Function of Random Variables 57
2.1.1 Cumulative distribution function technique 57
2.1.2 Transformation technique 62
2.1.3 Moment generating function technique 71
2.2 Statistics,Sample Mean and Sample Variance 73
2.2.1 Distribution of the sample mean 73
2.2.2 Distribution of the sample variance 74
2.3 The t and F Distributions 76
2.3.1 The t distribution 76
2.3.2 The F distribution 78
2.4 Order Statistics 81
2.4.1 Distribution of a single order statistic 81
2.4.2 Joint distribution of more order statistics 84
2.5 Limit Theorems 86
2.5.1 Convergency of a sequenceof distribution functions 86
2.5.2 Convergence in probability 91
2.5.3 Relationship of four classesof convergency 92
2.5.4 Law of large number 94
2.5.5 Central limit theorem 94
2.6 Some Challenging Questions 96
Exercise 2 99
Chapter 3 Point Estimation 102
3.1 Maximum Likelihood Estimator 102
3.1.1 Point estimator and point estimate 102
3.1.2 Joint density and likelihood function 104
3.1.3 Maximum likelihood estimate and maximum likelihood estimator 105
3.1.4 The invariance property of MLE 115
3.2 Moment Estimator 117
3.3 Bayesian Estimator 121
3.4 Properties of Estimators 125
3.4.1 Unbiasedness 125
3.4.2 Efficiency 126
3.4.3 Sufficiency 138
3.4.4 Completeness 146
3.5 Limiting Properties of MLE 151
3.6 Some Challenging Questions 153
Exercise 3 156
Chapter 4 Confidence Interval Estimation 162
4.1 Introduction 162
4.2 The Confidence Interval of Normal Mean 166
4.2.1 The variance is known 166
4.2.2 The variance is unknown 167
4.3 The Confidence Interval of the Difference of Two Normal Means 169
4.4 The Confidence Interval of Normal Variance 171
4.4.1 The mean is known 171
4.4.2 The mean is unknown 172
4.5 The Confidence Interval of the Ratio of Two Normal Variances 172
4.6 Large-Sample Confidence Intervals 174
4.7 The Shortest Confidence Interval 178
Exercise 4 180
Chapter 5 Hypothesis Testing 183
5.1 Introduction 183
5.1.1 Several basic notions 184
5.1.2 Type I error and Type II error 186
5.1.3 Power function 189
5.2 The Neyman-Pearson Lemma 191
5.2.1 Simple null hypothesis versus simple alternative 192
5.2.2 Composite hypotheses 199
5.3 Likelihood Ratio Test 203
5.3.1 Likelihood ratio statistic 203
5.3.2 Likelihood ratio test 205
5.4 Tests on Normal Means 211
5.4.1 One-sample normal test when variance is known 211
5.4.2 One-sample t test 215
5.4.3 Two-sample t test 217
5.5 Goodness of Fit Test 220
5.5.1 Introduction 220
5.5.2 The chi-square test for totally known distribution 222
5.5.3 The chi-square test for known distribution family with unknown parameters 226
Exercise 5 230
Chapter 6 Critical Regions and p-values for Skew Null Distributions 233
6.1 One-sample Chi-square Test on Normal Variance 233
6.2 Two-sample F Test on Normal Variances 238
Appendix A Basic Statistical Distributions 246
A.Discrete Distributions 246
A.Continuous Distributions 250
Appendix B A Unified Expectation Technique 256
B.Continuous Random Variables 257
B.Discrete Random Variables 277
Appendix C The Newton-Raphson and Fisher Scoring Algorithms 289
C.Newton's Method for Root Finding 289
C.Newton's Method for Calculating MLE 294
C.The Newton-Raphson Algorithm for High-dimensional Cases 299
C.The Fisher Scoring Algorithm 304
List of Figures 307
List of Tables 309
List of Acronyms 310
List of Symbols 311
References 315
Subject Index 317
