Why do we call test-set as a way for unbiased estimation?（为何我们说测试集是用来做无偏估计的？）

Why do we call test-set as a way for unbiased estimation?

#UnbiasedEstmator #DeepLearning #Test-set
For Deep Learning applications, we spilt the dataset into training / validation and test sets. The test set is used for unbiased estimation of the model skills (generalization errors).

A natural question then is whether or not these estimators are “good” in any sense. One measure of “good” is “unbiasedness.”

Cases in time series forecasting with DL models with MAE as the estimator

Assume the MAE (Mean Absolute Error) estimator is
$$\begin{array}{rl}\hat{\theta }& ={\mathbb{L}}_{MAE}(X_1,X_2,\cdots ,X_n)\\ & =\frac{1}{n}\sum _{i=1}^n|X_i-\widehat{X_i}|\end{array}$$
where $(X_1,X_2,\cdots )$ is the test set, and the widehat version is its estimations. If its mathematical expectations are:
$$E(\hat{\theta })=\hat{\theta }$$

Then, we call $\hat{\theta }$ is the unbiased estimator of MAE.

To prove that MAE is a unbiased estimator:

$$\begin{array}{rl}E(\hat{\theta })& =E(\frac{1}{n}\sum _{i=1}^n|X_i-\widehat{X_i}|)\\ & =\frac{1}{n}\sum _{i=1}^{n}E(|X_i-\widehat{X_i}|)\\ & =\frac{1}{n}\times n\times \hat{\theta }\\ & =\hat{\theta }\end{array}$$

Cases with MSE

MSE is also a unbiased estimator.

$$\begin{array}{rl}\hat{\theta }& ={\mathbb{L}}_{MSE}(X_1,X_2,\cdots ,X_n)\\ & =\frac{1}{n}\sum _{i=1}^n(X_i-\widehat{X_i}{)}^2\end{array}$$
$$\begin{array}{rl}E(\hat{\theta })& =E(\frac{1}{n}\sum _{i=1}^n(X_i-\widehat{X_i}{)}^2)\\ & =\frac{1}{n}E(\sum _{i=1}^n(X_i-\widehat{X_i}{)}^2)\\ & =\frac{1}{n}\sum _{i=1}^{n}E((X_i-\widehat{X_i}{)}^2)\\ & =\hat{\theta }\end{array}$$

Conclusion

To say that test set is used for unbiased estimation, it is not about the dataset itself, but about the estimator and estimated object, i.e., mean value and $\mu$, respectively.