Regularization of Deep or Distributed Models

正则化是建模，训练以及预测过程中的组成部分，它包括解释训练数据limitation and finiteness.

regularization strategies.

• put extra constraints on a machine learning model, such as adding restrictions on the parameter values.
• add extra terms in the objective function that can be thought of as corresponding to a soft constraint on the parameter values. If chosen
  carefully, these extra constraints and penalties can lead to improved performance on the test set.

• constraints and penalties are designed to encode specific kinds of prior knowledge.

  预测器的正则化在降低variance和增加bias之间做一个妥协。一个有效的正则是大幅度的降低variance，但并没有使得bias增大太多。