深度学习的数学：深度学习和卷积神经网络读书笔记

本文链接：https://blog.youkuaiyun.com/dashuniuniu/article/details/103334891

本文探讨了神经网络如何通过变换数据空间而不破坏其基本拓扑特性，重点介绍了tanh层的同胚性质。文章引用《Neural Networks, Manifolds, and Topology》的观点，解释了神经网络每一层如何拉伸和挤压空间，同时保持数据的连续性和连通性。深入讨论了CNN在图形处理中的应用，包括卷积、池化和全连接层的概念，以及它们如何利用局部信息提升训练效率。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

业余民科，垃圾内容勿看

神经网络与拓扑

在CNN的读书笔记之前，这里先记录一下神经网络与拓扑的关系，源于脑洞文章《Neural Networks, Manifolds, and Topology》，只要你站的角度够高，可以直击问题的本质。

With each layer, the network transforms the data, creating a new representation. We can look at the data in each of these representations and how the network classfies them. - 《Neural Networks, Manifolds, and Topology》

A $t a n h$ layer $t a n h$ (W $x$ + $b$ ) consists of:

A linear transformation by the “weight” matrix $W$
A translation by the vector $b$
Point-wise application of tanh

Each layer streches and squishes space, but it never cuts, breaks, or folds it. Intuitively, we can that it preserves topological properties. - 《Neural Networks, Manifolds, and Topology》

博客作者证明了 $t a n h$ (and $s i g m o i d$ and $s o f t p l u s$ but not ReLU)的layer具有homeomorphism（同胚）。