[ICML 2022]How Graph Neural Networks Enhance Convolutional Neural Networks Towards Mining the Topolo

最新推荐文章于 2025-04-03 20:03:59 发布

夏莉莉iy

最新推荐文章于 2025-04-03 20:03:59 发布

阅读量695

点赞数 9

分类专栏：论文精读文章标签：深度学习机器学习人工智能计算机视觉分类 cnn 神经网络

本文链接：https://blog.youkuaiyun.com/Sherlily/article/details/144777135

版权

论文精读专栏收录该内容

152 篇文章

订阅专栏

论文网址：How Graph Neural Networks Enhance Convolutional Neural Networks Towards Mining the Topological Structures from Histology

论文代码：GitHub - yiqings/HEGnnEnhanceCnn: Graph + Image Bimodal Deep Learning for Histology Image Classification

英文是纯手打的！论文原文的summarizing and paraphrasing。可能会出现难以避免的拼写错误和语法错误，若有发现欢迎评论指正！文章偏向于笔记，谨慎食用

2.3. Integrating CNN with GNN

2.4. Experiments and Results

2.5. Discussion

3. Reference

1. 心得

（1）Workshop啦，主要是标题装不下了

（2）很短的论文正文只有四页，可以直接导航到主模型图看我中文解释一分钟搞定

（3）附录有组织图片还有一堆表的，感兴趣的可以论文里面自取，这里没有附录

2. 论文逐段精读

2.1. Abstract

①Existing methods to fuse global image-level and cell-level: MLP or transformer

oncology n.[肿瘤] 肿瘤学 histology n.组织学 colorectal adj.结肠直肠的 gastric adj.胃的；胃部的

2.2. Introduction

①They comibine CNN and GNN on training

2.3. Integrating CNN with GNN

①Overall framework:

（我不是做这个方向的，我猜测，第一行：一个人的输入是一个组织切片，输出是是否患病？第二行是一个切片分成很多patch，然后第三行每个patch经过CNN提取特征，右边是每个patch的每个核一起全部变成一个图然后用GNN来卷，和CNN一样，GNN也是每个patch得到一个图的图级表示。最后把CNN和GNN每个patch的特征concat在一起就接MLP了。感觉是个很简单的模型，毕竟也比较早期的，写得还是很易于理解的没有弯弯绕绕）

②For stained histology whole-slide images (WSIs) with 224*224 resolution, they split each of them into non-overlaped patches

③Fusion strategy:

④Each patch can be regarded as a cell graph $\mathcal{G}=(\mathcal{V},\mathcal{E},\boldsymbol{X})$ , where $\mathcal{V}$ denotes node set, $\mathcal{E}$ is edge set, each nuclei region in graph is a node $v_i$

⑤Node features are extracted by CA2.5Net

⑥Edge is constructed by pair-wise Euclidean distance between nuclei centroids:

$w_{ij}:= \begin{cases} d_c/d(v_i,v_j), \, \, \, \, \, d(v_i,v_j)\leq d_c\, \, \, \, {pixels}, \\ 0, \, \, \, \, \, \, \, \, \, \, \, \, \, \, \, \, \, \, \, \, \, \, \, \, \, \, \, \, \, \, {otherwise} \end{cases}$

where $d(v_i,v_j)$ denotes Euclidean distance, $d_c$ denotes interaction between 2 cells. Adjacency matrix $A_{ij}=w_{ij}$

⑦Hidden representation updating:

$\boldsymbol{H}_{\mathcal{G}}^{\ell}=\mathrm{ReLU}\left(\mathrm{GraphConv}\left(\boldsymbol{A},\boldsymbol{H}^{\ell-1}\right)\right)$

⑧Alignment of CNN such as DenseNet:

$\left.\boldsymbol{H}_{\mathcal{I}}^{\ell}=\mathrm{ReLU}\left(\mathrm{Conv}(\mathrm{concat}[\boldsymbol{H}_{\mathcal{I}}^{1},\ldots,\boldsymbol{H}_{\mathcal{I}}^{\ell-1}]\right)\right)$

or ResNet:

$\boldsymbol{H}_{\mathcal{I}}^{\ell}=\mathrm{ReLU}\left(\mathrm{Conv}(\boldsymbol{H}^{\ell-1})+\boldsymbol{H}^{\ell-1}\right)$

⑨Combine output of CNN $\boldsymbol{H}_{\mathcal{I}}$ and GNN $\boldsymbol{H}_{\mathcal{G}}$ by MLP:

$\boldsymbol{H}_{o}=\mathrm{Linear}(\mathrm{MLPBlock}(...(\mathrm{MLPBlock}(\boldsymbol{H}_{c}))))$

where $H_{c}=\mathrm{concat}\left[H_{\mathcal{I}},H_{\mathcal{G}}\right]$ , $MLPBlock(H)=Dropout(ReLU(Linear(H)))$

⑩Combine output of CNN $\boldsymbol{H}_{\mathcal{I}}$ and GNN $\boldsymbol{H}_{\mathcal{G}}$ by Transformer:

$H_o=\mathrm{Linear}(\text{TransBlock}(\cdots(\text{TransBlock}(H_c))))$

2.4. Experiments and Results

①Datasets: CR-MSI (binary microsatellite instability (MSI) status classification), STAD-MSI (binary microsatellite instability (MSI) status classification), and GIST-PDL1 (binary Programmed Death-Ligand 1 (PD-L1) status binary classification)

②CNN backbones: MOBILENETV3, DENSENET, and RESNET

③Performance table:

2.5. Discussion

3. Reference

Shen, Y. et al. (2022) How Graph Neural Networks Enhance Convolutional Neural Networks Towards Mining the Topological Structures from Histology, ICML Workshop on Computational Biology. Baltimore, Maryland, USA.