Show and Tell: A Neural Image Caption Generator||谷歌图像描述实现

 

元学习论文总结||小样本学习论文总结

2017-2019年计算机视觉顶会文章收录 AAAI2017-2019 CVPR2017-2019 ECCV2018 ICCV2017-2019 ICLR2017-2019 NIPS2017-2019

 

---

 

论文：Show and Tell: A Neural Image Caption Generator

论文讲解：https://www.jianshu.com/p/3330a56f0d5e

论文实现：https://aistudio.baidu.com/?_=1554117309923#/projectdetail/51708

这里根据：https://github.com/ndscigdata/tensorflow-im2txt，在paddlepaddle上做了实现测试。

注意：一定要用tensorflow1.0.1，否则预训练模型无法匹配。

这里paddlepaddle配置tensorflow版本极其方便，赞b(￣▽￣)d

 

测试图像在：data以及data/images

 

根据上图（左到右，上到下）测试结果：

图像 COCO_val2014_000000224477.jpg 标题是:
  0) a man riding a wave on top of a surfboard . (概率=0.035672)
  1) a person riding a surf board on a wave (概率=0.016238)
  2) a man on a surfboard riding a wave . (概率=0.010146)
  图像 ep271.jpg 标题是:
  0) a woman is standing next to a horse . (概率=0.000759)
  1) a woman is standing next to a horse (概率=0.000647)
  2) a woman is standing next to a brown horse . (概率=0.000384)

 
图像 dog.jpg 标题是:
  0) a dog is eating a slice of pizza . (概率=0.000138)
  1) a dog is eating a slice of pizza on a plate . (概率=0.000047)
  2) a dog is sitting at a table with a pizza on it . (概率=0.000039)
  
图像 3.jpg 标题是:
  0) a close up of a person holding a pair of scissors (概率=0.001537)
  1) a close up of a person holding a pair of scissors . (概率=0.000223)
  2) a black and white image of a clock on a wall . (概率=0.000075)

  
图像 car.jpg 标题是:
  0) a car with a surfboard on top of it . (概率=0.000410)
  1) a car with a surfboard on top of it (概率=0.000130)
  2) a car parked on the beach with a surfboard on top of it . (概率=0.000061)
  
图像 chifan.jpg 标题是:
  0) a group of people sitting around a dinner table . (概率=0.009947)
  1) a group of people sitting around a table eating food . (概率=0.006874)
  2) a group of people sitting around a table with food . (概率=0.004062)
  
图像 huoche1.jpg 标题是:
  0) a train traveling down tracks next to a forest . (概率=0.001253)
  1) a train traveling down train tracks next to a forest . (概率=0.001045)
  2) a train on a train track with a sky background (概率=0.000953)
  
图像 huoche2.jpg 标题是:
  0) a passenger train that is pulling into a station . (概率=0.003332)
  1) a passenger train that is pulling into the station . (概率=0.001174)
  2) a train on a track near a platform (概率=0.001033)
  
图像 paobu.jpg 标题是:
  0) a woman is playing tennis on a court . (概率=0.001306)
  1) a woman is playing tennis on a court (概率=0.001043)
  2) a woman is holding a tennis racket on a court (概率=0.000733)
  
图像 plane.jpg 标题是:
  0) a large jetliner sitting on top of an airport runway . (概率=0.007427)
  1) a large jetliner sitting on top of an airport tarmac . (概率=0.007421)
  2) a large passenger jet sitting on top of an airport runway . (概率=0.004389)
  
图像 plane1.jpg 标题是:
  0) a large passenger jet flying through a blue sky . (概率=0.006770)
  1) a large jetliner flying through a blue sky . (概率=0.005401)
  2) a large passenger jet flying through a cloudy sky . (概率=0.001823)
  
图像 rabbit.jpg 标题是:
  0) a statue of a bear sitting on top of a table . (概率=0.000037)
  1) a statue of a bear sitting on top of a chair . (概率=0.000032)
  2) a statue of a bear sitting on a table . (概率=0.000028)
  
图像 sing.jpg 标题是:
  0) a woman holding a tennis racquet on a tennis court . 
  1) a woman holding a tennis racquet on a court . 
  2) a woman holding a tennis racket in her hand . 
  
图像 taolun.jpg 标题是:
  0) a group of people sitting around a table . 
  1) a group of people sitting around a table with a cake . 
  2) a group of people sitting around a table with laptops . 
  
图像 wudao.jpg 标题是:
  0) a couple of women standing next to each other . 
  1) a couple of kids that are sitting on a bed .
  2) a couple of kids that are sitting on a bed .