Deep系の論文の中で自分が読みたいor読んだものをメモ。 後半は後日更新予定。
後半その1:「モデルの圧縮」、「生成モデル」
後半その2: 画像系
過学習の防止・学習のアルゴリズム
Dropout
pdf: pdf
Srivastava, Nitish, et al. "Dropout: a simple way to prevent neural networks from overfitting." Journal of Machine Learning Research 15.1 (2014): 1929-1958.
Batch normalization
Ioffe, Sergey, and Christian Szegedy. "Batch normalization: Accelerating deep network training by reducing internal covariate shift." arXiv preprint arXiv:1502.03167 (2015).
Resnet
arxiv: [1512.03385] Deep Residual Learning for Image Recognition
He, Kaiming, et al. "Deep residual learning for image recognition." arXiv preprint arXiv:1512.03385 (2015).
www.slideshare.net
実装を比較されてます。非常にありがたい...
Layer Normalization
arxiv: [1607.06450] Layer Normalization
Ba, Jimmy Lei, Jamie Ryan Kiros, and Geoffrey E. Hinton. "Layer normalization." arXiv preprint arXiv:1607.06450 (2016).
Normalization Layers
arxiv: [1207.0580] Improving neural networks by preventing co-adaptation of feature detectors
www.slideshare.net
理論
Distilling the knowledge in a neural network
arXiv: [1503.02531] Distilling the Knowledge in a Neural Network
Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. "Distilling the knowledge in a neural network." arXiv preprint arXiv:1503.02531 (2015).
Deep neural networks are easily fooled: High confidence predictions for unrecognizable images
arXiv: [1412.1897] Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images
Nguyen, Anh, Jason Yosinski, and Jeff Clune. "Deep neural networks are easily fooled: High confidence predictions for unrecognizable images." 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2015.
画像・動画からのキャプション生成
Show, attend and tell: Neural image caption generation with visual attention
Xu, Kelvin, et al. "Show, attend and tell: Neural image caption generation with visual attention." arXiv preprint arXiv:1502.03044 2.3 (2015): 5.
Show and tell: A neural image caption generator
Vinyals, Oriol, et al. "Show and tell: A neural image caption generator." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015.
Deep visual-semantic alignments for generating image descriptions
Karpathy, Andrej, and Li Fei-Fei. "Deep visual-semantic alignments for generating image descriptions." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015.
www.slideshare.net