ICLR 2017会议论文。
摘要：
神经网络因为参数很多，所以很难训练。
Modern deep neural networks have a large number of parameters, making them very hard to train.
所以，分步骤训练参数。
We propose DSD, a dense-sparse-dense training ?ow, for regularizing deep neural networks and achieving better optimization performance. In the ?rst D (Dense) step, we train a dense network to learn connection weights and importance. In the S (Sparse) step, we regularize the network by pruning the unimportant connections with small weights and retraining the network given the sparsity constraint. In the ?nal D (re-Dense) step, we increase the model capacity by removing the sparsity constraint, re-initialize the pruned parameters from zero and retrain the whole dense network.
实验结果好。
Experiments show that DSD training can improve the performance for a wide range of CNNs, RNNs and LSTMs on the tasks of image classi?cation, caption generation and speech recognition
DSD:Dense-Sparse-Dense training for deep neural networks
原文地址：http://www.cnblogs.com/mengmengmiaomiao/p/7652779.html