To prune, or not to prune: exploring the efficacy of pruning for model compression

Model pruning seeks to induce sparsity in a deep neural network’s various connection matrices, thereby reducing the number of nonzero-valued parameters in the model. Recent reports (Han et al., 2015; Narang et al., 2017) prune deep networks at the cost of only a marginal loss in accuracy and achieve…

Michael Zhu arXiv.org

背景

Yin Guobing 尹国冰的博客

相关工作

90年代的剪枝通过将权重置零时网络损失函数增量的二阶泰勒级数近似来实现。最近的工作中，基于数量级的权重剪枝开始流行。这种方式简单易行，并且适用于大型网络与数据集。本文通过剪去最小数量级的权重来控制模型稀疏程度。这种策略不需要人工选择权重阈值，不仅适用于CNN也可以用于LSTM。

方案

$$s_t=s_f+(s_i-s_f)\left (1- \frac{t-t_0}{n \Delta t} \right)^3$$

总结

（全文完）

TensorFlow支持剪枝，轻参阅官方文档：

Trim insignificant weights | TensorFlow Model Optimization

TensorFlow

Pruning: Keras subclassed model increased support · Issue #155 · tensorflow/model-optimization

Currently the pruning API will throw an error when a subclassed model is passed to it. Users can get around this by diving into the subclassed models and applying pruning to individual Sequential/F…

tensorflow GitHub