The performance of the pruned model will have a certain degree of decline and you need to fine-tune it to improve its performance. Finetuning a pruned model is basically the same as training model from scratch, except that the hype-parameters are different, such as initial learning rate, learning rate decay type, etc.
When pruning and fine-tuning is done, an iteration of pruning is completed. In general, to achieve higher pruning rate without significant loss of performance, the model needs to be pruned several times. After every iteration of "prune-finetune", you need to make two changes to the commands before you run the next pruning:
- Modify the
--input_ckptflag to a checkpoint file generated in previous fine-tuning process. - Increase the value of
--sparsityflag to prune more in the next iteration.