The following are some tips for QAT.
- Keras Model
-
For Keras models, please set
backend.set_learning_phase(1)
before creating the float train graph, and setbackend.set_learning_phase(0)
before creating the float evaluation graph. Moreover,backend.set_learning_phase()
should be called afterbackend.clear_session()
. Tensorflow1.x QAT APIs are designed for Tensorflow native training APIs. Using Kerasmodel.fit()
APIs in QAT may lead to some "nodes not executed" issues. It is recommended to use QAT APIs in Tensorflow2 quantization tool with Keras APIs. - Dropout
- Experiments shows that QAT works better without dropout ops.
This tool does not support finetuning with dropouts at the moment and they
should be removed or disabled before running QAT. This can be done by
setting
is_training=false
when using tf.layers or calltf.keras.backend.set_learning_phase(0)
when using tf.keras.layers. - Hyper-parameter
- QAT is like float model training/finetuning, so the techniques for float model training/finetuning are also needed. The optimizer type and the learning rate curve are some important parameters to tune.