Searched refs:quantization_bit_range (Results 1 – 1 of 1) sorted by relevance
| /cmsis-nn-latest/Tests/UnitTest/ |
| D | conv_settings.py | 177 …def quantize_float_data(self, data=None, quantization_bit_range=8, quantization_type="affine", tf_… argument
188 scale = (data_max - data_min) / (pow(2, quantization_bit_range) - 1)
189 zero_point = -(round(data_max * scale)) - pow(2, quantization_bit_range - 1)
190 zero_point = max(zero_point, pow(quantization_bit_range - 1) - 1)
191 zero_point = min(zero_point, -pow(quantization_bit_range - 1))
195 scale = absolute_max / (pow(2, quantization_bit_range - 1) - 1)
252 … biases, quantization_bit_range=8, quantization_type="symmetric", tf_tensor=not self.generate_bias)
|
