nncf: NNCF2.5 When quantizing the model, an error occurred: "RuntimeError: Could not find the bias value of the node."
I have an ONNX model that contains convolutional layers but no fully connected layers. Upon inspection with Netron, I found that if a convolutional layer is not directly followed by a BatchNormalization layer, then the convolutional layer has both weights and biases. However, if a convolutional layer is directly followed by a BatchNormalization layer, it only has weights, and the BatchNormalization layer carries the bias. This is the structure of my model. I want to quantize it to int8 using NNCF. Currently, in the get_bias_value function in the nncf/quantization/algorithms/fast_bias_correction/onnx_backend.py code, I am encountering an error that says ‘Could not find the bias value of the node’. Do I now have to add a bias of 0 to all convolutional layers that do not have a bias, in order to avoid this error during quantization?
The code for quantization is:
import torch
import numpy as np
import onnx
from torchvision import datasets
from torchvision import transforms
import nncf
model_path = "/home/fp32_mainbody_onnx_bs/const_shape_pp_main_body.onnx"
normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
val_dataset = datasets.ImageFolder(
root=f"/home/resize_images_400",
transform=transforms.Compose(
[
transforms.Resize(640),
transforms.ToTensor(),
normalize,
]
),
)
val_loader = torch.utils.data.DataLoader(val_dataset, batch_size=1, shuffle=False)
model = onnx.load(model_path)
def transform_fn(data_item):
images, _ = data_item
scale = np.array([640 / 1800, 640 / 1800], dtype='float32').reshape(1, 2)
return {input_img_name: images.numpy(),input_scale_name: scale}
calibration_dataset = nncf.Dataset(val_loader, transform_fn)
onnx_quantized_model = nncf.quantize(model, calibration_dataset, subset_size=400)
int8_model_path = f"/home/int8_nncf_quant/int8_main_body.onnx"
onnx.save(onnx_quantized_model, int8_model_path)
The error message is:
Traceback (most recent call last): File “nncf_quant_mainbody.py”, line 40, in <module> onnx_quantized_model = nncf.quantize(model, calibration_dataset, subset_size=400) File “/home/.local/lib/python3.8/site-packages/nncf/quantization/quantize_model.py”, line 93, in quantize return quantize_impl( File “/home/.local/lib/python3.8/site-packages/nncf/telemetry/decorator.py”, line 71, in wrapped retval = fn(*args, **kwargs) File “/home/.local/lib/python3.8/site-packages/nncf/onnx/quantization/quantize_model.py”, line 68, in quantize_impl quantized_model = quantization_algorithm.apply(model, dataset=calibration_dataset) File “/home/.local/lib/python3.8/site-packages/nncf/quantization/algorithms/algorithm.py”, line 58, in apply return self._apply(model, statistic_points=None, dataset=dataset) File “/home/.local/lib/python3.8/site-packages/nncf/quantization/algorithms/post_training/algorithm.py”, line 188, in _apply modified_model = algorithm.apply(modified_model, statistic_points) File “/home/.local/lib/python3.8/site-packages/nncf/quantization/algorithms/algorithm.py”, line 63, in apply return self._apply(model, statistic_points) File “/home/.local/lib/python3.8/site-packages/nncf/quantization/algorithms/fast_bias_correction/algorithm.py”, line 136, in _apply for node, bias_value in tqdm(list(node_and_bias_value), desc=“Biases correction”): File “/home/.local/lib/python3.8/site-packages/nncf/quantization/algorithms/fast_bias_correction/algorithm.py”, line 128, in <genexpr> (node, self._backend_entity.get_bias_value(node, nncf_graph, model)) File “/home/.local/lib/python3.8/site-packages/nncf/quantization/algorithms/fast_bias_correction/onnx_backend.py”, line 86, in get_bias_value return get_bias_value(node, model) File “/home/.local/lib/python3.8/site-packages/nncf/onnx/graph/node_utils.py”, line 60, in get_bias_value raise RuntimeError(“Could not find the bias value of the node”) RuntimeError: Could not find the bias value of the node
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 38 (3 by maintainers)
Commits related to this issue
- [ONNX] Add search of tensors in nodes attributes; Add pattern (#1940) Changes: Add a function to properly search tensors in ONNX model; Add fused pattern for correct quantization scheme for a model... — committed to openvinotoolkit/nncf by kshpv a year ago
- [PTQ] Support for matmul [x,y]x[y] (#1973) ### Changes Fix calculation channel axes for matmul operation [x,y]x[y] ### Reason for changes Wrong channel axes calculation ### Related tickets ... — committed to openvinotoolkit/nncf by alexsu52 a year ago
- [PTQ] Added hswish metatype and LINEAR_ARITHMETIC_ACTIVATIONS_ARITHMETIC pattern (#1990) ### Changes Added hswish metatype and LINEAR_ARITHMETIC_ACTIVATIONS_ARITHMETIC pattern ### Reason for c... — committed to openvinotoolkit/nncf by alexsu52 a year ago
thanks @alexsu52
but
nncf.quantize(model, calibration_dataset, preset=nncf.QuantizationPreset.MIXED,fast_bias_correction=False,subset_size=2000,advanced_parameters=nncf.AdvancedQuantizationParameters(activations_range_estimator_params=nncf.RangeEstimatorParametersSet.MINMAX)this code is throwing an error:module 'nncf' has no attribute 'RangeEstimatorParametersSet'as same asactivations_range_estimator_params=RangeEstimatorParametersSet.MINMAXAnd I randomly selected representative images. Below is the download link.
https://drive.google.com/file/d/1CWwR4BzJSstEow4hEkNX9DvO5lgtO5Ys/view?usp=sharing
fp32 model: https://drive.google.com/file/d/14Xh9Uj4kEjxuWmGewGyT9b9dIHGjUp8V/view?usp=drive_link
Here is the quantization code I have, which ignores the post-processing layer of quantization:
@l-bat I have uploaded the model to Google Drive. Here is the link: https://drive.google.com/file/d/14Xh9Uj4kEjxuWmGewGyT9b9dIHGjUp8V/view?usp=drive_link