o 3Ih @s`ddlZddlZddlZddlZddlmZeeZddZ ddZ ddZ Gd d d Z dS) N)Conv1DcCs<|jj\}}tj||}|jjj|j_|jj|j_|S)N) weightshapetorchnnLineardataT contiguousbias)modulein_sizeout_sizelinearrh/home/air/sanwanet/gpt-api/venv/lib/python3.10/site-packages/onnxruntime/transformers/quantize_helper.py_conv1d_to_linears  rcCsNtdt|jD]}|j|}t|tr t|}||j|<q t|q dS)zsin-place This is for Dynamic Quantization, as Conv1D is not recognized by PyTorch, convert it to nn.Linear zreplace Conv1D with LinearN)loggerdebuglist_modules isinstancerrconv1d_to_linear)modelnamer rrrrrs     rcCs.t|dtjdd}td|S)Nztemp.p)rsave state_dictospathgetsizeremove)rsizerrr_get_size_of_pytorch_model's r#c@s,eZdZeejfddZedddZdS)QuantizeHelpercCsLt|tjj|tjjh|d}tdt|tdt||S)z{ Usage: model = quantize_model(model) TODO: mix of in-place and return, but results are different )dtypez'Size of full precision Torch model(MB):z"Size of quantized Torch model(MB):) rr quantizationquantize_dynamicrrrinfor#)rr%quantized_modelrrrquantize_torch_model/s z#QuantizeHelper.quantize_torch_modelFcCsddlm}ddlm}||jjdddtdtj |d||||dt j j id td |td tj |ddS) Nr)Path)r'T)parentsexist_okz&Size of full precision ONNX model(MB):rDefaultTensorType)use_external_data_format extra_optionszquantized model saved to:z!Size of quantized ONNX model(MB):)pathlibr+onnxruntime.quantizationr'parentmkdirrr(rrr onnx TensorProtoFLOAT)onnx_model_pathquantized_model_pathr/r+r'rrrquantize_onnx_model<s    z"QuantizeHelper.quantize_onnx_modelN)F)__name__ __module__ __qualname__ staticmethodrqint8r*r:rrrrr$.s  r$) loggingrr5rtransformers.modeling_utilsr getLoggerr;rrrr#r$rrrrs