o vi&@sddlZddlZddlZddlmZddlZddlZddlmZddl m Z m Z ddl m Z ddlmZddlmZmZeeZ    dd eeBejBdBd eeBdBdededededededededededBdeddfddZdS)N)Path)SymbolicShapeInference)extract_raw_data_from_modelhas_external_data)ReplaceUpsampleWithResize) ONNXModel)add_pre_process_metadata&save_and_reload_model_with_shape_inferF input_modeloutput_model_pathskip_optimizationskip_onnx_shapeskip_symbolic_shape auto_mergeint_maxguess_output_rankverbosesave_as_external_dataall_tensors_to_one_fileexternal_data_locationexternal_data_size_thresholdreturnc  Ks8|dur | dd}|dusJ|dusJdtjddD}t|}d}|sCtdt|tjr5|nt |}t |||||}|rdd|j D}t |d kr|d j}|d krtt|||j |d d }|j tjd |gtj||}t|}|s |st|d}| rtj||d| | ddnt||d}t|d}zEt}||_tjj |_!t|tjrt"|rt#dt$|\}}|%t&|t&||'}n |r| r|(ddtj)||dgd}~Wnt*yt+dt+t,-Ynw|}|s\|dur0t|d}| r(tj||d| | ddnt||d}t|tjrJtt|d}tj||d| | ddt|d}tj./||t |}Wdn 1sgwY|durt|tjrz|nt |}t0|| rtj||d| | | dddSt||dS)aShape inference and model optimization, in preparation for quantization. Args: input_model: Path to the input model file or ModelProto output_model_path: Path to the output model file skip_optimization: Skip model optimization step if true. This may result in ONNX shape inference failure for some models. skip_onnx_shape: Skip ONNX shape inference. Symbolic shape inference is most effective with transformer based models. Skipping all shape inferences may reduce the effectiveness of quantization, as a tensor with unknown shape can not be quantized. skip_symbolic_shape: Skip symbolic shape inference. Symbolic shape inference is most effective with transformer based models. Skipping all shape inferences may reduce the effectiveness of quantization, as a tensor with unknown shape can not be quantized. auto_merge: For symbolic shape inference, automatically merge symbolic dims when conflict happens. int_max: For symbolic shape inference, specify the maximum value for integer to be treated as boundless for ops like slice guess_output_rank: Guess output rank to be the same as input 0 for unknown ops verbose: Logs detailed info of inference, 0: turn off, 1: warnings, 3: detailed save_as_external_data: Saving an ONNX model to external data all_tensors_to_one_file: Saving all the external data to one file external_data_location: The file location to save the external file external_data_size_threshold: The size threshold for external data Ninput_model_pathzoutput_model_path is required.z pre.quant.)prefixz&Performing symbolic shape inference...cSs g|] }|jr |jdkr|qS)zai.onnx)domain).0opsetr g/home/air/biblejyuku/back/venv/lib/python3.10/site-packages/onnxruntime/quantization/shape_inference.py ^s z%quant_pre_process..rr zsymbolic_shape_inferred.onnxTF)rrsize_thresholdconvert_attributezoptimized.onnxzModelProto has external data not loaded into memory, ORT cannot create session. Please load external data before calling this function. See https://onnx.ai/onnx/repo-docs/ExternalData.html for more information.z7session.optimized_model_external_initializers_file_namezoptimized.onnx.dataCPUExecutionProvider) providerszYONNX Runtime Model Optimization Failed! Consider rerun with option `--skip_optimization'.zmodel_input.onnxzonnx_shape_inferred.onnx)rrlocationr&r')1poptempfileTemporaryDirectoryrloggerinfo isinstanceonnx ModelProtoloadr infer_shapes opset_importlenversionrrapplyremoveextendhelper make_opsetidversion_converterconvert_versionr str save_modelsave onnxruntimeSessionOptionsoptimized_model_filepathGraphOptimizationLevelORT_ENABLE_BASICgraph_optimization_levelr ValueErrorradd_external_initializerslistSerializeToStringadd_session_config_entryInferenceSession Exceptionerror traceback format_excshape_inferenceinfer_shapes_pathr )r rrrrrrrrrrrrdeprecated_kwargs quant_tmp_dir temp_pathmodel loaded_modelai_onnx_domain opset_versionopt_model_path sess_optionexternal_namesexternal_valuessessinferred_model_pathr r r!quant_pre_processs+                   v  ra) NNFFFFr FrFFNr )loggingr,rPpathlibrr1rB&onnxruntime.tools.symbolic_shape_inferr#onnxruntime.transformers.onnx_utilsrrfusionsr onnx_modelr quant_utilsr r getLogger__name__r.r?r2boolintrar r r r!sl