o 3Ih(@s@ddlmZddlZddlmZddlmZGdddeZdS) ) annotationsN) ONNXModel)FusioncsFeZdZdfdd Zdd d ZdddZdddZdddZZS) FusionGelumodelrcst|dddS)NGeluErf)super__init__)selfr __class__l/home/air/sanwanet/gpt-api/venv/lib/python3.10/site-packages/onnxruntime/quantization/fusions/fusion_gelu.pyr szFusionGelu.__init__erf_nodeonnx.NodeProtoinput_name_to_nodesdict[str, list[onnx.NodeProto]]output_name_to_nodedict[str, onnx.NodeProto]cCs@||||s||||s||||r|jdddSdS)z Interface function that tries to fuse a node sequence containing an Erf node into a single Gelu node. com.microsoftrN)fuse_1fuse_2fuse_3rset_opset_import)r rrrrrrfuses   zFusionGelu.fusereturnboolcCs|jd|vr dS||jd}t|dks|djdkrdS|d}||ds+dS|jd|vr4dS||jd}t|dksH|djdkrJdS|d}||dd|}|dur\dS|j|dd d dkrhdS|jd}|jd|jdkrydnd} ||j| kr||jd}t|dks|djdkrdS|d} || d sdS| jd} n"||d| |} | durdS|| d sdS|| jvrdS|jd} ||||| g} || | g||sdS|j | t j j d | |g| gd } d| _|j| dS)ay This pattern is from PyTorch model Fuse Gelu with Erf into one node: Pattern 1: +-------Mul(0.5)---------------------+ | | | v [root] --> Div -----> Erf --> Add --> Mul --> (B=1.4142...) (1) Pattern 2: +------------------------------------+ | | | v [root] --> Div -----> Erf --> Add --> Mul -->Mul --> (B=1.4142...) (1) (0.5) Note that constant input for Add and Mul could be first or second input: like either A=0.5 or B=0.5 is fine. rFrAddMulDivN -?MbP?delta?r nameinputsoutputsrToutputlenop_typehas_constant_input match_parentfind_constant_inputinputis_safe_to_fuse_nodesnodes_to_removeextendonnxhelper make_nodecreate_unique_node_namedomain nodes_to_addappend)r rrrchildren add_after_erf mul_after_erfdivsubgraph_inputanothermul_halfsubgraph_outputsubgraph_nodes fused_noderrrr#s^         zFusionGelu.fuse_1c Cs|jd|vr dS||jd}t|dks|djdkrdS|d}||ds+dS|jd|vr4dS||jd}t|dksH|djdkrJdS|d}||dsVdS|jd|vr_dS||jd}t|dkss|djdkrudS|d}||dd|}|durdSd} |j|d d d dkr||d d|} | durdS|| d sdS|jd} | |jvrdS|||||g} | r| | || |jdg||sdS|j | t j j d|| g|jdgd} d| _|j| dS)a& This pattern is from Keras model Fuse Gelu with Erf into one node: +------------------------------------------+ | | | v [root] --> Div -----> Erf --> Add --> Mul -->Mul (B=1.4142...) (A=1) (A=0.5) Note that constant input for Add and Mul could be first or second input: like either A=0.5 or B=0.5 is fine. rFrr r!r'r"Nr#r$r%Sqrtg@r r(rT)r-r.r/r0r1r2r3r=r4r5r6r7r8r9r:r;r<) r rrrr>r?r@mulrA sqrt_noderBrFrGrrrrzs^        zFusionGelu.fuse_2cCs|jd|vr dS||jd}t|dks|djdkrdS|d}||ds+dS|jd|vr4dS||jd}t|dksH|djdkrJdS|d}||dsVdS||dd|}|durddS|j|dd d }|dkrrdSd|} |j| } |jd|vrdS||jd}t|dks|djdkrdS|d} | jd| ks| jd| ksdS||||| g} || | jdg||sdS|j | t j j d | | g| jdgd } d | _|j| dS)a? This pattern is from TensorFlow model Fuse Gelu with Erf into one node: +----------------------------------------------+ | | | v [root] --> Mul -----> Erf --> Add --> Mul -->Mul (A=0.7071067690849304) (B=1) (B=0.5) Note that constant input for Add and Mul could be first or second input: like either A=0.5 or B=0.5 is fine. rFrr r!r'Ng`?r$r%r r(rTr,)r rrrr>r?rD first_muliroot_input_indexrBlast_mulrFrGrrrrs^      zFusionGelu.fuse_3)rr)rrrrrr)rrrrrrrr) __name__ __module__ __qualname__r rrrr __classcell__rrrrrs    WLr) __future__rr7 onnx_modelrfusionrrrrrrs