o `^hO@sJdZddlmZddlZddlmZddlmZm Z m Z ddl m Z ddl mZmZeejZdd Z  d@d d ZddZddZdAddZdBddZdCddZddZddZdDddZdDdd Zd!d"Zd#d$Z d%d&Z!d'd(Z"d)d*Z#d+d,Z$d-d.Z%dCd/d0Z&d1d2Z'd3d4Z(d5d6Z)dEd8d9Z*dEd:d;Z+dd?Z-dS)Fz+Functions used by least-squares algorithms.)copysignN)norm) cho_factor cho_solve LinAlgError)issparse)LinearOperatoraslinearoperatorc Cst||}|dkrtdt||}t|||d}|dkr&tdt||||}|t|| }||}||} || krI|| fS| |fS)aqFind the intersection of a line with the boundary of a trust region. This function solves the quadratic equation with respect to t ||(x + s*t)||**2 = Delta**2. Returns ------- t_neg, t_pos : tuple of float Negative and positive roots. Raises ------ ValueError If `s` is zero or `x` is not within the trust region. rz `s` is zero.z#`x` is not within the trust region.)npdot ValueErrorsqrtr) xsDeltaabcdqt1t2rX/home/air/shanriGPT/back/venv/lib/python3.10/site-packages/scipy/optimize/_lsq/common.pyintersect_trust_regions  r{Gz? c Csdd} ||} ||krt||d} |d| k} nd} | r2||| } t| |kr2| ddfSt| |}| rI| d| ||\}}| |}nd}|dusU| sa|dkratd|||d }n|}t|D]D}||ksq||kr|td|||d }| || ||\}}|dkr|}||}t|||}|||||8}t|||krnqg|| |d | } | |t| 9} | ||d fS) aSolve a trust-region problem arising in least-squares minimization. This function implements a method described by J. J. More [1]_ and used in MINPACK, but it relies on a single SVD of Jacobian instead of series of Cholesky decompositions. Before running this function, compute: ``U, s, VT = svd(J, full_matrices=False)``. Parameters ---------- n : int Number of variables. m : int Number of residuals. uf : ndarray Computed as U.T.dot(f). s : ndarray Singular values of J. V : ndarray Transpose of VT. Delta : float Radius of a trust region. initial_alpha : float, optional Initial guess for alpha, which might be available from a previous iteration. If None, determined automatically. rtol : float, optional Stopping tolerance for the root-finding procedure. Namely, the solution ``p`` will satisfy ``abs(norm(p) - Delta) < rtol * Delta``. max_iter : int, optional Maximum allowed number of iterations for the root-finding procedure. Returns ------- p : ndarray, shape (n,) Found solution of a trust-region problem. alpha : float Positive value such that (J.T*J + alpha*I)*p = -J.T*f. Sometimes called Levenberg-Marquardt parameter. n_iter : int Number of iterations made by root-finding procedure. Zero means that Gauss-Newton step was selected as the solution. References ---------- .. [1] More, J. J., "The Levenberg-Marquardt Algorithm: Implementation and Theory," Numerical Analysis, ed. G. A. Watson, Lecture Notes in Mathematics 630, Springer Verlag, pp. 105-116, 1977. cSsD|d|}t||}||}t|d|d |}||fS)zFunction of which to find zero. It is defined as "norm of regularized (by alpha) least-squares solution minus `Delta`". Refer to [1]_. r )rr sum)alphasufrrdenomp_normphi phi_primerrrphi_and_derivativejs  z2solve_lsq_trust_region..phi_and_derivativerFgNgMbP??r )EPSr rmaxranger abs)nmufrVr initial_alphartolmax_iterr&r! threshold full_rankp alpha_upperr$r% alpha_lowerr itratiorrrsolve_lsq_trust_region9sB1      r<cCsvzt|\}}t||f| }t|||dkr|dfWSWn ty(Ynw|d|d}|d|d}|d|d}|d|} |d|} t| | d||| d|d| || | | g} t| } t| t| } |t d| d| dd| dd| df}d tj |||dd t||} t | }|d d |f}|d fS) azSolve a general trust-region problem in 2 dimensions. The problem is reformulated as a 4th order algebraic equation, the solution of which is found by numpy.roots. Parameters ---------- B : ndarray, shape (2, 2) Symmetric matrix, defines a quadratic term of the function. g : ndarray, shape (2,) Defines a linear term of the function. Delta : float Radius of a trust region. Returns ------- p : ndarray, shape (2,) Found solution. newton_step : bool Whether the returned solution is the Newton step which lies within the trust region. r T)rr)rr))r)r)rr)r(axisNF) rrr r rarrayrootsrealisrealvstackrargmin)BgrRlowerr7rrrrfcoeffstvalueirrrsolve_trust_region_2ds0     6 6( rOcCsh|dkr ||}n||krdkrnnd}nd}|dkr&d|}||fS|dkr0|r0|d9}||fS)zUpdate the radius of a trust region based on the cost reduction. Returns ------- Delta : float New radius. ratio : float Ratio between actual and predicted reductions. rr)?g?g@r)ractual_reductionpredicted_reduction step_norm bound_hitr;rrrupdate_tr_radiuss  rUc Cs||}t||}|dur|t|||7}|d9}t||}|dura||}|t||7}dt||t||} |dur\|t|||7}| dt|||7} ||| fS||fS)aParameterize a multivariate quadratic function along a line. The resulting univariate quadratic function is given as follows:: f(t) = 0.5 * (s0 + s*t).T * (J.T*J + diag) * (s0 + s*t) + g.T * (s0 + s*t) Parameters ---------- J : ndarray, sparse matrix or LinearOperator shape (m, n) Jacobian matrix, affects the quadratic term. g : ndarray, shape (n,) Gradient, defines the linear term. s : ndarray, shape (n,) Direction vector of a line. diag : None or ndarray with shape (n,), optional Addition diagonal part, affects the quadratic term. If None, assumed to be 0. s0 : None or ndarray with shape (n,), optional Initial point. If None, assumed to be 0. Returns ------- a : float Coefficient for t**2. b : float Coefficient for t. c : float Free term. Returned only if `s0` is provided. Nr()r r ) JrGrdiags0vrrurrrrbuild_quadratic_1ds     r[c Csv||g}|dkrd||}||kr|krnn||t|}|||||}t|}||||fS)zMinimize a 1-D quadratic function subject to bounds. The free term `c` is 0 by default. Bounds must be finite. Returns ------- t : float Minimum point. y : float Minimum value. rg)appendr asarrayrE) rrlbubrrLextremumy min_indexrrrminimize_quadratic_1d.s     rccCs|jdkr||}t||}|dur|t|||7}n ||j}tj|ddd}|dur?|tj||ddd7}t||}d||S)aCompute values of a quadratic function arising in least squares. The function is 0.5 * s.T * (J.T * J + diag) * s + g.T * s. Parameters ---------- J : ndarray, sparse matrix or LinearOperator, shape (m, n) Jacobian matrix, affects the quadratic term. g : ndarray, shape (n,) Gradient, defines the linear term. s : ndarray, shape (k, n) or (n,) Array containing steps as rows. diag : ndarray, shape (n,), optional Addition diagonal part, affects the quadratic term. If None, assumed to be 0. Returns ------- values : ndarray with shape (k,) or float Values of the function. If `s` was 2-D, then ndarray is returned, otherwise, float is returned. r)Nr rr>r()ndimr r Tr)rVrGrrWJsrlrrrevaluate_quadraticEs      rhcCst||k||k@S)z$Check if a point lies within bounds.)r all)rr^r_rrr in_boundsosrjcCst|}||}t|}|tjtjddt||||||||||<Wdn1s9wYt|}|t||t | t fS)aCompute a min_step size required to reach a bound. The function computes a positive scalar t, such that x + s * t is on the bound. Returns ------- step : float Computed step. Non-negative value. hits : ndarray of int with shape of x Each element indicates whether a corresponding variable reaches the bound: * 0 - the bound was not hit. * -1 - the lower bound was hit. * 1 - the upper bound was hit. ignore)overN) r nonzero empty_likefillinferrstatemaximumminequalsignastypeint)rrr^r_non_zero s_non_zerostepsmin_steprrrstep_size_to_boundts      r|绽|=c Cstj|td}|dkrd|||k<d|||k<|S||}||}|tdt|}|tdt|}t||t||k@} d|| <t||t||k@} d|| <|S)aDetermine which constraints are active in a given point. The threshold is computed using `rtol` and the absolute value of the closest bound. Returns ------- active : ndarray of int with shape of x Each component shows whether the corresponding constraint is active: * 0 - a constraint is not active. * -1 - a lower bound is active. * 1 - a upper bound is active. dtyperr'r))r zeros_likerwrrr-isfiniteminimum) rr^r_r3active lower_dist upper_distlower_thresholdupper_threshold lower_active upper_activerrrfind_active_constraintss$  rc Cs|}t||||}t|d}t|d}|dkr4t||||||<t||||||<n&|||tdt||||<|||tdt||||<||k||kB}d||||||<|S)zShift a point to the interior of a feasible region. Each element of the returned vector is at least at a relative distance `rstep` from the closest bound. If ``rstep=0`` then `np.nextafter` is used. r'r)rr()copyrr rt nextafterrrr-) rr^r_rstepx_newr lower_mask upper_mask tight_boundsrrrmake_strictly_feasibles   rcCsxt|}t|}|dkt|@}||||||<d||<|dkt|@}||||||<d||<||fS)a4Compute Coleman-Li scaling vector and its derivatives. Components of a vector v are defined as follows:: | ub[i] - x[i], if g[i] < 0 and ub[i] < np.inf v[i] = | x[i] - lb[i], if g[i] > 0 and lb[i] > -np.inf | 1, otherwise According to this definition v[i] >= 0 for all i. It differs from the definition in paper [1]_ (eq. (2.2)), where the absolute value of v is used. Both definitions are equivalent down the line. Derivatives of v with respect to x take value 1, -1 or 0 depending on a case. Returns ------- v : ndarray with shape of x Scaling vector. dv : ndarray with shape of x Derivatives of v[i] with respect to x[i], diagonal elements of v's Jacobian. References ---------- .. [1] M.A. Branch, T.F. Coleman, and Y. Li, "A Subspace, Interior, and Conjugate Gradient Method for Large-Scale Bound-Constrained Minimization Problems," SIAM Journal on Scientific Computing, Vol. 21, Number 1, pp 1-23, 1999. rr'r))r ones_likerr)rrGr^r_rYdvmaskrrrCL_scaling_vectors  rc CsFt|||r |t|fSt|}t|}|}tj|td}||@}t||d||||||<||||k||<||@}t||d||||||<||||k||<||@}||}t ||||d||} ||t| d||| ||<| ||k||<t|} d| |<|| fS)z3Compute reflective transformation and its gradient.r~r r') rjr rrrrboolrrr remainder) rar^r_ lb_finite ub_finiter g_negativerrrLrGrrrreflective_transformations(    $ $ $ rc CstddddddddS)Nz${:^15}{:^15}{:^15}{:^15}{:^15}{:^15} Iterationz Total nfevCostCost reduction Step norm Optimalityprintformatrrrrprint_header_nonlinear!s  rcCsX|durd}n|d}|durd}n|d}t|d|d|d|||ddSNz z^15.2ez^15z^15.4er) iterationnfevcostcost_reductionrS optimalityrrrprint_iteration_nonlinear's,rcCstdddddddS)Nz{:^15}{:^15}{:^15}{:^15}{:^15}rrrrrrrrrrprint_header_linear6s  rcCsR|durd}n|d}|durd}n|d}t|d|d|||ddSrr)rrrrSrrrrprint_iteration_linear<s&rcCs t|tr ||S|j|S)z4Compute gradient of the least-squares cost function.) isinstancerrmatvecrer )rVrJrrr compute_gradNs   rcCsnt|rt|djddd}n tj|dddd}|dur+d||dk<nt||}d||fS)z5Compute variables scale based on the Jacobian matrix.r rr>r(Nr))rr r]powerrravelrr)rV scale_inv_old scale_invrrrcompute_jac_scaleVs"  rcDtfdd}fdd}fdd}tj|||dS)z#Return diag(d) J as LinearOperator.c|SN)matvecrrVrrrriz(left_multiplied_operator..matveccsddtjf|Sr)r newaxismatmatXrrrrlz(left_multiplied_operator..matmatcs|Sr)rrrrrrrosz)left_multiplied_operator..rmatvecrrrr rshaperVrrrrrrrleft_multiplied_operatore rcr)z#Return J diag(d) as LinearOperator.cst|Sr)rr rrrrrrzsz)right_multiplied_operator..matveccs|ddtjfSr)rr rrrrrr}rz)right_multiplied_operator..matmatcrrrrrrrrrz*right_multiplied_operator..rmatvecrrrrrrright_multiplied_operatorvrrcsFtj\}fdd}fdd}t||f||dS)zReturn a matrix arising in regularized least squares as LinearOperator. The matrix is [ J ] [ D ] where D is diagonal matrix with elements from `diag`. cst||fSr)r hstackrr)rVrWrrrsz(regularized_lsq_operator..matveccs*|d}|d}||Srr)rx1x2rVrWr/rrrs  z)regularized_lsq_operator..rmatvec)rr)r rr)rVrWr.rrrrrregularized_lsq_operators  rTcCs`|r t|ts |}t|r|j|j|jdd9_|St|tr*t||}|S||9}|S)zhCompute J diag(d). If `copy` is False, `J` is modified in place (unless being LinearOperator). clip)mode)rrrrdatatakeindicesrrVrrrrrright_multiplys rcCsr|r t|ts |}t|r |jt|t|j9_|St|tr,t ||}|S||ddtj f9}|S)zhCompute diag(d) J. If `copy` is False, `J` is modified in place (unless being LinearOperator). N) rrrrrr repeatdiffindptrrrrrrr left_multiplys rc CsD|||ko |dk}||||k}|r|rdS|rdS|r dSdS)z8Check termination condition for nonlinear least squares.rPr rNr) dFFdx_normx_normr;ftolxtolftol_satisfiedxtol_satisfiedrrrcheck_terminationsrcCsR|dd|d|d}t||tk<|dC}||d|9}t||dd|fS)z`Scale Jacobian and residuals for a robust loss function. Arrays are modified in place. r)r r(F)r)r*r)rVrJrhoJ_scalerrrscale_for_robust_loss_functions  r)Nrr)NN)rr)r})T).__doc__mathrnumpyr numpy.linalgr scipy.linalgrrr scipy.sparserscipy.sparse.linalgrr finfofloatepsr*rr<rOrUr[rcrhrjr|rrrrrrrrrrrrrrrrrrrrrsH    ' r3  3 *  ',"