o Th@s2ddlZddlZddlZddlZddlmZmZmZddlm Z m Z m Z m Z m Z mZddlZddlZddlmZddlmZddlmZddlmZmZgd ZGd d d eZd d ZddZddZddZ e!d"d"dZ#ddZ$    dHddZ%e$e%  dId!d"Z&Gd#d$d$Z'Gd%d&d&Z(Gd'd(d(Z)d)d*Z*Gd+d,d,e(Z+Gd-d.d.Z,d/"e#d0<Gd1d2d2e+Z-Gd3d4d4e-Z.Gd5d6d6e+Z/Gd7d8d8e+Z0Gd9d:d:e+Z1Gd;d<dd>e(Z3d?d@Z4e4dAe-Z5e4dBe.Z6e4dCe/Z7e4dDe1Z8e4dEe0Z9e4dFe2Z:e4dGe3Z;dS)JN)asarraydotvdot)normsolveinvqrsvd LinAlgError)get_blas_funcs)copy_if_needed)getfullargspec_no_self)scalar_search_wolfe1scalar_search_armijo) broyden1broyden2anderson linearmixing diagbroydenexcitingmixing newton_krylov BroydenFirstKrylovJacobianInverseJacobian NoConvergencec@seZdZdZdS)rz\Exception raised when nonlinear solver fails to converge within the specified `maxiter`.N)__name__ __module__ __qualname____doc__r r V/home/air/segue/gpt/backup/venv/lib/python3.10/site-packages/scipy/optimize/_nonlin.pyrsrcCst|SN)npabsolutemaxxr r r!maxnorm$r(cCs*t|}t|jtjst|tjdS|S)z:Return `x` as an array, of either floats or complex floatsdtype)rr# issubdtyper+inexactfloat64r&r r r! _as_inexact(sr/cCs(t|t|}t|d|j}||S)z;Return ndarray `x` as same array subclass and shape as `x0`__array_wrap__)r#reshapeshapegetattrr0)r'x0wrapr r r! _array_like0sr6cCs"t|s ttjSt|Sr")r#isfiniteallarrayinfrvr r r! _safe_norm7s r=z F : function(x) -> f Function whose root to find; should take and return an array-like object. xin : array_like Initial guess for the solution a iter : int, optional Number of iterations to make. If omitted (default), make as many as required to meet tolerances. verbose : bool, optional Print status to stdout on every iteration. maxiter : int, optional Maximum number of iterations to make. If more are needed to meet convergence, `NoConvergence` is raised. f_tol : float, optional Absolute tolerance (in max-norm) for the residual. If omitted, default is 6e-6. f_rtol : float, optional Relative tolerance for the residual. If omitted, not used. x_tol : float, optional Absolute minimum step size, as determined from the Jacobian approximation. If the step size is smaller than this, optimization is terminated as successful. If omitted, not used. x_rtol : float, optional Relative minimum step size. If omitted, not used. tol_norm : function(vector) -> scalar, optional Norm to use in convergence check. Default is the maximum norm. line_search : {None, 'armijo' (default), 'wolfe'}, optional Which type of a line search to use to determine the step size in the direction given by the Jacobian approximation. Defaults to 'armijo'. callback : function, optional Optional callback function. It is called on every iteration as ``callback(x, f)`` where `x` is the current solution and `f` the corresponding residual. Returns ------- sol : ndarray An array (of similar array type as `x0`) containing the final solution. Raises ------ NoConvergence When a solution was not found. ) params_basic params_extracCs|jr |jt|_dSdSr")r _doc_parts)objr r r!_set_docusrBkrylovFarmijoTc sT| durtn| } t|||| || d}tfdd}}t|tj}||}t|}t|}| | |||durQ|durJ|d}nd|j d}| durXd} n| d ur^d} | d vrft d d }d }d}d}t |D]}||||}|rnt|||}|j||d }t|dkrt d| rt||||| \}}}}nd}||}||}t|}|| || r| ||||d|d}||d|krt||}n t|t|||d}|}|rtjd|| ||ftjqr|r tt|d}| r%|j|||dkddd|d}t||fSt|S)a Find a root of a function, in a way suitable for large-scale problems. Parameters ---------- %(params_basic)s jacobian : Jacobian A Jacobian approximation: `Jacobian` object or something that `asjacobian` can transform to one. Alternatively, a string specifying which of the builtin Jacobian approximations to use: krylov, broyden1, broyden2, anderson diagbroyden, linearmixing, excitingmixing %(params_extra)s full_output : bool If true, returns a dictionary `info` containing convergence information. raise_exception : bool If True, a `NoConvergence` exception is raise if no solution is found. See Also -------- asjacobian, Jacobian Notes ----- This algorithm implements the inexact Newton method, with backtracking or full line searches. Several Jacobian approximations are available, including Krylov and Quasi-Newton methods. References ---------- .. [KIM] C. T. Kelley, "Iterative Methods for Linear and Nonlinear Equations". Society for Industrial and Applied Mathematics. (1995) https://archive.siam.org/books/kelley/fr16/ N)f_tolf_rtolx_tolx_rtoliterrcstt|Sr")r/r6flatten)zFr4r r!funcsznonlin_solve..funcrdTrDF)NrDwolfezInvalid line searchg?gH.?g?gMbP?)tolrz[Jacobian inversion yielded zero vector. This indicates a bug in the Jacobian approximation.?z%d: |F(x)| = %g; step %g z0A solution was found at the specified tolerance.z:The maximum number of iterations allowed has been reached.)rrS)nitfunstatussuccessmessage)r(TerminationConditionr/rJr# full_liker:r asjacobiansetupcopysize ValueErrorrangecheckminr_nonlin_line_searchupdater%sysstdoutwriteflushrr6 iteration) rMr4jacobianrIverbosemaxiterrErFrGrHtol_norm line_searchcallback full_outputraise_exception conditionrNr'dxFxFx_normgammaeta_max eta_tresholdetanrVrQs Fx_norm_neweta_Ainfor rLr! nonlin_solvezs-         r:0yE>{Gz?c sdg|gt|dgttdfdd fdd}|dkr.phics0t|d}||dd||S)NrF)r)abs)r{ds)rrdiffs_normr r!derphi#sz#_nonlin_line_search..derphirPr)xtolaminrD)rrR)T)rrr) rNr'rtrs search_typersminrr{phi1phi0rur ) rsrNrrrrrrr'r!rcs,       rcc@s.eZdZdZdddddefddZddZdS)rYz Termination condition for an iteration. It is terminated if - |F| < f_rtol*|F_0|, AND - |F| < f_tol AND - |dx| < x_rtol*|x|, AND - |dx| < x_tol NcCsx|dur ttjjd}|durtj}|durtj}|dur"tj}||_||_||_||_||_ ||_ d|_ d|_ dS)NgUUUUUU?r) r#finfor.epsr:rGrHrErFrrIf0_normri)selfrErFrGrHrIrr r r!__init__Js  zTerminationCondition.__init__cCs|jd7_||}||}||}|jdur||_|dkr$dS|jdur1d|j|jkSt||jkoJ||j|jkoJ||jkoJ||j|kS)NrrrS) rirrrIintrErFrGrH)rfr'rsf_normx_normdx_normr r r!rabs         zTerminationCondition.check)rrrrr(rrar r r r!rY=s   rYc@s:eZdZdZddZddZdddZd d Zd d Zd S)Jacobiana Common interface for Jacobians or Jacobian approximations. The optional methods come useful when implementing trust region etc., algorithms that often require evaluating transposes of the Jacobian. Methods ------- solve Returns J^-1 * v update Updates Jacobian to point `x` (where the function has residual `Fx`) matvec : optional Returns J * v rmatvec : optional Returns A^H * v rsolve : optional Returns A^-H * v matmat : optional Returns A * V, where V is a dense matrix with dimensions (N,K). todense : optional Form the dense Jacobian matrix. Necessary for dense trust region algorithms, and useful for testing. Attributes ---------- shape Matrix dimensions (M, N) dtype Data type of the matrix. func : callable, optional Function the Jacobian corresponds to cKsdgd}|D]\}}||vrtd||dur#t||||qt|dr0ddd}dSdS)N) rrdmatvecrmatvecrsolvematmattodenser2r+zUnknown keyword argument rcSs|dur td||S)Nz`dtype` must be None, was )r_r)rr+r]r r r! __array__sz$Jacobian.__init__..__array__NN)itemsr_setattrhasattr)rkwnamesnamevaluerr r r!rs zJacobian.__init__cCst|Sr")rrr r r!aspreconditionerszJacobian.aspreconditionerrcCtr"NotImplementedErrorrr<rQr r r!rzJacobian.solvecCdSr"r rr'rMr r r!rdrzJacobian.updatecCs>||_|j|jf|_|j|_|jjtjur|||dSdSr")rNr^r2r+ __class__r\rrdrr'rMrNr r r!r\s zJacobian.setupNr) rrrrrrrrdr\r r r r!r}s%  rc@s0eZdZdZddZeddZeddZdS) ra A simple wrapper that inverts the Jacobian using the `solve` method. .. legacy:: class See the newer, more consistent interfaces in :mod:`scipy.optimize`. Parameters ---------- jacobian : Jacobian The Jacobian to invert. Attributes ---------- shape Matrix dimensions (M, N) dtype Data type of the matrix. cCsB||_|j|_|j|_t|dr|j|_t|dr|j|_dSdS)Nr\r)rjrrrdrr\rr)rrjr r r!rs   zInverseJacobian.__init__cC|jjSr")rjr2rr r r!r2zInverseJacobian.shapecCrr")rjr+rr r r!r+rzInverseJacobian.dtypeN)rrrrrpropertyr2r+r r r r!rs rc stjjjttr StrttrStt j r\j dkr(t dt t jdjdkr>t dtfddfdddfd d dfd d jjd Stjrjdjdkrpt d tfd dfdddfdd dfdd jjd StdrtdrtdrttdtdjtdtdtdjjdStrGfdddt}|SttrttttttttdStd) zE Convert given object to one suitable for use as a Jacobian. rSzarray must have rank <= 2rrzarray must be squarec t|Sr")rr;Jr r! zasjacobian..ctj|Sr")rconjTr;rr r!rcrr")rr<rQrr r!rrcrr")rrrrrr r!rr)rrrrr+r2zmatrix must be squarecs|Sr"r r;rr r!rscsj|Sr"rrr;rr r!rscs |Sr"r rrspsolver r!rrcsj|Sr"rrrr r!rrr2r+rrrrrdr\)rrrrrdr\r+r2csLeZdZddZd fdd ZfddZd fdd Zfd d Zd S)zasjacobian..JaccSs ||_dSr"r&rr r r!rd zasjacobian..Jac.updatercs>|j}t|tjrt||Stj|r||StdNzUnknown matrix type) r' isinstancer#ndarrayrscipysparseissparser_rr<rQmrr r!rs     zasjacobian..Jac.solvecs<|j}t|tjrt||Stj|r||Stdr) r'rr#rrrrrr_rr<rrr r!r!s    zasjacobian..Jac.matveccsJ|j}t|tjrt|j|Stj |r!|j|St dr) r'rr#rrrrrrrr_rrr r!r*s   zasjacobian..Jac.rsolvecsH|j}t|tjrt|j|Stj |r |j|St dr) r'rr#rrrrrrrr_rrr r!r3s   zasjacobian..Jac.rmatvecNr)rrrrdrrrrr rr r!Jacs    r)rrrrrrrCz#Cannot convert object to a JacobianNr) rrlinalgrrrinspectisclass issubclassr#rndimr_ atleast_2drr2r+rrr3rcallablestrdictr BroydenSecondAnderson DiagBroyden LinearMixingExcitingMixingr TypeError)rrr rr!r[sf            ' r[c@s$eZdZddZddZddZdS)GenericBroydencCsjt||||||_||_t|dr1|jdur3t|}|r,dtt|d||_dSd|_dSdSdS)Nalpha?rrR)rr\last_flast_xrrrr%)rr4f0rNnormf0r r r!r\Ms zGenericBroyden.setupcCrr"rrr'rrsdfrdf_normr r r!_update[rzGenericBroyden._updatec Cs@||j}||j}|||||t|t|||_||_dSr")rrrr)rr'rrrsr r r!rd^s   zGenericBroyden.updateN)rrrr\rrdr r r r!rLs rc@seZdZdZddZeddZeddZdd Zd d Z dd dZ dddZ ddZ dddZ ddZddZddZd ddZdS)! LowRankMatrixz A matrix represented as .. math:: \alpha I + \sum_{n=0}^{n=M} c_n d_n^\dagger However, if the rank of the matrix reaches the dimension of the vectors, full matrix representation will be used thereon. cCs(||_g|_g|_||_||_d|_dSr")rcsrrzr+ collapsed)rrrzr+r r r!rqs  zLowRankMatrix.__init__c Cs\tgd|dd|g\}}}||}t||D]\}} || |} ||||j| }q|S)N)axpyscaldotcr)r zipr^) r<rrrrrrwcdar r r!_matvecys  zLowRankMatrix._matveccCs t|dkr ||Stddg|dd|g\}}|d}|tjt||jd}t|D]\}} t|D]\} } ||| f|| | 7<q6q.tjt||jd} t|D] \} } || || | <qW| |} t|| } ||} t|| D] \} }|| | | j | } qu| S)Evaluate w = M^-1 vrrrNrr*) lenr r#identityr+ enumeratezerosrrr^)r<rrrrrc0Airjrqrqcr r r!_solves$   zLowRankMatrix._solvecCs.|jdur t|j|St||j|j|jS)zEvaluate w = M vN)rr#rrrrrrrr<r r r!rs zLowRankMatrix.matveccCs:|jdurt|jj|St|t|j|j|j S)zEvaluate w = M^H vN) rr#rrrrrrrrrr r r!rs zLowRankMatrix.rmatvecrcCs,|jdur t|j|St||j|j|jS)rN)rrrrrrrrr r r!rs  zLowRankMatrix.solvecCs8|jdurt|jj|St|t|j|j|j S)zEvaluate w = M^-H vN) rrrrrrr#rrrrr r r!rs zLowRankMatrix.rsolvecCst|jdur|j|dddf|dddf7_dS|j||j|t|j|jkr8|dSdSr")rrrappendrrr^collapse)rrrr r r!rs .   zLowRankMatrix.appendNcCs|durtjd|ddd|durtjd|ddd|jdur&|jS|jtj|j|jd}t|j |j D]\}}||dddf|dddf 7}q9|S)NzJLowRankMatrix is scipy-internal code, `dtype` should only be None but was z (not handled)) stacklevelzILowRankMatrix is scipy-internal code, `copy` should only be None but was r*) warningswarnrrr#rrzr+rrrr)rr+r]Gmrrr r r!rs$ *zLowRankMatrix.__array__cCs&tj|td|_d|_d|_d|_dS)z0Collapse the low-rank matrix to a full-rank one.)r]N)r#r9r rrrrrr r r!rs zLowRankMatrix.collapsecCsH|jdurdS|dks Jt|j|kr"|jdd=|jdd=dSdS)zH Reduce the rank of the matrix by dropping all vectors. Nrrrrrrrankr r r!restart_reduces   zLowRankMatrix.restart_reducecCsN|jdurdS|dks Jt|j|kr%|jd=|jd=t|j|ksdSdS)zK Reduce the rank of the matrix by dropping oldest vectors. Nrrrr r r! simple_reduces  zLowRankMatrix.simple_reducec Cs6|jdurdS|}|dur|}n|d}|jr!t|t|jd}tdt||d}t|j}||kr6dSt|jj}t|jj}t |dd\}}t ||j }t |dd\} } } t |t | }t || j }t|D]} |dd| f|j| <|dd| f|j| <qp|j|d=|j|d=dS) a Reduce the rank of the matrix by retaining some SVD components. This corresponds to the "Broyden Rank Reduction Inverse" algorithm described in [1]_. Note that the SVD decomposition can be done by solving only a problem whose size is the effective rank of this matrix, which is viable even for large problems. Parameters ---------- max_rank : int Maximum rank of this matrix after reduction. to_retain : int, optional Number of SVD components to retain when reduction is done (ie. rank > max_rank). Default is ``max_rank - 2``. References ---------- .. [1] B.A. van der Rotten, PhD thesis, "A limited memory Broyden method to solve high-dimensional systems of nonlinear equations". Mathematisch Instituut, Universiteit Leiden, The Netherlands (2003). https://web.archive.org/web/20161022015821/http://www.math.leidenuniv.nl/scripties/Rotten.pdf NrSrreconomic)modeF) full_matrices)rrrbrr%r#r9rrrrrr rr`r]) rmax_rank to_retainrrrCDRUSWHkr r r! svd_reduces0    zLowRankMatrix.svd_reducerrr")rrrrr staticmethodrrrrrrrrrrrrr r r r!rfs"        ra alpha : float, optional Initial guess for the Jacobian is ``(-1/alpha)``. reduction_method : str or tuple, optional Method used in ensuring that the rank of the Broyden matrix stays low. Can either be a string giving the name of the method, or a tuple of the form ``(method, param1, param2, ...)`` that gives the name of the method and values for additional parameters. Methods available: - ``restart``: drop all matrix columns. Has no extra parameters. - ``simple``: drop oldest matrix column. Has no extra parameters. - ``svd``: keep only the most significant SVD components. Takes an extra parameter, ``to_retain``, which determines the number of SVD components to retain when rank reduction is done. Default is ``max_rank - 2``. max_rank : int, optional Maximum rank for the Broyden matrix. Default is infinity (i.e., no rank reduction). broyden_paramsc@sVeZdZdZdddZddZdd Zdd d Zd dZdddZ ddZ ddZ dS)ral Find a root of a function, using Broyden's first Jacobian approximation. This method is also known as "Broyden's good method". Parameters ---------- %(params_basic)s %(broyden_params)s %(params_extra)s See Also -------- root : Interface to root finding algorithms for multivariate functions. See ``method='broyden1'`` in particular. Notes ----- This algorithm implements the inverse Jacobian Quasi-Newton update .. math:: H_+ = H + (dx - H df) dx^\dagger H / ( dx^\dagger H df) which corresponds to Broyden's first Jacobian update .. math:: J_+ = J + (df - J dx) dx^\dagger / dx^\dagger dx References ---------- .. [1] B.A. van der Rotten, PhD thesis, "A limited memory Broyden method to solve high-dimensional systems of nonlinear equations". Mathematisch Instituut, Universiteit Leiden, The Netherlands (2003). https://math.leidenuniv.nl/scripties/Rotten.pdf Examples -------- The following functions define a system of nonlinear equations >>> def fun(x): ... return [x[0] + 0.5 * (x[0] - x[1])**3 - 1.0, ... 0.5 * (x[1] - x[0])**3 + x[1]] A solution can be obtained as follows. >>> from scipy import optimize >>> sol = optimize.broyden1(fun, [0, 0]) >>> sol array([0.84116396, 0.15883641]) Nrestartcst|_d_|durtj}|_t|trdn |dd|d}|df|dkr<fdd_ dS|dkrJfdd_ dS|d krXfd d_ dSt d |d ) Nr rrr c jjSr")r rr  reduce_paramsrr r!r z'BroydenFirst.__init__..simplecr#r")r rr r$r r!rr&r"cr#r")r rr r$r r!rr&zUnknown rank reduction method '') rrrr r#r:rrr_reducer_)rrreduction_methodrr r$r!rs$   zBroydenFirst.__init__cCs.t||||t|j |jd|j|_dS)Nr)rr\rrr2r+r rr r r!r\szBroydenFirst.setupcCs t|jSr")rr rr r r!rrzBroydenFirst.todensercCs>|j|}t|s||j|j|j|j|S|Sr") r rr#r7r8r\rrrN)rrrQrr r r!rs  zBroydenFirst.solvecC |j|Sr")r rrrr r r!r zBroydenFirst.matveccCr,r")r rrrrQr r r!rr.zBroydenFirst.rsolvecCr,r")r rr-r r r!rr.zBroydenFirst.rmatvecc CsD||j|}||j|}|t||} |j|| dSr")r)r rrrr rr'rrsrrrr<rrr r r!rs  zBroydenFirst._update)Nr"Nr) rrrrrr\rrrrrrr r r r!rJs 4   rc@seZdZdZddZdS)raK Find a root of a function, using Broyden's second Jacobian approximation. This method is also known as "Broyden's bad method". Parameters ---------- %(params_basic)s %(broyden_params)s %(params_extra)s See Also -------- root : Interface to root finding algorithms for multivariate functions. See ``method='broyden2'`` in particular. Notes ----- This algorithm implements the inverse Jacobian Quasi-Newton update .. math:: H_+ = H + (dx - H df) df^\dagger / ( df^\dagger df) corresponding to Broyden's second method. References ---------- .. [1] B.A. van der Rotten, PhD thesis, "A limited memory Broyden method to solve high-dimensional systems of nonlinear equations". Mathematisch Instituut, Universiteit Leiden, The Netherlands (2003). https://web.archive.org/web/20161022015821/http://www.math.leidenuniv.nl/scripties/Rotten.pdf Examples -------- The following functions define a system of nonlinear equations >>> def fun(x): ... return [x[0] + 0.5 * (x[0] - x[1])**3 - 1.0, ... 0.5 * (x[1] - x[0])**3 + x[1]] A solution can be obtained as follows. >>> from scipy import optimize >>> sol = optimize.broyden2(fun, [0, 0]) >>> sol array([0.84116365, 0.15883529]) c Cs:||}||j|}||d} |j|| dSNrS)r)r rrr0r r r!rs  zBroydenSecond._updateN)rrrrrr r r r!rs 2rc@s4eZdZdZdddZddd Zd d Zd d ZdS)ra Find a root of a function, using (extended) Anderson mixing. The Jacobian is formed by for a 'best' solution in the space spanned by last `M` vectors. As a result, only a MxM matrix inversions and MxN multiplications are required. [Ey]_ Parameters ---------- %(params_basic)s alpha : float, optional Initial guess for the Jacobian is (-1/alpha). M : float, optional Number of previous vectors to retain. Defaults to 5. w0 : float, optional Regularization parameter for numerical stability. Compared to unity, good values of the order of 0.01. %(params_extra)s See Also -------- root : Interface to root finding algorithms for multivariate functions. See ``method='anderson'`` in particular. References ---------- .. [Ey] V. Eyert, J. Comp. Phys., 124, 271 (1996). Examples -------- The following functions define a system of nonlinear equations >>> def fun(x): ... return [x[0] + 0.5 * (x[0] - x[1])**3 - 1.0, ... 0.5 * (x[1] - x[0])**3 + x[1]] A solution can be obtained as follows. >>> from scipy import optimize >>> sol = optimize.anderson(fun, [0, 0]) >>> sol array([0.84116588, 0.15883789]) NrcCs2t|||_||_g|_g|_d|_||_dSr")rrrMrsrrvw0)rrr4r3r r r!rAs  zAnderson.__init__rc Cs|j |}t|j}|dkr|Stj||jd}t|D] }t|j||||<qzt |j |}Wnt yI|jdd=|jdd=|YSwt|D]}||||j||j|j|7}qN|SNrr*) rrrsr#emptyr+r`rrrrr ) rrrQrsrzdf_frrvrr r r!rJs"       (zAnderson.solvec Cs,| |j}t|j}|dkr|Stj||jd}t|D] }t|j||||<qtj||f|jd}t|D]<}t|D]5}t|j||j||||f<||krs|j dkrs|||ft|j||j||j d|j8<q>q8t ||} t|D]} || | |j| |j| |j7}q~|S)Nrr*rS) rrrsr#r6r+r`rrr4r) rrrsrzr7rbrrrvrr r r!ras&     6  (zAnderson.matvecc Cs|jdkrdS|j||j|t|j|jkr/|jd|jdt|j|jkst|j}tj||f|jd}t |D])} t | |D]!} | | krU|j d} nd} d| t |j| |j| || | f<qIqB|t |dj 7}||_dS)Nrr*rSr)r3rsrrrpopr#rr+r`r4rtriurrr) rr'rrsrrrrzrrrwdr r r!rxs&        ( zAnderson._update)Nrr2r)rrrrrrrrr r r r!rs  F  rc@sVeZdZdZdddZddZddd Zd d Zdd d ZddZ ddZ ddZ dS)ra, Find a root of a function, using diagonal Broyden Jacobian approximation. The Jacobian approximation is derived from previous iterations, by retaining only the diagonal of Broyden matrices. .. warning:: This algorithm may be useful for specific problems, but whether it will work may depend strongly on the problem. Parameters ---------- %(params_basic)s alpha : float, optional Initial guess for the Jacobian is (-1/alpha). %(params_extra)s See Also -------- root : Interface to root finding algorithms for multivariate functions. See ``method='diagbroyden'`` in particular. Examples -------- The following functions define a system of nonlinear equations >>> def fun(x): ... return [x[0] + 0.5 * (x[0] - x[1])**3 - 1.0, ... 0.5 * (x[1] - x[0])**3 + x[1]] A solution can be obtained as follows. >>> from scipy import optimize >>> sol = optimize.diagbroyden(fun, [0, 0]) >>> sol array([0.84116403, 0.15883384]) NcCt|||_dSr"rrrrrr r r!r  zDiagBroyden.__init__cCs6t||||tj|jdfd|j|jd|_dS)Nrrr*)rr\r#fullr2rr+rrr r r!r\s&zDiagBroyden.setuprcC | |jSr"rr/r r r!rr.zDiagBroyden.solvecC | |jSr"rBr-r r r!rr.zDiagBroyden.matveccC| |jSr"rrr/r r r!rzDiagBroyden.rsolvecC| |jSr"rEr-r r r!rrFzDiagBroyden.rmatveccCst|j Sr")r#diagrrr r r!rr)zDiagBroyden.todensecCs(|j||j|||d8_dSr1rBrr r r!rs(zDiagBroyden._updater"r rrrrrr\rrrrrrr r r r!rs (   rc@sNeZdZdZdddZdddZdd Zdd d Zd d ZddZ ddZ dS)ra Find a root of a function, using a scalar Jacobian approximation. .. warning:: This algorithm may be useful for specific problems, but whether it will work may depend strongly on the problem. Parameters ---------- %(params_basic)s alpha : float, optional The Jacobian approximation is (-1/alpha). %(params_extra)s See Also -------- root : Interface to root finding algorithms for multivariate functions. See ``method='linearmixing'`` in particular. NcCr<r"r=r>r r r!rr?zLinearMixing.__init__rcCrCr"rr/r r r!rr.zLinearMixing.solvecCrAr"rJr-r r r!rr.zLinearMixing.matveccCs| t|jSr"r#rrr/r r r!rzLinearMixing.rsolvecCs| t|jSr"rKr-r r r!rrLzLinearMixing.rmatveccCstt|jdd|jS)Nr)r#rHr@r2rrr r r!rszLinearMixing.todensecCrr"r rr r r!rrzLinearMixing._updater"r) rrrrrrrrrrrr r r r!rs    rc@sVeZdZdZdddZddZdd d Zd d Zdd dZddZ ddZ ddZ dS)ra Find a root of a function, using a tuned diagonal Jacobian approximation. The Jacobian matrix is diagonal and is tuned on each iteration. .. warning:: This algorithm may be useful for specific problems, but whether it will work may depend strongly on the problem. See Also -------- root : Interface to root finding algorithms for multivariate functions. See ``method='excitingmixing'`` in particular. Parameters ---------- %(params_basic)s alpha : float, optional Initial Jacobian approximation is (-1/alpha). alphamax : float, optional The entries of the diagonal Jacobian are kept in the range ``[alpha, alphamax]``. %(params_extra)s NrRcCs t|||_||_d|_dSr")rrralphamaxbeta)rrrNr r r!r#s  zExcitingMixing.__init__cCs2t||||tj|jdf|j|jd|_dSr5)rr\r#r@r2rr+rOrr r r!r\)s"zExcitingMixing.setuprcCrCr"rOr/r r r!r-r.zExcitingMixing.solvecCrAr"rPr-r r r!r0r.zExcitingMixing.matveccCrGr"rOrr/r r r!r3rFzExcitingMixing.rsolvecCrDr"rQr-r r r!r6rFzExcitingMixing.rmatveccCstd|jS)NrM)r#rHrOrr r r!r9rFzExcitingMixing.todensecCsL||jdk}|j||j7<|j|j|<tj|jd|j|jddS)Nr)out)rrOrr#cliprN)rr'rrsrrrincrr r r!r<szExcitingMixing._update)NrRrrIr r r r!rs    rc@sHeZdZdZ  dddZdd Zd d Zdd dZddZddZ dS)ra Find a root of a function, using Krylov approximation for inverse Jacobian. This method is suitable for solving large-scale problems. Parameters ---------- %(params_basic)s rdiff : float, optional Relative step size to use in numerical differentiation. method : str or callable, optional Krylov method to use to approximate the Jacobian. Can be a string, or a function implementing the same interface as the iterative solvers in `scipy.sparse.linalg`. If a string, needs to be one of: ``'lgmres'``, ``'gmres'``, ``'bicgstab'``, ``'cgs'``, ``'minres'``, ``'tfqmr'``. The default is `scipy.sparse.linalg.lgmres`. inner_maxiter : int, optional Parameter to pass to the "inner" Krylov solver: maximum number of iterations. Iteration will stop after maxiter steps even if the specified tolerance has not been achieved. inner_M : LinearOperator or InverseJacobian Preconditioner for the inner Krylov iteration. Note that you can use also inverse Jacobians as (adaptive) preconditioners. For example, >>> from scipy.optimize import BroydenFirst, KrylovJacobian >>> from scipy.optimize import InverseJacobian >>> jac = BroydenFirst() >>> kjac = KrylovJacobian(inner_M=InverseJacobian(jac)) If the preconditioner has a method named 'update', it will be called as ``update(x, f)`` after each nonlinear step, with ``x`` giving the current point, and ``f`` the current function value. outer_k : int, optional Size of the subspace kept across LGMRES nonlinear iterations. See `scipy.sparse.linalg.lgmres` for details. inner_kwargs : kwargs Keyword parameters for the "inner" Krylov solver (defined with `method`). Parameter names must start with the `inner_` prefix which will be stripped before passing on the inner method. See, e.g., `scipy.sparse.linalg.gmres` for details. %(params_extra)s See Also -------- root : Interface to root finding algorithms for multivariate functions. See ``method='krylov'`` in particular. scipy.sparse.linalg.gmres scipy.sparse.linalg.lgmres Notes ----- This function implements a Newton-Krylov solver. The basic idea is to compute the inverse of the Jacobian with an iterative Krylov method. These methods require only evaluating the Jacobian-vector products, which are conveniently approximated by a finite difference: .. math:: J v \approx (f(x + \omega*v/|v|) - f(x)) / \omega Due to the use of iterative matrix inverses, these methods can deal with large nonlinear problems. SciPy's `scipy.sparse.linalg` module offers a selection of Krylov solvers to choose from. The default here is `lgmres`, which is a variant of restarted GMRES iteration that reuses some of the information obtained in the previous Newton steps to invert Jacobians in subsequent steps. For a review on Newton-Krylov methods, see for example [1]_, and for the LGMRES sparse inverse method, see [2]_. References ---------- .. [1] C. T. Kelley, Solving Nonlinear Equations with Newton's Method, SIAM, pp.57-83, 2003. :doi:`10.1137/1.9780898718898.ch3` .. [2] D.A. Knoll and D.E. Keyes, J. Comp. Phys. 193, 357 (2004). :doi:`10.1016/j.jcp.2003.08.010` .. [3] A.H. Baker and E.R. Jessup and T. Manteuffel, SIAM J. Matrix Anal. Appl. 26, 962 (2005). :doi:`10.1137/S0895479803422014` Examples -------- The following functions define a system of nonlinear equations >>> def fun(x): ... return [x[0] + 0.5 * x[1] - 1.0, ... 0.5 * (x[1] - x[0]) ** 2] A solution can be obtained as follows. >>> from scipy import optimize >>> sol = optimize.newton_krylov(fun, [0, 0]) >>> sol array([0.66731771, 0.66536458]) Nlgmres c Ks`||_||_ttjjjtjjjtjjjtjjj tjjj tjjj d |||_ t||jd|_|j tjjjurI||jd<d|jd<|jddnG|j tjjjtjjjtjjj fvrb|jddn.|j tjjjur||jd<d|jd<|jd g|jd d |jd d |jdd|D]\}}|dstd|||j|dd<qdS)N)bicgstabgmresrUcgsminrestfqmr)rlr3r"rrlatolrouter_kouter_vprepend_outer_vTstore_outer_AvFinner_zUnknown parameter )preconditionerrrrrrrXrYrUrZr[r\getmethod method_kw setdefaultgcrotmkr startswithr_) rrrf inner_maxiterinner_Mr^rkeyrr r r!rsD       zKrylovJacobian.__init__cCs<t|j}t|j}|jtd|td||_dS)Nr)rr4r%rromega)rmxmfr r r!_update_diff_steps z KrylovJacobian._update_diff_stepcCslt|}|dkr d|S|j|}||j|||j|}tt|s4tt|r4td|S)Nrz$Function returned non-finite results) rrnrNr4rr#r8r7r_)rr<nvscr+r r r!rs  zKrylovJacobian.matvecrcCsNd|jvr|j|j|fi|j\}}|S|j|j|fd|i|j\}}|S)Nrtol)rgrfop)rrhsrQsolr~r r r!rs  zKrylovJacobian.solvecCsD||_||_||jdurt|jdr |j||dSdSdS)Nrd)r4rrqrdrrd)rr'rr r r!rds  zKrylovJacobian.updatecCst||||||_||_tjj||_|j dur%t |j j d|_ ||jdur>t|jdr@|j|||dSdSdS)Nrr\)rr\r4rrrraslinearoperatorrurr#rr+rrqrdr)rr'rrNr r r!r\s   zKrylovJacobian.setup)NrUrVNrWr) rrrrrrqrrrdr\r r r r!rGse /  rcCst|j}|\}}}}}}} tt|t| d|} ddd| D} | r,d| } ddd| D} | r<| d} |rEtd|d} | t|| |j| d} i}| t t | |||}|j |_ t ||S) a Construct a solver wrapper with given name and Jacobian approx. It inspects the keyword arguments of ``jac.__init__``, and allows to use the same arguments in the wrapper function, in addition to the keyword arguments of `nonlin_solve` Nz, cSsg|] \}}|d|qS=r .0rr<r r r! z#_nonlin_wrapper..cSsg|] \}}|d|qSryr r{r r r!r} r~zUnexpected signature a def %(name)s(F, xin, iter=None %(kw)s, verbose=False, maxiter=None, f_tol=None, f_rtol=None, x_tol=None, x_rtol=None, tol_norm=None, line_search='armijo', callback=None, **kw): jac = %(jac)s(%(kwkw)s **kw) return nonlin_solve(F, xin, jac, iter, verbose, maxiter, f_tol, f_rtol, x_tol, x_rtol, tol_norm, line_search, callback) )rrjackwkw)_getfullargspecrlistrrjoinr_rrrdglobalsexecrrB)rr signatureargsvarargsvarkwdefaults kwonlyargs kwdefaults_kwargskw_strkwkw_strwrappernsrNr r r!_nonlin_wrappers,    rrrrrrrr) rCNFNNNNNNrDNFT)rDrr)s|      (4  -@H'`Mp@D.?K ,