o "`^h2 @sxddlZddlZddlZddlZddlZejddkr4ddlmZddlm Z m Z ddl m Z m Z mZnddlmZddlm Z ddlm Z m Z ddl m Z mZz ddlmZd ZWneykddlmZd ZYnwgd Zd Zd ZdZdZdZdZedZesedZesdZeej edZ!ze!"Wn e#yYnwdZ$zhej%ej&e'dZ(z*ej e(dZ)ddl*Z*e*+e)dZ,zdde,-DZ.We,/[,ne,/[,wWn/e#y ej e(dZ)e+e)Z,dde,-DZ.Wdn 1swYYnwWn e#ye$gZ.YnwddZ0dd Z1d0d!d"Z2d#d$Z3  d1d,d-Z4d.d/Z5dS)2N) LWPCookieJar)Requesturlopen) quote_plusurlparseparse_qs)r)rr) BeautifulSoupTF)searchluckyget_random_user_agentget_tbszhttps://www.google.%(tld)s/zuhttps://www.google.%(tld)s/search?hl=%(lang)s&q=%(query)s&btnG=Google+Search&tbs=%(tbs)s&safe=%(safe)s&cr=%(country)szrhttps://www.google.%(tld)s/search?hl=%(lang)s&q=%(query)s&start=%(start)d&tbs=%(tbs)s&safe=%(safe)s&cr=%(country)szhttps://www.google.%(tld)s/search?hl=%(lang)s&q=%(query)s&num=%(num)d&btnG=Google+Search&tbs=%(tbs)s&safe=%(safe)s&cr=%(country)sz~https://www.google.%(tld)s/search?hl=%(lang)s&q=%(query)s&num=%(num)d&start=%(start)d&tbs=%(tbs)s&safe=%(safe)s&cr=%(country)s)hlqnumbtnGstarttbssafecrHOMEUSERHOME.z.google-cookiez2Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0)zuser_agents.txt.gzrbcCg|]}|qSstrip.0_rrS/home/air/shanriGPT/back/venv/lib/python3.10/site-packages/googlesearch/__init__.py nr"zuser_agents.txtcCrrrrrrr!r"ur#cCs ttS)za Get a random user agent string. :rtype: str :return: Random user agent string. )randomchoiceuser_agents_listrrrr!r {s r cCs|d}|d}dtS)z Helper function to format the tbs parameter. :param datetime.date from_date: Python date object. :param datetime.date to_date: Python date object. :rtype: str :return: Dates encoded in tbs format. z%m/%d/%Yz-cdr:1,cd_min:%(from_date)s,cd_max:%(to_date)s)strftimevars) from_dateto_daterrr!r s  r cCs|durt}t|}|d|t||rt|}n t}t||d}t||| }| zt W|St yEY|Sw)aL Request the given URL and return the response page, using the cookie jar. :param str url: URL to retrieve. :param str user_agent: User agent for the HTTP requests. Use None for the default. :param bool verify_ssl: Verify the SSL certificate to prevent traffic interception attacks. Defaults to True. :rtype: str :return: Web page retrieved for the given URL. :raises IOError: An exception is raised on error. :raises urllib2.URLError: An exception is raised on error. :raises urllib2.HTTPError: An exception is raised on error. Nz User-Agent)context) USER_AGENTr add_header cookie_jaradd_cookie_headerrssl_create_unverified_contextextract_cookiesreadclosesave Exception)url user_agent verify_sslrequestresponser+htmlrrr!get_pages&       r=cCshz)|drt|d}t|jdd}t|d}|jr$d|jvr'|WSWdSWdSty3YdSw)Nz/url?httprrgoogle) startswithrrquerynetlocr6)linkorrr! filter_results    rEcomen0off @c  cst} d}t|}| si} tD] }|| vrtd|qttt| | |r9|dkr3tt}nt t}n|dkrCt t}nt t}|rO||kr|}| D]\}}t|}t|}|d||f}qUt |t|| | }tr}t|d}nt|}z |jddd}Wnty|jd d}|r||d}Ynw|D]7}z|d }Wn tyYqwt|}|sqt|}|| vrq| ||V|d 7}|r||krd Sq||krd S||7}|dkrtt}nt t}|rO||ksOd Sd S) a Search the given query string using Google. :param str query: Query string. Must NOT be url-encoded. :param str tld: Top level domain. :param str lang: Language. :param str tbs: Time limits (i.e "qdr:h" => last hour, "qdr:d" => last 24 hours, "qdr:m" => last month). :param str safe: Safe search. :param int num: Number of results per page. :param int start: First result to retrieve. :param int stop: Last result to retrieve. Use None to keep searching forever. :param float pause: Lapse to wait between HTTP requests. A lapse too long will make the search slow, but a lapse too short may cause Google to block your IP. Your mileage may vary! :param str country: Country or region to focus the search on. Similar to changing the TLD, but does not yield exactly the same results. Only Google knows why... :param dict extra_params: A dictionary of extra HTTP GET parameters, which must be URL encoded. For example if you don't want Google to filter similar results you can set the extra_params to {'filter': '0'} which will append '&filter=0' to every query. :param str user_agent: User agent for the HTTP requests. Use None for the default. :param bool verify_ssl: Verify the SSL certificate to prevent traffic interception attacks. Defaults to True. :rtype: generator of str :return: Generator (iterator) that yields found URLs. If the stop parameter is None the iterator will loop forever. rzQGET parameter "%s" is overlapping with the built-in GET parameterrJz&%s=%sz html.parserr )idagbarhrefN)setrurl_parameterskeys ValueErrorr=url_homer( url_next_pageurl_next_page_num url_searchurl_search_numitemstimesleepis_bs4r findfindAllAttributeErrorclearKeyErrorrEhashadd)rAtldlangrrrrstoppausecountry extra_paramsr8r9hashescount builtin_paramr7 last_countkvr<soupanchorsrOrNrChrrr!r s%                r cOstt|i|S)z Shortcut to single-item search. Same arguments as the main search function, but the return value changes. :rtype: str :return: URL found by Google. )nextr )argskwargsrrr!r os r )NT) rFrGrHrIrJrNrKrLNNT)6osr$sysr\r0 version_infohttp.cookiejarrurllib.requestrr urllib.parserrr cookieliburlliburllib2bs4r r^ ImportError__all__rVrYrWrZrXrSgetenv home_folderpathjoinr.loadr6r,abspathsplit__file__install_folderuser_agents_filegzipopenfp readlinesr&r4r r r=rEr r rrrr!s               '