o
\hK: @ s d Z dZdgZddlmZ ddlZddlZddlmZm Z m
Z
m Z m Z ddl
mZmZ ddlmZmZmZmZmZ d ZG d
d d eeZG d d deZdS )
zCUse the HTMLParser library to parse HTML files that aren't too bad.MITHTMLParserTreeBuilder )
HTMLParserN)CDataComment DeclarationDoctypeProcessingInstruction)EntitySubstitution
UnicodeDammit)DetectsXMLParsedAsHTMLParserRejectedMarkupHTMLHTMLTreeBuilderSTRICTz html.parserc @ s| e Zd ZdZdZdZdd Zdd Zdd Zd d d Z d d
dZ
dd Z dd Z dd Z
dd Zdd Zdd Zdd Zd S ) BeautifulSoupHTMLParserzA subclass of the Python standard library's HTMLParser class, which
listens for HTMLParser events and translates them into calls
to Beautiful Soup's tree construction API.
ignorereplacec O s: | d| j| _tj| g|R i | g | _| dS )a Constructor.
:param on_duplicate_attribute: A strategy for what to do if a
tag includes the same attribute more than once. Accepted
values are: REPLACE (replace earlier values with later
ones, the default), IGNORE (keep the earliest value
encountered), or a callable. A callable must take three
arguments: the dictionary of attributes already processed,
the name of the duplicate attribute, and the most recent value
encountered.
on_duplicate_attributeN)popREPLACEr r __init__ already_closed_empty_element_initialize_xml_detector)selfargskwargs r Z/home/air/segue/gemini/backup/venv/lib/python3.10/site-packages/bs4/builder/_htmlparser.pyr . s z BeautifulSoupHTMLParser.__init__c C s t |)N)r
)r messager r r errorJ s z BeautifulSoupHTMLParser.errorc C s | j ||dd}| | dS )zHandle an incoming empty-element tag.
This is only called when the markup looks like .
:param name: Name of the tag.
:param attrs: Dictionary of the tag's attributes.
F)handle_empty_elementN)handle_starttag
handle_endtag)r nameattrstagr r r handle_startendtagZ s z*BeautifulSoupHTMLParser.handle_startendtagTc C s i }|D ]0\}}|du rd}||v r.| j }|| jkrn|d| jfv r'|||< n |||| n|||< d}q| \} }
| jj|dd|| |
d} | r[| jr[|r[| j|dd | j | | j
du rg| | dS dS )a3 Handle an opening tag, e.g. ''
:param name: Name of the tag.
:param attrs: Dictionary of the tag's attributes.
:param handle_empty_element: True if this tag is known to be
an empty-element tag (i.e. there is not expected to be any
closing tag).
N z"")
sourceline sourceposF)check_already_closed) r IGNOREr getpossoupr" is_empty_elementr# r append _root_tag_root_tag_encountered) r r$ r% r! attr_dictkeyvalueon_dupe attrvaluer) r* r& r r r r" i s0
z'BeautifulSoupHTMLParser.handle_starttagc C s. |r|| j v r| j | dS | j| dS )zHandle a closing tag, e.g. ''
:param name: A tag name.
:param check_already_closed: True if this tag is expected to
be the closing portion of an empty-element tag,
e.g. ''.
N)r remover. r# )r r$ r+ r r r r# s z%BeautifulSoupHTMLParser.handle_endtagc C s | j | dS )z4Handle some textual data that shows up between tags.N)r. handle_datar datar r r r9 s z#BeautifulSoupHTMLParser.handle_datac C s | drt|dd}n| dr t|dd}nt|}d}|dk rN| jjdfD ]!}|s1q,z
t|g|}W q, tyM } zW Y d}~q,d}~ww |skzt|}W n t t
fyj } zW Y d}~nd}~ww |pnd}| | dS )zHandle a numeric character reference by converting it to the
corresponding Unicode character and treating it as textual
data.
:param name: Character number, possibly in hexadecimal.
x XN z windows-1252u �)
startswithintlstripr. original_encoding bytearraydecodeUnicodeDecodeErrorchr
ValueError
OverflowErrorr9 )r r$ real_namer; encodinger r r handle_charref s2
z&BeautifulSoupHTMLParser.handle_charrefc C s0 t j|}|dur
|}nd| }| | dS )zHandle a named entity reference by converting it to the
corresponding Unicode character(s) and treating it as textual
data.
:param name: Name of the entity reference.
Nz&%s)r
HTML_ENTITY_TO_CHARACTERgetr9 )r r$ characterr; r r r handle_entityref s
z(BeautifulSoupHTMLParser.handle_entityrefc C s& | j | j | | j t dS )zOHandle an HTML comment.
:param data: The text of the comment.
N)r. endDatar9 r r: r r r handle_comment s
z&BeautifulSoupHTMLParser.handle_commentc C s6 | j |tdd }| j | | j t dS )zYHandle a DOCTYPE declaration.
:param data: The text of the declaration.
zDOCTYPE N)r. rR lenr9 r r: r r r handle_decl s
z#BeautifulSoupHTMLParser.handle_declc C sN | drt}|tdd }nt}| j | j| | j| dS )z{Handle a declaration of unknown type -- probably a CDATA block.
:param data: The text of the declaration.
zCDATA[N)upperr@ r rT r r. rR r9 )r r; clsr r r unknown_decl s
z$BeautifulSoupHTMLParser.unknown_declc C s0 | j | j | | | | j t dS )z\Handle a processing instruction.
:param data: The text of the instruction.
N)r. rR r9 _document_might_be_xmlr r: r r r handle_pi s
z!BeautifulSoupHTMLParser.handle_piN)T)__name__
__module__ __qualname____doc__r, r r r r' r" r# r9 rM rQ rS rU rX rZ r r r r r $ s
7(
r c sR e Zd ZdZdZdZeZeee gZ
dZ d fdd Z d ddZ
d d
Z ZS )
r zpA Beautiful soup `TreeBuilder` that uses the `HTMLParser` parser,
found in the Python standard library.
FTNc sp t }dD ]}||v r||}|||< qtt| jdi | |p#g }|p'i }|| d|d< ||f| _dS )a Constructor.
:param parser_args: Positional arguments to pass into
the BeautifulSoupHTMLParser constructor, once it's
invoked.
:param parser_kwargs: Keyword arguments to pass into
the BeautifulSoupHTMLParser constructor, once it's
invoked.
:param kwargs: Keyword arguments for the superclass constructor.
)r Fconvert_charrefsNr )dictr superr r update parser_args)r rc
parser_kwargsr extra_parser_kwargsargr5 __class__r r r * s
z HTMLParserTreeBuilder.__init__c c s^ t |tr|dddfV dS |g}|g}||g}t|||d|d}|j|j|j|jfV dS )a Run any preliminary steps necessary to make incoming markup
acceptable to the parser.
:param markup: Some markup -- probably a bytestring.
:param user_specified_encoding: The user asked to try this encoding.
:param document_declared_encoding: The markup itself claims to be
in this encoding.
:param exclude_encodings: The user asked _not_ to try any of
these encodings.
:yield: A series of 4-tuples:
(markup, encoding, declared encoding,
has undergone character replacement)
Each 4-tuple represents a strategy for converting the
document to Unicode and parsing it. Each strategy will be tried
in turn.
NFT)known_definite_encodingsuser_encodingsis_htmlexclude_encodings)
isinstancestrr markuprC declared_html_encoding contains_replacement_characters) r ro user_specified_encodingdocument_declared_encodingrl ri rj
try_encodingsdammitr r r prepare_markupC s$
z$HTMLParserTreeBuilder.prepare_markupc
C sb | j \}}t|i |}| j|_z || | W n ty+ } zt|d}~ww g |_dS )z{Run some incoming markup through some parsing process,
populating the `BeautifulSoup` object in self.soup.
N)rc r r. feedcloseAssertionErrorr
r )r ro r r parserrL r r r rw t s
zHTMLParserTreeBuilder.feed)NN)NNN)r[ r\ r] r^ is_xml picklable
HTMLPARSERNAMEr r featuresTRACKS_LINE_NUMBERSr rv rw
__classcell__r r rg r r s
1)r^ __license____all__ html.parserr syswarnings bs4.elementr r r r r
bs4.dammitr
r bs4.builderr r
r r r r} r r r r r r s z