o i~a@sddlZddlZddlmZmZddlmZddlmZm Z ddl Z ddl m Z m Z mZeddeDZed deDZed de DZeed d gBZed ZhdZedZiZddZGdddZGdddeZGdddeZGdddZGdddZddZ dS)N)BytesIOStringIO)Path) ascii_lettersascii_uppercase)EOF ReparseErrorspace_characterscc|]}|VqdSNencode.0itemrb/home/develop/edictos_py/central_edictos/env/lib/python3.10/site-packages/tinyhtml5/inputstream.py rccr r r rrrrr rccr r r rrrrrr>   z[ - -/:-@\[-`{-~]cKst|trt|dkrt|rtt|fi|St|tr,t|fi|Stt|dr7|dn|trCt|fi|St |fi|S)Nreadr) isinstancestrlenris_fileHTMLUnicodeInputStream read_texthasattrr9HTMLBinaryInputStream)sourcekwargsrrrHTMLInputStream&s" rDc@sZeZdZdZddZddZddZdd Zd d Zd d Z ddZ dddZ ddZ dS)r>zProvides a Unicode stream of characters to the HTMLTokenizer. This class takes care of character encoding and removing or replacing incorrect byte-sequences and also provides column and line tracking. cKs.dg|_tddf|_|||_|dS)zInitialise the HTMLInputStream. Create a normalized stream from source for use by tinyhtml5. source can be either a file-object, local filename or a string. rutf-8certainN) new_lineslookup_encodingencoding open_streamstreamreset)selfrBrCrrr__init__9s   zHTMLUnicodeInputStream.__init__cCs.d|_d|_d|_g|_d|_d|_d|_dS)Nr)chunk chunk_size chunk_offseterrorsprevious_number_linesprevious_number_columns_buffered_characterrMrrrrLIs zHTMLUnicodeInputStream.resetcCst|dr|St|S)zuProduce a file object from source. source can be either a file object, local filename or a string. r9)r@rrMrBrrrrJWsz"HTMLUnicodeInputStream.open_streamcCsZ|j}|dd|}|j|}|dd|}|dkr#|j|}||fS||d}||fS)N rr)rPcountrTrfindrU)rMoffsetrP number_lines position_linelast_line_positionposition_columnrrr _position_s   z HTMLUnicodeInputStream._positioncCs||j\}}|d|fS)z9Return (line, col) of the current position in the stream.r)rbrR)rMlinecolumnrrrpositionjs zHTMLUnicodeInputStream.positioncCs6|j|jkr |s tS|j}|j|}|d|_|S)zlRead one character from the stream or queue if available. Return EOF when EOF is reached. r)rRrQ read_chunkrrP)rMrR characterrrrrgos   z HTMLUnicodeInputStream.charactercCs||j\|_|_d|_d|_d|_|jd}|jr%|j|}d|_n|s)dSt |dkrPt |d}|dksEd|krCd krPnn |d|_|dd}t t t |D]}|jd qY|d d }|d d }||_t ||_dS)NrOri(FrrZ iizinvalid-codepointz rY T)rbrQrTrUrPrRrKr9rVr<ordrangeinvalid_unicode_refindallrSappendreplace)rMdatalast_rrrrfs0           z!HTMLUnicodeInputStream.read_chunkFcCszt||f}Wn+ty3ddd|D}|s d|}td|d}|}t||f<Ynwg} ||j|j}|durK|j|jkrJn*n| }||jkrc| |j|j|||_n| |j|jd| ssnq7d|S) aReturn a string of characters from the stream. String goes up to but does not include any character in 'characters' or EOF. 'characters' must be a container that supports the 'in' method and iteration over its characters. rOcSsg|] }dt|dqS)z\x02x)rj)rrgrrr sz6HTMLUnicodeInputStream.chars_until..^[z]+TN) characters_until_regexKeyErrorjoinrecompilematchrPrRrQendrnrf)rM charactersoppositeregexresultr|r}rrr chars_untils4      z"HTMLUnicodeInputStream.chars_untilcCsZ|tur)|jdkr||j|_|jd7_dS|jd8_|j|j|ks+JdSdS)Nrr)rrRrPrQ)rMcharrrrungets   zHTMLUnicodeInputStream.ungetN)F) __name__ __module__ __qualname____doc__rNrLrJrbrergrfrrrrrrr>1s  % .r>csXeZdZdZ   dddZfddZdd Zd d Zd d ZddZ ddZ Z S)rAzProvide a binary stream of characters to the HTMLTokenizer. This class takes care of character encoding and removing or replacing incorrect byte-sequences and also provides column and line tracking. N windows-1252cKsX|||_d|_||_||_||_||_||_||_ |j ddus&J| dS)Nir) rJ raw_streamnumber_bytes_metaoverride_encodingtransport_encodingsame_origin_parent_encodinglikely_encodingdefault_encodingdetermine_encodingrIrL)rMrBrrrrrrCrrrrNs   zHTMLBinaryInputStream.__init__cs*|jdjj}||jd|_tdS)Nrro)rI codec_info streamreaderrrKsuperrL)rMr __class__rrrLszHTMLBinaryInputStream.resetcCs0t|drt|dr|r|S|}t|S)Nr9seekable)r@rr9rrXrrrrJ s z!HTMLBinaryInputStream.open_streamcCs|df}|ddur|St|jdf}|ddur|St|jdf}|ddur,|S|df}|ddur:|St|jdf}|ddurQ|djdsQ|St|jdf}|ddur`|St|j df}|dduro|StddfS)NrFr tentativezutf-16r) detect_bomrHrrdetect_encoding_metarname startswithrr)rMrIrrrrs,         z(HTMLBinaryInputStream.determine_encodingcCs|jddks Jt|}durdS|jdvr$td}|dus"JdS||jdkr5|jddf|_dS|jd|df|_|td|jdd|)NrrFutf-16beutf-16lerErzEncoding changed from z to )rIrHrrseekrLr )rM new_encodingrrrchange_encoding8s   z%HTMLBinaryInputStream.change_encodingc Cstjdtjdtjdtjdtjdi}|jd}t|t sJdD]}| |d|}r9|j |t |Sq |j d dS) zAttempt to detect at BOM at the start of the stream. If an encoding can be determined from the BOM return the name of the encoding otherwise return None. rErrzutf-32lezutf-32be)rNr) codecsBOM_UTF8 BOM_UTF16_LE BOM_UTF16_BE BOM_UTF32_LE BOM_UTF32_BErr9r:bytesgetrrH)rMbomsstringrrIrrrrHs   z HTMLBinaryInputStream.detect_bomcCsV|j|j}t|tsJt|}|jd|}|dur)|jdvr)t d}|S)z1Report the encoding declared by the meta element.rNrrE) rr9rr:rEncodingParserr get_encodingrrH)rMbufferparserrIrrrres z*HTMLBinaryInputStream.detect_encoding_meta)NNNNr) rrrrrNrLrJrrrr __classcell__rrrrrAs (rAc@szeZdZdZddZddZddZdd Zd d Zd d Z e e eZ e ddZ e fddZddZddZddZdS) EncodingByteszBytes-like object with an associated position and various extra methods. If the position is ever greater than the string length then an exception is raised. cCst|tsJt||Sr )r:r__new__lower)clsvaluerrrr{szEncodingBytes.__new__cCs d|_dS)NrZrb)rMrrrrrNs zEncodingBytes.__init__cCs0|jd}|_|t|krt|||dSNrrbr< StopIterationrMrerrr__next__s zEncodingBytes.__next__cCs |jd|_}|||dSrrrrrrpreviousszEncodingBytes.previouscCs"|jt|kr ttd||_dSNr)rbr<rmaxrrrr set_positionszEncodingBytes.set_positioncCs&|jt|kr t|jdkr|jSdSrrrWrrr get_positions  zEncodingBytes.get_positioncCs||j|jdSr)rerWrrr current_byteszEncodingBytes.current_bytecCsR|j}|t|kr$|||d}||vr||_|S|d7}|t|ks ||_dS)zSkip past a list of characters.rNrer<rbrMr~rergrrrskips  zEncodingBytes.skipcCsR|j}|t|kr$|||d}||vr||_|S|d7}|t|ks ||_dSrrrrrr skip_untils  zEncodingBytes.skip_untilcCs(|||j}r|jt|7_|S)zLook for a sequence of bytes at the start of a string. If the bytes are found return True and advance the position to the byte after the match. Otherwise return False and leave the position alone. )rrer<)rMrrrrr match_bytesszEncodingBytes.match_bytescCs6z|||jt|d|_WdStytw)zLook for the next sequence of bytes matching a given sequence. If a match is found advance the position to the last byte of the match. rT)indexrer<rb ValueErrorr)rMrrrrjump_tos  zEncodingBytes.jump_toN)rrrrrrNrrrrpropertyrerspace_characters_bytesrrrrrrrrrss     rc@sXeZdZdZddZddZddZdd Zd d Zd d Z ddZ ddZ ddZ dS)rz@Mini parser for detecting character encoding from meta elements.cCst||_d|_dSr )rrprIrMrprrrrNs  zEncodingParser.__init__c Csd|jvrdS|j|j|j|j|j|jd}|jD]A}d}z|jdWn ty1Y|j Sw|D]\}}|j |rSz|}Wn tyRd}Ynwq6|sZ|j Sq|j S)NrprrWrrrrs zEncodingParser.handle_commentcCs|jjtvrdSd}d} |}durdS|ddkr/|ddk}|r.|dur.||_dSn?|ddkrG|d}t|}|durF||_dSn'|ddkrntt|d}|}durnt|}|durn|rl||_dS|}q ) NTFrs http-equivrs content-typecharsetscontent) rprr get_attributerIrHContentAttributeParserrparse)rM has_pragmapending_encoding attributetentative_encodingcodeccontent_parserrrrrs<      zEncodingParser.handle_metacCs |jddS)NFend_tag)handle_possible_tagrWrrrr z(EncodingParser.handle_possible_start_tagcCst|j|jddS)NTr)nextrprrWrrrrs  z&EncodingParser.handle_possible_end_tagcCs\|j}|jtvr|r||dS|t}|dkr#|dS |dur- dSq$)NTr)rprascii_letters_bytesrrrspaces_angle_bracketsr)rMrrprgrrrrs   z"EncodingParser.handle_possible_tagcCr)NrrrWrrrr5rzEncodingParser.handle_othercCs|j}|ttdgB}|dust|dksJ|dvrdSg}g} |dkr+|r+n0|tvr4|}n'|dvr?d|dfS|tvrK||n |durQdS||t |}q$|dkrj| d|dfSt ||}|}d vr t |}||krt |d|d|fS|tvr||n||qy|d krd|dfS|tvr||n |durdS|| t |}|t vrd|d|fS|tvr||n |durdS||q) z{Return a (name, value) pair for the next attribute in the stream. If no attribute is found, return None. /Nr)rNT=)rr)'"r) rprr frozensetr<ryascii_uppercase_bytesrnrrrr)rMrprgattribute_nameattribute_valuequoterrrr8sl       zEncodingParser.get_attributeN) rrrrrNrrrrrrrrrrrrrs! rc@seZdZddZddZdS)rcCst|tsJ||_dSr )r:rrprrrrrNs zContentAttributeParser.__init__cCszy|jd|jjd7_|j|jjdksWdS|jjd7_|j|jjdvrS|jj}|jjd7_|jj}|j|rP|j||jjWSWdS|jj}z|jt|j||jjWWStyy|j|dYWSwtyYdSw)Nrrr)rr)rprrerrrrr)rMr old_positionrrrrs2         zContentAttributeParser.parseN)rrrrNrrrrrrs rcCs\t|trz|d}Wn tyYdSw|dur,zt|WSty+YdSwdS)zReturn the Python codec name corresponding to an encoding. Return None if the string doesn't correspond to a valid encoding. asciiN)r:rdecodeUnicodeDecodeError webencodingslookupAttributeError)rIrrrrHs    rH)!rrziorrpathlibrrrrr constantsrr r rrrrrr{rlnon_bmp_invalid_codepointsascii_punctuation_rerwrDr>rArrrrrHrrrrs:  3Y< &