File manager - Edit - /opt/gsutil/third_party/chardet/chardet/__pycache__/utf1632prober.cpython-39.pyc
Back
a Y�d9! � @ s< d dl mZmZ ddlmZ ddlmZ G dd� de�ZdS )� )�List�Union� )� CharSetProber)� ProbingStatec s e Zd ZdZdZdZdd�� fdd� Zdd�� fdd � Zee d�d d ��Z ee d�d d ��Z e d�dd�Z e d�dd�Zed�dd�Zed�dd�Zed�dd�Zed�dd�Zee dd�dd �Zee dd �d d �Zeeef ed �d!d"�Z eed�d#d$��Z e d�d%d&�Z � Z S )'� UTF1632Proberad This class simply looks for occurrences of zero bytes, and infers whether the file is UTF16 or UTF32 (low-endian or big-endian) For instance, files looking like ( [nonzero] )+ have a good probability to be UTF32BE. Files looking like ( [nonzero] )+ may be guessed to be UTF16BE, and inversely for little-endian varieties. � g�G�z�?N)�returnc sj t � �� d| _dgd | _dgd | _tj| _g d�| _d| _ d| _ d| _ d| _ d| _ d| _| �� d S )Nr � �r r r r F)�super�__init__�position� zeros_at_mod�nonzeros_at_modr � DETECTING�_state�quad�invalid_utf16be�invalid_utf16le�invalid_utf32be�invalid_utf32le�'first_half_surrogate_pair_detected_16be�'first_half_surrogate_pair_detected_16le�reset��self�� __class__� �8/opt/gsutil/third_party/chardet/chardet/utf1632prober.pyr ) s zUTF1632Prober.__init__c sb t � �� d| _dgd | _dgd | _tj| _d| _d| _ d| _ d| _ d| _ d| _ g d�| _d S )Nr r Fr )r r r r r r r r r r r r r r r r r r r r 8 s zUTF1632Prober.resetc C s4 | � � r dS | �� rdS | �� r$dS | �� r0dS dS )Nzutf-32bezutf-32lezutf-16bezutf-16lezutf-16)�is_likely_utf32be�is_likely_utf32le�is_likely_utf16be�is_likely_utf16ler r r r � charset_nameF s zUTF1632Prober.charset_namec C s dS )N� r r r r r �languageS s zUTF1632Prober.languagec C s t d| jd �S )N� �?g @��maxr r r r r �approx_32bit_charsW s z UTF1632Prober.approx_32bit_charsc C s t d| jd �S )Nr( g @r) r r r r �approx_16bit_charsZ s z UTF1632Prober.approx_16bit_charsc C sj | � � }|| jkoh| jd | | jkoh| jd | | jkoh| jd | | jkoh| jd | | jkoh| j S �Nr r � � )r+ �MIN_CHARS_FOR_DETECTIONr �EXPECTED_RATIOr r �r Z approx_charsr r r r! ] s ����z UTF1632Prober.is_likely_utf32bec C sj | � � }|| jkoh| jd | | jkoh| jd | | jkoh| jd | | jkoh| jd | | jkoh| j S r- )r+ r0 r r1 r r r2 r r r r" g s ����z UTF1632Prober.is_likely_utf32lec C sV | � � }|| jkoT| jd | jd | | jkoT| jd | jd | | jkoT| j S )Nr r/ r r. )r, r0 r r1 r r r2 r r r r# q s ����z UTF1632Prober.is_likely_utf16bec C sV | � � }|| jkoT| jd | jd | | jkoT| jd | jd | | jkoT| j S )Nr r. r r/ )r, r0 r r1 r r r2 r r r r$ { s ����z UTF1632Prober.is_likely_utf16le)r r c C s� |d dksL|d dksL|d dkrR|d dkrRd|d krHdkrRn nd| _ |d dks�|d dks�|d dkr�|d dkr�d|d kr�dkr�n nd| _d S ) z� Validate if the quad of bytes is valid UTF-32. UTF-32 is valid in the range 0x00000000 - 0x0010FFFF excluding 0x0000D800 - 0x0000DFFF https://en.wikipedia.org/wiki/UTF-32 r r � �� r. �� Tr/ N)r r )r r r r r �validate_utf32_characters� s4 � � � ��� � � � ��� z'UTF1632Prober.validate_utf32_characters)�pairr c C s� | j sNd|d kr dkr*n nd| _ qxd|d krBdkrxn qxd| _n*d|d krfdkrrn nd| _ nd| _| js�d|d kr�dkr�n nd| _q�d|d kr�dkr�n q�d| _n*d|d kr�dkr�n nd| _nd| _d S ) a9 Validate if the pair of bytes is valid UTF-16. UTF-16 is valid in the range 0x0000 - 0xFFFF excluding 0xD800 - 0xFFFF with an exception for surrogate pairs, which must be in the range 0xD800-0xDBFF followed by 0xDC00-0xDFFF https://en.wikipedia.org/wiki/UTF-16 r4 r �� T�� r5 Fr N)r r r r )r r7 r r r �validate_utf16_characters� s z'UTF1632Prober.validate_utf16_characters)�byte_strr c C s� |D ]�}| j d }|| j|< |dkrX| �| j� | �| jdd� � | �| jdd� � |dkrt| j| d7 < n| j| d7 < | j d7 _ q| jS )Nr r/ r r. r )r r r6 r: r r �state)r r; �cZmod4r r r �feed� s zUTF1632Prober.feedc C sF | j tjtjhv r| j S | �� dkr.tj| _ n| jdkr@tj| _ | j S )Ng�������?i )r r �NOT_ME�FOUND_IT�get_confidencer r r r r r< � s zUTF1632Prober.statec C s( | � � s | �� s | �� s | �� r$dS dS )Ng333333�?g )r$ r# r" r! r r r r rA � s �����z UTF1632Prober.get_confidence) �__name__� __module__� __qualname__�__doc__r0 r1 r r �property�strr% r' �floatr+ r, �boolr! r" r# r$ r �intr6 r: r �bytes� bytearrayr r> r< rA � __classcell__r r r r r s* r N)�typingr r � charsetproberr �enumsr r r r r r �<module> s
| ver. 1.4 |
Github
|
.
| PHP 8.2.28 | Generation time: 0.02 |
proxy
|
phpinfo
|
Settings