File manager - Edit - /home/u478019808/domains/bestandroidphones.store/public_html/static/img/logo/gcs_json_media.cpython-39.pyc.tar
Back
opt/gsutil/gslib/__pycache__/gcs_json_media.cpython-39.pyc 0000644 00000046650 15025233507 0017577 0 ustar 00 a +(Wg�q � @ s� d Z ddlmZ ddlmZ ddlmZ ddlmZ ddlZddlZddlZddl Z ddl Z ddl Z ddl m Z ddl mZ dd l mZ dd lmZ dd lmZ dd lmZ dd lmZ ddlmZ ddlmZ ddlm Z ddlm Z ddl m Z ddl Z ddl m!Z! e j"�re#Z$edd� �Z%G dd� de&�Z'G dd� de&�Z(dd� Z)G d d � d e&�Z*d d � Z+G d d!� d!e j,�Z-G d"d#� d#e j,�Z.dS )$zEMedia helper functions and classes for Google Cloud Storage JSON API.� )�absolute_import)�print_function)�division)�unicode_literalsN)� http_client)�urllib)� cStringIO)� exceptions)�BadRequestException)� LazyWrapper)�ProgressCallbackWithTimeout)�DEBUGLEVEL_DUMP_REQUESTS)�SSL_TIMEOUT_SEC)�TRANSFER_BUFFER_SIZE)�UTF8)� text_util)� parse_uric C s t �d�S )Nz\d+)�re�compile� r r �#/opt/gsutil/gslib/gcs_json_media.py�<lambda>2 � r c @ s2 e Zd ZdZdd� Zedd� �Zejdd� �ZdS )�BytesTransferredContainera Container class for passing number of bytes transferred to lower layers. For resumed transfers or connection rebuilds in the middle of a transfer, we need to rebuild the connection class with how much we've transferred so far. For uploads, we don't know the total number of bytes uploaded until we've queried the server, but we need to create the connection class to pass to httplib2 before we can query the server. This container object allows us to pass a reference into Upload/DownloadCallbackConnection. c C s d| _ d S )Nr �Z-_BytesTransferredContainer__bytes_transferred��selfr r r �__init__@ s z"BytesTransferredContainer.__init__c C s | j S �Nr r r r r �bytes_transferredC s z+BytesTransferredContainer.bytes_transferredc C s || _ d S r r �r �valuer r r r G s N)�__name__� __module__� __qualname__�__doc__r �propertyr �setterr r r r r 5 s r c @ s, e Zd ZdZeddddfdd�Zdd� ZdS )�$UploadCallbackConnectionClassFactoryz�Creates a class that can override an httplib2 connection. This is used to provide progress callbacks and disable dumping the upload payload during debug statements. It can later be used to provide on-the-fly hash digestion during upload. r Nc C s( || _ || _|| _|| _|| _|| _d S r )�bytes_uploaded_container� buffer_size� total_size�progress_callback�logger�debug)r r) r* r+ r, r- r. r r r r T s z-UploadCallbackConnectionClassFactory.__init__c sH | j �| j� | j�| j�| j�| j�G � �����fdd�dtj�}|S )z/Returns a connection class that overrides send.c sj e Zd ZdZ�ZdZ� ZdZ�ZdZ dZ dZ dZ dd� Z d��fdd � Z��fd d �Zd��fd d� ZdS )zYUploadCallbackConnectionClassFactory.GetConnectionClass.<locals>.UploadCallbackConnectionz&Connection class override for uploads.FN� g �?c _ s& t |d< tjj| g|�R i |�� d S �N�timeout�r �httplib2�HTTPSConnectionWithTimeoutr �r �args�kwargsr r r r z s zbUploadCallbackConnectionClassFactory.GetConnectionClass.<locals>.UploadCallbackConnection.__init__c s� | j �d� tjr| j }n6g }| j D ]*}t|t�r>|�|� q$|�|�t�� q$d� |�}t |�}� t kr|�r|�� d| � | j dd�= t|t �r�||7 }d}| j||d� |dur�| �|� dS )z�Send the currently buffered request and clear the buffer. Appends an extra \r\n to the buffer. Args: message_body: if specified, this is appended to the request. )r r s zsend: %sN)�num_metadata_bytes)�_buffer�extend�six�PY2� isinstance�bytes�append�encoder �join�lenr r. �str�send)r � message_body�encode_chunked�items�item�msgr8 �� outer_debug� outer_loggerr r � _send_output� s& zfUploadCallbackConnectionClassFactory.GetConnectionClass.<locals>.UploadCallbackConnection._send_outputc s^ |dkr:d� dd� |D ��}|| _� tkr�r��d� n�|dkrzz"td� dd� |D ���}|| _W q� tyv Y q�0 nj|dkr�zNd� d d� |D ��}t� �|�}t |�d kr�t|d �t|d � d | _ W n ty� Y n0 | jd k�rD| j�rD| j �rD| j t | j� | _ d| _d | _d | _ � tk�rD��rD��d| j � t jj| |g|�R � d S )a� Overrides HTTPConnection.putheader. Send a request header line to the server. For example: h.putheader('Accept', 'text/html'). This override records the content encoding, length, and range of the payload. For uploads where the content-range difference does not match the content-length, progress printing will under-report progress. These headers are used to calculate a multiplier to correct the progress. For example: the content-length for gzip transport encoded data represents the compressed size of the data while the content-range difference represents the uncompressed size. Dividing the content-range difference by the content-length gives the ratio to multiply the progress by to correctly report the relative progress. Args: header: The header. *values: A set of values for the header. zcontent-encodingr/ c S s g | ] }t |��qS r �rC ��.0�vr r r � <listcomp>� r zwUploadCallbackConnectionClassFactory.GetConnectionClass.<locals>.UploadCallbackConnection.putheader.<locals>.<listcomp>z4send: Using gzip transport encoding for the request.�content-lengthc S s g | ] }t |��qS r rN rO r r r rR � r z content-rangec S s g | ] }t |��qS r rN rO r r r rR � r � r �gzipNz&send: Setting progress modifier to %s.)rA �header_encodingr r. �int� header_length� ValueError� DECIMAL_REGEX�findallrB � header_range�float� size_modifierr �HTTPSConnection� putheader)r �header�valuesr! �rangesrJ r r r` � sB � " ��zcUploadCallbackConnectionClassFactory.GetConnectionClass.<locals>.UploadCallbackConnection.putheaderr c s | j s,d| _ � r,t�� �| _| j�| jj� t|tj�rBt |�}n t|tj �rZt� |�}n|}|� | j �}|�rtjr�tj�| |� n.t|t�r�tj�| |� ntj�| |�t�� t|�}|r�||kr�||8 }d}n ||8 }d}| j�rt|| j �}| j�|� |� | j �}qjdS )a Overrides HTTPConnection.send. Args: data: string or file-like object (implements read()) of data to send. num_metadata_bytes: number of bytes that consist of metadata (headers, etc.) not representing the data being uploaded. Tr N)�processed_initial_bytesr �callback_processor�Progressr) r r= r; � text_typer � binary_type�BytesIO�read�GCS_JSON_BUFFER_SIZEr< r3 r4 rD r> r@ r rB rW r^ )r �datar8 Z full_bufferZpartial_bufferZsent_data_bytes)�outer_progress_callback�outer_total_sizer r rD � sD �� � z^UploadCallbackConnectionClassFactory.GetConnectionClass.<locals>.UploadCallbackConnection.send)NF)r )r" r# r$ r% r) rd rk re �sizerV rX r\ r^ r rM r` rD r �Zouter_buffer_sizeZ outer_bytes_uploaded_containerrK rL rm rn r r �UploadCallbackConnectionk s '?rq )r) r* r+ r, r- r. r3 r4 )r rq r rp r �GetConnectionClassb s 2z7UploadCallbackConnectionClassFactory.GetConnectionClass�r" r# r$ r% r r rr r r r r r( L s � r( c s* | j � dddtjdf� fdd� }|| _ dS )z�Wraps upload_http so we only use our custom connection_type on PUTs. POSTs are used to refresh oauth tokens, and we don't want to process the data sent in those requests. Args: upload_http: httplib2.Http instance to wrap �GETNc s. |dks|dkr|}nd }� | |||||d�S )N�PUT�POST��method�body�headers� redirections�connection_typer )�urirx ry rz r{ r| Zoverride_connection_type�Z request_origr r � NewRequest* s �z)WrapUploadHttpRequest.<locals>.NewRequest)�requestr3 �DEFAULT_MAX_REDIRECTS)� upload_httpr r r~ r �WrapUploadHttpRequest s �r� c @ s* e Zd ZdZedddfdd�Zdd� ZdS )�&DownloadCallbackConnectionClassFactorya� Creates a class that can override an httplib2 connection. This is used to provide progress callbacks, disable dumping the download payload during debug statements, and provide on-the-fly hash digestion during download. On-the-fly digestion is particularly important because httplib2 will decompress gzipped content on-the-fly, thus this class provides our only opportunity to calculate the correct hash for an object that has a gzip hash in the cloud. r Nc C s" || _ || _|| _|| _|| _d S r )r* r+ r, � digesters�bytes_downloaded_container)r r� r* r+ r, r� r r r r J s z/DownloadCallbackConnectionClassFactory.__init__c s G � fdd�dt j�}|S )z6Returns a connection class that overrides getresponse.c sB e Zd ZdZ� jZ� jZ� jZ � j Z dZ dZ dd� Zddd�ZdS ) z]DownloadCallbackConnectionClassFactory.GetConnectionClass.<locals>.DownloadCallbackConnectionz(Connection class override for downloads.FNc _ s& t |d< tjj| g|�R i |�� d S r0 r2 r5 r r r r b s zfDownloadCallbackConnectionClassFactory.GetConnectionClass.<locals>.DownloadCallbackConnection.__init__c sB t j���}|jt jt jfvr"|S |j� d� �fdd� }||_|S )a Wraps an HTTPResponse to perform callbacks and hashing. In this function, self is a DownloadCallbackConnection. Args: buffering: Unused. This function uses a local buffer. Returns: HTTPResponse object with wrapped read function. Nc s� | r | t kr td| t f ��n| p$t } �jsXd�_�jrXt�j�j��_�j��jj � � | �}t |�}�jrz�j�|� �j r��j D ]}�j | � |� q�|S )a� Overrides HTTPConnection.getresponse.read. This function only supports reads of TRANSFER_BUFFER_SIZE or smaller. Args: amt: Integer n where 0 < n <= TRANSFER_BUFFER_SIZE. This is a keyword argument to match the read function it overrides, but it is required. Returns: Data read from HTTPConnection. z7Invalid HTTP read size %s during download, expected %s.T) r r rd rm r rn re rf � outer_bytes_downloaded_containerr rB �outer_digesters�update)�amtrl � read_length�alg�Zorig_read_funcr r r rj w s0 ���� zwDownloadCallbackConnectionClassFactory.GetConnectionClass.<locals>.DownloadCallbackConnection.getresponse.<locals>.read)N)r �HTTPConnection� getresponse�status�OK�PARTIAL_CONTENTrj )r � bufferingZ orig_responserj r r� r r� f s �%ziDownloadCallbackConnectionClassFactory.GetConnectionClass.<locals>.DownloadCallbackConnection.getresponse)F)r" r# r$ r% r+ rn r� r� r, rm r� r� rd re r r� r r r r �DownloadCallbackConnectionY s r� )r3 r4 )r r� r r r rr V s Gz9DownloadCallbackConnectionClassFactory.GetConnectionClassrs r r r r r� ? s � r� c s@ dd� }| j � dddtjdf� fdd� }t�|| �| _|| _ | S )z�Overrides download request functions for an httplib2.Http object. Args: download_http: httplib2.Http.object to wrap / override. Returns: Wrapped / overridden httplib2.Http object. c s� � �fdd�| j D �} | r*t| �d d p,d} | rB| �|�||� | �|�|||�\} } | r�| �| |�r�| �|�||� | �|�|||�\} } d| _| jdk�r| �� �|| | �D ]R}|�|�||� | �|�|||�\} } | jdkr�| j �|� |�| |� �qq�| j �s$|dv �s$| jdk�r�| j �r�| jd v �r�|�r�d | v�rb| jd k�rbt � d | | ��d | v �r�| d }t |�\}}}}}|du �r�tj�||�| d < | jd k�r�|dv �r�| d | d< d| v�r�|| d< t �|| | | j| � d|v �r�|d= d|v �r |d= d|v �r"| j�s"|d= d | v �r�| d }t�| �}d|v�rP||d< |}| jdv �rhd}d}| j|||||d |jd�\} } || _nt �d| | ��n<| jdv �r�|dv �r�d| v �r�|| d< t �|| | | j| � | | fS )zlDo the actual request using the connection object. Also follow one level of redirects if necessary. c s& g | ] }|� � ��r|���|f�qS r )�inscope�depth)rP �auth��host� request_urir r rR � s �zDWrapDownloadHttpRequest.<locals>.OverrideRequest.<locals>.<listcomp>r rT Ni� )rt �HEAD�/ )�, �- �. r� i3 �locationr� z:Redirected but the response is missing a Location: header.r� z-x-permanent-redirect-urlzcontent-locationz if-none-matchzif-modified-since� authorization)r� r� rt )ry rz r{ r| z4Redirected more times than redirection_limit allows.)�� �� )�authorizations�sortedr� � _conn_request�response� _stale_digestr� �_auth_from_challenger? �follow_all_redirects�follow_redirectsr3 �RedirectMissingLocationr r �parse�urljoin� _updateCache�cache� forward_authorization_headers�copy�deepcopy� __class__�previous� RedirectLimit)r �connr� � absolute_urir� rx ry rz r{ �cachekey�authsr� r� �contentr� r� �scheme� authority�path�query�fragment� old_response�redirect_methodr r� r �OverrideRequest� s� � � � � �� � � � � �z0WrapDownloadHttpRequest.<locals>.OverrideRequestrt Nc s4 |dkr � | ||||d d�S � | |||||d�S d S )Nrv rw r )r} rx ry rz r{ r| r~ r r r s ��z+WrapDownloadHttpRequest.<locals>.NewRequest)r� r3 r� �types� MethodType�_request)� download_httpr� r r r~ r �WrapDownloadHttpRequest� s _� r� c @ s e Zd ZdZdd� ZdS )�HttpWithNoRetriesa< httplib2.Http variant that does not retry. httplib2 automatically retries requests according to httplib2.RETRIES, but in certain cases httplib2 ignores the RETRIES value and forces a retry. Because httplib2 does not handle the case where the underlying request body is a stream, a retry may cause a non-idempotent write as the stream is partially consumed and not reset before the retry occurs. Here we override _conn_request to disable retries unequivocally, so that uploads may be retried at higher layers that properly handle stream request bodies. c C s� z0t |d�r |jd u r |�� |�||||� W n� tjyF � Y n� tjyp |�� t� d|j ��Y n� tj j y� |�� � Y nv tj y� } z>d}t |d�r�t|d�d }n|j}|tjjkrЂ W Y d }~n(d }~0 tj�y |�� � Y n0 z |�� }W n& tj tjf�y6 |�� � Y nB0 d} |dk�rP|�� n|�� } t�|�}|dk�rxt�|| �} || fS )N�sock� Unable to find the server at %sr r6 r/ r� )�hasattrr� �connectr� �socketr1 �gaierror�closer3 �ServerNotFoundErrorr� �ssl�SSLError�error�getattr�errno� ECONNREFUSEDr � HTTPExceptionr� rj �Response�_decompressContent) r r� r� rx ry rz �e�errr� r� r r r r� 9 sJ � z HttpWithNoRetries._conn_requestN)r" r# r$ r% r� r r r r r� + s r� c sB e Zd ZdZ� fdd�Zedd� �Zejdd� �Zdd� Z� Z S ) �HttpWithDownloadStreama� httplib2.Http variant that only pushes bytes through a stream. httplib2 handles media by storing entire chunks of responses in memory, which is undesirable particularly when multiple instances are used during multi-threaded/multi-process copy. This class copies and then overrides some httplib2 functions to use a streaming copy approach that uses small memory buffers. Also disables httplib2 retries (for reasons stated in the HttpWithNoRetries class doc). c s* d | _ t�� | _tt| �j|i |�� d S r )�_stream�logging� getLogger�_logger�superr� r )r r6 �kwds�r� r r r r s z HttpWithDownloadStream.__init__c C s | j S r �r� r r r r �streamw s z HttpWithDownloadStream.streamc C s || _ d S r r� r r r r r� { s c C sr z0t |d�r |jd u r |�� |�||||� W n� tjyF � Y n� tjyp |�� t� d|j ��Y n� tj j y� |�� � Y nv tj y� } z>d}t |d�r�t|d�d }n|j}|tjjkrЂ W Y d }~n(d }~0 tj�y |�� � Y n0 z |�� }W n: tj tjf�yJ } z|�� � W Y d }~�n(d }~0 0 d} |dk�rn|�� t�|�}n�|dk�rL|jtjtjfv �rLd } t |d��r�|�d �} |} d} | �t�} | �r�| jd u �r�t � d ��t � | j| � | t | �7 } n�q��q�| d u�r@t!| �t!| �k�r@| j"�#t$j%d | | |� |j&d = t'| �|j&d < t�|�}n |�� } t�|�}t�(|| �} || fS ) Nr� r� r r6 r/ r� rt rI rS z5Cannot exercise HttpWithDownloadStream with no streamzlOnly got %s bytes out of content-length %s for request URI %s. Resetting content-length to match bytes read.))r� r� r� r� r� r1 r� r� r3 r� r� r� r� r� r� r� r� r r� r� r� r� r� r� � getheaderrj r r� �apitools_exceptions�InvalidUserInputErrorr � write_to_fdrB �longr� �logr� �DEBUGrI rC r� )r r� r� rx ry rz r� r� r� r� �content_lengthZ http_streamZ bytes_readZnew_datar r r r� � s� � � � �� z$HttpWithDownloadStream._conn_request) r" r# r$ r% r r& r� r' r� � __classcell__r r r� r r� e s r� )/r% � __future__r r r r r� r� r r� r� r; � six.movesr r r �apitools.base.pyr r� �gslib.cloud_apir �gslib.lazy_wrapperr �gslib.progress_callbackr �gslib.utils.constantsr r r r � gslib.utilsr r3 r �PY3rW r� rZ �objectr r( r� r� r� �Httpr� r� r r r r �<module> sH T d :
| ver. 1.4 |
Github
|
.
| PHP 8.2.28 | Generation time: 0.02 |
proxy
|
phpinfo
|
Settings