One Hat Cyber Team
Your IP :
216.73.216.183
Server IP :
23.137.84.82
Server :
Linux srv25.usacloudserver.us 5.14.0-570.39.1.el9_6.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 4 05:08:52 EDT 2025 x86_64
Server Software :
LiteSpeed
PHP Version :
8.1.33
Buat File
|
Buat Folder
Eksekusi
Dir :
~
/
lib64
/
python3.9
/
site-packages
/
borg
/
__pycache__
/
Edit File:
remote.cpython-39.pyc
a HZ�h�� � @ s� d dl Z d dlZd dlZd dlZd dlZd dlZd dlZd dlZd dlZd dl Z d dl Z d dl Z d dl Z d dl Z d dlZd dlmZmZ ddlmZ ddlmZ ddlT ddlmZmZ ddlmZ dd lmZ dd lm Z dd lm Z dd lm Z dd lm Z ddlm Z ddlm!Z!m"Z" ddl#m$Z$m%Z% ddlm&Z& ddl'm(Z( ddl)m*Z*m+Z+ ddl,m-Z- ddl.m/Z/ e$e0�Z#dZ1e*e�Z2d\Z3Z4Z5Z6dZ7dZ8dd� Z9G d d � d e�Z:G d d � d e:�Z;G d d!� d!e�Z<G d"d#� d#e�Z=G d$d%� d%e�Z>G d&d'� d'e�Z?d(d)d*d*d*d+d,d-d-d.d*d*d/d0d*d1d2�Z@d3d4� ZAG d5d6� d6�ZBG d7d8� d8�ZCd9d:� ZDG d;d<� d<�ZEd=d>� ZFG d?d@� d@�ZGG dAdB� dBeG�ZHdCddddCdD�dEdF�ZIdS )G� N)�Popen�PIPE� )� __version__)� Compressor)�*)�Error�IntegrityError)� bin_to_hex)� get_base_dir)�get_limited_unpacker)�replace_placeholders)�sysinfo)�format_file_size)� safe_unlink)�prepare_subprocess_env� ignore_sigint)� create_logger� setup_logging)�msgpack)� Repository)� parse_version�format_version)�xxh64)�EfficientCollectionQueue� )� i� m� a� r�d 皙�����?c C sJ t |� }}|rFt�| |�}||8 }|s*qF||d� }t�|d � q |S )z;os.write wrapper so we do not lose data for partial writes.Ng��&� .>)�len�os�write�time�sleep)�fd�data�amountZ remaining�count� r+ �1/usr/lib64/python3.9/site-packages/borg/remote.py�os_write0 s r- c @ s e Zd ZdZdS )�ConnectionClosedz Connection closed by remote hostN��__name__� __module__� __qualname__�__doc__r+ r+ r+ r, r. C s r. c @ s e Zd ZdZdS )�ConnectionClosedWithHintz$Connection closed by remote host. {}Nr/ r+ r+ r+ r, r4 G s r4 c @ s e Zd ZdZdS )�PathNotAllowedz Repository path not allowed: {}Nr/ r+ r+ r+ r, r5 K s r5 c @ s e Zd ZdZdS )�InvalidRPCMethodzRPC method {} is not validNr/ r+ r+ r+ r, r6 O s r6 c @ s e Zd ZdZdS )�!UnexpectedRPCDataFormatFromClientz4Borg {}: Got unexpected RPC data format from client.Nr/ r+ r+ r+ r, r7 S s r7 c s e Zd ZdZ� fdd�Z� ZS )�!UnexpectedRPCDataFormatFromServerz.Got unexpected RPC data format from server: {}c sd z|� � d d� }W n> tyR |d d� }dd� |D �}t�d�|�d�}Y n0 t� �|� d S )N� c S s g | ] }d | �qS )z%02Xr+ )�.0Zbyter+ r+ r, � <listcomp>_ � z>UnexpectedRPCDataFormatFromServer.__init__.<locals>.<listcomp>� �0 )�decode�UnicodeDecodeError�textwrap�fill�join�super�__init__)�selfr( �� __class__r+ r, rE Z s z*UnexpectedRPCDataFormatFromServer.__init__)r0 r1 r2 r3 rE � __classcell__r+ r+ rG r, r8 W s r8 )�repair� save_space)rK r+ )�limit�marker)�idr( �rN )�keydata)� client_data)�path�create� lock_wait�lock� exclusive� append_only)�next_unreserved� start_nonce)�check�commit�rollback�destroy�__len__�list�put�get�delete�save_key�load_key� break_lock� negotiate�open�get_free_nonce�commit_nonce_reservationc s � fdd�� D �S )Nc s i | ]}|� � � | �qS r+ )r? )r: �k��dr+ r, � <dictcomp>� r< z decode_keys.<locals>.<dictcomp>r+ rk r+ rk r, � decode_keys� s rn c @ sR e Zd ZdZdd� Zdd� Zdd� Zdd � Zd d � Zd d � Z ddd�Z dd� Z dS )�RepositoryServer)r^ rZ r[ rb r] ra r_ �scanrf rg r` r\ rc rd re rh ri �inject_exceptionc C s, d | _ || _|| _|| _|| _td�| _d S )N�1.0.8)� repository�restrict_to_paths�restrict_to_repositoriesrW � storage_quotar �client_version)rF rt ru rW rv r+ r+ r, rE � s zRepositoryServer.__init__c sr z � fdd�t t| �D �W S tyl |dkrPt� �dkrPt| dd� }n� � fdd�t |�D � Y S 0 dS )z5Translate from positional protocol to named protocol.c s i | ]\}}|� | �qS r+ r+ �r: �pos�name��argvr+ r, rm � r< z8RepositoryServer.positional_to_named.<locals>.<dictcomp>rg � Nc s i | ]\}}|� | �qS r+ r+ rx r{ r+ r, rm � r< )� enumerate� compatMap� IndexErrorr" )rF �methodr| �mappingr+ r{ r, �positional_to_named� s z$RepositoryServer.positional_to_namedc s$ t t�|�j�� � �fdd��D �S )zWRemove unknown named parameters from call, because client did (implicitly) say it's ok.c s i | ]}|� v r|�| �qS r+ r+ �r: rz �Zknown�kwargsr+ r, rm � r< z0RepositoryServer.filter_args.<locals>.<dictcomp>)�set�inspect� signature� parameters)rF �fr� r+ r� r, � filter_args� s z RepositoryServer.filter_argsc C s� t j�� }t j�� }t j�� }t�|d� t�|d� t�|d� td�}t�|gg g d�\}}}|�r�t�|t �}|s�| j d ur�| j � � nt |d� t��� � d S |�|� |D �]�} t| t�r�d} | t } | t �� } t| t �} n^t| t��r*t| �dk�r*d} | \}} } } | �� } | �| | �} n | j d u�r@| j � � tt��zd| | j v�r^t | ��zt | | �}W n t �y� t | j | �}Y n0 | � || �} |f i | ��}W �n� t!�yh } �z�| �r�t"�#|j$|�}t"j%t �&� � }d}t|t'��r|�(� g}|j"}t|t)j*t)j+t,f��rnt-�.d�/|�� z2t0�1t| d|j$j2d |j3d |d |d |d t4� i�}W nL t5�y� t0�1t| d|j$j2d dd� |j3D �d |d |d |d t4� i�}Y n0 t ||� n�t|t)j*t)j+t,f��r�nft|t'��r�|j"�r�t-j6nt-j7}|�(� }nt-j6}d|j$j2 }t"�8� � dt4� � �}t-�9|� t-�:||� d}t |t0�1d| |j$j2|f�� W Y d }~q�d }~0 0 | �r�t |t0�1t| t;|i�� q�t |t0�1d| d |f�� q�|rJ| j � � d S qJd S )NFTZserver� z<Borg {}: Got connection close before repository was opened. r} � � exception_class� exception_args� exception_full� exception_short� exception_trace� sysinfoc S s$ g | ] }t |tttf�r |nd �qS �N)� isinstance�str�bytes�int)r: �xr+ r+ r, r; s �z*RepositoryServer.serve.<locals>.<listcomp>z%s Exception in RPC callz3Remote Exception (see remote log for the traceback)r )<�sys�stdin�fileno�stdoutr# � set_blockingr �select�read�BUFSIZErs �closer- �formatr �encode�feedr� �dict�MSGID�MSGr? rn �ARGS�tupler" r� r7 � rpc_methodsr6 �getattr�AttributeErrorr� � BaseException� traceback�format_exception_onlyrH �format_exception�exc_infor � get_messager � DoesNotExist� AlreadyExistsr5 �logging�debugrC r �packbr0 �argsr � TypeError�ERROR�DEBUG� format_exc�error�log�RESULT)rF �stdin_fd� stdout_fd� stderr_fd�unpacker�r�w�esr( �unpacked� dictFormat�msgidr� r� �_r� �res�eZex_shortZex_fullZex_trace�msgZ tb_log_level�tb�excr+ r+ r, �serve� s� � � �� 2 zRepositoryServer.servec C sZ |t kr t S t|t�rL|d | _t�t�d�j�}tdd|d� t � d� nt | _dt iS )N� client_version� T)Zis_serve�json�levelz2Initialized logging system for JSON-based protocol�server_version) �RPC_PROTOCOL_VERSIONr� r� rw r� Z getLevelName� getLoggerr� r �loggerr� � BORG_VERSION)rF rQ r� r+ r+ r, rf / s zRepositoryServer.negotiatec C s� t |t�rt�|�}|�d�r8tj�t� |dd � �}nr|�d�r\tj�t� |dd � �}nN|�d�r|tj�|dd � �}n.|�d�r�tj�|�}n|�d�r�|dd � }tj� |�S ) Nz/~/� z~/r z/~r �~z/./) r� r� r# �fsdecode� startswithrR rC r � expanduser�realpath)rF rR r+ r+ r, � _resolve_path@ s z RepositoryServer._resolve_pathFNTc C s� t �d|� | �|�}t �d|� tj�|d�}| jrn| jD ](} tj�tj�| �d�} |�| �r< qnq<t |��| j r�| j D ]&} tj�tj�| �d�} | |krz q�qzt |��| r�| j p�|}t |||||| j ||d�| _| j�� | jjS )Nz Resolving repository path %rz Resolved repository path to %rr� )rT rU rW rv rV �make_parent_dirs)r� r� r� r# rR rC rt r� r� r5 ru rW r rv rs � __enter__rN ) rF rR rS rT rU rV rW r� Z path_with_sepZrestrict_to_pathZrestrict_to_path_with_sepZrestrict_to_repositoryZ restrict_to_repository_with_sepr+ r+ r, rg P s2 � zRepositoryServer.openc C s� |� � }d}d}|dkr$t�|��n�|dkr8t�|��np|dkrLt�|��n\|dkr^t|��nJ|dkrptd��n8|d kr�t�||��n"|d kr�t|��n|d kr�d d d S ) Nz test stringz test string2r� r� � CheckNeededr r5 Zfoo�ObjectNotFoundr6 Zdivider ) r? r r� r� r� r r5 r� r6 )rF �kind�s1�s2r+ r+ r, rq s s& z!RepositoryServer.inject_exception)FNTNFF) r0 r1 r2 r� rE r� r� r� rf r� rg rq r+ r+ r+ r, ro � s k � #ro c @ s e Zd Zdd� Zdd� ZdS )�SleepingBandwidthLimiterc C s0 |r&t |t �| _t�� | _| j| _nd | _d S r� )r� �RATELIMIT_PERIOD� ratelimitr% � monotonic�ratelimit_last�ratelimit_quota)rF rL r+ r+ r, rE � s z!SleepingBandwidthLimiter.__init__c C s� | j r�t�� }| jt |krN| j| j 7 _| jd| j krHd| j | _|| _| jdkr�| jt | }t�|� | j| j 7 _t�� | _t|�| jkr�|d | j� }t� ||�}| j r�| j|8 _|S )Nr r ) r� r% r� r� r� r� r& r" r# r$ )rF r' �to_sendZnowZtosleep�writtenr+ r+ r, r$ � s$ z SleepingBandwidthLimiter.writeN)r0 r1 r2 rE r$ r+ r+ r+ r, r� � s r� c s � �fdd�}|S )aS Check version requirements and use self.call to do the remote method call. <since> specifies the version in which borg introduced this method. Calling this method when connected to an older version will fail without transmitting anything to the server. Further kwargs can be used to encode version specific restrictions: <previously> is the value resulting in the behaviour before introducing the new parameter. If a previous hardcoded behaviour is parameterized in a version, this allows calls that use the previously hardcoded behaviour to pass through and generates an error if another behaviour is requested by the client. E.g. when 'append_only' was introduced in 1.0.7 the previous behaviour was what now is append_only=False. Thus @api(..., append_only={'since': parse_version('1.0.7'), 'previously': False}) allows calls with append_only=False for all version but rejects calls using append_only=True on versions older than 1.0.7. <dontcare> is a flag to set the behaviour if an old version is called the new way. If set to True, the method is called without the (not yet supported) parameter (this should be done if that is the more desirable behaviour). If False, an exception is generated. E.g. before 'threshold' was introduced in 1.2.0a8, a hardcoded threshold of 0.1 was used in commit(). c s t �� �� ��fdd��}|S )Nc s6 t �� �}|j| g|�R i |��}i }i }|j�� D ]Z\}}|dkrFq4||jv rx|dkrh|j| ||< q�|j| ||< q4|j|jur4|j||< q4| j�k r�| � � j t ������� D ]j\}} | d | jkr�q�d| v r�|| | d kr�q�| � dd�r�q�| � � j � d|� d|| ��t | d ���q�| j � j |fi |��S ) NrF �wait�since� previously�dontcareFr= �=)r� r� Zbindr� �itemsZ arguments�default�emptyr� �RPCServerOutdatedr0 r ra �call) rF r� r� ZsigZ bound_argsZnamedZextrarz ZparamZ restriction)r� �kwargs_decoratorr� r+ r, �do_rpc� s4 �z&api.<locals>.decorator.<locals>.do_rpc)� functools�wraps)r� r� �r� r� )r� r, � decorator� s zapi.<locals>.decoratorr+ )r� r� r r+ r r, �api� s #r c @ sf e Zd Zg ZG dd� de�ZG dd� de�ZdZdUdd �Z d d � Z d d � Z dd� Z dd� Z edd� �Zdd� Zdd� Zdd� Zdd� ZdVd d �Zeed �ed �dd �ed!�dd �d"�dWd#d$��Zeed �ed%�d&d �d'�dXd(d)��Zeed �ed*�ddd+�ed,�d-dd+�ed*�ddd+�d.�dYd/d0��Zeed �d1�d2d3� �Zeed �d1�d4d5� �Zeed �d1�d6d7� �Z eed �d1�dZd8d9��Z eed:�d1�d[d;d<��Z d=d>� Z d\d?d@�Z eed �d1�d]dAdB��Z!eed �d1�d^dCdD��Z"eed �d1�dEdF� �Z#eed �d1�dGdH� �Z$eed �d1�dIdJ� �Z%eed �d1�dKdL� �Z&eed �d1�dMdN� �Z'dOdP� Z(d_dQdR�Z)dSdT� Z*dS )`�RemoteRepositoryc @ sL e Zd Zdd� Zdd� Zedd� �Zedd� �Zed d � �Zed d � �Z d S )zRemoteRepository.RPCErrorc C s || _ d S r� )r� )rF r� r+ r+ r, rE � s z"RemoteRepository.RPCError.__init__c C s( d| j v r d�| j d ��� S | jS d S )Nr� � )r� rC r? �exception_class�rF r+ r+ r, r� � s z%RemoteRepository.RPCError.get_messagec C s | j �dd�S )Nr� T)r� ra r r+ r+ r, r� � s z#RemoteRepository.RPCError.tracebackc C s | j d �� S )Nr� �r� r? r r+ r+ r, r � s z)RemoteRepository.RPCError.exception_classc C s. d| j v r d�| j d ��� S | �� d S d S )Nr� r z4 Remote Exception (see remote log for the traceback))r� rC r? r� r r+ r+ r, �exception_full� s z(RemoteRepository.RPCError.exception_fullc C s d| j v r| j d �� S dS d S )Nr� r� r r r+ r+ r, r � s z!RemoteRepository.RPCError.sysinfoN) r0 r1 r2 rE r� �propertyr� r r r r+ r+ r+ r, �RPCError� s r c @ s( e Zd ZdZedd� �Zedd� �ZdS )z"RemoteRepository.RPCServerOutdatedz2Borg server is too old for {}. Required version {}c C s | j d S �Nr �r� r r+ r+ r, r� s z)RemoteRepository.RPCServerOutdated.methodc C s | j d S )Nr r r r+ r+ r, �required_version s z3RemoteRepository.RPCServerOutdated.required_versionN)r0 r1 r2 r3 r r� r r+ r+ r+ r, r� s r� FNTc s� | �_ �_g �_d�_d�_d�_tdt��_d�_ i �_ t � �_ i �_ i �_d �_t|rl|jrl|jd nd��_|r�|jr�|jd d nd�_td��_td��_d �_|�_|jdk} t | d�} �� || �} | s�� |�| } t � d | � t!| dt"t"t"| t#d ��_�jj$�%� �_&�jj'�%� �_(�jj)�%� �_*t+�,�j&d � t+�,�j(d � t+�,�j*d � �j(�j*g�_-�j&�j(�j*g�_.�zz��/d d dt0ii�} W n t1�y� t2d�d �Y n0 | t3k�r�d �_4n4t5| t6��r�d| v �r�d�_4| d �_n t7d| ��� ������fdd�} �j4�r(| � njz | � W n^ �j8�y� } zB|j9dk�rT� d}t:j)�;|� td��_dt<d< | � W Y d }~n d }~0 0 W n t7�y� ��=� � Y n0 d S )Nr i r< i Zclientrr Z __testsuite__)�systemzSSH command line: %s)�bufsizer� r� �stderr�envZ preexec_fnFrf rQ r� z Is borg working on the server?s server_versionTz8Server insisted on using unsupported protocol version %sc s" �j �jj����� �d��_d S )N)rR rS rT rU rV rW r� )rg �locationrR rN r+ �rW rS rV rU rT r� rF r+ r, �do_openK s �z*RemoteRepository.__init__.<locals>.do_openr� ao Please note: If you see a TypeError complaining about the number of positional arguments given to open(), you can ignore it if it comes from a borg version < 1.0.7. This TypeError is a cosmetic side effect of the compatibility code borg clients >= 1.0.7 have to support older borg servers. This problem will go away as soon as the server has been upgraded to 1.0.7+. z1.0.6)rR rS rT rU rg )>r Z _location� preload_idsr� �rx_bytes�tx_bytesr r� r� �stderr_received�chunkid_to_msgidsr� �ignore_responses� responses�async_responses� shutdown_timer� Zupload_ratelimitr� Z upload_buffer�upload_buffer_size_limitr r� r r� �p�_args�hostr �borg_cmd�ssh_cmdr� r� r r r r� r� r� r� r� r r� r# r� �r_fds�x_fdsr� r� r. r4 r� r� r� r� � Exceptionr r r� r$ r r� )rF r rS rV rT rU rW r� r� �testingr r" �versionr �errr� r+ r r, rE s| � zRemoteRepository.__init__c C s> t | j�r t�dt | j�f � | jr:| �� ds:J d��d S )Nz2still %d cached responses left in RemoteRepositoryFz&cleanup happened in Repository.__del__)r" r r� r� r r� r r+ r+ r, �__del__k s zRemoteRepository.__del__c C s d| j j� d| j�� � d�S )N�<r= �>)rH r0 r Zcanonical_pathr r+ r+ r, �__repr__r s zRemoteRepository.__repr__c C s | S r� r+ r r+ r+ r, r� u s zRemoteRepository.__enter__c C sv zH|d ur t �� d | _| �� W t�dt| j�t| j�| j � | � � n(t�dt| j�t| j�| j � | � � 0 d S )N� zDRemoteRepository: %s bytes sent, %s bytes received, %d messages sent) r% r� r r\ r� r� r r r r� r� �rF �exc_typeZexc_valZexc_tbr+ r+ r, �__exit__x s � ��zRemoteRepository.__exit__c C s t | j�S r� )r rN r r+ r+ r, �id_str� s zRemoteRepository.id_strc C s2 g }|dur�t �� }|�t j�r,|�d� n^|�t j�rD|�d� nF|�t j�rRn8|�t j�rj|�d� n |�t j�r�|�d� nt d��|j D ]*}d|vr�d| }d |v r�|�d | � q�d |v r�|j r�|�d |j � g }|�r |t j d ddg | | j S |j�ptj�dd�}t|�}||dg | S dS )z return a borg serve command lineNz--debugz--infoz--errorz --criticalz log level missing, fix this code�.z borg.debug.rs z--debug-topic=%srv z--storage-quota=%sz-mz borg.archiverr� ZBORG_REMOTE_PATH�borg)r� r� Z isEnabledForr� �append�INFOZWARNINGr� �CRITICAL� ValueErrorZ debug_topicsrv r� � executable�extra_test_args� remote_pathr# �environra r )rF r� r' ZoptsZ root_loggerZtopicZenv_varsr; r+ r+ r, r" � s6 zRemoteRepository.borg_cmdc C sl | j jptj�dd�}t�|�}|jr8|dt|j�g7 }|j rX|� |j � d|j � �� n|� d|j � |S )zEreturn a ssh command line that can be prefixed to a borg command lineZBORG_RSHZsshz-p�@�%s) r �rshr# r< ra �shlex�split�portr� �userr5 r! )rF r r? r� r+ r+ r, r# � s zRemoteRepository.ssh_cmdc s � fdd�t | D �S )Nc s g | ] }� | �qS r+ r+ r� �r� r+ r, r; � r< z8RemoteRepository.named_to_positional.<locals>.<listcomp>)r )rF r� r� r+ rD r, �named_to_positional� s z$RemoteRepository.named_to_positionalc K s& | j ||gfi |��D ] }| S d S r� �� call_many)rF �cmdr� �kw�respr+ r+ r, r� � s zRemoteRepository.callc # s: |s|dkrd S � fdd�}� fdd�}� fdd�}t |�}g } |rHdn� j} |� |s^|�r$� jr�t�� � jkr�t�d t| �t� j�� d S | r�zH� j � | d �} | � d� d | v r�|| � n| t V | s�|s�W d S W q� t y� Y q�Y q�0 q�|dk�rbz� j� � \} } W n0 t �y@ |�r6� j�r6Y �qbnY d S Y q�0 d | v �rV|| � q�| t V q�� j�s�|�sx� j�r�t| �tk �r�� jg} ng } t�� j| � jd �\}}}|�r�td ��|D �]�}|� ju �r�t�|t�}|�s�t� �� j t|�7 _ � j � |� � j D ]�} t | t ��r(| t! } nPt | t"��rpt| �d k�rp| \}} }}|�rbt!| d |i} n t!| t |i} nt#|��| � jv �r�� j�$| � d | v �r�| � j| < n| t d u�r�| � j| < n | � j | < �qn�|� j%u �r�t�|d�}|�s�t� �� j t|�7 _ � j&�r � j&| }d� _&|j'dd�}|�rJ|d �(d��sJ|� � � _&|D ]}t)|�*� � �qN�q�|rTt� j�| k�r |�s�� j�r t| �tk �r |�r�|�r�|dk�s�J d��|d d � j+v �r�| �,||� d�d �� n�|� d�}|dk�r|d � j+v �r| �,||d �� nh� j-d 7 _-| �,� j-� � j.�r^� j�/t0�1t!� j-t2|t3|i�� n$� j�/t0�1d � j-|� �4||�f�� � j�sj� j�rj� j� d�}d|i}� j-d 7 _-� j+�5|g ��,� j-� � j.�r� j�/t0�1t!� j-t2dt3|i�� n$� j�/t0�1d � j-d� �4d|�f�� �qj|� qT� jt6| �O _d S )Nr c sx � j rtz4� j�� j� j �� �} � j| 7 _� j �| � W n8 tyr } z |jtj tj fvr^� W Y d }~n d }~0 0 d S r� ) r� r� r$ r� Z peek_frontr Z pop_front�OSError�errnoZEAGAINZ EWOULDBLOCK)r� r� r r+ r, � send_buffer� s z/RemoteRepository.call_many.<locals>.send_bufferc s& � j | �d�}� j | s"� j | = |S r )r �pop)Zchunkidr� r r+ r, �pop_preload_msgid� s z5RemoteRepository.call_many.<locals>.pop_preload_msgidc s| | d � � }d| v}| �d�}|dkr8t�� jj���n@|dkrRt�� jj���n&|dkrlt�� jj���n |dkr�|r�td��nt|d � � ��n�|d kr�|r�t� d��nt� |d � � ��n�|d kr�|r�t d ��nt |d � � ��n�|d k�rt� |d � � ��nr|d k�rB|�r(t� d� jj��nt� |d � � � jj��n6|dk�rn|�r\t d��nt |d � � ��n � �| ��d S )Nr� r� r� r� r� r z(not available)r �AtticRepositoryr5 z (unknown)�ParentPathDoesNotExistr� r6 )r? ra r r� r Z processedr� r� r rP r5 rQ r� r6 r )r� r� Z old_serverr� r r+ r, � handle_error� s@ z0RemoteRepository.call_many.<locals>.handle_errorr zPshutdown_time reached, shutting down with %d waiting_for and %d async_responses.r� r zFD exception occurredr} i � r< T)�keepends���)� r ra z&is_preload is only supported for 'get'rN )7r_ r r r% r� r� r� r" r r rN r� �KeyError�popitemr r� r � MAX_INFLIGHTr� r� r$ r% r& r� r# r� r� r. r r� r� r� r� r� r� r8 �remover� r � splitlines�endswith�handle_remote_liner? r r5 r� r� Z push_backr r� r� r� rE � setdefaultr� )rF rH �callsr� � is_preloaded� async_waitrM rO rR Z waiting_forZmaximum_to_sendr� r� Zw_fdsr� r� r� r' r( r� r� r� �lines�liner� Zchunk_idr+ r r, rG � s� ) � $ , "$ "(zRemoteRepository.call_manyz1.0.0z1.0.7)r� r� z1.1.9)r� rW r� c C s dS �z;actual remoting is done via self.call in the @api decoratorNr+ )rF rR rS rT rU rV rW r� r+ r+ r, rg � s zRemoteRepository.openz1.2.0a4r )r� � max_durationc C s dS rc r+ )rF rJ rK rd r+ r+ r, rZ � s zRemoteRepository.checkz1.2.0a0)r� r� r� z1.2.0a8r! )r� �compact� threshold�cleanup_commitsc C s dS rc r+ )rF rK re rf rg r+ r+ r, r[ � s zRemoteRepository.commit)r� c C s dS rc r+ r r+ r+ r, r\ � s zRemoteRepository.rollbackc C s dS rc r+ r r+ r+ r, r] � s zRemoteRepository.destroyc C s dS rc r+ r r+ r+ r, r^ � s zRemoteRepository.__len__c C s dS rc r+ �rF rL rM r+ r+ r, r_ � s zRemoteRepository.listz1.1.0b3c C s dS rc r+ rh r+ r+ r, rp � s zRemoteRepository.scanc C s | � |g�D ] }| S d S r� )�get_many)rF rN rJ r+ r+ r, ra � s zRemoteRepository.getc c s$ | j ddd� |D �|d�E d H d S )Nra c S s g | ] }d |i�qS rO r+ )r: rN r+ r+ r, r; � r< z-RemoteRepository.get_many.<locals>.<listcomp>)r_ rF )rF �idsr_ r+ r+ r, ri � s zRemoteRepository.get_manyc C s dS rc r+ )rF rN r( r� r+ r+ r, r` � s zRemoteRepository.putc C s dS rc r+ )rF rN r� r+ r+ r, rb � s zRemoteRepository.deletec C s dS rc r+ )rF rP r+ r+ r, rc � s zRemoteRepository.save_keyc C s dS rc r+ r r+ r+ r, rd � s zRemoteRepository.load_keyc C s dS rc r+ r r+ r+ r, rh � s z RemoteRepository.get_free_noncec C s dS rc r+ )rF rX rY r+ r+ r, ri � s z)RemoteRepository.commit_nonce_reservationc C s dS rc r+ r r+ r+ r, re � s zRemoteRepository.break_lockc C s2 | j r.| j j�� | j j�� | j �� d | _ d S r� )r r� r� r� r� r r+ r+ r, r� � s zRemoteRepository.closec C s" | j dg d|d�D ] }| S d S )Nr T)r^ r� r` rF )rF r� rJ r+ r+ r, �async_response� s z RemoteRepository.async_responsec C s | j |7 _ d S r� )r )rF rj r+ r+ r, �preload� s zRemoteRepository.preload)FFNTFFN)TFT)FNTFFF)FFr )FTr! F)NN)NN)F)T)T)T)+r0 r1 r2 r: r& r r r� r� rE r* r- r� r1 r r2 r" r# rE r� rG r r rg rZ r[ r\ r] r^ r_ rp ra ri r` rb rc rd rh ri re r� rk rl r+ r+ r+ r, r � s� " � W . = � � � � r c C s | � d�sJ �| �d��rbt�| �}|d dvrFt�d|d | � dS |d dkr�tt|d tj�}t |t �srJ �t� |d �}d |d |d < t� d �jr�||� � kr�t j�t�|�d � n|�|d|d � n�|d �d��r t� d�}|� � tjk�r t� d �j�r>d |v �r&d |d |d < t j�t�|�d � n"d |v �r t j�d |d d � n�| �d��r�| �dd�\}}}tt|tj�}|�d��r�t�||�� � n&|�dd�\}}t� |��||�� � n8t� d �j�r�t� d��d | �� � nt j�d | � dS )zv Handle a remote log line. This function is remarkably complex because it handles multiple wire formats. )� r� �{�type)Zprogress_messageZprogress_percent� log_messagez3Dropped remote log message with unknown type %r: %sNrp Z levelnamerz zRemote: �messager4 r� r> Z progress_zborg.output.progressrm z$LOG r= r zRemote:r r� )r[ r� r� �loadsr� Zwarningr� r� r7 r� r� r� ZgetEffectiveLevelr� r r$ �dumpsr� r6 rA �rstrip�strip)rb r� r� Z target_loggerZprogress_loggerr� Zlognamer+ r+ r, r\ � sB r\ c @ sL e Zd ZdZddd�Zdd� Zdd� Zd d � Zd d � Zddd�Z dd� Z dS )�RepositoryNoCacheaR A not caching Repository wrapper, passes through to repository. Just to have same API (including the context manager) as RepositoryCache. *transform* is a callable taking two arguments, key and raw repository data. The return value is returned from get()/get_many(). By default, the raw repository data is returned. Nc C s || _ |pdd� | _d S )Nc S s |S r� r+ )�keyr( r+ r+ r, �<lambda>4 r< z,RepositoryNoCache.__init__.<locals>.<lambda>)rs � transform)rF rs ry r+ r+ r, rE 2 s zRepositoryNoCache.__init__c C s d S r� r+ r r+ r+ r, r� 6 s zRepositoryNoCache.closec C s | S r� r+ r r+ r+ r, r� 9 s zRepositoryNoCache.__enter__c C s | � � d S r� )r� r/ r+ r+ r, r1 < s zRepositoryNoCache.__exit__c C s t | j|gdd��S )NF��cache)�nextri �rF rw r+ r+ r, ra ? s zRepositoryNoCache.getTc c s. t || j�|��D ]\}}| �||�V qd S r� )�ziprs ri ry )rF �keysr{ rw r( r+ r+ r, ri B s zRepositoryNoCache.get_manyc C s d S r� r+ r r+ r+ r, �log_instrumentationF s z%RepositoryNoCache.log_instrumentation)N)T) r0 r1 r2 r3 rE r� r� r1 ra ri r� r+ r+ r+ r, rv ) s rv c s\ e Zd ZdZd� fdd� Zdd� Zdd� Zd d � Zd d � Zd d� Z dd� Z ddd�Z � Z S )�RepositoryCacheaf A caching Repository wrapper. Caches Repository GET operations locally. *pack* and *unpack* complement *transform* of the base class. *pack* receives the output of *transform* and should return bytes, which are stored in the cache. *unpack* receives these bytes and should return the initial data (as returned by *transform*). Nc sv t � �||� |pdd� | _|p&dd� | _t� | _tjdd�| _| � � d| _ d| _ d| _ d| _ d| _d| _d| _d S )Nc S s | S r� r+ �r( r+ r+ r, rx X r< z*RepositoryCache.__init__.<locals>.<lambda>c S s | S r� r+ r� r+ r+ r, rx Y r< z borg-cache-)�prefixr g )rD rE �pack�unpackr� r{ �tempfileZmkdtemp�basedir�query_size_limit�size�hits�misses� slow_misses�slow_lat� evictions�enospc)rF rs r� r� ry rG r+ r, rE V s zRepositoryCache.__init__c C s& t �| j�j}tt|d d��| _d S )Ng �?l )�shutil� disk_usager� �freer� �min� size_limit)rF Zavailable_spacer+ r+ r, r� f s z RepositoryCache.query_size_limitc C s t j�| jt|��S r� )r# rR rC r� r r} r+ r+ r, � key_filenamej s z RepositoryCache.key_filenamec C sn | � � td| j �}| j|krj| jrj| j�� }| �|�}| jt�|�j 8 _t� |� | j d7 _ qd S )Ng�������?r ) r� r� r� r� r{ rN r� r# �stat�st_size�unlinkr� )rF Z target_sizerw �filer+ r+ r, �backoffm s zRepositoryCache.backoffc C s | � ||�}|s|S | �|�}| �|�}z8t|d��}|�|� W d � n1 sT0 Y W nj ty� } zRz t|� W n ty� Y n0 |jtj kr�| j d7 _ | � � n� W Y d }~n<d }~0 0 | j t |�7 _ | j�|� | j | jkr�| � � |S )N�wbr )ry r� r� rg r$ rK r �FileNotFoundErrorrL �ENOSPCr� r� r� r" r{ �addr� ) rF rw r( r{ � transformed�packedr� r' Zos_errorr+ r+ r, � add_entryw s, , zRepositoryCache.add_entryc C s> t �dt| j�t| j�t| j�| j| j| j | j | j | j � d S )NzyRepositoryCache: current items %d, size %s / %s, %d hits, %d misses, %d slow misses (+%.1fs), %d evictions, %d ENOSPC hit) r� r� r" r{ r r� r� r� r� r� r� r� r� r r+ r+ r, r� � s �z#RepositoryCache.log_instrumentationc C s" | � � | j�� t�| j� d S r� )r� r{ �clearr� �rmtreer� r r+ r+ r, r� � s zRepositoryCache.closeTc # s$ � fdd�|D �}t |� j�|��}|D ]�}|� jv r�� �|�}t|d��.}� jd7 _� �|�� �V W d � n1 s~0 Y q(|D ]6\}} ||kr�� � || |�} � j d7 _ | V q(q�t � � } � j� |�} � jt � � | 7 _� � || |�} � jd7 _| V q(|D ]} �qd S )Nc s g | ]}|� j vr|�qS r+ rz )r: rw r r+ r, r; � r< z,RepositoryCache.get_many.<locals>.<listcomp>�rbr )r~ rs ri r{ r� rg r� r� r� r� r� r% � perf_counterra r� r� ) rF r r{ Z unknown_keysZrepository_iteratorrw r� r' Zkey_r( r� Zt0r� r+ r r, ri � s, 0 zRepositoryCache.get_many)NNN)T) r0 r1 r2 r3 rE r� r� r� r� r� r� ri rI r+ r+ rG r, r� J s r� F)�decrypted_cacher� r� ry � force_cachec s� |r|s|s|rt d��nB|r\|�t�d�� td��� �fdd�}� �fdd�}�fdd �}t| t�sj|rxt| |||�S t| |�S d S ) a� Return a Repository(No)Cache for *repository*. If *decrypted_cache* is a key object, then get and get_many will return a tuple (csize, plaintext) instead of the actual data in the repository. The cache will store decrypted data, which increases CPU efficiency (by avoiding repeatedly decrypting and more importantly MAC and ID checking cached objects). Internally, objects are compressed with LZ4. z:decrypted_cache and pack/unpack/transform are incompatiblez=I8sZlz4c s& | \}}�� |�}� �|t|��| S r� )�compressr� r )r( �csize� decrypted� compressed�� cache_struct� compressorr+ r, r� � s z cache_if_remote.<locals>.packc sP t | �} � �| d � j� �\}}| � jd � }|t|�krBtd��|��|�fS )Nz)detected corrupted data in metadata cache)� memoryviewr� r� r r � decompress)r( r� Zchecksumr� r� r+ r, r� � s z cache_if_remote.<locals>.unpackc s t |�}� �| |�}||fS r� )r" Zdecrypt)Zid_r( r� r� )rw r+ r, ry � s z"cache_if_remote.<locals>.transformN)r8 �struct�Structr r� r r� rv )rs r� r� r� ry r� r+ )r� r� rw r, �cache_if_remote� s r� )JrL r� r� r� r� r# r� r@ r� r� r� r� rA r% r� � subprocessr r r� r r� r Z constantsZhelpersr r r r r r r r r r r r� r r r rs r r( r r Zalgorithms.checksumsr Zhelpers.datastructr r0 r� r� r� r� r� r� rX r� r- r. r4 r5 r6 r7 r8 r rn ro r� r r r\ rv r� r� r+ r+ r+ r, �<module> s� #� y : J!p
Simpan