o Rh8@stddlZddlZddlmZmZmZmZddlmZm Z m Z m Z GdddeZ GdddeZ Gdd d eZdS) N)CompleteMultipartUploadTaskCreateMultipartUploadTaskSubmissionTaskTask)ChunksizeAdjustercalculate_range_parameter get_callbacksget_filtered_dictc @sxeZdZdZdddddddd d d Zgd Zgd ZgdZddZddZ ddZ ddZ ddZ ddZ ddZdS)CopySubmissionTaskz+Task for submitting tasks to execute a copyIfMatchIfModifiedSince IfNoneMatchIfUnmodifiedSinceSSECustomerKeySSECustomerAlgorithmSSECustomerKeyMD5 RequestPayerExpectedBucketOwner) CopySourceIfMatchCopySourceIfModifiedSinceCopySourceIfNoneMatchCopySourceIfUnmodifiedSinceCopySourceSSECustomerKeyCopySourceSSECustomerAlgorithmCopySourceSSECustomerKeyMD5rr) rrrrrrrrrrrr) rrrrrrrMetadataDirectiveTaggingDirective)rrrrrc Cs|jjdur9|jj}||j}|j}|D]\} } | |jvr'| ||j| <q|jj di|} |j | d|jj|j krK| |||||dS| |||||dS)a :param client: The client associated with the transfer manager :type config: s3transfer.manager.TransferConfig :param config: The transfer config associated with the transfer manager :type osutil: s3transfer.utils.OSUtil :param osutil: The os utility associated to the transfer manager :type request_executor: s3transfer.futures.BoundedExecutor :param request_executor: The request executor associated with the transfer manager :type transfer_future: s3transfer.futures.TransferFuture :param transfer_future: The transfer future associated with the transfer request that tasks are being submitted for N ContentLength)metasize call_args)_get_head_object_request_from_copy_source copy_source extra_argsitemsEXTRA_ARGS_TO_HEAD_ARGS_MAPPING source_client head_objectprovide_transfer_sizemultipart_threshold_submit_copy_request_submit_multipart_request) selfclientconfigosutilrequest_executortransfer_futurer!head_object_requestr$paramvalueresponserrS/home/air/sanwanet/backup_V2/venv/lib/python3.10/site-packages/s3transfer/copies.py_submitPs6     zCopySubmissionTask._submitc CsL|jj}t|d}|j|t|j||j|j|j|j ||jj ddddS)Nprogress)r.r#bucketkeyr$ callbacksr T)transfer_coordinator main_kwargsis_final) rr!r_transfer_coordinatorsubmitCopyObjectTaskr#r:r;r$r )r-r.r/r0r1r2r!progress_callbacksrrr7r+s"  z'CopySubmissionTask._submit_copy_requestcCsx|jj}i}|jD] \}} ||jvr| ||<q |j|t|j||j|j |dd} |j } t } | | |jj } tt|jj t| } g}t|d}td| dD]F}||j}t| |d| |jj |d<|| |d| |jj }|jd}||j|t|j||j|j|j |||||d d| id qU||j}|j|t|j||j|j |d| |d d d dS) N)r.r:r;r$)r=r>r9CopySourceRangeChecksumAlgorithm) r.r#r:r; part_numberr$r<r checksum_algorithm upload_id)r=r>pending_main_kwargs)rIpartsT)r=r>rJr?)rr!r$r%CREATE_MULTIPART_ARGS_BLACKLISTr@rArr:r;multipart_chunksizeradjust_chunksizer intmathceilfloatrrange_extra_upload_part_argsr_get_transfer_sizegetappend CopyPartTaskr#_extra_complete_multipart_argsr)r-r.r/r0r1r2r!create_multipart_extra_argsr4valcreate_multipart_future part_sizeadjuster num_parts part_futuresrCrGextra_part_argsr rHcomplete_multipart_extra_argsrrr7r,s    z,CopySubmissionTask._submit_multipart_requestcCs(t|tr t|Std|t|f)NzZExpecting dictionary formatted: {"Bucket": bucket_name, "Key": key} but got %s or type %s.) isinstancedictcopy TypeErrortype)r-r#rrr7r"s   zl  r c@seZdZdZddZdS)rBzTask to do a nonmultipart copyc Cs0|jd|||d||D]}||dqdS)a :param client: The client to use when calling PutObject :param copy_source: The CopySource parameter to use :param bucket: The name of the bucket to copy to :param key: The name of the key to copy to :param extra_args: A dictionary of any extra arguments that may be used in the upload. :param callbacks: List of callbacks to call after copy :param size: The size of the transfer. This value is passed into the callbacks ) CopySourceBucketKeybytes_transferredNr) copy_object) r-r.r#r:r;r$r<r callbackrrr7_main3s zCopyObjectTask._mainNrorprqrrrzrrrr7rB0s rBc@seZdZdZ dddZdS)rXz)Task to upload a part in a multipart copyNc Csx|jd|||||d|} |D]} | | dq| dd} | |d}| r:d| }|| dvr:| d|||<|S) a< :param client: The client to use when calling PutObject :param copy_source: The CopySource parameter to use :param bucket: The name of the bucket to upload to :param key: The name of the key to upload to :param upload_id: The id of the upload :param part_number: The number representing the part of the multipart upload :param extra_args: A dictionary of any extra arguments that may be used in the upload. :param callbacks: List of callbacks to call after copy part :param size: The size of the transfer. This value is passed into the callbacks :param checksum_algorithm: The algorithm that was used to create the multipart upload :rtype: dict :returns: A dictionary representing a part:: {'Etag': etag_value, 'PartNumber': part_number} This value can be appended to a list to be used to complete the multipart upload. If a checksum is in the response, it will also be included. )rsrtruUploadId PartNumberrvCopyPartResultETag)rr}ChecksumNr)upload_part_copyupper)r-r.r#r:r;rIrGr$r<r rHr6ryetag part_metadatachecksum_memberrrr7rzLs(&    zCopyPartTask._mainrir{rrrr7rXIs rX)rerPs3transfer.tasksrrrrs3transfer.utilsrrrr r rBrXrrrr7s