roboto.storage.object_store.s3
Module Contents
ProgressCallbackInvoker
Bases: boto3.s3.transfer.BaseSubscriber
Invoke a provided callback via a subscriber.
Taken from https://github.com/boto/boto3/blob/develop/boto3/s3/transfer.py#L505C1-L516C42
ProgressCallbackInvoker.on_progress()
Callback to be invoked when progress is made on transfer
This callback can be useful for:
- Recording and displaying progress
Parameters
future s3transfer.The TransferFuture representing the requested transfer.
bytes_transferred intThe number of bytes transferred for that invocation of the callback. Note that a negative amount can be provided, which usually indicates that an in-progress request needed to be retried and thus progress was rewound.
S3Store
Bases: roboto.storage.object_store.object_store.ObjectStore
Base class for protocol classes.
Protocol classes are defined as:
class Proto(Protocol):
def meth(self) -> int:
...Such classes are primarily used with static type checkers that recognize structural subtyping (static duck-typing).
For example:
class C:
def meth(self) -> int:
return 0
def func(x: Proto) -> int:
return x.meth()
func(C()) # Passes static type checkSee PEP 544 for details. Protocol classes decorated with @typing.runtime_checkable act as simple-minded runtime protocols that check only the presence of given attributes, ignoring their type signatures. Protocol classes can be generic, they are defined as:
class GenProto[T](Protocol):
def meth(self) -> T:
...Parameters
s3_client botocore.transfer_config Optional[boto3.S3Store.create()
Factory function to assemble an S3Store with refreshable credentials.
Parameters
credential_provider roboto.Return type
S3Store.get()
Downloads a file from a cloud URI to a local path.
Parent directories of the destination path are created automatically if they don’t exist.
Parameters
source_uri strFull URI (e.g., ‘s3://my-bucket/folder/data.csv’)
destination pathlib.Local path where the file should be saved.
on_progress Optional[roboto.Optional callback to be periodically called with the number of bytes downloaded.
Return type
S3Store.put()
Uploads a local file to a specific cloud URI.
Parameters
source pathlib.Local path to the file.
destination_uri strFull URI (e.g., ‘s3://my-bucket/folder/data.csv’)
on_progress Optional[roboto.Optional callback to be periodically called with the number of bytes uploaded.
Return type