torchsig.utils.file_handlers.zarr.ZarrFileHandler

class torchsig.utils.file_handlers.zarr.ZarrFileHandler(root: str, batch_size: int = 1)

Bases: TorchSigFileHandler

Handler for reading and writing data to/from a Zarr file format.

This class extends the TorchSigFileHandler and provides functionality to handle reading, writing, and managing Zarr-based storage for dataset samples.

datapath_filename

The name of the folder used to store the data in Zarr format.

Type:

str

Methods

exists	Checks if the Zarr file exists at the specified path.
load
setup
size	Return size of dataset
static_load	Loads a sample from the Zarr file at the specified index (without instantiating a ZarrFileHandler)
teardown
write	Writes a sample (data and targets) to the Zarr file at the specified index.

Attributes

datapath_filename_base

__init__(root: str, batch_size: int = 1)

Initializes the ZarrFileHandler

Parameters:

• root (str) – Where to write dataset on disk.

• batch_size (int, optional) – Size fo each batch write. Defaults to 1.

exists() → bool

Checks if the Zarr file exists at the specified path.

Returns:

True if the Zarr file exists, otherwise False.

Return type:

bool

write(batch_idx: int, batch: Any) → None

Writes a sample (data and targets) to the Zarr file at the specified index.

Parameters:

• idx (int) – The index at which to store the data in the Zarr file.

• data (np.ndarray) – The data to write to the Zarr file.

• targets (Any) – The corresponding targets to write as metadata for the sample.

Notes

If the index is greater than the current size of the array, the array is expanded to accommodate the new sample.

static size(dataset_path: str) → int

Return size of dataset

Parameters:

dataset_path (str) – path to dataset on disk

Returns:

size of dataset

Return type:

int

static static_load(filename: str, idx: int) → Tuple[ndarray, List[Dict[str, Any]]]

Loads a sample from the Zarr file at the specified index (without instantiating a ZarrFileHandler)

Parameters:

• filename (str) – Path to the directory containing the Zarr file.

• idx (int) – The index of the sample to load.

Returns:

The data and the associated metadata for the sample.

Return type:

Tuple[np.ndarray, List[Dict[str, Any]]]

Raises:

IndexError – If the index is out of bounds.

__repr__() → str

Return repr(self).

__str__() → str

Return str(self).